Please Read Accelerating Face Detection on GMSL-based Camera with Vitis AI Part 3

The Vitis Accelerator build flow that is used in this tutorial takes a base platform and links in various Xilinx Object (.xo) files that make up the FPGA accelerators to generate a final FPGA binary file (.xclbin). This tutorial uses a single .xo for the DPU. To build the DPU accelerator, run

make repos/Vitis-AI/DPU-TRD/prj/Vitis/binary_container_1/dpu.xo

The target is shown below

The first step is to copy dpu/dpu_conf.vh into the build directory. This file contains the desired configuration for the DPU IP. This file already exists in Vitis-AI repo, but for demonstration purposes, this tutorial builds a different configuration. Namely, this design uses a B2304 configuration while everything else is default.

The second step is to build dpu.xo using the makefile that’s provided in the Vitis-AI repo.

The next build step is to create the Vitis platform. A Vitis platform contains the base Vivado project built previously and various components from the Petalinux project such as the boot files, linux image, and sysroot. To create the Vitis platform, run the following command:

source <xrt_install>/xrt/setup.sh

make vitis_platform_workspace/vitis_ai_multicam_plat/export/vitis_ai_multicam_plat/vitis_ai_multicam_plat.xpfm

This target is shown below

The main task is to source scripts/build_vitis_platform.tcl using Xilinx Software Command Line Tool (xsct). scripts/build_vitis_platform.tcl is shown below

This script executes several Vitis commands. First, a platform project is created based on the .xsa file from a previous step. Note that this design builds the application using petalinux. If it is preferred to build the application in Vitis itself, the -sysroot flag should be added to the domain create call so that the application will build against the customized rootfs. Then a Linux domain is created pointing to the linux image from the petalinux build. Some additional properties are set for the domain and then the platform is generated to create the .xpfm file.

Link the dpu.xo file with the base platform to create the final FPGA binary (.xclbin) with the following command

                make repos/Vitis-AI/DPU-TRD/prj/Vitis/binary_container_1/dpu.xclbin

which calls Vitis v++ as shown below

This step takes dpu/prj_config file as an input which informs Vitis how to connect the DPU accelerator to the rest of the platform

The freqHz properties of the clocks match the clocking wizard actual output frequencies. The connections are made according to the sptag properties in the block design.

The next step is to build the neural network model itself. This involves calling Vitis-AI dlet command (inside the tools docker container) to generate a DPU Configuration File (DCF) and then generate a DPU executable for the specific model that is used in this tutorial (densebox). Build it with

                make repos/Vitis-AI/Rebuild-Model-Zoo/compiled_output/cf_densebox_wider_360_640_1.11G/dpu_densebox.elf

This make target is rather complicated to prepare the file structure for ease of calling things inside docker container given the existing Vitis-AI repository organization

The first task is to extract the .hwh file from system.hdf in the Vivado project. This file is used by the Vitis AI tools to parse various information about DPU configuration. This file is an input to DLet Vitis-AI tool which creates the DCF.

Next, scripts/launch_docker.sh is copied into the Vitis-AI repository. This file is similar to repos/Vitis-AI/docker_run.sh which comes with the Vitis-AI repository, only slightly modified to

1. Add -d switch to run the docker container in the background
2. Add --name model_builder switch so we can connect to it and execute a command inside the docker container at a later time

scripts/build_model.sh is also copied into the Vitis-AI repository. This script is actually executed inside the docker container

This script first sets up the Vitis-AI tools environment, calls dlet, and then runs scripts/rebuild-model-zoo.py. rebuild-model-zoo.py calls vai_c_caffe to generate the DPU executable.

Now that the directory structure is correct, launch_docker.sh is executed to spin up the docker container. Then build_model.sh is executed inside that container. Finally, the docker container is stopped and extraneous files are cleaned up.

The last step in the makefile is to copy all the necessary binaries to the sd_card directory.

                make sd_card

Bootgen is also called during this step to create the final BOOT.BIN that contains the bitstream created by Vitis v++.

To run the demonstration, follow the steps from the Quick Start section starting at the Prepare the SD Card step.

This tutorial has demonstrated all steps necessary for building a custom Vitis AI platform to accelerate real-time neural network processing on live camera stream. Beginning with the FPGA platform design, the unique considerations for enabling DPU accelerator were shown. The software design was explained next in order to highlight the steps necessary to enable DPU interfacing in Linux. Finally, the Vitis platform was developed and integrated with the DPU executable outputs from Vitis AI to complete the design. Having finished this tutorial, the system designer now has all the skills required to leverage Xilinx’s unique machine learning acceleration technology for deployment in custom designs.