FRED Runtime

The FRED runtime is the reference implementation of the FRED framework for the GNU/Linux operating system. It has been designed to run on the Xilinx Zynq-7000 and Zynq UltraScale+ SoC FPGAs platforms. The FRED runtime consists of a system support design, automated by DART, and a set of software support components.

System support design

The FRED support design is a reference design for the FPGA side of the SoC that has been designed to support the deployment of dynamically-reconfigured hardware accelerators. The support design divides the FPGA into two regions: a static region and a reconfigurable region. The static region contains the logic needed to realize the communication infrastructure, namely a set of AXI Interconnects, which the user can extend by adding other support modules depending on the specific needs. The reconfigurable region is organized into a set of statically defined slots that are logically grouped into partitions.

FRED support design

FRED support design

Software support

The FRED software support comprises software components in charge of managing the FPGA and implementing the FRED scheduling policy on top of the system support design. The software support has been designed in a modular fashion, relying as much as possible on user space implementation to improve maintainability, safety, and expandability. The central component of the software support is a user-space server process, named the FRED server, which manages acceleration requests from Linux processes (and threads) according to the FRED scheduling policy. Linux processes and hardware accelerators share data through a zero-copy mechanism implemented using physically contiguous (uncached) memory buffers. The FRED server relies on two custom kernel modules and the UIO framework for controlling the hardware accelerators.

FRED software support

FRED software support

The FRED software stack figure introduces FRED runtime components. At the application level we see that it possible to write applications with C/C++/Python programming languages. It is also possible to write applications using ROS2 and Xilinx Vitis AI frameworks (both currently under development). Still in user space, we have the fred_lib, which is linked with the application to have access to the fred_server.

FRED software stack

FRED software stack

The FRED server initiates the FPGA support during the initialization phase and then manages Linux processes and threads requests. Internally, the FRED server uses I/O multiplexing to monitor all hardware and software component events from a single event loop. The FRED server communicates with the software processes using a simple client-server messaging protocol based on Unix domain socket. From a user perspective, the interactions between the software process and the FRED server are abstracted by fred_lib, which is available in C and Python.

In kernel space, two Linux kernel modules, called fred_buffctl and fpga_mgr, were developed/modified to abstract the access to the FPGA fabric. The fred_buffctl module is used to allocate the contiguous memory buffers used to share data between software processes and dynamically-reconfigured hardware accelerators. The fpga_mgr module manages the device reconfiguration in an optimized way with respect to the Xilinx’s stock driver.

OS Support

It is necessary to have a Linux image with the basic setup and requirements to run the FRED Framework. For this, we created two Yocto layers called meta-fred and meta-retis that can be used with Petalinux v2020.2 to create the tested Linux image. meta-fred cross-compile all software part of FRED runtime. Using this is Yocto layer is the recommended way to compile FRED runtime. Alternatively, meta-retis provides all the Kernel setup and software compilation/analysis/debugging tooling to compile and test FRED software directly in the board. meta-retis also provides setup for running/debugging real-time Linux systems using, for instance, PREEMPT-RT, perf, ftrace, stress-ng, etc. So, the user can decide which approach to use, but the recommendation is to use both Yocto layers for an increased and more flexible design/debug capability. Finally, fred-framework is a meta repository that combines all FRED Framework, facilitating compilation in the board;

Testing FRED runtime

FRED runtime is available for download. Further instructions can be found in FRED repository and in the Getting Started section.

Reference