diff --git a/docs/solutions/reference-designs/adrd8012-01Z/adrd8012-01z_01-block-diagram.png b/docs/solutions/reference-designs/adrd8012-01Z/adrd8012-01z_01-block-diagram.png deleted file mode 100644 index 26d9c3f7c98..00000000000 --- a/docs/solutions/reference-designs/adrd8012-01Z/adrd8012-01z_01-block-diagram.png +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:b9ca0bbc89c3953d441c9a92d00bf1ae481e766497e783bf7b1223922a10d585 -size 46128 diff --git a/docs/solutions/reference-designs/adrd8012-01Z/adrd8012-01z_bd.svg b/docs/solutions/reference-designs/adrd8012-01Z/adrd8012-01z_bd.svg new file mode 100644 index 00000000000..79275c665f6 --- /dev/null +++ b/docs/solutions/reference-designs/adrd8012-01Z/adrd8012-01z_bd.svg @@ -0,0 +1 @@ +MAX96724SIOASIOBMIPI D-PHY4 LanesUARTDDRxSPII2CInterruptsZynqMPSoCTimerK26 Industrial SOMCSI-2 INTERFACEAD9545SIOCSIODCSI1CSI2SIOASIOBCSI-2 INTERFACESIOCSIODCSI1CSI2I2C controllerIPMIPI D-PHY4 LanesXilinx MIPI CSI-2ReceiverSubsystemXilinx MIPI CSI-2ReceiverSubsystemAXISYUV422-4ppcAXISYUV422-4ppcXilinxAXIS Switch..XilinxAXIS Switch..RTP NetworkingStackCorundumNICSFP+CageFSYNCMFPs+ I2CFSYNCMFPs+ I2CAXI PWMGENIPMAX96724AXISAXISNotes:*ppc–pixelsperclock,AXISSwitch–switchlogictodividesingleportAXISoutputtofourAXISportsaccordingtoTDESTfield64b64b64b64b \ No newline at end of file diff --git a/docs/solutions/reference-designs/adrd8012-01Z/img_1252_1_.jpg b/docs/solutions/reference-designs/adrd8012-01Z/img_1252_1_.jpg deleted file mode 100644 index 654c9b64045..00000000000 --- a/docs/solutions/reference-designs/adrd8012-01Z/img_1252_1_.jpg +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:5d8da15ab4eb448a857b579bf67f024b07c78fe55ab2a53bbdcede315a338c12 -size 104794 diff --git a/docs/solutions/reference-designs/adrd8012-01Z/index.rst b/docs/solutions/reference-designs/adrd8012-01Z/index.rst index d1793e10b78..b6ceea2b92b 100644 --- a/docs/solutions/reference-designs/adrd8012-01Z/index.rst +++ b/docs/solutions/reference-designs/adrd8012-01Z/index.rst @@ -1,9 +1,7 @@ -.. imported from: https://wiki.analog.com/resources/eval/user-guides/adrd8012-01z - ADRD8012-01Z ============ -FPGA-based 8x GMSL to 10 Gb Ethernet Adapter. +FPGA-based 8 x GMSL2 cameras to 10 Gb Ethernet Edge Compute Platform Overview --------- @@ -11,27 +9,34 @@ Overview .. figure:: adrd8012-01z_angle-evaluation-board.jpg :width: 600 px - ADRD8012-01Z GMSL Board + ADRD8012-01Z board -The **ADRD8012-01Z** is an edge compute platform enabling low latency data -transfer from eight :adi:`Gigabit Multimedia Serial Link™ (GMSL) ` -interfaces on to a 10 Gb Ethernet link. The target applications include -autonomous robots and vehicles where machine vision and real-time sensor fusion +**ADRD8012-01Z** is an edge compute platform enabling low latency data +transfer from two :adi:`Gigabit Multimedia Serial Link™ (GMSL) ` +deserializers (resulting in an example of up to 8 GMSL2-enabled camera modules) on to a 10 Gb Ethernet link. +The target applications include autonomous robots and vehicles where machine vision and real-time sensor fusion is critical. Some of the main features and benefits include: -- 8x GMSL2 camera interfaces with up to 6 Gbps/channel -- 10 Gbps SFP+ Ethernet interface -- Precision Time Protocol for synchronization with host systems and other edge devices +- 2x GMSL2 deserializers with up to 6 Gbps/SerDes link +- 10 GbE-capable SFP+ connector +- Precision Time Protocol (PTP) for synchronization with host systems and other edge devices - Embedded processing capabilities using the on-board `AMD Kria K26 System-on-Module `__ -- ROS2 compliant +- ROS2 compliant video streaming design - Open-source embedded Linux software and FPGA design - Advanced camera triggering functions and control features -.. figure:: adrd8012-01z_01-block-diagram.png +The block design and the RTP networking stack made using the FPGA region are presented below: + +.. figure:: adrd8012-01z_bd.svg + :width: 800 px + + ADRD8012-01Z Block Design + +.. figure:: rtp-networking-stack.svg :width: 800 px - ADRD8012-01Z Simplified Block Diagram + RTP networking stack Specifications -------------- @@ -48,7 +53,7 @@ Specifications | I/O | 16 general purpose I/O pins with software | | | configurable functionality, 3.3V voltage level | +-----------------------+-----------------------------------------------------+ -| GMSL | 2x Quad Fakra connectors supporting 8 x GMSL | +| GMSL | 2x Quad Fakra connectors supporting 8 x GMSL2 | | | camera interfaces | +-----------------------+-----------------------------------------------------+ | Processing | | @@ -65,9 +70,8 @@ Specifications +-----------------------+-----------------------------------------------------+ | Operating System | Linux OS | +-----------------------+-----------------------------------------------------+ -| Network data protocol | RTP over UDP with software implementation and | -| | option for licensable FPGA accelerated RTP & UDP | -| | stack | +| Network data protocol | Open-sourced FPGA-accelerated Real-Time Transport | +| | for uncompressed video over UDP/IPv4 implementation | +-----------------------+-----------------------------------------------------+ @@ -78,19 +82,25 @@ Required Hardware ~~~~~~~~~~~~~~~~~~ - 1 x :adi:`ADRD8012-01Z ` -- 8 x `Tier IV C1 cameras `__ -- 8 x Fakra cables +- Up to 8 x Fakra cables - 2 x Quad-based mini-Fakra cables - 1 x 16 GB SD card -- 1 x PC with 10G ethernet card +- 1 x PC with 10 GbE NIC - 1 x SFP+ Ethernet cable -FPGA SD Card Image +Example GMSL2 camera options +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- up to 8 x `Tier IV C1 cameras `__ +- up to 4 x `Tier IV C2 cameras `__ +- `Intel RealSense D457 cameras `__ + +SD Card Image ~~~~~~~~~~~~~~~~~~~ -.. admonition:: Download +.. admonition:: SD card image that contains example setups using previously mentioned GMSL2-enabled cameras - `SD card image `__ + `Download `__ After downloading the file, extract the compressed image and write it to the SD card using `Balena Etcher `__ or @@ -109,7 +119,7 @@ the corresponding position, as indicated in the following image: .. figure:: img_1242_1_.jpg :width: 400 px - Boot Mode Switches Position + Boot mode switches for SD card boot Connect the Quad-based mini-Fakra cables to the corresponding connectors on the board. These will connect the cameras to the corresponding deserializers. @@ -117,18 +127,21 @@ board. These will connect the cameras to the corresponding deserializers. .. figure:: img_1247_1_.jpg :width: 400 px - Connecting the Fakra Cables + Quad mini-Fakra cabbles connection to board's deserializers -Connect a SFP+ cable to the corresponding SFP port on the board. +Connect an SFP+ cable to the corresponding SFP+ port on the board. .. figure:: img_1244_1_.jpg :width: 400 px - Connecting the SFP Cable + SFP+ cable connection to board's cage Finally, you will need to connect a USB/micro-USB cable to the micro-USB port located on the board. After that, you will be able to connect to the first USB COM port that appears on the serial terminal, with a baud rate of **115200**. +Besides this wired connection, after the Linux system on the board is +completely initialized, you will be able to use the network-related connection +through the 10 GbE interface (eth0 on the board) and leveraging ssh SW support. .. note:: @@ -148,7 +161,11 @@ COM port that appears on the serial terminal, with a baud rate of **115200**. .. important:: - Both server and client should have the same MTU + Both server and client should have the same MTU. + Default camera setup is 4 x Tier IV C1 on P1 deserializer (connected to CSI-2 RX subsystem 0). + In case of Intel RealSense D457 camera, only the example setup with 1 camera module on P1 is present, + and it contains the hardcoded 0/1 sensors enabled for virtual channels 0 and 1 (as these are streaming + RAW color format - UYVY). .. shell:: @@ -173,37 +190,92 @@ COM port that appears on the serial terminal, with a baud rate of **115200**. .. shell:: - #Configure the video pipeline and the cameras - $cd /home/analog/Workspace/K26 - $./media_cfg_des1/2/12.sh - #(depending on the desired deserializer or 12 for the case when there are two deserializers with 4 cameras) + #(Optional, if you don't know the destination MAC address of the remote side - the NIC of the PC + #which is locally connected to the board) + #Ping the PC's IPv4 address + $ping + #Check the ARP table of the board's NIC + $arp -a + #You will see the matches between IPv4 and MAC addresses + #IPv4 MAC + +FPGA-accelerated RTP networking stack setup +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. shell:: + + #Configure the instantiated RTP engines and RTP session mux logic + #using the files created by the sysfs implementation - which represent + #configurable fields of the implemented protocols + #RTP engines - devices present in /sys/devices/ + #e.g. (L2) - MAC sublayer of Ethernet v2 standard + $echo 0xaabbccddeeff > dest_mac_address + $echo 0x010203040506 > src_mac_address + #e.g. (L3) - IPv4 + $echo 0x0a2a0014 > dest_ipv4_address + $echo 0x0a2a0018 > src_ipv4_address + #e.g. (L4) - UDP + $echo 5004 > dest_udp_port + $echo 5004 > src_udp_port + #e.g. (L7) - RTP + $echo 1920 > num_pixels_per_line + $echo 1280 > num_lines + #In addition, the video format can be converted from YUYV to UYVY (as ordering methods + #in YUV422) + $echo 1 > convert_yuyv_to_uyvy + + #Start the video transmission using the FPGA-accelerated RTP stack by setting the + #start_transfer bit of the RTP session mux driver + $echo 1 > start_transfer + +Video subsystem configuration and streaming startup +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. shell:: - #Start streaming to another host - $cd /home/analog/Workspace/gstreamer - #(depending on the number of cameras - 4 or 8 cameras - 1 or 2 deserializers) - $./stream_1des_4cams/2des_8cams.sh [HOST_IP_ADDRESS of the x86-based workstation can be modified/by default is set to 10.42.0.106 in this script] + #Configure the video image format for GMSL SerDes/MIPI CSI-2 receiver subdevices and FPS of the camera + #module + #some of the Tier IV C1,C2 and RealSense D457 setups contain pre-implemented scripts for the configuration + #for the subdevs of the instantiated media devices. The name included in media_cfg_X is related to the + #sensor - Tier IV C1 contains isx021, C2 - imx490 and the d457 remains the same as the camera name + + $cd /home/analog/config_streaming + $./media_cfg_isx021_4p1.sh + + #Start the transmission from the sensor devices + $cd /home/analog/config_streaming + #(depending on the number of used deserializers - each one will instantiate one video and media device) + $./start_streaming_TierIV_C1_videodev0_1 .. note:: - In order to stop all this processes generated by the streaming-related - scripts, you can use the Linux pidof command to see what are the IDs of this - gstreamer-related instaces, and after that kill these ones by using Linux kill - command, in the following way: + The video streaming using the FPGA-accelerated RTP stack is started automatically when + the RTP engines and session mux instances are configured as indicated before. On the other side, + the streaming from the sensors is realized using the v4l2-ctl command executed on the corresponding + video devices. The v4l2-related commands depending on the hardware connectivity are present in this + SW configurations for video subsystem and streaming-related section. In order to stop all this + processes generated by the streaming-related scripts, you can use the Linux pidof command to see + what are the IDs of this v4l2-ctl-related instaces, and after that kill these ones by using Linux kill + command, in the following way: .. shell:: - #That is the command to show the ID numbers of the opened processes and will show you 8 numbers - like in the following line - $pidof gst-launch-1.0 - $800 799 798 797 796 795 794 793 - $sudo kill 800 799 797 796 795 794 793 + #Pidof output when having 2 video devices on which the streaming is started + $pidof v4l2-ctl + $800 799 + $sudo kill 800 799 -Now the streams are running on ports 5004 to 5007, depending on the configured number of cameras. +The video streaming is done using the previously configured UDP source/destination ports. -Install Gstreamer on x86 -~~~~~~~~~~~~~~~~~~~~~~~~ +Remote target setup +~~~~~~~~~~~~~~~~~~~ -Depending on the Linux distribution of your x86 workstation, you can install +To decode the RTP-based video streaming from the ADRD8012-01z system, you can use various +video streaming frameworks which supports the RFC-compliant RTP for uncompressed video standard +(RFC4175). This section presents the example setup using the GStreamer framework running on an +Linux distro. +Depending on the Linux distribution of your x86/arm64 workstation, you can install Gstreamer by using the corresponding package manager. For example, on Ubuntu you can use the following command: @@ -216,103 +288,102 @@ you can use the following command: More details about Gstreamer installation can be found `here `__. -Displaying the Video --------------------- +The following examples serves as commands used to decode the RTP-based video streaming from the +cameras from 1/2 deserializers [for a Tier IV C1 setup] -On the receiving side, `Gstreamer `__ -must be installed. +Single Deserializer (4 C1 cameras) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +If destination UDP ports were set to 5004-5007, use the following commands for 4 Tier IV C1s +[1920x1280]: -Single Deserializer (4 cameras) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Now open 4 instances of Gstreamer for each port(5004-5007). - -**On x86 workstation** +**On remote target** .. shell:: - $gst-launch-1.0 udpsrc caps=“application/x-rtp, sampling=YCbCr-4:2:2, - depth=(string)8, width=(string)1920, height=(string )1080” port=“5004” ! - rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink + $gst-launch-1.0 udpsrc caps="application/x-rtp, sampling=YCbCr-4:2:2, \ + depth=(string)8, width=(string)1920, height=(string )1280" port="5004" ! \ + rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink \ text-overlay=true sync=false - $gst-launch-1.0 udpsrc caps=“application/x-rtp, sampling=YCbCr-4:2:2, - depth=(string)8, width=(string)1920, height=(string )1080” port=“5005” ! - rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink + $gst-launch-1.0 udpsrc caps="application/x-rtp, sampling=YCbCr-4:2:2, \ + depth=(string)8, width=(string)1920, height=(string )1280" port="5005" ! \ + rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink \ text-overlay=true sync=false - $gst-launch-1.0 udpsrc caps=“application/x-rtp, sampling=YCbCr-4:2:2, - depth=(string)8, width=(string)1920, height=(string )1080” port=“5006” ! - rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink + $gst-launch-1.0 udpsrc caps="application/x-rtp, sampling=YCbCr-4:2:2, \ + depth=(string)8, width=(string)1920, height=(string )1080” port="5006" ! \ + rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink \ text-overlay=true sync=false - $gst-launch-1.0 udpsrc caps=“application/x-rtp, sampling=YCbCr-4:2:2, - depth=(string)8, width=(string)1920, height=(string )1080” port=“5007” ! - rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink + $gst-launch-1.0 udpsrc caps="application/x-rtp, sampling=YCbCr-4:2:2, \ + depth=(string)8, width=(string)1920, height=(string )1280" port="5007" ! \ + rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink \ text-overlay=true sync=false -.. figure:: img_1252_1_.jpg - Gstreamer Video Display for Single Deserializer (4 Cameras) +Two Deserializers (8 C1 cameras) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -2 x Deserializers (8 cameras) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +If destination UDP ports were set to 5004-5011, use the following commands for 8 Tier IV C1s +[1920x1280]: -Now open 8 instances of Gstreamer for each port(5004-5011). - -**On x86 workstation** +**On remote target** .. shell:: - $gst-launch-1.0 udpsrc caps=“application/x-rtp, sampling=YCbCr-4:2:2, - depth=(string)8, width=(string)1920, height=(string )1080” port=“5004” ! - rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink + $gst-launch-1.0 udpsrc caps="application/x-rtp, sampling=YCbCr-4:2:2, \ + depth=(string)8, width=(string)1920, height=(string )1280" port="5004" ! \ + rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink \ text-overlay=true sync=false - $gst-launch-1.0 udpsrc caps=“application/x-rtp, sampling=YCbCr-4:2:2, - depth=(string)8, width=(string)1920, height=(string )1080” port=“5005” ! - rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink + $gst-launch-1.0 udpsrc caps="application/x-rtp, sampling=YCbCr-4:2:2, \ + depth=(string)8, width=(string)1920, height=(string )1280" port="5005" ! \ + rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink \ text-overlay=true sync=false - $gst-launch-1.0 udpsrc caps=“application/x-rtp, sampling=YCbCr-4:2:2, - depth=(string)8, width=(string)1920, height=(string )1080” port=“5006” ! - rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink + $gst-launch-1.0 udpsrc caps="application/x-rtp, sampling=YCbCr-4:2:2, \ + depth=(string)8, width=(string)1920, height=(string )1280" port="5006" ! \ + rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink \ text-overlay=true sync=false - $gst-launch-1.0 udpsrc caps=“application/x-rtp, sampling=YCbCr-4:2:2, - depth=(string)8, width=(string)1920, height=(string )1080” port=“5007” ! - rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink + $gst-launch-1.0 udpsrc caps="application/x-rtp, sampling=YCbCr-4:2:2, \ + depth=(string)8, width=(string)1920, height=(string )1280" port="5007" ! \ + rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink \ text-overlay=true sync=false - $gst-launch-1.0 udpsrc caps=“application/x-rtp, sampling=YCbCr-4:2:2, - depth=(string)8, width=(string)1920, height=(string )1080” port=“5008” ! - rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink + $gst-launch-1.0 udpsrc caps="application/x-rtp, sampling=YCbCr-4:2:2, \ + depth=(string)8, width=(string)1920, height=(string )1280" port="5008" ! \ + rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink \ text-overlay=true sync=false - $gst-launch-1.0 udpsrc caps=“application/x-rtp, sampling=YCbCr-4:2:2, - depth=(string)8, width=(string)1920, height=(string )1080” port=“5009” ! - rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink + $gst-launch-1.0 udpsrc caps="application/x-rtp, sampling=YCbCr-4:2:2, \ + depth=(string)8, width=(string)1920, height=(string )1280" port="5009" ! \ + rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink \ text-overlay=true sync=false - $ gst-launch-1.0 udpsrc caps=“application/x-rtp, sampling=YCbCr-4:2:2, - depth=(string)8, width=(string)1920, height=(string )1080” port=“5010” ! - rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink + $ gst-launch-1.0 udpsrc caps="application/x-rtp, sampling=YCbCr-4:2:2, \ + depth=(string)8, width=(string)1920, height=(string )1280" port="5010" ! \ + rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink \ text-overlay=true sync=false - $gst-launch-1.0 udpsrc caps=“application/x-rtp, sampling=YCbCr-4:2:2, - depth=(string)8, width=(string)1920, height=(string )1080” port=“5011” ! - rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink + $gst-launch-1.0 udpsrc caps="application/x-rtp, sampling=YCbCr-4:2:2, \ + depth=(string)8, width=(string)1920, height=(string )1280" port="5011" ! \ + rtpvrawdepay ! videoconvert ! fpsdisplaysink video-sink=xvimagesink \ text-overlay=true sync=false -User Guides ------------ +.. + Enable after adding new content + + User Guides + ----------- -.. toctree:: - :titlesonly: - :maxdepth: 1 + .. toctree:: + :titlesonly: + :maxdepth: 1 + :glob: - production_testing/index + */index Help and Support ---------------- diff --git a/docs/solutions/reference-designs/adrd8012-01Z/rtp-networking-stack.svg b/docs/solutions/reference-designs/adrd8012-01Z/rtp-networking-stack.svg new file mode 100644 index 00000000000..1e1a5640b55 --- /dev/null +++ b/docs/solutions/reference-designs/adrd8012-01Z/rtp-networking-stack.svg @@ -0,0 +1 @@ +RTP Networking StackRTP Engine64bRTP EngineRTP EngineRTP EngineRTP EngineRTP EngineRTP EngineRTP Engine64b64b64b64b64b64b64bRTP Session MUX64b64b64b64b64b64b64b64b64bRTP VideoTransmissionHandlerCorundum NICApplication CoreImplementationNIC DataAXISAXIS64b64bWrapper toPHY layerAXIS with TDATA on x b,e.g.64b = 64 bits \ No newline at end of file