Writeback driver support for DM8168

From RidgeRun Developer Connection
Jump to: navigation, search

RidgeRun provides support for the DM816x writeback driver. This driver is used to access the OSD and blended video and graphics images out of the hardware compositor.

You can think of the writeback driver as giving you the ability to do a screen shot of the currently displayed video frame overlayed with the QT or other OSD text and graphics. Some people this of the writeback driver as giving you the ability to do a video screen capture, similar to the way some programs run on a personal computer. No matter how you choose to think of the writeback driver, it gives you access to each video frame with the OSD overlayed on top.

Writeback driver

The DM816x SoC supports a feedback data path (which is called writeback) in DVO1. You can see it in the following image.

HDVPSS block.png
Figure 2. DM816x HDVPSS Block Diagram

To manage the stream of video frames coming from the writeback path, we add an extra video device /dev/video7. So, your system is ready to capture using V4L2 and you will be able to capture from both VIP0 and VIP1 video input nodes and from the writeback path. If the V4L2 display driver is enabled, the video nodes would have the following map:

/dev/video0 → Capture VIP 0 Port A
/dev/video1 → Display 0
/dev/video2 → Display 1
/dev/video3 → Display 2
/dev/video4 → Capture VIP 0 Port B
/dev/video5 → Capture VIP 1 Port A
/dev/video6 → Capture VIP 1 Port B
/dev/video7 → Capture SC5 WB2

If you are not using the V4L2 display driver, you need to use this mapping instead:

/dev/video0 → Capture VIP 0 Port A
/dev/video1 → Capture VIP 0 Port B
/dev/video2 → Capture VIP 1 Port A
/dev/video3 → Capture VIP 1 Port B
/dev/video4 → Capture SC5 WB2

Enabling the writeback driver

To enable the writeback driver, run make config and select:

    --> Proprietary software
        - Enable Writeback driver support

Then run make and make install as usual. Now, your SDK is ready to capture data from the writeback path.

In order to use the writeback driver we need to display something on DVO1 (HDMI). If you are thinking about using DVO2, you need to tie DVO2 and DVO1 and then use the writeback driver:

insmod just in the EVM
insmod  lib/modules/2.6.37-ridgerun/kernel/drivers/video/ti81xx/sii9022a/sii9022a.ko

echo 0 > /sys/devices/platform/vpss/graphics0/enabled
echo 0 > /sys/devices/platform/vpss/display0/enabled   
echo 0 > /sys/devices/platform/vpss/display1/enabled 
echo 5 > /sys/devices/platform/vpss/system/tiedvencs  
echo 2:hdmi,dvo2 > /sys/devices/platform/vpss/graphics0/nodes  
echo vcompmux:hdmi,dvo2 > /sys/devices/platform/vpss/video0/nodes  
echo 1 > /sys/devices/platform/vpss/display0/enabled
echo 1 > /sys/devices/platform/vpss/display1/enabled
echo 1 > /sys/devices/platform/vpss/graphics0/enabled

GStreamer pipelines

Snapshot pipeline

gst-launch -e v4l2src device=/dev/video7 num-buffers=1 always-copy=false queue-size=8  ! 'video/x-raw-yuv,format=(fourcc)YUY2,width=1920,height=1080,framerate=(fraction)60/1' ! omxbufferalloc \
numBuffers=10 ! queue  ! ffmpegcolorspace ! jpegenc ! jifmux ! filesink location=writeback.jpeg

Videotestsrc with writeback pipeline

gst-launch videotestsrc  ! "video/x-raw-yuv, format=(fourcc)NV12, width=1920, height=1080, framerate=60/1, buffer-count-requested=4" ! omx_mdeiscaler name=d d.src_00 !  'video/x-raw-yuv, \
width=(int)1920, height=(int)1080' ! v4l2sink sync=false device=/dev/video1 d.src_01  ! fakesink silent=true -v &

V42 source with writeback pipeline

gst-launch -e v4l2src device=/dev/video7 num-buffers=1 always-copy=false queue-size=8  ! 'video/x-raw-yuv,format=(fourcc)YUY2,width=1920,height=1080,framerate=(fraction)60/1' ! rrbufferalloc \
numBuffers=10 ! queue  ! ffmpegcolorspace ! jpegenc ! jifmux ! filesink location=writeback.jpeg

RTSP streaming V4L2 source with writeback pipeline

gst-launch v4l2src device=/dev/video7 always-copy=false queue-size=8 ! 'video/x-raw-yuv, format=(fourcc)YUY2, width=1920, height=1080, framerate=(fraction)30/1, buffer-count-requested=10' ! \
omxbufferalloc numBuffers=10 ! omx_noisefilter ! gstperf ! omx_h264enc output-buffers=10 input-buffers=10 i-period=30 force-idr-period=90 bitrate=5000000 ! rr_h264parser singleNalu=true \
! video/x-h264, mapping=/stream1 ! rtspsink name=sink service=554

In the host side, you can use vlc or the next pipeline:

IPADDR=10.251.101.114
gst-launch-0.10 rtspsrc location=rtsp://$IPADDR:/stream1 ! rtph264depay ! queue ! ffdec_h264 ! fpsdisplaysink sync=false

It is important to mention that the output on the writeback path is in YUY2 format. So, we use the omx_noisefilter to convert the image from YUY2 to NV12.


RTSP streaming, video test source with writeback pipeline

Videotestsrc + Writeback + RTSP

gst-launch v4l2src device=/dev/video7 always-copy=false queue-size=8 ! 'video/x-raw-yuv, format=(fourcc)YUY2, width=1920, height=1080, framerate=(fraction)30/1, buffer-count-requested=10' ! \
omxbufferalloc numBuffers=10 ! omx_noisefilter ! omx_h264enc output-buffers=10 input-buffers=10 i-period=30 force-idr-period=90 bitrate=5000000 ! rr_h264parser singleNalu=true  ! \
video/x-h264, mapping=/stream1 ! rtspsink name=sink service=554 videotestsrc  ! "video/x-raw-yuv, format=(fourcc)NV12, width=1920, height=1080, framerate=15/1, buffer-count-requested=4" ! \
omx_mdeiscaler name=d d.src_00 !  'video/x-raw-yuv, width=(int)1920, height=(int)1080' ! v4l2sink sync=false device=/dev/video1 d.src_01  ! fakesink silent=true -v

In the host side, you can use vlc or the next pipeline (remember to change the ip address):

IPADDR=10.251.101.114
gst-launch-0.10 rtspsrc location=rtsp://$IPADDR:/stream1 ! rtph264depay ! queue ! ffdec_h264 ! fpsdisplaysink sync=false

Triple capture with writeback pipeline

Capture two 1080p@60fps and one 1080p@60fsp writeback.

gst-launch -e v4l2src device=/dev/video0 always-copy=false queue-size=8  ! 'video/x-raw-yuv,format=(fourcc)NV12,width=1920,height=1080,framerate=(fraction)60/1' ! omxbufferalloc \
numBuffers=10 ! gstperf  ! fakesink v4l2src device=/dev/video5 always-copy=false queue-size=8  ! 'video/x-raw-yuv,format=(fourcc)NV12,width=1920,height=1080,framerate=(fraction)60/1' ! \
 omxbufferalloc numBuffers=10 ! gstperf  ! fakesink v4l2src device=/dev/video7 always-copy=false queue-size=8  ! 'video/x-raw-yuv,format=(fourcc)NV12,width=1920, \
height=1080,framerate=(fraction)60/1' ! omxbufferalloc numBuffers=10 ! gstperf ! fakesink 

Triple capture with writeback pipeline with display

Capture two 1080p@60fps and one 1080p@60fps writeback capture with two displays 1080 and one HDMI component output.

gst-launch -e v4l2src device=/dev/video0 always-copy=false queue-size=8  ! 'video/x-raw-yuv,format=(fourcc)YUY2,width=1920,height=1080,framerate=(fraction)60/1' ! omxbufferalloc \
numBuffers=10 ! gstperf  ! v4l2sink sync=false device=/dev/video1  sync=false v4l2src device=/dev/video5 always-copy=false queue-size=8  ! 'video/x-raw-yuv,format=(fourcc)YUY2, \
width=1920,height=1080,framerate=(fraction)60/1' ! omxbufferalloc numBuffers=10 ! gstperf  ! v4l2sink sync=false device=/dev/video2  v4l2src device=/dev/video7 always-copy=false queue-size=8  ! \
'video/x-raw-yuv,format=(fourcc)YUY2,width=1920,height=1080,framerate=(fraction)60/1' ! omxbufferalloc numBuffers=10 ! gstperf ! fakesink 

Same functionality but displaying writeback with low latency.

gst-launch -e v4l2src device=/dev/video0 always-copy=false queue-size=8  ! 'video/x-raw-yuv,format=(fourcc)YUY2,width=1920,height=1080,framerate=(fraction)60/1' ! omxbufferalloc \
numBuffers=10 ! gstperf  ! v4l2sink sync=false device=/dev/video1  sync=false v4l2src device=/dev/video7 always-copy=false queue-size=8  ! 'video/x-raw-yuv,format=(fourcc)YUY2, \
width=1920,height=1080,framerate=(fraction)60/1' ! omxbufferalloc numBuffers=10 ! gstperf  ! v4l2sink enable-last-buffer=false sync=false device=/dev/video2  v4l2src device=/dev/video5 \
always-copy=false queue-size=8  ! 'video/x-raw-yuv,format=(fourcc)YUY2,width=1920,height=1080,framerate=(fraction)60/1' ! omxbufferalloc numBuffers=10 ! gstperf ! fakesink 

Capture 2x1080p@60fps + 1x1080p@60fps Writeback capture + 2 display 1080 - component, HDMI + RTSP

gst-launch -e v4l2src device=/dev/video0 always-copy=false queue-size=8  ! 'video/x-raw-yuv,format=(fourcc)YUY2,width=1920,height=1080,framerate=(fraction)60/1' ! omxbufferalloc \
numBuffers=10 !gstperf  ! v4l2sink sync=false device=/dev/video1  sync=false v4l2src device=/dev/video5 always-copy=false queue-size=8  ! 'video/x-raw-yuv,format=(fourcc)YUY2, \
width=1920,height=1080,framerate=(fraction)60/1' ! omxbufferalloc numBuffers=10 ! gstperf  ! v4l2sink sync=false device=/dev/video2  v4l2src device=/dev/video7 always-copy=false \
queue-size=8  ! 'video/x-raw-yuv,format=(fourcc)YUY2,width=1920,height=1080,framerate=(fraction)60/1' ! omxbufferalloc numBuffers=10 ! gstperf ! omx_noisefilter ! omx_h264enc \
output-buffers=10 input-buffers=10 i-period=30 force-idr-period=90 bitrate=5000000 ! rr_h264parser singleNalu=true  ! video/x-h264, mapping=/stream1 ! rtspsink name=sink service=554

In the host side, you can use vlc or the next pipeline (remember to change the ip address):

gst-launch-0.10 rtspsrc location=rtsp://10.251.101.114:/stream1 ! rtph264depay ! queue ! ffdec_h264 ! fpsdisplaysink sync=false

Articles related to this board

Spectrum Digital DM8168 EVM
Getting Started Guide for DM8168 EVM
RidgeRun 2011Q2 SDK User Guide
RidgeRun Turrialba SDK User Guide