DM36x Dual Resizer

From RidgeRun Developer Connection
Revision as of 12:36, 27 February 2013 by Tfischer (Talk | contribs)

Jump to: navigation, search

Introduction

The DM365 and DM368 include an IPIPE dual output hardware resizer as part of the Video Processing Front End (VPFE). This means each input video frame can be resized to two different output buffers. The resizer can also perform a color space conversion. TI describes the funcationality as:

Programmable down or up-sampling filter for both horizontal and vertical directions with range from 1/16x to 16x, in which the filter outputs two images with different magnification simultaneously.

RidgeRun sells dual output resizer support as an SDK add-on.

References

Linux driver

The DM36x Linux driver set already support the dual output hardware resizer. Minor changes are needed to be able to determine the size of the frames produced by resizer B.

The dm365_ipipe.ko Linux kernel driver needs to be extended by adding a struct imp_hw_interface get_out_pixel_format. This is needed to be able to determine the size of the frame from hardware resizer B. In addition, a defect may need to be fixed so that the default configuration parameters are only copied in the not configured state.

In vpfe_capture, additional logic is needed to determine the size of the frame from hardware resizer B.

Kernel memory pools

When using the dual resizer you need to use a bigger input buffer size than the one negotiated on GStreamer capabilities so you will need to change the input buffer size on the make config menu. For example:

make config
  -> Architecture configurations
    -> Maximum Video Input Buffer Size
      -> 1080P

Compile and install the SDK after making this change.

GStreamer gst-dual-resizer element

A new GStreamer plug-in was created to allow use of the dual output resizer. A new element named 'resizer' supports two on-request source pads, named rszA and rszB and operates similar to a GStreamer tee element. The resizer is designed to be used right after v4l2src element. Each GSTBuf from v4l2src contains the output from both channel A and B. The resizer element simply splits those combined frames into separate video pipelines. the capabilities of each downstream pipeline determines the color space, width, and height used to configure the resizer configuration.


Element details are show below.

Factory Details:
  Long name:    DM36x Resizer
  Class:        Control/Video
  Description:  Configure dm36x resizers and get both resizers video
  Author(s):    Melissa Montero <<melissa.montero@ridgerun.com>>
  Rank:         none (0)

Plugin Details:
  Name:                 resizer
  Description:          Configure dm36x resizers and get both resizers video
  Filename:             /usr/lib/gstreamer-0.10/libgstresizer.so
  Version:              0.10.0
  License:              LGPL
  Source module:        gst-resizer
  Binary package:       GStreamer
  Origin URL:           http://gstreamer.net/

GObject
 +----GstObject
       +----GstElement
             +----GstResizer

Pad Templates:
  SRC template: 'rszA'
    Availability: On request
      Has request_new_pad() function: gst_resizer_request_new_pad
    Capabilities:
      video/x-raw-yuv
                 format: UYVY
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw-yuv
                 format: NV12
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]

  SRC template: 'rszB'
    Availability: On request
      Has request_new_pad() function: gst_resizer_request_new_pad
    Capabilities:
      video/x-raw-yuv
                 format: { UYVY, NV12 }
                  width: [ 1, 640 ]
                 height: [ 1, 480 ]
              framerate: [ 0/1, 2147483647/1 ]

  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw-yuv
                 format: UYVY
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw-yuv
                 format: NV12
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]


Element Flags:
  no flags set

Element Implementation:
  Has change_state() function: gst_resizer_change_state
  Has custom save_thyself() function: gst_element_save_thyself
  Has custom restore_thyself() function: gst_element_restore_thyself

Element has no clocking capabilities.
Element has no indexing capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Implementation:
      Has chainfunc(): gst_resizer_chain
      Has custom eventfunc(): gst_pad_event_default
      Has custom queryfunc(): gst_pad_query_default
      Has custom iterintlinkfunc(): gst_pad_iterate_internal_links_default
      Has getcapsfunc(): gst_resizer_get_caps
      Has setcapsfunc(): gst_resizer_set_caps
      Has acceptcapsfunc(): gst_pad_acceptcaps_default
    Pad Template: 'sink'

Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: null Current: "resizer0"


Example applications

Simplist GStreamer pipeline

The first line gets video frames from V4L2 and passes each frame to the resizer. The second line is the pipeline that processes the data from resizer channel A output, in this case scaling to 1280 x 720 in the NV12 colorspace. In the third line, the pipeline pocesses the data from resizer channel B output, in this case scaling to 320 x 240 also in the NV12 colorspace. In both cases the video data is just tossed. Notice pipeline needs to use separate queue elements in each branch to provide separate threads for each branch, otherwise a blocked dataflow in one branch would stall the other branches.

gst-launch -e v4l2src always-copy=false chain-ipipe=false ! resizer name=rs \
              rs.rszA ! queue ! 'video/x-raw-yuv,format=(fourcc)NV12,width=1280,height=720' ! fakesink \
              rs.rszB ! queue ! 'video/x-raw-yuv,format=(fourcc)NV12,width=320,height=240' ! fakesink


GStreamer face detect pipeline

If you are using the [DM36x Face Detection] element, you can avoid ARM based image scaling by using the hardware accelerated resizer element.

gst-launch -e v4l2src always-copy=false chain-ipipe=false ! dmaiaccel ! resizer name=rs \
              rs.rszA ! queue ! 'video/x-raw-yuv,format=(fourcc)NV12,width=1280,height=720' ! dmaienc_h264 encodingpreset=2 targetbitrate=3600000 ! queue ! qtmux  ! filesink location=test.mov 
              rs.rszB ! queue ! 'video/x-raw-yuv,format=(fourcc)NV12,width=320,height=192' ! dm365facedetect  draw-square=true  no-copy=true ! dmaiperf ! TIDmaiVideoSink enable-last-buffer=false videoStd=720P_60

GStreamer RTP streaming and saving to file pipeline

If you are using a slower data rate Internet connection, such as GSM, you may want to save the video at a high resolution and simultaneously stream the video at a lower resolution.

gst-launch -e v4l2src chain-ipipe=false always-copy=false input-src=camera ! 'video/x-raw-yuv, width=640, height=480, format=(fourcc)NV12' ! resizer name=rsz rsz.rszA ! queue ! 'video/x-raw-yuv, width=640, height=480, format=(fourcc)NV12' ! dmaiaccel ! dmaiperf ! dmaienc_h264 encodingpreset=2 ratecontrol=2 bytestream=true targetbitrate=3000000 ! queue ! qtmux ! filesink location=/test.mp4 async=false  rsz.rszB ! queue ! 'video/x-raw-yuv, width=320, height=240, format=(fourcc)NV12, framerate=(fraction)30/1' ! fakesink enable-last-buffer=false async=false