RidgeRun Auto exposure/Auto white balance library for DM368 and DM365
Video capture quality can be enhance with image processing, like auto white balance(AWB) and auto exposure algorithms(AE):
- Auto exposure performs automatic adjustments of the image brightness according to the amount of light that reaches the camera sensor.
- Auto white balance automatically compensates color differences based on lighting so white actually appears white.
Some camera sensors don't include auto white balance and/or auto exposure processing, so RidgeRun offers a library with AE and AWB algorithms called librraew. This library has been developed for the DM365 platform. The DM365 VPFE has a module H3A designed to support control loops for auto focus, auto white balance and auto exposure by collecting statistics about the imaging/video data. There are two blocks in this module:
- Auto focus engine
- Auto exposure and auto white balance engine
The librraew only use the auto exposure and auto white balance engine. This engine divides the frames into two dimensional blocks of pixels referred as windows. The engine can provide some image/video metrics:
- Accumulation of clipped pixels along with all non-saturated pixels in each window per color.
- Accumulation of the sum of squared pixels per color.
- Minimum and maximum pixels values in each window per color.
Three applications are required to support the auto exposure and auto white-balance (AEW) adjustments:
- Ipiped, a D-Bus server for controlling and configuring the camera sensor, the dm365 video processor and the aew library.
- Librraew, a library that includes auto white balance and auto exposure algorithms.
- Ipipe-client, an D-Bus client that can be used to invoke any of the methods supported by the Ipiped.
Ipiped must run in background:
Ipiped registers with D-Bus and waits until ipipe-client requests to execute a method.
Ipipe-client is a Dbus client that use commands to invoke methods of the ipiped, so ipiped must be running to use ipipe-client. A command can required arguments depending of the functionality. Ipipe-client has two operation modes, you can ask to execute a single command or you can open an interactive console to execute a group of commands.
To execute a single command, you can use the following command line syntax
ipipe-client <command> <argument 1> ... <argument n>
To get into the interactive console, you have to run ipipe-client without any command. Then to execute a command you only need to use the command and the required arguments.
ipipe-client ipipe-client$ <command 1> <argument 1> ... <argument n> ipipe-client$ <command 2> <argument 1> ... <argument n>
To quit the interactive console you can use quit or exit.
In order to know the commands that are available run:
or get into the interactive console and execute help.
This shows a description of each command, as follows:
Command Description help Displays the help text for all the possible commands or a specific command. set-debug Enable/Disable debug messages. init-aew Initialize AEW algorithms. stop-aew End AEW algorithm. shell Execute a shell command(shell_cmd) using interactive console. ping Show if ipipe-daemon is alive. quit Quit from the interactive console. exit Exit from the interactive console. get-video-processor Show the video processor that is being used. get-sensor Show the sensor that is being used. run-config-script Execute a group of ipipe-client commands. set-previewer-mode Configure previewer on continuous or one-shot mode. set-bayer-pattern Sets R/Gr/Gb/B color pattern to the previewer. set-digital-gain Sets red (R), green (G) and blue gains (G) on the ipipe. get-digital-gain Returns the gain value for each color component(RGB). set-luminance Brightness(Br) and contrast(C) adjustment. get-luminance Returns the value of the Brightness(Br) and contrast(C) adjustment. flip-vertical Flips the image vertically(on the sensor). flip-horizontal Flips the image horizontally (on the sensor). set-exposure Sets the effective shutter time of the sensor for the light integration. get-exposure Gets the exposure time of the sensor in us. set-sensor-gain Sets red(R), green(G) and blue(B) gain directly on the sensor. get-sensor-gain Gets sensor red(R), green(G) and blue(B).
If you want more detailed information about a command execute:
ipipe-client help <command>
Auto exposure and auto white balance adjustments can be started with an ipipe-client's command called init-aew. Init-aew requires some arguments to define the algorithms and other parameters. To see the arguments required you can request for help that show you the list as follows:
Command: init-aew Syntax: init-aew <WB> <AE> <G> <EM> <T[us]> <fps> <seg> <width> <height> Description: Initialize AEW algorithms Arguments: WB: white balance algorithm, the options are: G -for gray world algorithm W -for retinex algorithm N -for none AE: auto exposure algorithm, the options are EC -for electronic centric N -for none G: gain type, the options are: S -to use the sensor gain D -to use the digital EM: exposure metering method, the options are: S -for spot metering that take into account the light information of a small portion in the center of the frame and the rest is ignored P -for partial metering that take into account the light information of a larger portion in the center of the frame (10% - 15%) and the rest of the frame is ignored. C -for center weighted metering that take into account the light information coming from the entire frame with emphasis placed on the center area A -for average metering that take into account the light information from the entire frame without weighting SG -for segmented metering that divides the frame on 6 pieces and weighting them to avoid backlighting T: wait time in us, specifies the time between algorithm adjustments, max value=1s=1000000us fps: minimum frame rate seg: frame segmentation factor, each frame is segmented into regions, this factor represents the percentage of the maximum number of possible regions width: captured video/image horizontal size height: captured video/image vertical size
Also you can stop automatic adjustments with the command stop-aew
Some of the init-aew arguments need to be explained in more detail:
- WB: as you see above this parameter indicates the AWB algorithm you want to use.
- The Gray World algorithm is based on the assumption that given an image with sufficient amount of color variation, the average reflectance of the scene is achromatic(gray). If the scene has a color that dominates, the results of the algorithm may not be satisfactory.
- The White Patch algorithm assumes that the maximum response in an image is caused by a perfect reflector, so represents the color of the illumination. This algorithm can be used on images without saturated pixels.
- AE: indicates the AE algorithm, for the moment there is only one algorithm: Electronic centric(EC). This algorithm uses the user defined metering system to get the scene illumination and tries to approach the brightness level to the mid-tone of intensity.
- EM: this parameter defines the brightness metering system that is going to be used for the auto exposure algorithm.
- S: Averages the light information of a small portion in the center of the frame (1% - 5% of the entire image) the rest of the frame is ignored. This system is good for scenes that have a principal object and you are only interested on it, you don't care the background, even more on very high contrast scenes
- P: Average the light information of a larger portion in the center of the frame (10% - 15% of the entire image) the rest of the frame is ignored. It is generally used when very bright or very dark areas on the edges of the frame would otherwise influence on a bad way the scene illumination.
- C: Average the light information coming from the entire frame with emphasis placed on the center area. This algorithm can be used when you want that the whole scene be well illuminated and not be affected for the small edges brightness variations. The subjects of the picture must be at the center of the image. But, if a backlight is present into the scene the central part results darker than the rest of the scene and unpleasant underexposed foreground is produced.
- A: Average the light information coming from the entire frame without weighting any particular portion of the metered area. This algorithm can be used on scenes that not have a principal object and you want an average illumination. If the scene has a high contrast, the algorithm cause under or over exposure of parts of the scene.
- SG: This algorithm is designed for scenes that have a principal object in backlighting condition. Emphasize the luminance of the main object according to the degree of backlighting. Divides the frame on 6 pieces and weighting them.
- T: the time between interactions defines how fast the algorithm can adjust the scene parameters. If you don't need fast changes you can use a greater time to get less CPU usage.
- seg: this factor is related with the amount of CPU usage and the auto-adjustments precision. If you use a high segmentation percentage you will have greater CPU usage but you will get more precision on the adjustments.