Image statistics

Previous tutorial: Plug-ins' functions

 

There were already number of previous tutorials discussing as different image processing filter plug-ins, as two source image processing filter plug-ins. No matter what is the number of source images, all image processing filter plug-ins are aimed to change image in one or another way. They may provide result of their filtering as a new image or may change the source image itself. But they still do supposed to work with pixels trying to calculate new values for them. However, in some cases there is a need to process an image without making any changes to its pixels' data, but instead calculate some values, which could be then used for further image processing. For example, the Otsu Threshold plug-in performs image thresholding according to the algorithm developed by Nobuyuki Otsu - it automatically finds a threshold value and then applies it to the given image. What if somebody needs only to calculate that threshold value without further changing the image? Or maybe another plug-in, which could be used to find coordinates of some object on an image, without touching it in any way.

To address the need of image processing routines, which don't change an image in any way, but only calculate some values, a new plug-in type was added into version 1.2.2 of Computer Vision Sandbox. The name of the plug-in type may sound a bit confusing though - image processing plug-ins. The application already had image processing filter plug-ins. And now it gets image processing plug-ins. Something should of been done with the naming most probably. But this is what it is now.

One of the few image processing plug-ins provided with version 1.2.2 of Computer Vision Sandbox is Image Statistics plug-in. Once it is applied to an image, it calculates set of statistical values, like histograms of RGB channels for color images or histogram of intensities for grayscale images, minimum and maximum values of each channel, mean and standard deviation, etc. These statistical values can then be used to guide different image processing filters, which would change an image appropriately, or can be just used for informational purpose. To start with it, let's have a look at how to get RGB histograms and display them for debug purposes. Note: the script below supposes that sandbox configuration has a camera and one video repeater. The script runs for the camera object, while video repeater is used to show histograms.

local math = require 'math'

-- Use video repeater to output images with histograms
repeater1 = Host.CreatePluginInstance( 'VideoRepeaterPush' )
repeater1:SetProperty( 'id', '1' )

-- Create instances of plug-ins to use
imageDrawing = Host.CreatePluginInstance( 'ImageDrawing' )
imageStats   = Host.CreatePluginInstance( 'ImageStatistics' )

function Main( )
    image  = Host.GetImage( )
    width  = image:Width( )
    height = image:Height( )

    -- Create an image to draw statistics on
    -- (using same size as of source image; hopefully it is enough)
    statsImage = Image.Create( width, height, 'RGB24' )

    -- Processes source image and gather statistics values
    imageStats:ProcessImage( image )

    -- Draw RGB gistograms
    DrawHistogram( statsImage, imageStats:GetProperty( 'redHistogram' ),
                   imageStats:GetProperty( 'redRange' ),
                   imageStats:GetProperty( 'redMean' ),
                   'FF0000', 0, 0, height / 3 )
    DrawHistogram( statsImage, imageStats:GetProperty( 'greenHistogram' ),
                   imageStats:GetProperty( 'greenRange' ),
                   imageStats:GetProperty( 'greenMean' ),
                   '00FF00', 0, height / 3, height / 3 )
    DrawHistogram( statsImage, imageStats:GetProperty( 'blueHistogram' ),
                   imageStats:GetProperty( 'blueRange' ),
                   imageStats:GetProperty( 'blueMean' ),
                   '0000FF', 0, height / 3 * 2, height / 3 )

    -- Push the image with histograms into repeater
    repeater1:ProcessImage( statsImage )
    statsImage:Release( )
end

-- Draw a histogram on the specified image
function DrawHistogram( image, histogram, range, mean, color, x, y, height )
    lineLevel = y + height - 1
    histogramHeight = height - 3

    imageDrawing:CallFunction( 'DrawLine', image,
                               { x, lineLevel }, { x + 255, lineLevel }, color )

    -- Find histogram value with most hits
    max = 0
    for i = 1, 256 do
        if histogram[i] > max then
            max = histogram[i]
        end
    end

    -- Draw the histogram itself
    for i = 1, 256 do
        valueHeight = math.floor( histogramHeight * histogram[i] / max )
        if valueHeight > 0 then
            x1 = x + i - 1
            y1 = y + histogramHeight
            y2 = y1 - valueHeight + 1
            imageDrawing:CallFunction( 'DrawLine', image, { x1, y1 }, { x1, y2 }, color )
        end
    end

    -- Also output min/max/mean
    mean = math.floor( mean * 10 ) / 10
    
    imageDrawing:CallFunction( 'DrawText', image,
        "Min : " .. tostring( range[1] ), { 258, y + 1 }, color, '00000000' )
    imageDrawing:CallFunction( 'DrawText', image,
        "Max : " .. tostring( range[2] ), { 258, y + 11 }, color, '00000000' )
    imageDrawing:CallFunction( 'DrawText', image,
        "Mean: " .. tostring( mean ), { 258, y + 21 }, color, '00000000' )
end

As the script above demonstrates, first an image processing plug-in must be used to process some image. Once it is done, the values it calculated can be retrieved by using GetProperty() API.

One of the applications for the statistical values we've obtained can be histogram stretching, which can be performed by using Levels Linear image processing filter plug-in. As the picture above shows, none of the RGB channels have pixel values in the full [0, 255] range. However, it can be changed with the Levels Linear plug-ins, which will increase contrast of the image.

-- Create instances of plug-ins to use
levelLinear  = Host.CreatePluginInstance( 'LevelsLinear' )
...

function Main( )
    ...
    -- Set input ranges to the ones found by Image Statistics
    -- Output ranges are set to [0, 255] by default
    levelLinear:SetProperty( 'redIn',   imageStats:GetProperty( 'redRange' ) )
    levelLinear:SetProperty( 'greenIn', imageStats:GetProperty( 'greenRange' ) )
    levelLinear:SetProperty( 'blueIn',  imageStats:GetProperty( 'blueRange' ) )

    levelLinear:ProcessImageInPlace( image )
    ...
end

However, in some cases it might be required to stretch histogram even more. Not to the [min, max] range, but to some other range. In this case the provided histograms can be examined to decide what could be a better range to stretch. One thing to do might be to find a range, which contains 95% (or whatever percentage we are interested in) of the histogram values, for example. This is done by dropping 2.5% of values from each side of a histogram, which gives us a smaller range containing the rest of 95%. To simplify things, the Image Statistics plug-in may find such range for us. The range of interest is specified by the Range To Find property (set before processing an image), while the actual found ranges can be then obtained from Red/Green/Blue Range Found properties.

imageStats:SetProperty( 'rangeToFind', 95 )

function Main( )
    ...
    -- Set input ranges to the ones found by Image Statistics
    levelLinear:SetProperty( 'redIn',   imageStats:GetProperty( 'redRangeFound' ) )
    levelLinear:SetProperty( 'greenIn', imageStats:GetProperty( 'greenRangeFound' ) )
    levelLinear:SetProperty( 'blueIn',  imageStats:GetProperty( 'blueRangeFound' ) )
    
    ...
end

And below is the result of stretching the above image to the range containing 95% of histogram. As it demonstrates, the contrast has increased even more.

The above image may look like its histogram was a bit too overstretched and so contrast was increased too much. However, below is another sample image, which has histogram of a very different shape.

The above histogram shows that most of its values are concentrated at the right end - the picture is quite bright and so most of the pixels' values have high values. At the same time the histogram spans through the entire [0, 255] range, which means doing histogram stretching to the [min, max] range as we did before will not give any change at all for this image. Below is the result of applying Levels Linear to the 85% range and does not look so overstretched. It just looks fine. Note: the obtained range actually covers only 92.5% of the histogram. This is caused by the fact that when asking for a range covering 85%, the Image Statistics plug-in tries to drop same amount from both side of a histogram. Which is 7.5% in this case. However, number of 255 values is much greater than this and so they are not excluded from the final range.

There are few more statistical values we can retrieve out of the Image Statistics plug-in. Suppose we have a pre-processed image like the one shown below, which contains some objects of interest with everything else removed. Such images have a lot of black colored background, which appeared due to removing unwanted parts of an image. Calculating statistics for such image may give quite misleading results.

As the above histogram indicates, the image has lots of 0 values. Are we really interested about it? Not really, since we did the pre-processing which introduced it. It would be more helpful to see histograms for the objects we've got. Well, this part is easy. All we need is just to set first histogram value to 0, before finding the value with most hits and rendering the histogram (see the script at the beginning of the tutorial). This way we eliminate the influence of all those black pixels.

Also, check the minimum pixel value and mean value. Again they don't help much. The minimum value for every RGB channel of the above image is 0. It would be better to get minimum value for the objects. Similar about mean value - it is the overall mean value for the entire image, but we would prefer to have the mean value for the objects of our interest. So basically what we are looking is same minimum, mean and standard deviation values, but calculated only for non-zero values. And this is what the Image Statistics plug-in provides through Reg/Green/Blue Range Excluding 0, Reg/Green/Blue Mean Excluding 0 and Reg/Green/Blue StdDev Excluding 0 properties. All we need to do is just to query right property depending on what we need.

Below picture demonstrates a more meaningful result for our pre-processed image - just a small modification to histograms for resetting zero value and using statistical values, which were obtained without taking zeros into account.

That is it about image statics for now. As it was demonstrated it is quite a powerful tool, which can be used as for image enhancement, as for getting information about objects of interest. There are more possible statistical values can be gathered, which may allow doing other interesting things as well. But we'll leave those for future versions of the Computer Vision Sandbox and their tutorials.

 

Next tutorial: Controlling overloaded video processing graph

Copyright © 2015-2019 Andrew Kirillov