Sandbox variables

Previous tutorial: Releasing resources and garbage collection

 

In one of the previous tutorials it was demonstrated how to put images on top a video source with a bit of scripting. But it would be more interesting to put one video source into another. This, however, requires some way of communication between video processing graphs of video sources - a way to make video frames of one video source available for another. This can be achieved using sandbox variables.

Starting from API revision 3, the Lua scripting plug-in provides scripting API, which enables scripts to store variables on the host's side (application running the scripts). This allows scripts exchanging different variables - a script running for one video source may set variables, while another script running for another video source may get those variables and do something about them. The type of variables, which are allowed to be stored at host, are Lua scalars, arrays and image objects. The latter is what we need to get one video source displayed on top of another one.

Before going into coding, below is a quick example of sandbox view, which runs 4 video sources, but uses 2 cells for display - the left cell shows an IP camera put on top of another IP camera, while the cell on the right shows IP camera put on top of a video source capturing system screen (content of Internet Explorer in this case).

The script for video source, which we want to put on top of another video source (or combine in any other possible way) is very simple. All it needs to do is set its current video frame to some host variable using the Host.SetVariable() API: 

function Main( )
    Host.SetVariable( 'camera1', Host.GetImage( ) )
end

The script for the second video source, which we want to combine with another source, is a bit longer, but no way more complicated. First it uses Host.GetVariable() API to get an image from a host variable, then resizes it and finally puts it on top of the current video frame. And since we get a copy of the host image, we better release it afterwards to free memory. 

-- Create instance of resize plug-in
resizeImagePlugin = Host.CreatePluginInstance( 'ResizeImage' )
resizeImagePlugin:SetProperty( 'interpolation', 1 )

function Main( )
    -- Get current video frame
    image = Host.GetImage( )
    -- Get image to embed
    anotherImage = Host.GetVariable( 'camera1' )

    if anotherImage then
        width  = image:Width( )
        height = image:Height( )

        smallWidth  = width  / 2;
        smallHeight = height / 2;

        -- Resize the image coming from another video source
        resizeImagePlugin:SetProperty( 'width',  smallWidth )
        resizeImagePlugin:SetProperty( 'height', smallHeight )
        resizedImage = resizeImagePlugin:ProcessImage( anotherImage )

        -- Put resized image of another video source on top current one
        image:PutImage( resizedImage, width - smallWidth, 0 )

        -- Release image we no longer need
        anotherImage:Release( )
        resizedImage:Release( )
    end
end

Just putting one image on top of another is one of the simplest thigs to do. There can be many other different ways of processing two images. Some of those may require the new type plug-ins, which can process two images together to achieve their goal. That is a topic for a different tutorial though. Below, however, is a quick sample of how two video sources are embedded into quadrilaterals of another video source.

And below is a bit of code to demonstrate embedding one image into quadrilateral of another image: 

-- Create instance of the plug-in to embed images into quadrilateral
embedQuadPlugin = Host.CreatePluginInstance( 'EmbedQuadrilateral' )

function Main( )
    -- Get current video frame
    image = Host.GetImage( )
    width  = image:Width( )
    height = image:Height( )

    smallWidth  = 2 * width  / 5
    smallHeight = 2 * height / 5

    -- Get image to embed
    anotherImage = Host.GetVariable( 'camera1' )

    if anotherImage then
        embedQuadPlugin:SetProperty( 'point1', { 0, 0 } )
        embedQuadPlugin:SetProperty( 'point2', { smallWidth,   smallHeight / 4 } )
        embedQuadPlugin:SetProperty( 'point3', { smallWidth, ( smallHeight / 4 ) * 3 } )
        embedQuadPlugin:SetProperty( 'point4', { 0, smallHeight } )

        embedQuadPlugin:ProcessImageInPlace( image, anotherImage )
        anotherImage:Release( )
    end
end

 

Next tutorial: Two source image processing