Performing Partial Image Selection

Overview

A digital image consists of a number of visual groups formed by visually similar pixels. These visual groups are referred to as objects, and any given image contains a number of objects that appear together to define that image. Segmentation is the process by which an image is divided into object regions, with each region identifying a similar set of objects. Once these object regions are identified, features such as color, texture, and shape are extracted from each of the object regions.

A MediaObject contains the following information:

• a copy of the image that was analyzed

• a segmentation mask, which is a 2D-byte array that maps each pixel in the image to a specific class. Each byte in the array contains a value of 0, 1, or 2 corresponding to the class to which the image’s pixels were mapped.

Because of the advanced segmentation technique built into eVe, you can enable users to identify specific objects within an image and accurately search for both a whole image and for parts (or objects) of an image.

The following describes how you can use eVe to enable partial image selection for users:

• Determine the classes on which the user is clicking

• Extract the signatures for the classes from the corresponding MediaObject

• Construct a "dummy" MediaObject containing the signatures

• Pass the "dummy" MediaObject to search as the source

The following steps describe in detail how to program eVe for partial image selection within your application:

1. Obtain a source image and display. Use the methods in the ImageManager interface to perform this step.

2. Obtain the segmentation mask for the source image and display. Use the methods in the ImageManager interface to perform this step.

3. Obtain the 2D byte array corresponding to the segmentation mask.
   (byte segmentationMask[][] = (byte[][]) mediaObject.getProperty("segmentationMask")))

4. When the user selects the output from either step 1 or step 2, capture the x,y coordinates of the mouse click and compare these against the results from step 3 to determine the class in which the user is interested.

5. When users activate the search function, use eVe to perform the following:

a. construct a new dummy media object

b. iterate over the particular classes of interest:

Vector newColorSignatures = new Vector();

Vector newRegionSignatures = new Vector();

etc.

c. for each class selected, code:

newColorSignatures.add(source.getIndex(Eve.COLOR,<selected class>);

newRegionSignatures.add(source.getIndex(Eve.REGION,<selected class>);

dummyMediaObject.setIndex(Eve.COLOR,newColorSignatures);

dummyMediaObject.setIndex(Eve.REGION,newRegionSignatures);

d. call the media collection's search method:

SearchResults results[] = mediaCollection.search(dummyMediaObject,<your search parameters>);

Code Samples

To Get A Segmentation Mask Image:

MediaObject mo=mc.getMediaObject(12345); //get a MediaObject somehow.

Image i=Eve.newImageManager().getSegmentationMaskImage(moo);

To Search for One Region in an Image:

String regs="01"; //this indicates we want to search using only regions 0 and 1, or "red" and "green"

 //we would set this to "02" for "red" and "blue".

 //or "12" for "green" and "blue".

 //for an understanding of these scare-quoted colors, look at a segmentation mask image.

MediaObject newmo=Eve.newMediaObject(); //create a temporary media object, currently empty.

Vector[] vs={mo.getIndex(Eve.COLOR), mo.getIndex(Eve.TEXTURE), mo.getIndex(Eve.SHAPE), mo.getIndex(Eve.REGION)};

//create a working vector containing all indexes for all search axes.

int[] ins={Eve.COLOR, Eve.TEXTURE, Eve.SHAPE, Eve.REGION};

//this is a temporary array to allow us to reduce code by looping.

/*

Now, for each basic search axis,

*/

for (int i=0;i<4;i++){

/*

insert into the new MediaObject only the indexes for the regions we have selected, thus creating an "artificial" query object that reflects only our chosen regions.

*/

Vector newvec=new Vector();

Vector oldvec=vs[i];

for (int j=0;j<oldvec.size();j++){

if (regs==null || regs.length()==0 || regs.indexOf((j)+"")!=-1){

//here, add the index but only if the index is one of our selected indexes.

//TO BE CLEAR, oldvec is a vector that contains as many indexes as there

//are regions in the image - one index per region, in red, green, blue

//order. Note that there may be fewer regions, depending on the

//EveContext used during analysis or the Eve.properties file used during

//analysis.

newvec.addElement(oldvec.elementAt(j));

}

}

newmo.setIndex(ins[i], newvec);

}

mo=newmo;