human computer interaction

Color Predicates

It’s been a long time since I posted something here. But here it is.

I started my current project with a simple objective- I wanted to build a completely natural human- computer interaction system. Basically, my final aim is to just have a few cameras and microphones work as the input devices instead of a mouse and keyboard. So, with this in mind, I started out with my first objective- to segment out and track the hand in a video stream.
The first problem that I encountered was that skin as such is not a single color but a heady mixture of reds, yellows and white.The problem is simplified a bit in the HSV colorspace, but not enough. A simple thresholding program will not work at all given such few constraints. So, while researching on how to solve this problem, I got a paper mentioning color predicates.

Now, what are color predicates? Color predicates, very simply put are a map in the colorspace. A simple region within which every color included is taken as being part of the colors to be segmented and outside of which are the rejected colors. So, I decided that this was the way to go. Took me a couple of days but I was able to build a simple color predicate training program.

My way of implementing color predicates:

I used a matrix of 255×255 to represent the Hue and Saturation values possible. I then provided several images of my hand cropped onto a white background ( Used GIMP – awesome piece of software ). Then I scanned each image pixel by pixel and rejecting white, I simply incremented the value at each H,S index of the matrix if I found a color other than white. Now using this simple algorithm, I trained a color predicate for around 20 images taken in various illumination schemes.  With a few manipulations for rejecting low values, I then used the color predicate for simple thresholding of images in my video stream.


The best part of this exercise was when I looked at the HSV map thus created in a text editor….amazingly, there were two different regions which could be clearly seen, corresponding to the red spectrum and the yellow spectrum.

The code for the above exercise:

//Color Predicates ver 1.2


int colorp[256][256];
IplImage *img, *imghsv;
int h,s,v;
int main()

int i,j;

FILE *fp = fopen("cp2.bin", "r");
fread(colorp[i], sizeof(int), 256, fp);

CvScalar Data;


FILE* fp2=fopen("cp2.bin","w");
fwrite(colorp[i], sizeof(int), 256*256, fp2);


return 0;


In the next post, I will be describing my attempt to use the created color predicate.