Project Title
A Dynamical Systems Approach to Visual Attention based on Saliency Maps

Focal attention on a scene is thought to be controlled by competition between weighted modalities. A bottom-up model of attention biases toward stimuli with the highest saliency, while a top-down model integrates selection based on an external goal. The combination of modalities into a single saliency map directs attentional shift via a winner take all network.

The purpose of this project is to learn color and motion modality weights in a supervised fashion and then use these weights to predict future attentional shift. A commercial eye tracking system monitors focus of a test subject while the weighted model controls gaze of a pan-tilt camera.

The software system is written in C++ and image processing uses the OpenCV library on a x86 Linux platform. Multithreading with Unix pthreads and breaking the application into components allows a focus on future extensibility for additional modalities and applications.

Following is a chronological list of goals to accomplish the project:

• Color and motion feature extraction.
• Feature extraction from a real time video input.
• Combining features into a saliency map.
• Integrate camera for goal directed movement and focus.
• Supervised learning of modality saliency weights.

Project Timeline

Project Results & Conclusions
Thesis