AVISION

Lab View  
HOME PEOPLE RESEARCH PUBLICATIONS COURSES SPONSORS    

RESEARCH  HIGHLIGHTS

     Projects

     Demos

     Software

     Datasets

  ABOUT US

     Seminars

     Lab

     News

 

  LINKS

Igert

Vision

Ece

Ucsb

Melvyl

 

 

 

 

Project: Real-Time Color Image Guidance System 

PEOPLE  

Daryl Robert Fortney, B.S.Manjunath

OBJECTIVE

This project focus is on implementation of a real-time color image guidance system. Areas of research include image acquisition, color-space transformation and classification, statistical object segmentation, predictive object tracking, and camera control.  This system is operated on a PC platform including a color video camera, a digital signal-processing card containing a Texas Instruments C4x DSP and color image capture, and a serial port interface to a robotic tripod mechanism. This system was first used as a commercial instrument-tracking interface to a medical robotics application, and was later modified for use as a research tool and front-end to a larger system for face detection and recognition. 

EXAMPLES

Tracking Medical Instrument Tags - An example Descriptor File setup for tracking medical instrument tags is \CODE\SYSTEM\INST.DSC.  In this file, the Hue parameter has been set to identify colors between the range of 170 through 210.  This represents a shade of blue with exceptional results for the chosen medical instrument tag as previously discussed.

Tracking Human Faces - An example Descriptor File setup for tracking human faces based on Caucasian skin color is \CODE\SYSTEM\FACE.DSC.  In this file the Hue parameter has been set to identify colors between the range of 60 through 80.  This represents a shade of pink akin to Caucasian skin color.

Software:

The user interface software was developed using the Borland C++ V3.11 development environment.  Source code is available for download from here.  The main project file is named AVI_PC.PRJ.  The components of this project were written in C++, and include the main program AVI_PC.CPP which is supported by the files: IOL.CPP a generic IO Library, PTU.CPP a Pan-Tilt Unit driver, UART.CPP a serial port driver, and LIA.CPP and FASTLIA.ASM Link Interface Adapter Protocol drivers.  Also provided is SLU.CPP for controlling an optional servo lens unit.  The resulting executable file is AVI_PC.EXE.

The AVI_C40 component is responsible for executing the image processing algorithms in real-time.  It achieves this by first communicating with the PC-based component through the LIA Protocol for updated user-interface parameter and control information, second processing the image frame according to the image guidance algorithms as researched, and third sending the image processing results to the PC for camera control. Details of the image guidance algorithms as researched during the course of this paper are presented next.

The image processing software was developed using the Code Composer V3.05 development environment.  Source code is available for download from here.  The main project file is named AVI_C40.MAK.  The components of this project were written in mixed C and assembly language, and include the main program AVI_C40.C which is supported by the files: CLS_HSV.ASM the color space transformation and classification code, and VIDEO.C and CHARSET.C for overlaying and displaying the image processing results.  The output file is AVI_C40.OUT and is loaded by the PC-Based component to the DSP during system initialization.

 

AVISION MATLAB Tool

Throughout this project, MATLAB was used extensively as a test-bed for the development of the image processing algorithms.  The result is a GUI based MATLAB tool termed AV for Active Vision image analysis.  References to this application are made throughout the contents of this paper. Using this tool in conjunction with hours of NTSC medical video camera footage allowed the determination and setting of the various Descriptor File parameters as specified in Chapter 8 of complete report. The source code for this tool is located in the Matlab.zip file.  In order to run the tool this folder must be referenced by the PATHDEF variable of the MATLAB configuration file \MATLAB\TOOLBOX\LOCAL\PATHDEF.M.

  System Usage - This system is initialized and executed by the AVI_PC.EXE application running on the PC.  The command line syntax of this application is as follows: AVI_PC          [filename.dsc] 

Where, [filename.dsc] is an optionally specified Descriptor File. A Descriptor File is a text-based file used to specify both the target application and the default values of each of the target application recognized operating parameters.  It is a text file listing first, the name of the DSP target application, second the ISA port addresses of the DSP card, third a descriptive comment, and finally an ordered list of each application recognized parameter name followed by the minimum, default, and maximum parameter values.  Refer to the default Descriptor File named AVISION.DSC for an example.

The ordered and complete list of recognized parameters of the AVI_C40 application are listed in the table below:

 

Parameter Name

Min Value

Max Value

Description

Image Acquisition Parameters

IMG_MODE

0

1

Perform Acquisition

0=Off

1=On

IMG_OVR

0

1

Store Image w/ Overlay

0=Off

1=On

IMG_X_S

0

511

X Dimension Start Pixel

IMG_X_E

0

511

X Dimension End Pixel

IMG_X_R

1

511

X Dimension Sample Rate

IMG_Y_S

0

479

Y Dimension Start Pixel

IMG_Y_E

0

479

Y Dimension End Pixel

IMG_Y_R

1

479

Y Dimension Sample Rate

Classification Parameters

CLS_MODE

0

1

Perform Classification

0=Off

1=On

CLS_X_R

1

511

X Dimension Sample Rate

CLS_Y_R

1

479

Y Dimension Sample Rate

CLS_V_LO

0

255

Value Threshold (Low)

CLS_S_LO

0

255

Saturation Threshold (Low)

CLS_H_LO

0

255

Hue Threshold (Low)

CLS_H_HI

0

255

Hue Threshold (High)

Object Tracking Parameters

TRK_MODE

0

2

Perform Tracking

0=Off

1=On (Single Object)

2=On (Multiple Object)

TRK_Zi

0

999

Required Mass To Begin Tracking

TRK_Zo

0

999

Required Mass To Continue Tracking

TRK_SRCt

0

999

Time (ms) To Search For Lost Object

TRK_XSIZE

0

511

X Dimension Bounding Box Maximum Size

TRK_YSIZE

0

479

Y Dimension Bounding Box Maximum Size

Camera Control Parameters

CNT_MODE

0

1

Perform Camera Control

0=Off

1=On

CNT_SENS

0

10

b - Sensitivity Curve Selector

CNT_GAIN

0

999

a - Gain Curve Multiplier

Table 1 User Interface Adjustable Parameters

After initializing the Pan-Tilt Unit, AVI_PC loads and initializes the specified DSP target application AVI_C40 to the DSP card using the default parameter values as specified in the Descriptor File.  These parameters are then listed on the console where the user may select and adjust their values using the number pad.

In order to efficiently store and retrieve multi-dimensional image frames and sequence data including overlay information along side tracking data, a unique file format was developed.  This format is termed the Program Application Interface (PAI) format.  Using this format many different types of information may be stored.  The specific format for Color Images is diagrammed below:

 

Figure 11 PAI Image File Format

Image data from the camera may be output either to the results monitor or to a specified PAI file.  The default setting is to output the images to the results monitor.  To redirect the output to a specified file, select O for output and then enter F for file, followed by the path and filename.  Be aware that a great deal of data is generated very quickly using this method. Image data for processing may be input either from the camera directly, or from a specified PAI file.  The default setting is to input images from the camera.  To redirect the input from a specified file, select I for input and then enter F for file, followed by the path and filename. The image processing algorithms may be run continuously or a specified number of times. The default setting is to run continuously. To run the image processing loop specified number of times, select R for run mode and then enter S for step followed by the number of steps.  This feature is useful for storing a limited number of images to file as described above. While the processor is running, the user may adjust any parameter values.  Adjustments are made by either using the number pad to scroll the arrow through the parameter list and selecting new values, or by editing the Descriptor File.  Next we present two example Descriptor Files which preset the system to track medical instrument tags and human faces.

 

 

 

If you have any comments or questions, please send them to web administrator.

Image Processing and Vision Research Lab, Electrical and Computer Engineering 

Department,  University of California at Santa Barbara, Santa Barbara, CA 93106.