Object-based Retrieval in Image Databases


Sitaram Bhagavathy, Shawn Newsam, Lei Wang, B.S.Manjunath


Our objective is develop models to describe geo-spatial object classes (e.g. airports, golf courses, harbors, etc.) in order to enable tasks such as object detection, recognition, and segmentation. Such objects cannot be effectively modeled using traditional shape-based or contour-based techniques because they lack homogeneity of features, tight spatial constraints, and well-defined boundaries. We created statistical models for various geo-spatial objects with image texture as the visual feature. The choice of texture as a feature is justifiable because many geographic processes (e.g. agricultural fields, parking lots, etc.), that result in object formation, can be effectively characterized by it. The model characterizes object classes in terms of texture motifs and their spatial distribution. Texture motifs are the textures that are characteristic of an object class. For example, in the case of golf courses, the texture motifs are the texture signatures corresponding to trees and fairways.

We used a Gaussian mixture model (GMM) to learn the texture motifs for an object class. This model captures the wide variation of visual features within the class, which results in the formation of clusters in the feature space. Using texture samples drawn from several training examples, we trained a GMM using the expectation-maximization (EM) algorithm. The dominant Gaussians in the GMM represent the texture motifs. Given a new object image from the same class, we can use the model to learn the distribution of texture motifs in it. We can also use the model for object classification by computing the probability that an unlabeled object belongs to that class. We improved the GMM-based model by learning the spatial arrangement of texture motifs within a class. We used a hidden Markov modeling (HMM) framework, wherein the states of the model capture the texture motifs and the state transition probabilities capture the adjacency relationships between the motifs. The distribution of the texture features (observed outputs) are once again modeled as a GMM. We used horizontal and vertical sequences of texture features as training data for the HMM. Using the model thus created, we obtained a better performance, in terms of texture motif labeling and object classification, than that of the previous model.


These materials are presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each authors copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder. Lei Wang and B. S. Manjunath, "A SEMANTIC REPRESENTATION FOR IMAGE RETRIEVAL," IEEE International Conference on Image Processing (ICIP), Barcelona, Spain, September 2003. [abstract] S. Newsam, S. Bhagavathy, and B.S. Manjunath, "Object Localization Using Texture Motifs and Markov Random Fields," IEEE International Conference on Image Processing (ICIP), Barcelona, Spain, September 2003. [abstract] S. Newsam, L. Wang, S. Bhagavathy, and B. S. Manjunath, "Using Texture to Annotate Remote Sensed Datasets," Proceedings of 3rd International Symposium on Image and Signal Processing and Analysis (ISPA), September 2003, Rome, Italy, September 2003. [abstract] J. Tesic, S. Newsam, and B.S. Manjunath, "Mining Image Datasets using Perceptual Association Rules," Proceedings of SIAM Sixth Workshop on Mining Scientific and Engineering Datasets in conjunction with the Third SIAM International Conference (SDM), San Francisco, California, May 2003. [abstract] S. Newsam, S. Bhagavathy, and B. S. Manjunath, "Modeling Object Classes in Aerial Images Using Hidden Markov Models," Proceedings of International Conference on Image Processing (ICIP), Rochester, NY, USA, September 2002. [abstract] S. Bhagavathy, S. Newsam, and B. S. Manjunath, "Modeling Object Classes in Aerial Images Using Texture Motifs," Proceedings of IEEE International Conference on Pattern Recognition (ICPR), Québec City, August 2002. [abstract]


This work is supported in part by an ONR/AASERT #N00014-98-1-0515 grant.