Mining Events in Image Databases


Jelena Tesic, Shawn Newsam, B.S.Manjunath


As technology advances and more visual data are available, we need more effective systems to handle the image data understanding. The framework must efficiently summarize information contained in the image data; it must provide scalability with respect to the nature, size and dimension of a dataset; and it must offer simple representations of the results and relationships discovered in the dataset. Humans can instantly answer the question ``Is this highway going through a desert?'' just by looking at an aerial photograph of a region. This query, essentially formulated as a high-level concept, cannot be answered by most existing intelligent image analysis systems. Meaningful semantic analysis and knowledge extraction require data representations that are understandable at a conceptual level.

This work introduces a novel approach to spatial event representation and analysis for large image datasets. Image features are classified using supervised and unsupervised learning techniques. Spatial relationships between the labeled features are summarized using Spatial Event Cubes (SEC). SEC are shown to be effective for visualizing non-obvious dataset spatial characteristics such as frequently occurring land-cover arrangements in aerial images. SEC support perceptual association rule mining approach and provide efficient pruning for generating higher order candidate itemsets. We demonstrate the use and possible applications of proposed framework on a large collection of aerial videos of Amazonia and large collection aerial photos of Santa Barbara region.


These materials are presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each authors copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder. S. Newsam, J. Tesic, L.Wang and B.S. Manjunath, "Issues in Mining Video Datasets," in Proceedings of SPIE International Symposium On Electronic Imaging, Storage and Retrieval Methods and Applications for Multimedia, San Jose, California, January 2004. J. Tesic, S. Newsam, and B.S. Manjunath, "Mining Image Datasets using Perceptual Association Rules," Proceedings of SIAM Sixth Workshop on Mining Scientific and Engineering Datasets in conjunction with the Third SIAM International Conference (SDM), San Francisco, California, May 2003. [abstract] J. Tesic, S. Newsam, and B.S. Manjunath, "Scalable Spatial Event Representation," Proceedings of IEEE International Conference on Multimedia and Expo (ICME), Lausanne, Switzerland, August 2002. [abstract]


This research was supported in part by The Institute of Scientific Computing Research (ISCR) award under the auspices of the U.S. Department of Energy by the Lawrence Livermore National Laboratory under contract No.W-7405- ENG-48, ONR# N00014-01-1-0391, NSF Instrumentation #EIA-9986057, and NSF Infrastructure #EIA-0080134.