Spatio-Temporal Querying of Video Content Using SQL for Quantizable Video Databases

Multimedia database modeling and representation play an important role for efficient storage and retrieval of multimedia. Modeling of semantic video content that enables spatio-temporal queries is one of the challenging tasks. A video is called as “quantizable” if the instants of a video are enough for a person to imagine the missing scenes properly. A semantic query for quantizable videos can be defined in a more flexible way using spatio-temporal instants. In this paper, we provide a semantic modeling and retrieval system, termed as G-SMART. Firstly, the videos are quantized according to semantic events. Then semantic instants and events of the video that include objects, events, and locations are provided as a grammar-based string. This linear string representation enables both the spatial and temporal retrieval of the video using Structured Query Language (SQL). The redundancy in this linear representation is reduced by using data reduction properties such as removal of implied information. Various types of queries such as event-object-location, event- location, object-location, event-object, current-next event, projection and semantic event are supported by G-SMART. A graphical user interface is designed to build queries and view the query results. GSMART enables multimodal presentation by displaying the query results in the form images and videos. We show our results on a tennis video database.