Transductive Multi-Modality Video Semantic Concept Detection with Tensor Representation
基于张量表示的直推式多模态视频语义概念检测

A higher-order tensor framework for video analysis and understanding is proposed in this paper.In this framework,image frame,audio and text are represented,which are the three modalities in video shots as data points by the 3rd-order tensor.Then a subspace embedding and dimension reduction method is proposed,which explicitly considers the manifold structure of the tensor space from temporal-sequenced associated co-occurring multimodal media data in video.It is called TensorShot approach.Transductive learning uses a large amount of unlabeled data together with the labeled data to build better classifiers.A transductive support tensor machines algorithm is proposed to train effective classifier.This algorithm preserves the intrinsic structure of the submanifold where tensorshots are sampled,and is also able to map out-of-sample data points directly.Moreover,the utilization of unlabeled data improves classification ability.Experimental results show that this method improves the performance of video semantic concept detection.