Learning Image-Text Associations

Web information fusion can be defined as the problem of collating and tracking information related to specific topics on the World Wide Web. Whereas most existing work on web information fusion has focused on text-based multidocument ummarization,this paper concerns the topic of image and text association, a cornerstone of cross-media web information fusion. Specifically, we present two learning methods for discovering the underlying associations between images and texts based on small training data sets. The first method based on vague transformation measures the information similarity between the visual features and the textual features through a set of predefined domain-specific information categories. Another method uses a neural network to learn direct mapping between the visual and textual features by automatically and incrementally summarizing the associated features into a set of information templates. Despite their distinct approaches, our experimental results on a terrorist domain document set show that both methods are capable of learning associations between images and texts from a small training data set.

Proposed system:

The task of identifying image-text associations can be cast into an information retrieval (IR) problem .In this paper, we present two methods, following the multilingual retrieval paradigm for learning image text associations. The first method is a textual-visual similarity model with the use of a statistical vague transformation technique for extracting associations between images and texts. As vague transformation typically requires large training data sets and tends to be computationally intensive, we employ a set of domain-specific information categories for indirectly matching the textual and visual information at the semantic level. With a small number of domain information categories, the training data sets for vague transformation need not be large and the computation cost can be minimized. In addition, as each information category summarizes a set of data samples,

implicit image-text associations can be captured

The performance of learning association rule is highly dependent on the number of items (e.g., image features and the number of lexical terms). Although existing methods that learning association rules between image features and high-level semantic concepts are applicable for small set of concepts/ keywords, they may encounter problems when mining association rules on images and free texts where a large amount of different terms exist. This may not only cause significant increasing in the learning times but also result in a great number of association rules which may also lower the performance during the process of annotating images as more rules need to be considered and consolidated.

 

Algorithms used:

 1)  Based on vague transformation measures

 2)  A neural network

SYSTEM REQUIREMENT

Hardware Requirements

       Processor                     :           Pentium III / IV

            Hard Disk                   :           40 GB

            Ram                             :           256 MB

Monitor                       :           15VGA Color

            Mouse                         :           Ball / Optical

            Keyboard                    :           102 Keys

Software Requirements

Operating System       :           Windows XP professional

Front End                    :           Microsoft Visual Studio .Net 2005

Language                    :           Visual C#.Net

Back End                    :           SQL Server 2000