Documents as multiple overlapping windows into grids of counts
In text analysis documents are represented as disorganized bags of words, models of count features are typically based on mixing a small number of topics \cite{lda,sam}. Recently, it has been observed that for many text corpora documents evolve into one another in a smooth way, with some features dropping and new ones being introduced. The counting grid \cite{cgUai} models this spatial metaphor literally: it is multidimensional grid of word distributions learned in such a way that a document's own distribution of features can be modeled as the sum of the histograms found in a window into the grid. The major drawback of this method is that it is essentially a mixture and all the content much be generated by a single contiguous area on the grid. This may be problematic especially for lower dimensional grids. In this paper, we overcome to this issue with the \emph{Componential Counting Grid} which brings the componential nature of topic models to the basic counting grid. We also introduce a generative kernel based on the document's grid usage and a visualization strategy useful for understanding large text corpora. We evaluate our approach on document classification and multimodal retrieval obtaining state of the art results on standard benchmarks.
Code (0)
등록된 구현이 없습니다.
Tasks
Document ClassificationRetrievalTopic ModelsSimilar Papers 제목 키워드 기반
Accurate Supervised and Semi-Supervised Machine Reading for Long Documents
We introduce a hierarchical architecture for machine reading capable of extracting precise information from long documents. The model divides the document into small, overlapping windows and encodes all windows in parall…
DecoderQuestion AnsweringReading ComprehensionSubject Cross Validation in Human Activity Recognition
K-fold Cross Validation is commonly used to evaluate classifiers and tune their hyperparameters. However, it assumes that data points are Independent and Identically Distributed (i.i.d.) so that samples used in the train…
Activity RecognitionHuman Activity RecognitionHierarchical learning of grids of microtopics
The counting grid is a grid of microtopics, sparse word/feature distributions. The generative model associated with the grid does not use these microtopics individually. Rather, it groups them in overlapping rectangular …
DiversityGeneral ClassificationWindformer:Bi-Directional Long-Distance Spatio-Temporal Network For Wind Speed Prediction
Wind speed prediction is critical to the management of wind power generation. Due to the large range of wind speed fluctuations and wake effect, there may also be strong correlations between long-distance wind turbines. …
ManagementTime SeriesParallel Context Windows for Large Language Models
When applied to processing long text, Large Language Models (LLMs) are limited by their context window. Existing efforts to address this limitation involve training specialized architectures, and cannot be easily applied…
In-Context LearningPlaying the Game of 2048Question AnsweringRetrieval