Knowledge extraction from the learning of sequences in a long short term memory (LSTM) architecture
We introduce a general method to extract knowledge from a recurrent neural network (Long Short Term Memory) that has learnt to detect if a given input sequence is valid or not, according to an unknown generative automaton. Based on the clustering of the hidden states, we explain how to build and validate an automaton that corresponds to the underlying (unknown) automaton, and allows to predict if a given sequence is valid or not. The method is illustrated on artificial grammars (Reber's grammar variations) as well as on a real use-case whose underlying grammar is unknown.
Code (0)
등록된 구현이 없습니다.
Tasks
ClusteringvalidSimilar Papers 제목 키워드 기반
KEDformer:Knowledge Extraction Seasonal Trend Decomposition for Long-term Sequence Prediction
Time series forecasting is a critical task in domains such as energy, finance, and meteorology, where accurate long-term predictions are essential. While Transformer-based models have shown promise in capturing temporal …
Time SeriesTime Series ForecastingKeyphrases Extraction from User-Generated Contents in Healthcare Domain Using Long Short-Term Memory Networks
We propose keyphrases extraction technique to extract important terms from the healthcare user-generated contents. We employ deep learning architecture, i.e. Long Short-Term Memory, and leverage word embeddings, medical …
Question AnsweringText ClassificationText SummarizationWord EmbeddingsAutomatic Open Knowledge Acquisition via Long Short-Term Memory Networks with Feedback Negative Sampling
Previous studies in Open Information Extraction (Open IE) are mainly based on extraction patterns. They manually define patterns or automatically learn them from a large corpus. However, these approaches are limited when…
Implicit RelationsOpen Information ExtractionSentenceSequence Graph Transform (SGT): A Feature Extraction Function for Sequence Data Mining (Extended Version)
The ubiquitous presence of sequence data across fields such as the web, healthcare, bioinformatics, and text mining has made sequence mining a vital research area. However, sequence mining is particularly challenging bec…
ClusteringMeta-Learning for Neural Network-based Temporal Point Processes
Human activities generate various event sequences such as taxi trip records, bike-sharing pick-ups, crime occurrence, and infectious disease transmission. The point process is widely used in many applications to predict …
Meta-LearningPoint ProcessesPrediction