Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning
Large Language Models (LLMs) excel at reasoning and planning when trained on chainof-thought (CoT) data, where the step-by-step thought process is explicitly outlined by text tokens. However, this results in lengthy inputs where many words support textual coherence rather than core reasoning information, and processing these inputs consumes substantial computation resources. In this work, we propose a hybrid representation of the reasoning process, where we partially abstract away the initial reasoning steps using latent discrete tokens generated by VQ-VAE, significantly reducing the length of reasoning traces. We explore the use of latent trace abstractions in two scenarios: 1) training the model from scratch for the Keys-Finding Maze problem, 2) fine-tuning LLMs on this hybrid data with an extended vocabulary including unseen latent tokens, for both logical and mathematical reasoning problems. To facilitate effective learning, we introduce a simple training procedure that randomly mixes latent and text tokens, which enables fast adaptation to new latent tokens. Our approach consistently outperforms the baselines methods in various benchmarks.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingMathematical ReasoningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Improving Latent Reasoning in LLMs via Soft Concept Mixing
Unlike human reasoning in abstract conceptual spaces, large language models (LLMs) typically reason by generating discrete tokens, which potentially limit their expressive power. The recent work Soft Thinking has shown t…
Reinforcement LearningExploring Unbiased Deepfake Detection via Token-Level Shuffling and Mixing
The generalization problem is broadly recognized as a critical challenge in detecting deepfakes. Most previous work believes that the generalization gap is caused by the differences among various forgery methods. However…
DeepFake DetectionFace SwappingPositionUnmixing-Guided Spatial-Spectral Mamba with Clustering Tokens for Hyperspectral Image Classification
Although hyperspectral image (HSI) classification is critical for supporting various environmental applications, it is a challenging task due to the spectral-mixture effect, the spatial-spectral heterogeneity and the dif…
Hyperspectral Image ClassificationToken-Label Alignment for Vision Transformers
Data mixing strategies (e.g., CutMix) have shown the ability to greatly improve the performance of convolutional neural networks (CNNs). They mix two images as inputs for training and assign them with a mixed label with …
image-classificationImage ClassificationSemantic SegmentationTransfer LearningLeveraging Latent Visual Reasoning in Silence
Latent visual reasoning involves visual evidence more directly in multimodal reasoning by inserting continuous latent tokens before textual generation. However, the necessity of these latent tokens at inference remains a…
Reinforcement LearningMultimodal ReasoningSpatial ReasoningVisual Grounding