LAMBADA
1개 벤치마크 · 논문 30편 · 이 태스크의 논문 보기 →
Benchmarks
BIG-bench
Most implemented
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
Universal Transformers
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
The LAMBADA dataset: Word prediction requiring a broad discourse context
Training Compute-Optimal Large Language Models
Residual Shuffle-Exchange Networks for Fast Processing of Long Sequences
Papers
Matryoshka Model Learning for Improved Elastic Student Models
Industry-grade ML models are carefully designed to meet rapidly evolving serving constraints, which requires significant resources for model development. In this paper, we propose MatTA, a framework for training multiple…
LAMBADAMathmodelAdaGC: Improving Training Stability for Large Language Model Pretraining
Large Language Models (LLMs) face increasing loss spikes during scaling, undermining training stability and final performance. While gradient clipping mitigates this issue, traditional global approaches poorly handle par…
LAMBADALanguage ModelingLanguage ModellingLarge Language ModelExplaining and Improving Contrastive Decoding by Extrapolating the Probabilities of a Huge and Hypothetical LM
Contrastive decoding (CD) (Li et al., 2023) improves the next-token distribution of a large expert language model (LM) using a small amateur LM. Although CD is applied to various LMs and domains to enhance open-ended tex…
LAMBADAText GenerationBeyond Autoregression: Fast LLMs via Self-Distillation Through Time
Autoregressive (AR) Large Language Models (LLMs) have demonstrated significant success across numerous tasks. However, the AR modeling paradigm presents certain limitations; for instance, contemporary autoregressive LLMs…
Automated Theorem ProvingCode GenerationLAMBADANatural Language UnderstandingSpectra: Surprising Effectiveness of Pretraining Ternary Language Models at Scale
Rapid advancements in GPU computational power has outpaced memory capacity and bandwidth growth, creating bottlenecks in Large Language Model (LLM) inference. Post-training quantization is the leading method for addressi…
GPULAMBADALanguage ModellingLarge Language Model+1SymBa: Symbolic Backward Chaining for Structured Natural Language Reasoning
To improve the performance and explainability of LLM-based natural language reasoning, structured reasoning can be applied to generate explicitly structured proofs. Among different methods for structured reasoning, we sp…
Arithmetic ReasoningGSM8KLAMBADA