Papers 16k
“16k” 태그가 달린 논문 146편 · 필터 해제
UniCode$^2$: Cascaded Large-scale Codebooks for Unified Multimodal Understanding and Generation
Unified multimodal large language models (MLLMs) have shown promise in jointly advancing multimodal understanding and generation, with visual codebooks discretizing images into tokens for autoregressive modeling. Existin…
16kMSTAR: Box-free Multi-query Scene Text Retrieval with Attention Recycling
Scene text retrieval has made significant progress with the assistance of accurate text localization. However, existing approaches typically require costly bounding box annotations for training. Besides, they mostly adop…
16kRetrievalText RetrievalHow Far Are We from Optimal Reasoning Efficiency?
Large Reasoning Models (LRMs) demonstrate remarkable problem-solving capabilities through extended Chain-of-Thought (CoT) reasoning but often produce excessively verbose and redundant reasoning traces. This inefficiency …
16kBenchmarkingNumerical IntegrationFlashDMoE: Fast Distributed MoE in a Single Kernel
The computational sparsity of Mixture-of-Experts (MoE) models enables sub-linear growth in compute cost as model size increases, thus offering a scalable path to training massive neural networks. However, existing implem…
16kCPUGPUMixture-of-Experts+1FAMA: The First Large-Scale Open-Science Speech Foundation Model for English and Italian
The development of speech foundation models (SFMs) like Whisper and SeamlessM4T has significantly advanced the field of speech processing. However, their closed nature--with inaccessible training data and code--poses maj…
16kSpecExtend: A Drop-in Enhancement for Speculative Decoding of Long Sequences
Speculative decoding is a widely adopted technique for accelerating inference in large language models (LLMs), but its performance degrades on long inputs due to increased attention cost and reduced draft accuracy. We in…
16kLong-Context UnderstandingUI-Genie: A Self-Improving Approach for Iteratively Boosting MLLM-based Mobile GUI Agents
In this paper, we introduce UI-Genie, a self-improving framework addressing two key challenges in GUI agents: verification of trajectory outcome is challenging and high-quality training data are not scalable. These chall…
16kMonarchAttention: Zero-Shot Conversion to Fast, Hardware-Aware Structured Attention
Transformers have achieved state-of-the-art performance across various tasks, but suffer from a notable quadratic complexity in sequence length due to the attention mechanism. In this work, we propose MonarchAttention --…
16k4kTraining Long-Context LLMs Efficiently via Chunk-wise Optimization
While long-context large language models (LLMs) exhibit remarkable document processing capabilities, their prohibitively high training costs often hinder customized applications. To mitigate this issue, we propose \texti…
16kGPUPSC: Extending Context Window of Large Language Models via Phase Shift Calibration
Rotary Position Embedding (RoPE) is an efficient position encoding approach and is widely utilized in numerous large language models (LLMs). Recently, a lot of methods have been put forward to further expand the context …
16kPositionAchieving Scalable Robot Autonomy via neurosymbolic planning using lightweight local LLM
PDDL-based symbolic task planning remains pivotal for robot autonomy yet struggles with dynamic human-robot collaboration due to scalability, re-planning demands, and delayed plan availability. Although a few neurosymbol…
16k8kTask PlanningFalseReject: A Resource for Improving Contextual Safety and Mitigating Over-Refusals in LLMs via Structured Reasoning
Safety alignment approaches in large language models (LLMs) often lead to the over-refusal of benign queries, significantly diminishing their utility in sensitive scenarios. To address this challenge, we introduce FalseR…
16kBenchmarkingSafety AlignmentKL3M Tokenizers: A Family of Domain-Specific and Character-Level Tokenizers for Legal, Financial, and Preprocessing Applications
We present the KL3M tokenizers, a family of specialized tokenizers for legal, financial, and governmental text. Despite established work on tokenization, specialized tokenizers for professional domains remain understudie…
16k4k8kOptical Character Recognition (OCR)NSF-SciFy: Mining the NSF Awards Database for Scientific Claims
We present NSF-SciFy, a large-scale dataset for scientific claim extraction derived from the National Science Foundation (NSF) awards database, comprising over 400K grant abstracts spanning five decades. While previous d…
16kAbstract generationClaim VerificationX-LRM: X-ray Large Reconstruction Model for Extremely Sparse-View Computed Tomography Recovery in One Second
Sparse-view 3D CT reconstruction aims to recover volumetric structures from a limited number of 2D X-ray projections. Existing feedforward methods are constrained by the limited capacity of CNN-based architectures and th…
16kCT ReconstructionEvaluating the Suitability of Different Intraoral Scan Resolutions for Deep Learning-Based Tooth Segmentation
Intraoral scans are widely used in digital dentistry for tasks such as dental restoration, treatment planning, and orthodontic procedures. These scans contain detailed topological information, but manual annotation of th…
16k2k4k8k+2EpMAN: Episodic Memory AttentioN for Generalizing to Longer Contexts
Recent advances in Large Language Models (LLMs) have yielded impressive successes on many language tasks. However, efficient processing of long contexts using LLMs remains a significant challenge. We introduce \textbf{Ep…
16kDecoderQuestion AnsweringRetrieval-augmented GenerationCLOVER: A Test Case Generation Benchmark with Coverage, Long-Context, and Verification
Software testing is a critical aspect of software development, yet generating test cases remains a routine task for engineers. This paper presents a benchmark, CLOVER, to evaluate models' capabilities in generating and c…
16k4ksoftware testingFairness through Difference Awareness: Measuring Desired Group Discrimination in LLMs
Algorithmic fairness has conventionally adopted the mathematically convenient perspective of racial color-blindness (i.e., difference unaware treatment). However, we contend that in a range of important settings, group d…
16kDescriptiveFairnessM+: Extending MemoryLLM with Scalable Long-Term Memory
Equipping large language models (LLMs) with latent-space memory has attracted increasing attention as they can extend the context window of existing language models. However, retaining information from the distant past r…
16kGPULong-Context UnderstandingText Generation