Position: Ideas Should be the Center of Machine Learning Research
Machine learning research increasingly bifurcates into two disconnected modes: benchmark-driven engineering that prioritizes metrics over understanding, and idealized theory that often fails to transfer to modern systems. In this position paper, we argue that the field focuses too heavily on these endpoints, neglecting the central scientific object: the idea. We propose an Ideas First framework in which ideas are valued for the behavioral signatures they predict in modern models, and these signatures are tested through tailored experiments designed to detect the relevant patterns rather than to win leaderboards. This shift not only bridges the gap between theory and practice but also promotes equity by removing the "complexity premium," enabling rigorous scientific contributions from researchers with modest computational, financial, and human resources. Ultimately, we advocate for a research culture centered on ideas, treating benchmarks and theorems as instruments for testing mechanistic hypotheses rather than as ends in themselves.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Human-centered NLP Fact-checking: Co-Designing with Fact-checkers using Matchmaking for AI
While many Natural Language Processing (NLP) techniques have been proposed for fact-checking, both academic research and fact-checking organizations report limited adoption of such NLP work due to poor alignment with fac…
Fact CheckingMisinformationMedical Large Language Model Benchmarks Should Prioritize Construct Validity
Medical large language models (LLMs) research often makes bold claims, from encoding clinical knowledge to reasoning like a physician. These claims are usually backed by evaluation on competitive benchmarks; a tradition …
Clinical KnowledgeLanguage ModelingLanguage ModellingLarge Language ModelPosition: Evaluation of Visual Processing Should Be Human-Centered, Not Metric-Centered
This position paper argues that the evaluation of modern visual processing systems should no longer be driven primarily by single-metric image quality assessment benchmarks, particularly in the era of generative and perc…
Image Quality AssessmentImage RestorationPosition: Machine Learning for Heart Transplant Allocation Policy Optimization Should Account for Incentives
The allocation of scarce donor organs constitutes one of the most consequential algorithmic challenges in healthcare. While the field is rapidly transitioning from rigid, rule-based systems to machine learning and data-d…
Causal InferenceMachine Learning: a Lecture Note
This lecture note is intended to prepare early-year master's and PhD students in data science or a related discipline with foundational ideas in machine learning. It starts with basic ideas in modern machine learning wit…
Meta-LearningModel Selection