paper-with-me

BIG-bench

Beyond the Imitation Game Benchmark

홈페이지 · 논문 349편

The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to probe large language models and extrapolate their future capabilities. Big-bench include more than 200 tasks. Image source: https://arxiv.org/pdf/2206.04615.pdf

Texts English

벤치마크

Multi-task Language Understanding on BBH-nlp 결과 30개
Multi-task Language Understanding on BBH-alg 결과 14개
Common Sense Reasoning on BIG-bench (Causal Judgment) 결과 9개
Common Sense Reasoning on BIG-bench (Date Understanding) 결과 9개
Common Sense Reasoning on BIG-bench (Disambiguation QA) 결과 9개
Logical Reasoning on BIG-bench (Formal Fallacies Syllogisms Negation) 결과 9개
Logical Reasoning on BIG-bench (Penguins In A Table) 결과 9개
Logical Reasoning on BIG-bench (Reasoning About Colored Objects) 결과 9개
Logical Reasoning on BIG-bench (Temporal Sequences) 결과 9개
Multiple Choice Question Answering (MCQA) on BIG-bench (Hyperbaton) 결과 9개
Multiple Choice Question Answering (MCQA) on BIG-bench (Movie Recommendation) 결과 9개
Multiple Choice Question Answering (MCQA) on BIG-bench (Navigate) 결과 9개
Multiple Choice Question Answering (MCQA) on BIG-bench (Ruin Names) 결과 9개
Common Sense Reasoning on BIG-bench (Sports Understanding) 결과 8개
Sarcasm Detection on BIG-bench (SNARKS) 결과 8개
Word Sense Disambiguation on BIG-bench (Anachronisms) 결과 6개
Common Sense Reasoning on BIG-bench (Winowhy) 결과 4개
Crass AI on BIG-bench 결과 4개
Intent Recognition on BIG-bench 결과 4개
Logical Reasoning on BIG-bench (Logic Grid Puzzle) 결과 4개
Logical Reasoning on BIG-bench (StrategyQA) 결과 4개
Multiple Choice Question Answering (MCQA) on BIG-bench (Novel Concepts) 결과 4개
Auto Debugging on Big-bench Lite 결과 3개
Common Sense Reasoning on BIG-bench (Known Unknowns) 결과 3개
Language Modelling on BIG-bench-lite 결과 3개
Memorization on BIG-bench (Hindu Knowledge) 결과 3개
Analogical Similarity on BIG-bench 결과 2개
Analytic Entailment on BIG-bench 결과 2개
Common Sense Reasoning on BIG-bench (Logical Sequence) 결과 2개
Crash Blossom on BIG-bench 결과 2개
Dark Humor Detection on BIG-bench 결과 2개
Discourse Marker Prediction on BIG-bench 결과 2개
Empirical Judgments on BIG-bench 결과 2개
English Proverbs on BIG-bench 결과 2개
Entailed Polarity on BIG-bench 결과 2개
Epistemic Reasoning on BIG-bench 결과 2개
Evaluating Information Essentiality on BIG-bench 결과 2개
Fantasy Reasoning on BIG-bench 결과 2개
Figure Of Speech Detection on BIG-bench 결과 2개
GRE Reading Comprehension on BIG-bench 결과 2개
General Knowledge on BIG-bench 결과 2개
Human Organs Senses Multiple Choice on BIG-bench 결과 2개
Identify Odd Metapor on BIG-bench 결과 2개
Implicatures on BIG-bench 결과 2개
Implicit Relations on BIG-bench 결과 2개
Irony Identification on BIG-bench 결과 2개
LAMBADA on BIG-bench 결과 2개
Logical Args on BIG-bench 결과 2개
Logical Reasoning on BIG-bench (Logical Fallacy Detection) 결과 2개
Mathematical Induction on BIG-bench 결과 2개
Metaphor Boolean on BIG-bench 결과 2개
Misconceptions on BIG-bench 결과 2개
Moral Permissibility on BIG-bench 결과 2개
Movie Dialog Same Or Different on BIG-bench 결과 2개
Nonsense Words Grammar on BIG-bench 결과 2개
Odd One Out on BIG-bench 결과 2개
Phrase Relatedness on BIG-bench 결과 2개
Physical Intuition on BIG-bench 결과 2개
Presuppositions As NLI on BIG-bench 결과 2개
Question Selection on BIG-bench 결과 2개
Riddle Sense on BIG-bench 결과 2개
Sentence Ambiguity on BIG-bench 결과 2개
Similarities Abstraction on BIG-bench 결과 2개
Timedial on BIG-bench 결과 2개
Understanding Fables on BIG-bench 결과 2개
on BIG-bench (Hyperbaton) 결과 1개
on BIG-bench (Navigate) 결과 1개
on BIG-bench (Ruin Names) 결과 1개
on BIG-bench (SNARKS) 결과 1개
on Big-bench Hard 결과 1개
Abstract Algebra on BIG-bench 결과 1개
Anatomy on BIG-bench 결과 1개
Astronomy on BIG-bench 결과 1개
BIG-bench Machine Learning on BIG-bench 결과 1개
Business Ethics on BIG-bench 결과 1개
Clinical Knowledge on BIG-bench 결과 1개
College Mathematics on BIG-bench 결과 1개
College Medicine on BIG-bench 결과 1개
Computer Security on BIG-bench 결과 1개
Econometrics on BIG-bench 결과 1개
Elementary Mathematics on BIG-bench 결과 1개
FEVER (2-way) on BIG-bench 결과 1개
FEVER (3-way) on BIG-bench 결과 1개
Formal Logic on BIG-bench 결과 1개
Global Facts on BIG-bench 결과 1개
High School European History on BIG-bench 결과 1개
High School Geography on BIG-bench 결과 1개
High School Government and Politics on BIG-bench 결과 1개
High School Macroeconomics on BIG-bench 결과 1개
High School Mathematics on BIG-bench 결과 1개
High School Microeconomics on BIG-bench 결과 1개
High School Psychology on BIG-bench 결과 1개
High School US History on BIG-bench 결과 1개
High School World History on BIG-bench 결과 1개
Human Aging on BIG-bench 결과 1개
Human Sexuality on BIG-bench 결과 1개
International Law on BIG-bench 결과 1개
Jurisprudence on BIG-bench 결과 1개
Logical Fallacies on BIG-bench 결과 1개
Management on BIG-bench 결과 1개
Marketing on BIG-bench 결과 1개
Medical Genetics on BIG-bench 결과 1개
Miscellaneous on BIG-bench 결과 1개
Moral Disputes on BIG-bench 결과 1개
Moral Scenarios on BIG-bench 결과 1개
Natural Questions on BIG-bench 결과 1개
Nutrition on BIG-bench 결과 1개
Philosophy on BIG-bench 결과 1개
Prehistory on BIG-bench 결과 1개
Professional Accounting on BIG-bench 결과 1개
Professional Law on BIG-bench 결과 1개
Professional Medicine on BIG-bench 결과 1개
Professional Psychology on BIG-bench 결과 1개
Public Relations on BIG-bench 결과 1개
RACE-h on BIG-bench 결과 1개
RACE-m on BIG-bench 결과 1개
Security Studies on BIG-bench 결과 1개
Sociology on BIG-bench 결과 1개
TriviaQA on BIG-bench 결과 1개
US Foreign Policy on BIG-bench 결과 1개
Virology on BIG-bench 결과 1개
World Religions on BIG-bench 결과 1개