Quantifying the Complexity of Standard Benchmarking Datasets for Long-Term Human Trajectory Prediction
Methods to quantify the complexity of trajectory datasets are still a missing piece in benchmarking human trajectory prediction models. In order to gain a better understanding of the complexity of trajectory prediction tasks and following the intuition, that more complex datasets contain more information, an approach for quantifying the amount of information contained in a dataset from a prototype-based dataset representation is proposed. The dataset representation is obtained by first employing a non-trivial spatial sequence alignment, which enables a subsequent learning vector quantization (LVQ) stage. A large-scale complexity analysis is conducted on several human trajectory prediction benchmarking datasets, followed by a brief discussion on indications for human trajectory prediction and benchmarking.
Code (0)
등록된 구현이 없습니다.
Tasks
BenchmarkingPredictionQuantizationTrajectory PredictionSimilar Papers 제목 키워드 기반
How Hungry is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Inference
This paper introduces a novel infrastructure-aware benchmarking framework for quantifying the environmental footprint of LLM inference across 30 state-of-the-art models as deployed in commercial data centers. Our framewo…
BenchmarkingWelQrate: Defining the Gold Standard in Small Molecule Drug Discovery Benchmarking
While deep learning has revolutionized computer-aided drug discovery, the AI community has predominantly focused on model innovation and placed less emphasis on establishing best benchmarking practices. We posit that wit…
BenchmarkingDrug DiscoveryAssumed Identities: Quantifying Gender Bias in Machine Translation of Gender-Ambiguous Occupational Terms
Machine Translation (MT) systems frequently encounter gender-ambiguous occupational terms, where they must assign gender without explicit contextual cues. While individual translations in such cases may not be inherently…
BenchmarkingMachine TranslationTranslationClimateCause: Complex and Implicit Causal Structures in Climate Reports
Understanding climate change requires reasoning over complex causal networks. Yet, existing causal discovery datasets predominantly capture explicit, direct causal relations. We introduce ClimateCause, a manually expert-…
Efficiently Quantifying Individual Agent Importance in Cooperative MARL
Measuring the contribution of individual agents is challenging in cooperative multi-agent reinforcement learning (MARL). In cooperative MARL, team performance is typically inferred from a single shared global reward. Arg…
BenchmarkingMulti-agent Reinforcement Learning