paper-with-me

Papers

Evaluating Human Trajectory Prediction with Metamorphic Testing

2024-07-26 · Helge Spieker, Nassim Belmecheri, Arnaud Gotlieb, Nadjib Lazaar

The prediction of human trajectories is important for planning in autonomous systems that act in the real world, e.g. automated driving or mobile robots. Human trajectory prediction is a noisy process, and no prediction does precisely match any future trajectory. It is therefore approached as a stochastic problem, where the goal is to minimise the error between the true and the predicted trajectory. In this work, we explore the application of metamorphic testing for human trajectory prediction. Metamorphic testing is designed to handle unclear or missing test oracles. It is well-designed for human trajectory prediction, where there is no clear criterion of correct or incorrect human behaviour. Metamorphic relations rely on transformations over source test cases and exploit invariants. A setting well-designed for human trajectory prediction where there are many symmetries of expected human behaviour under variations of the input, e.g. mirroring and rescaling of the input data. We discuss how metamorphic testing can be applied to stochastic human trajectory prediction and introduce the Wasserstein Violation Criterion to statistically assess whether a follow-up test case violates a label-preserving metamorphic relation.

📄 PDF Abstract BibTeX arXiv:2407.18756

Code (0)

등록된 구현이 없습니다.

Tasks

PredictionTrajectory Prediction

Similar Papers 제목 키워드 기반

Metamorphic Testing of Multimodal Human Trajectory Prediction

2025-09-01 · Helge Spieker, Nadjib Lazaar, Arnaud Gotlieb, Nassim Belmecheri arxiv

Context: Predicting human trajectories is crucial for the safety and reliability of autonomous systems, such as automated vehicles and mobile robots. However, rigorously testing the underlying multimodal Human Trajectory…

Semantic SegmentationTrajectory Prediction

MT-Teql: Evaluating and Augmenting Consistency of Text-to-SQL Models with Metamorphic Testing

2020-12-21 · Pingchuan Ma, Shuai Wang

Text-to-SQL is a task to generate SQL queries from human utterances. However, due to the variation of natural language, two semantically equivalent utterances may appear differently in the lexical level. Likewise, user p…

Text to SQLText-To-SQL

LLMORPH: Automated Metamorphic Testing of Large Language Models

2026-03-24 · Steven Cho, Stefano Ruberto, Valerio Terragni arxiv

Automated testing is essential for evaluating and improving the reliability of Large Language Models (LLMs), yet the lack of automated oracles for verifying output correctness remains a key challenge. We present LLMORPH,…

MORTAR: Multi-turn Metamorphic Testing for LLM-based Dialogue Systems

2024-12-20 · Guoxiang Guo, Aldeida Aleti, Neelofar Neelofar, Chakkrit Tantithamthavorn 외

With the widespread application of LLM-based dialogue systems in daily life, quality assurance has become more important than ever. Recent research has successfully introduced methods to identify unexpected behaviour in …

Object-based Metamorphic Testing through Image Structuring

2020-02-12 · Adrian Wildandyawan, Yasuharu Nishi

Testing software is often costly due to the need of mass-producing test cases and providing a test oracle for it. This is often referred to as the oracle problem. One method that has been proposed in order to alleviate t…

Object