paper-with-me

Papers

Comparing Machine Learning Algorithms by Union-Free Generic Depth

2023-12-20 · Hannah Blocher, Georg Schollmeyer, Malte Nalenz, Christoph Jansen

We propose a framework for descriptively analyzing sets of partial orders based on the concept of depth functions. Despite intensive studies in linear and metric spaces, there is very little discussion on depth functions for non-standard data types such as partial orders. We introduce an adaptation of the well-known simplicial depth to the set of all partial orders, the union-free generic (ufg) depth. Moreover, we utilize our ufg depth for a comparison of machine learning algorithms based on multidimensional performance measures. Concretely, we provide two examples of classifier comparisons on samples of standard benchmark data sets. Our results demonstrate promisingly the wide variety of different analysis approaches based on ufg methods. Furthermore, the examples outline that our approach differs substantially from existing benchmarking approaches, and thus adds a new perspective to the vivid debate on classifier comparison.

📄 PDF Abstract BibTeX arXiv:2312.12839

Code (1)

hannahblo/comparing_algorithms_using_ufg_depth 공식 구현

Tasks

Benchmarking

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Similar Papers 제목 키워드 기반

Depth Functions for Partial Orders with a Descriptive Analysis of Machine Learning Algorithms

2023-04-19 · Hannah Blocher, Georg Schollmeyer, Christoph Jansen, Malte Nalenz

We propose a framework for descriptively analyzing sets of partial orders based on the concept of depth functions. Despite intensive studies of depth functions in linear and metric spaces, there is very little discussion…

BenchmarkingDescriptive

A note on the connectedness property of union-free generic sets of partial orders

2023-04-19 · Georg Schollmeyer, Hannah Blocher

This short note describes and proves a connectedness property which was introduced in Blocher et al. [2023] in the context of data depth functions for partial orders. The connectedness property gives a structural insight…

Partial Rankings of Optimizers

2024-02-26 · Julian Rodemann, Hannah Blocher

We introduce a framework for benchmarking optimizers according to multiple criteria over various test functions. Based on a recently introduced union-free generic depth function for partial orders/rankings, it fully expl…

Benchmarking

The no-free-lunch theorems of supervised learning

2022-02-09 · Tom F. Sterkenburg, Peter D. Grünwald

The no-free-lunch theorems promote a skeptical conclusion that all possible machine learning algorithms equally lack justification. But how could this leave room for a learning theory, that shows that some algorithms are…

Inductive BiasLearning TheoryPhilosophy

Incremental Domain Adaptation for Neural Machine Translation in Low-Resource Settings

2019-08-01 · WS 2019 8 · Marimuthu Kalimuthu, Michael Barz, Daniel Sonntag

We study the problem of incremental domain adaptation of a generic neural machine translation model with limited resources (e.g., budget and time) for human translations or model training. In this paper, we propose a nov…

Active LearningDomain AdaptationInformativenessMachine Translation+3