One Model is Not Enough: Ensembles for Isolated Sign Language Recognition

2022-07-04 · Sensors 2022 7 · Marek Hrúz, Ivan Gruber, Jakub Kanis, Matyáš Boháček, Miroslav Hlaváč, Zdeněk Krňoul

In this paper, we dive into sign language recognition, focusing on the recognition of isolated signs. The task is defined as a classification problem, where a sequence of frames (i.e., images) is recognized as one of the given sign language glosses. We analyze two appearance-based approaches, I3D and TimeSformer, and one pose-based approach, SPOTER. The appearance-based approaches are trained on a few different data modalities, whereas the performance of SPOTER is evaluated on different types of preprocessing. All the methods are tested on two publicly available datasets: AUTSL and WLASL300. We experiment with ensemble techniques to achieve new state-of-the-art results of 73.84% accuracy on the WLASL300 dataset by using the CMA-ES optimization method to find the best ensemble weight parameters. Furthermore, we present an ensembling technique based on the Transformer model, which we call Neural Ensembler.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Sign Language Recognition

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Position-Wise Feed-Forward Layer 설명 없음

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Residual Connection 설명 없음

MAEBE: Multi-Agent Emergent Behavior Framework

2025-06-03 · Sinem Erisken, Timothy Gothard, Martin Leitgab, Ram Potham

Traditional AI safety evaluations on isolated LLMs are insufficient as multi-agent AI ensembles become prevalent, introducing novel emergent risks. This paper introduces the Multi-Agent Emergent Behavior Evaluation (MAEB…

Ensembles of Localised Models for Time Series Forecasting

2020-12-30 · Rakshitha Godahewa, Kasun Bandara, Geoffrey I. Webb, Slawek Smyl 외

With large quantities of data typically available nowadays, forecasting models that are trained across sets of time series, known as Global Forecasting Models (GFM), are regularly outperforming traditional univariate for…

Clustering Time Series Time Series Analysis Time Series Forecasting

Word separation in continuous sign language using isolated signs and post-processing

2022-04-02 · Razieh Rastgoo, Kourosh Kiani, Sergio Escalera

. Continuous Sign Language Recognition (CSLR) is a long challenging task in Computer Vision due to the difficulties in detecting the explicit boundaries between the words in a sign sentence. To deal with this challenge, …

Sentence Sign Language Recognition

Exploiting Ensemble Learning for Cross-View Isolated Sign Language Recognition

2025-02-04 · Fei Wang, Kun Li, Yiqi Nie, Zhangling Duan 외

In this paper, we present our solution to the Cross-View Isolated Sign Language Recognition (CV-ISLR) challenge held at WWW 2025. CV-ISLR addresses a critical issue in traditional Isolated Sign Language Recognition (ISLR…

Ensemble Learning Sign Language Recognition

A Transformer Model for Boundary Detection in Continuous Sign Language

2024-02-22 · Razieh Rastgoo, Kourosh Kiani, Sergio Escalera

Sign Language Recognition (SLR) has garnered significant attention from researchers in recent years, particularly the intricate domain of Continuous Sign Language Recognition (CSLR), which presents heightened complexity …

Boundary Detection Sign Language Recognition