Bumblebee: Foundation Model for Particle Physics Discovery

2024-12-10 · Andrew J. Wildridge, Jack P. Rodgers, Ethan M. Colbert, Yao Yao, Andreas W. Jung, Miaoyuan Liu

Bumblebee is a foundation model for particle physics discovery, inspired by BERT. By removing positional encodings and embedding particle 4-vectors, Bumblebee captures both generator- and reconstruction-level information while ensuring sequence-order invariance. Pre-trained on a masked task, it improves dileptonic top quark reconstruction resolution by 10-20% and excels in downstream tasks, including toponium discrimination (AUROC 0.877) and initial state classification (AUROC 0.625). The flexibility of Bumblebee makes it suitable for a wide range of particle physics applications, especially the discovery of new particles.

📄 PDF Abstract BibTeX arXiv:2412.07867

Code (0)

등록된 구현이 없습니다.

Tasks

model

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Multi-Head Attention 설명 없음

Weight Decay 설명 없음

WordPiece 설명 없음

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

EveNet: A Foundation Model for Particle Collision Data Analysis

2026-01-23 · Ting-Hsiang Hsu, Bai-Hong Zhou, Qibin Liu, Yue Xu 외 arxiv

While deep learning is transforming data analysis in high-energy physics, computational challenges limit its potential. We address these challenges in the context of collider physics by introducing EveNet, an event-level…

Self-Supervised Learning

Cross-Domain Transfer with Particle Physics Foundation Models: From Jets to Neutrino Interactions

2026-04-14 · Gregor Krzmanc, Vinicius Mikuni, Benjamin Nachman, Callum Wilkinson arxiv

Future AI-based studies in particle physics will likely start from a foundation model to accelerate training and enhance sensitivity. As a step towards a general-purpose foundation model for particle physics, we investig…

Binary Classification

Simulate Bumblebee and Extend It to Support LE Coded PHY in BLE version 5

2023-05-17 · Hao Zhao

Bumblebee is an innovative backscatter system that leverages productive Bluetooth Low Energy (BLE) carriers to generate ZigBee transmissions. It improves upon previous systems like FreeRider and Interscatter by eliminati…

The evolution of AI from image interpretation toward scientific inference in nanoparticle electron microscopy

2026-07-11 · Evropi Toulkeridou, Jiafei Li, Leonardo Lari, Panagiotis Grammatikopoulos arxiv

Artificial intelligence (AI) is transforming electron microscopy by enabling quantitative analysis of increasingly large and complex datasets for nanoparticle characterization. Recent advances in machine learning (ML) an…

Self-Supervised Learning

OmniJet-$α$: The first cross-task foundation model for particle physics

2024-03-08 · Joschka Birk, Anna Hallin, Gregor Kasieczka

Foundation models are multi-dataset and multi-task machine learning methods that once pre-trained can be fine-tuned for a large variety of downstream applications. The successful development of such general-purpose model…

Jet Tagging Transfer Learning