Anchoring to Exemplars for Training Mixture-of-Expert Cell Embeddings

2021-12-06 · Siqi Wang, Manyuan Lu, Nikita Moshkov, Juan C. Caicedo, Bryan A. Plummer

Analyzing the morphology of cells in microscopy images can provide insights into the mechanism of compounds or the function of genes. Addressing this task requires methods that can not only extract biological information from the images, but also ignore technical variations, ie, changes in experimental procedure or differences between equipments used to collect microscopy images. We propose Treatment ExemplArs with Mixture-of-experts (TEAMs), an embedding learning approach that learns a set of experts that are specialized in capturing technical variations in our training set and then aggregates specialist's predictions at test time. Thus, TEAMs can learn powerful embeddings with less technical variation bias by minimizing the noise from every expert. To train our model, we leverage Treatment Exemplars that enable our approach to capture the distribution of the entire dataset in every minibatch while still fitting into GPU memory. We evaluate our approach on three datasets for tasks like drug discovery, boosting performance on identifying the true mechanism of action of cell treatments by 5.5-11% over the state-of-the-art.

📄 PDF Abstract BibTeX arXiv:2112.03208

Code (0)

등록된 구현이 없습니다.

Tasks

Drug DiscoveryGPUMixture-of-Experts

SkillMoV: Mixture-of-View Routing with Prototype-Conditioned Gating for Unified Multi-View Proficiency Estimation

2026-06-16 · Edoardo Bianchi, Antonio Liotta arxiv

Estimating human proficiency from video is a key challenge for automated skill assessment, with applications in sports coaching, music pedagogy, surgical training, and workplace learning. Existing approaches often focus …

HEBATRON: A Hebrew-Specialized Open-Weight Mixture-of-Experts Language Model

2026-05-11 · Noam Kayzer, Dan Revital, Ori Bar Joseph, Smadar Arvatz 외 arxiv

We present Hebatron, a Hebrew-specialized open-weight large language model built on the NVIDIA Nemotron-3 sparse Mixture-of-Experts architecture. Training employs a three-phase easy-to-hard curriculum with continuous ant…

GC-MoE: Genomics-Guided Cell-Type-Specific Mixture of Experts for Histology-Based Single-Cell Spatial Transcriptomics

2026-06-01 · Kaito Shiku, Ahtisham Fazeel Abbasi, Ryoma Bise, Yuichiro Iwashita 외 arxiv

Histology-based single-cell spatial transcriptomics (ST) estimation aims to predict gene expression for individual cells from histopathological images and cell locations, reducing the need for costly single-cell ST measu…

Split-on-Share: Mixture of Sparse Experts for Task-Agnostic Continual Learning

2026-01-24 · Fatema Siddika, Md Anwar Hossen, Tanwi Mallick, Ali Jannesari arxiv

Continual learning in Large Language Models (LLMs) is hindered by the plasticity-stability dilemma, where acquiring new capabilities often leads to catastrophic forgetting of previous knowledge. Existing methods typicall…

parameter-efficient fine-tuning Continual Learning

SDG-MoE: Signed Debate Graph Mixture-of-Experts

2026-05-08 · Stepan Kulibaba, Kirill Labzin, Artem Dzhalilov, Roman Pakhomov 외 arxiv

Sparse MoE models achieve a good balance between capacity and compute by routing each token to a small subset of experts. However, in most MoE architectures, once a token is routed, the selected experts process it indepe…