Rapid Adaptation with Conditionally Shifted Neurons

2017-12-28 · ICML 2018 7 · Tsendsuren Munkhdalai, Xingdi Yuan, Soroush Mehri, Adam Trischler

We describe a mechanism by which artificial neural networks can learn rapid adaptation - the ability to adapt on the fly, with little data, to new tasks - that we call conditionally shifted neurons. We apply this mechanism in the framework of metalearning, where the aim is to replicate some of the flexibility of human learning in machines. Conditionally shifted neurons modify their activation values with task-specific shifts retrieved from a memory module, which is populated rapidly based on limited task experience. On metalearning benchmarks from the vision and language domains, models augmented with conditionally shifted neurons achieve state-of-the-art results.

📄 PDF Abstract BibTeX arXiv:1712.09926

Code (0)

등록된 구현이 없습니다.

Tasks

Few-Shot Image Classification

Methods 이 논문이 사용한 방법론

Average Pooling 설명 없음

Residual Connection 설명 없음

ReLU How Do I Communicate to Expedia? How Do I Communicate to Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Live Support & Special Travel…

1x1 Convolution A 1 x 1 Convolution is a convolution with some special properties in that it can be used for dimensionality reduction,…

Batch Normalization 설명 없음

Bottleneck Residual Block A Bottleneck Residual Block is a variant of the residual block that utilises 1x1 convolutions to create a bottleneck. The…

Global Average Pooling Global Average Pooling is a pooling operation designed to replace fully connected layers in classical CNNs. The idea is to generate one feature map for each corresponding…

Residual Block Residual Blocks are skip-connection blocks that learn residual functions with reference to the layer inputs, instead of learning unreferenced functions. They were introduced…

Interneurons accelerate learning dynamics in recurrent neural networks for statistical adaptation

2022-09-21 · David Lipshutz, Cengiz Pehlevan, Dmitri B. Chklovskii

Early sensory systems in the brain rapidly adapt to fluctuating input statistics, which requires recurrent communication between neurons. Mechanistically, such recurrent communication is often indirect and mediated by lo…

Modeling Neural Activity with Conditionally Linear Dynamical Systems

2025-02-25 · Victor Geadah, Amin Nejatbakhsh, David Lipshutz, Jonathan W. Pillow 외

Neural population activity exhibits complex, nonlinear dynamics, varying in time, over trials, and across experimental conditions. Here, we develop Conditionally Linear Dynamical System (CLDS) models as a general-purpose…

Bayesian Inference

Graph Embedding with Shifted Inner Product Similarity and Its Improved Approximation Capability

2018-10-04 · Akifumi Okuno, Geewook Kim, Hidetoshi Shimodaira

We propose shifted inner-product similarity (SIPS), which is a novel yet very simple extension of the ordinary inner-product similarity (IPS) for neural-network based graph embedding (GE). In contrast to IPS, that is lim…

Graph Embedding

Efficient Spike-Coding with Multiplicative Adaptation in a Spike Response Model

2012-12-01 · NeurIPS 2012 12 · Sander M. Bohte

Neural adaptation underlies the ability of neurons to maximize encoded information over a wide dynamic range of input stimuli. While adaptation is an intrinsic feature of neuronal models like the Hodgkin-Huxley model, th…

The Role of Excitatory Parvalbumin-positive Neurons in the Tectofugal Pathway of Pigeon (Columba livia) Hierarchical Visual Processing

2025-07-21 · Shan Lu, Xiaoteng Zhang, Yueyang Cang, Shihao Pan 외 arxiv

The visual systems of birds and mammals exhibit remarkable organizational similarities: the dorsal ventricular ridge (DVR) demonstrates a columnar microcircuitry that parallels the cortical architecture observed in mamma…