Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation
Convolutional Neural Networks (CNNs) can be shifted across 2D images or 3D videos to segment them. They have a fixed input size and typically perceive only small local contexts of the pixels to be classified as foreground or background. In contrast, Multi-Dimensional Recurrent NNs (MD-RNNs) can perceive the entire spatio-temporal context of each pixel in a few sweeps through all pixels, especially when the RNN is a Long Short-Term Memory (LSTM). Despite these theoretical advantages, however, unlike CNNs, previous MD-LSTM variants were hard to parallelize on GPUs. Here we re-arrange the traditional cuboid order of computations in MD-LSTM in pyramidal fashion. The resulting PyraMiD-LSTM is easy to parallelize, especially for 3D data such as stacks of brain slice images. PyraMiD-LSTM achieved best known pixel-wise brain image segmentation results on MRBrainS13 (and competitive results on EM-ISBI12).
Code (0)
등록된 구현이 없습니다.
Tasks
Brain Image SegmentationImage SegmentationSemantic SegmentationSimilar Papers 제목 키워드 기반
Multi-Head Highly Parallelized LSTM Decoder for Neural Machine Translation
One of the reasons Transformer translation models are popular is that self-attention networks for context modelling can be easily parallelized at sequence level. However, the computational complexity of a self-attention …
DecoderMachine TranslationTranslationParallelizable Stack Long Short-Term Memory
Stack Long Short-Term Memory (StackLSTM) is useful for various applications such as parsing and string-to-tree neural machine translation, but it is also known to be notoriously difficult to parallelize for GPU training …
GPUMachine TranslationTranslationTraining RNNs as Fast as CNNs
Common recurrent neural network architectures scale poorly due to the intrinsic difficulty in parallelizing their state computations. In this work, we propose the Simple Recurrent Unit (SRU) architecture, a recurrent uni…
General ClassificationLanguage ModelingLanguage ModellingQuestion Answering+3FastTrees: Parallel Latent Tree-Induction for Faster Sequence Encoding
Inducing latent tree structures from sequential data is an emerging trend in the NLP research landscape today, largely popularized by recent methods such as Gumbel LSTM and Ordered Neurons (ON-LSTM). This paper proposes …
Language ModelingLanguage ModellingMachine TranslationNatural Language Inference+1Fast and Accurate Entity Recognition with Iterated Dilated Convolutions
Today when many practitioners run basic NLP on the entire web and large-volume traffic, faster methods are paramount to saving time and energy costs. Recent advances in GPU hardware have led to the emergence of bi-direct…
Computational EfficiencyGPUNamed Entity Recognition (NER)NER+1