HyperSeg: Patch-wise Hypernetwork for Real-time Semantic Segmentation
We present a novel, real-time, semantic segmentation network in which the encoder both encodes and generates the parameters (weights) of the decoder. Furthermore, to allow maximal adaptivity, the weights at each decoder block vary spatially. For this purpose, we design a new type of hypernetwork, composed of a nested U-Net for drawing higher level context features, a multi-headed weight generating module which generates the weights of each block in the decoder immediately before they are consumed, for efficient memory utilization, and a primary network that is composed of novel dynamic patch-wise convolutions. Despite the usage of less-conventional blocks, our architecture obtains real-time performance. In terms of the runtime vs. accuracy trade-off, we surpass state of the art (SotA) results on popular semantic segmentation benchmarks: PASCAL VOC 2012 (val. set) and real-time semantic segmentation on Cityscapes, and CamVid. The code is available: https://nirkin.com/hyperseg.
Code (1)
Tasks
DecoderDichotomous Image SegmentationReal-Time Semantic SegmentationSegmentationSemantic SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Unsupervised Dialogue Topic Segmentation in Hyperdimensional Space
We present HyperSeg, a hyperdimensional computing (HDC) approach to unsupervised dialogue topic segmentation. HDC is a class of vector symbolic architectures that leverages the probabilistic orthogonality of randomly dra…
SegmentationHypernetwork Dismantling via Deep Reinforcement Learning
Network dismantling aims to degrade the connectivity of a network by removing an optimal set of nodes. It has been widely adopted in many real-world applications such as epidemic control and rumor containment. However, c…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)HyperSeg: Hybrid Segmentation Assistant with Fine-grained Visual Perceiver
This paper aims to address universal segmentation for image and video perception with the strong reasoning ability empowered by Visual Large Language Models (VLLMs). Despite significant progress in current unified se…
Reasoning SegmentationSegmentationUniversal SegmentationVideo Segmentation+2HN-MVTS: HyperNetwork-based Multivariate Time Series Forecasting
Accurate forecasting of multivariate time series data remains a formidable challenge, particularly due to the growing complexity of temporal dependencies in real-world scenarios. While neural network-based models have ac…
Multivariate Time Series ForecastingHyperSeg: Towards Universal Visual Segmentation with Large Language Model
This paper aims to address universal segmentation for image and video perception with the strong reasoning ability empowered by Visual Large Language Models (VLLMs). Despite significant progress in current unified segmen…
Language ModelingLarge Language ModelOpen Vocabulary Semantic SegmentationPanoptic Segmentation+9