GeneFormer: Learned Gene Compression using Transformer-based Context Modeling
With the development of gene sequencing technology, an explosive growth of gene data has been witnessed. And the storage of gene data has become an important issue. Traditional gene data compression methods rely on general software like G-zip, which fails to utilize the interrelation of nucleotide sequence. Recently, many researchers begin to investigate deep learning based gene data compression method. In this paper, we propose a transformer-based gene compression method named GeneFormer. Specifically, we first introduce a modified transformer structure to fully explore the nucleotide sequence dependency. Then, we propose fixed-length parallel grouping to accelerate the decoding speed of our autoregressive model. Experimental results on real-world datasets show that our method saves 29.7% bit rate compared with the state-of-the-art method, and the decoding speed is significantly faster than all existing learning-based gene compression methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Data CompressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Variable Rate Image Compression via N-Gram Context based Swin-transformer
This paper presents an N-gram context-based Swin Transformer for learned image compression. Our method achieves variable-rate compression with a single model. By incorporating N-gram context into the Swin Transformer, we…
Image ReconstructionImage CompressionEfficient Contextformer: Spatio-Channel Window Attention for Fast Context Modeling in Learned Image Compression
Entropy estimation is essential for the performance of learned image compression. It has been demonstrated that a transformer-based entropy model is of critical importance for achieving a high compression ratio, however,…
Image CompressionContextformer: A Transformer with Spatio-Channel Attention for Context Modeling in Learned Image Compression
Entropy modeling is a key component for high-performance image compression algorithms. Recent developments in autoregressive context modeling helped learning-based methods to surpass their classical counterparts. However…
Image CompressionMS-SSIMSSIMFoundation Models Meet Imbalanced Single-Cell Data When Learning Cell Type Annotations
With the emergence of single-cell foundation models, an important question arises: how do these models perform when trained on datasets having an imbalance in cell type distribution due to rare cell types or biased sampl…
Cell Entity Annotationimbalanced classificationLarge Language ModelSingle-cell modelingSparse autoencoders reveal organized biological knowledge but minimal regulatory logic in single-cell foundation models: a comparative atlas of Geneformer and scGPT
Background: Single-cell foundation models such as Geneformer and scGPT encode rich biological information, but whether this includes causal regulatory logic rather than statistical co-expression remains unclear. Sparse a…