paper-with-me

홈 › Papers

Hierarchical-embedding autoencoder with a predictor (HEAP) as efficient architecture for learning long-term evolution of complex multi-scale physical systems

2025-05-24 · Alexander Khrabry, Edward Startsev, Andrew Powis, Igor Kaganovich

We propose a novel efficient architecture for learning long-term evolution in complex multi-scale physical systems which is based on the idea of separation of scales. Structures of various scales that dynamically emerge in the system interact with each other only locally. Structures of similar scale can interact directly when they are in contact and indirectly when they are parts of larger structures that interact directly. This enables modeling a multi-scale system in an efficient way, where interactions between small-scale features that are apart from each other do not need to be modeled. The hierarchical fully-convolutional autoencoder transforms the state of a physical system not just into a single embedding layer, as it is done conventionally, but into a series of embedding layers which encode structures of various scales preserving spatial information at a corresponding resolution level. Shallower layers embed smaller structures on a finer grid, while deeper layers embed larger structures on a coarser grid. The predictor advances all embedding layers in sync. Interactions between features of various scales are modeled using a combination of convolutional operators. We compare the performance of our model to variations of a conventional ResNet architecture in application to the Hasegawa-Wakatani turbulence. A multifold improvement in long-term prediction accuracy was observed for crucial statistical characteristics of this system.

📄 PDF Abstract BibTeX arXiv:2505.18857

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Average Pooling 설명 없음
Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…
Global Average Pooling Global Average Pooling is a pooling operation designed to replace fully connected layers in classical CNNs. The idea is to generate one feature map for each corresponding…
Kaiming Initialization 설명 없음
Max Pooling Max Pooling is a pooling operation that calculates the maximum value for patches of a feature map, and uses it to create a downsampled (pooled) feature map. It is usually…

Similar Papers 제목 키워드 기반

Improving Image Autoencoder Embeddings with Perceptual Loss

2020-01-10 · Gustav Grund Pihlgren, Fredrik Sandin, Marcus Liwicki

Autoencoders are commonly trained using element-wise loss. However, element-wise loss disregards high-level structures in the image which can lead to embeddings that disregard them as well. A recent improvement to autoen…

Hierarchical JEPA Meets Predictive Remote Control in Beyond 5G Networks

2026-01-28 · Abanoub M. Girgis, Ibtissam Labriji, Mehdi Bennis arxiv

In wireless networked control systems, ensuring timely and reliable state updates from distributed devices to remote controllers is essential for robust control performance. However, when multiple devices transmit high-d…

Not All Operations Contribute Equally: Hierarchical Operation-Adaptive Predictor for Neural Architecture Search

2021-01-01 · ICCV 2021 10 · Ziye Chen, Yibing Zhan, Baosheng Yu, Mingming Gong 외

Graph-based predictors have recently shown promising results on neural architecture search (NAS). Despite their efficiency, current graph-based predictors treat all operations equally, resulting in biased topological…

AllNeural Architecture Search

LLM-guided Hierarchical Search for End-to-end Reasoning Intensive Retrieval

2025-10-15 · Nilesh Gupta, Wei-Cheng Chang, Ngot Bui, Cho-Jui Hsieh 외 arxiv

Search systems are increasingly used for reasoning-intensive queries, where what makes a document relevant requires understanding or reasoning over the query-document relation rather than relying on surface vocabulary or…

Dynamic Ensemble of Low-fidelity Experts: Mitigating NAS "Cold-Start"

2023-02-02 · Junbo Zhao, Xuefei Ning, Enshu Liu, Binxin Ru 외

Predictor-based Neural Architecture Search (NAS) employs an architecture performance predictor to improve the sample efficiency. However, predictor-based NAS suffers from the severe ``cold-start'' problem, since a large …

Neural Architecture Search