Context-Driven Performance Modeling for Causal Inference Operators on Neural Processing Units
The proliferation of large language models has driven demand for long-context inference on resource-constrained edge platforms. However, deploying these models on Neural Processing Units (NPUs) presents significant challenges due to architectural mismatch: the quadratic complexity of standard attention conflicts with NPU memory and compute patterns. This paper presents a comprehensive performance analysis of causal inference operators on a modern NPU, benchmarking quadratic attention against sub-quadratic alternatives including structured state-space models and causal convolutions. Our analysis reveals a spectrum of critical bottlenecks: quadratic attention becomes severely memory-bound with catastrophic cache inefficiency, while sub-quadratic variants span from compute-bound on programmable vector cores to memory-bound by data movement. These findings provide essential insights for co-designing hardware-aware models and optimization strategies to enable efficient long-context inference on edge platforms.
Code (0)
등록된 구현이 없습니다.
Tasks
Causal InferenceSimilar Papers 제목 키워드 기반
Causal Discovery from Conditionally Stationary Time Series
Causal discovery, i.e., inferring underlying causal relationships from observational data, has been shown to be highly challenging for AI systems. In time series modeling context, traditional causal discovery methods mai…
Causal DiscoveryCausal InferenceProbabilistic Deep LearningTime Series+1On Variational Inference for User Modeling in Attribute-Driven Collaborative Filtering
Recommender Systems have become an integral part of online e-Commerce platforms, driving customer engagement and revenue. Most popular recommender systems attempt to learn from users' past engagement data to understand b…
AttributeBIG-bench Machine LearningCausal InferenceCollaborative Filtering+2From Correlation to Causation: Understanding Climate Change through Causal Analysis and LLM Interpretations
This research presents a three-step causal inference framework that integrates correlation analysis, machine learning-based causality discovery, and LLM-driven interpretations to identify socioeconomic factors influencin…
Causal InferenceDecision MakingUsing causal inference to avoid fallouts in data-driven parametric analysis: a case study in the architecture, engineering, and construction industry
The decision-making process in real-world implementations has been affected by a growing reliance on data-driven models. We investigated the synergetic pattern between the data-driven methods, empirical domain knowledge,…
Causal InferenceDecision Makingfeature selectionSTOAT: Spatial-Temporal Probabilistic Causal Inference Network
Spatial-temporal causal time series (STC-TS) involve region-specific temporal observations driven by causally relevant covariates and interconnected across geographic or network-based spaces. Existing methods often model…
Causal Inference