Positional Encoding in the Context of Memristor-Based Analog Computation for Automatic Speech Recognition
Memristors provide a new chance for resource-efficient computation of neural models for natural language processing by enabling analog execution of vector-matrix-multiplication. Yet, computations on these devices are currently subject to larger distortion, both in weight programming and execution. In this work, we identify large output values of transformed positional encodings to cause major degradation within analog-to-digital conversion (ADC) as part of memristor-based computation. By adjusting the proportion of weight and precision bits of the ADC of specific memristor layers, we reduce the degradation of the execution by ~50% relative, while keeping the estimated energy consumption stable. Additionally, we investigate scenarios where the ADC cannot be modified. In that case the degradation can be reduced by ~30% relative after removing encoding-related linear transformations.
Code (0)
등록된 구현이 없습니다.
Tasks
Speech RecognitionSimilar Papers 제목 키워드 기반
Neuron inspired data encoding memristive multi-level memory cell
Mapping neuro-inspired algorithms to sensor backplanes of on-chip hardware require shifting the signal processing from digital to the analog domain, demanding memory technologies beyond conventional CMOS binary storage u…
Analog Neural Computing with Super-resolution Memristor Crossbars
Memristor crossbar arrays are used in a wide range of in-memory and neuromorphic computing applications. However, memristor devices suffer from non-idealities that result in the variability of conductive states, making p…
Super-ResolutionMemristor models for machine learning
In the quest for alternatives to traditional CMOS, it is being suggested that digital computing efficiency and power can be improved by matching the precision to the application. Many applications do not need the high pr…
BIG-bench Machine LearningAnalog simulator of integro-differential equations with classical memristors
An analog computer makes use of continuously changeable quantities of a system, such as its electrical, mechanical, or hydraulic properties, to solve a given problem. While these devices are usually computationally more …
Brain-Inspired Reservoir Computing Using Memristors with Tunable Dynamics and Short-Term Plasticity
Recent advancements in reservoir computing research have created a demand for analog devices with dynamics that can facilitate the physical implementation of reservoirs, promising faster information processing while cons…