paper-with-me

Papers

Low latency optical-based mode tracking with machine learning deployed on FPGAs on a tokamak

2023-11-30 · Yumou Wei, Ryan F. Forelli, Chris Hansen, Jeffrey P. Levesque, Nhan Tran, Joshua C. Agar, Giuseppe Di Guglielmo, Michael E. Mauel, Gerald A. Navratil

Active feedback control in magnetic confinement fusion devices is desirable to mitigate plasma instabilities and enable robust operation. Optical high-speed cameras provide a powerful, non-invasive diagnostic and can be suitable for these applications. In this study, we process fast camera data, at rates exceeding 100kfps, on $\textit{in situ}$ Field Programmable Gate Array (FPGA) hardware to track magnetohydrodynamic (MHD) mode evolution and generate control signals in real-time. Our system utilizes a convolutional neural network (CNN) model which predicts the $n$=1 MHD mode amplitude and phase using camera images with better accuracy than other tested non-deep-learning-based methods. By implementing this model directly within the standard FPGA readout hardware of the high-speed camera diagnostic, our mode tracking system achieves a total trigger-to-output latency of 17.6$\mu$s and a throughput of up to 120kfps. This study at the High Beta Tokamak-Extended Pulse (HBT-EP) experiment demonstrates an FPGA-based high-speed camera data acquisition and processing system, enabling application in real-time machine-learning-based tokamak diagnostic and control as well as potential applications in other scientific domains.

📄 PDF Abstract BibTeX arXiv:2312.00128

Code (2)

m3-learning/hls4ml-frame-grabbers 공식 구현
fastmachinelearning/hls4ml-frame-grabbers

Tasks

Diagnostic

Similar Papers 제목 키워드 기반

Low Latency Gaze Tracking via Latent Optical Sensing

2026-05-18 · Yidan Zheng, Matheus Souza, Kaizhang Kang, Qiang Fu 외 arxiv

We present a real-time gaze tracking system that directly acquires task-relevant latent features using a fully passive optical encoder. Instead of forming and processing full-resolution images, our approach leverages a m…

Gaze Estimation

A Survey on Event-based Optical Marker Systems

2025-04-29 · Nafiseh Jabbari Tofighi, Maxime Robic, Fabio Morbidi, Pascal Vasseur

The advent of event-based cameras, with their low latency, high dynamic range, and reduced power consumption, marked a significant change in robotic vision and machine perception. In particular, the combination of these …

object-detectionObject DetectionPose EstimationSurvey

Optical Flow-Guided 6DoF Object Pose Tracking with an Event Camera

2025-12-24 · Zibin Liu, Banglei Guan, Yang Shang, Shunkun Liang 외 arxiv

Object pose tracking is one of the pivotal technologies in multimedia, attracting ever-growing attention in recent years. Existing methods employing traditional cameras encounter numerous challenges such as motion blur, …

Pose Tracking

Metro-haul Project Vertical Service Demo: Video Surveillance Real-time Low-latency Object Tracking

2020-06-08 · Annika Dochhan, Johannes K. Fischer, Bodo Lent, Achim Autenrieth 외

We report on the EU H2020 project METRO-HAUL use-case demonstration, including flexible allocation of storage and computing resources in different network locations and deployment of a network slice instance through a pr…

Object Tracking

Low Latency Visual Inertial Odometry with On-Sensor Accelerated Optical Flow for Resource-Constrained UAVs

2024-06-19 · Jonas Kühne, Michele Magno, Luca Benini

Visual Inertial Odometry (VIO) is the task of estimating the movement trajectory of an agent from an onboard camera stream fused with additional Inertial Measurement Unit (IMU) measurements. A crucial subtask within VIO …

Optical Flow Estimation