paper-with-me

홈 › Papers

Towards Augmented Microscopy with Reinforcement Learning-Enhanced Workflows

2022-08-04 · Michael Xu, Abinash Kumar, James M. LeBeau

Here, we report a case study implementation of reinforcement learning (RL) to automate operations in the scanning transmission electron microscopy (STEM) workflow. To do so, we design a virtual, prototypical RL environment to test and develop a network to autonomously align the electron beam without prior knowledge. Using this simulator, we evaluate the impact of environment design and algorithm hyperparameters on alignment accuracy and learning convergence, showing robust convergence across a wide hyperparameter space. Additionally, we deploy a successful model on the microscope to validate the approach and demonstrate the value of designing appropriate virtual environments. Consistent with simulated results, the on-microscope RL model achieves convergence to the goal alignment after minimal training. Overall, the results highlight that by taking advantage of RL, microscope operations can be automated without the need for extensive algorithm design, taking another step towards augmenting electron microscopy with machine learning methods.

📄 PDF Abstract BibTeX arXiv:2208.02865

Code (1)

lebeaugroup/diffalignrl 공식 구현 pytorch

Tasks

reinforcement-learningReinforcement LearningReinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Test 설명 없음
ALIGN In the ALIGN method, visual and language representations are jointly trained from noisy image alt-text data. The image and text encoders are learned via contrastive loss…

Similar Papers 제목 키워드 기반

Microscope 2.0: An Augmented Reality Microscope with Real-time Artificial Intelligence Integration

2018-11-21 · Po-Hsuan Cameron Chen, Krishna Gadepalli, Robert MacDonald, Yun Liu 외

The brightfield microscope is instrumental in the visual examination of both biological and physical samples at sub-millimeter scales. One key clinical application has been in cancer histopathology, where the microscopic…

Diagnostic

Global Voxel Transformer Networks for Augmented Microscopy

2020-08-05 · Zhengyang Wang, Yaochen Xie, Shuiwang Ji

Advances in deep learning have led to remarkable success in augmented microscopy, enabling us to obtain high-quality microscope images without using expensive microscopy hardware and sample preparation techniques. Howeve…

Deep Learning

Creating and troubleshooting microscopy analysis workflows: common challenges and common solutions

2024-03-06 · Beth A Cimini

As microscopy diversifies and becomes ever-more complex, the problem of quantification of microscopy images has emerged as a major roadblock for many researchers. All researchers must face certain challenges in turning m…

EAA: Automating materials characterization with vision language model agents

2026-02-17 · Ming Du, Yanqi Luo, Srutarshi Banerjee, Michael Wojcik 외 arxiv

We present Experiment Automation Agents (EAA), a vision-language-model-driven agentic system designed to automate complex experimental microscopy workflows. EAA integrates multimodal reasoning, tool-augmented action, and…

Multimodal Reasoning

The Rise of Data-Driven Microscopy powered by Machine Learning

2024-01-10 · Leonor Morgado, Estibaliz Gómez-de-Mariscal, Hannah S. Heil, Ricardo Henriques

Optical microscopy is an indispensable tool in life sciences research, but conventional techniques require compromises between imaging parameters like speed, resolution, field-of-view, and phototoxicity. To overcome thes…