Integrating Artificial Intelligence and Augmented Reality in Robotic Surgery: An Initial dVRK Study Using a Surgical Education Scenario
Robot-assisted surgery has become progressively more and more popular due to its clinical advantages. In the meanwhile, the artificial intelligence and augmented reality in robotic surgery are developing rapidly and receive lots of attention. However, current methods have not discussed the coherent integration of AI and AR in robotic surgery. In this paper, we develop a novel system by seamlessly merging artificial intelligence module and augmented reality visualization to automatically generate the surgical guidance for robotic surgery education. Specifically, we first leverage reinforcement leaning to learn from expert demonstration and then generate 3D guidance trajectory, providing prior context information of the surgical procedure. Along with other information such as text hint, the 3D trajectory is then overlaid in the stereo view of dVRK, where the user can perceive the 3D guidance and learn the procedure. The proposed system is evaluated through a preliminary experiment on surgical education task peg-transfer, which proves its feasibility and potential as the next generation of robot-assisted surgery education solution.
Code (1)
Similar Papers 제목 키워드 기반
Do You See What I See: Using Augmented Reality and Artificial Intelligence
Do You See What I See: Using Augmented Reality and Artificial Intelligence We explore the challenges of real-world applications of augmented reality (AR) and artificial intelligence (AI) through experiments that demons…
Keypoint EstimationPose EstimationAR Visualization System for Ship Detection and Recognition Based on AI
Augmented reality technology has been widely used in industrial design interaction, exhibition guide, information retrieval and other fields. The combination of artificial intelligence and augmented reality technology ha…
Information Retrievalobject-detectionObject DetectionRetrieval+1KHAIT: K-9 Handler Artificial Intelligence Teaming for Collaborative Sensemaking
In urban search and rescue (USAR) operations, communication between handlers and specially trained canines is crucial but often complicated by challenging environments and the specific behaviors canines are trained to ex…
Edge-computingobject-detectionObject DetectionWhere is your place, Visual Place Recognition?
Visual Place Recognition (VPR) is often characterized as being able to recognize the same place despite significant changes in appearance and viewpoint. VPR is a key component of Spatial Artificial Intelligence, enabling…
Visual Place RecognitionRetrieval and Localization with Observation Constraints
Accurate visual re-localization is very critical to many artificial intelligence applications, such as augmented reality, virtual reality, robotics and autonomous driving. To accomplish this task, we propose an integrate…
Autonomous DrivingImage RetrievalRetrievalSemantic Segmentation