DT/MARS-CycleGAN: Improved Object Detection for MARS Phenotyping Robot
Robotic crop phenotyping has emerged as a key technology to assess crops' morphological and physiological traits at scale. These phenotypical measurements are essential for developing new crop varieties with the aim of increasing productivity and dealing with environmental challenges such as climate change. However, developing and deploying crop phenotyping robots face many challenges such as complex and variable crop shapes that complicate robotic object detection, dynamic and unstructured environments that baffle robotic control, and real-time computing and managing big data that challenge robotic hardware/software. This work specifically tackles the first challenge by proposing a novel Digital-Twin(DT)MARS-CycleGAN model for image augmentation to improve our Modular Agricultural Robotic System (MARS)'s crop object detection from complex and variable backgrounds. Our core idea is that in addition to the cycle consistency losses in the CycleGAN model, we designed and enforced a new DT-MARS loss in the deep learning model to penalize the inconsistency between real crop images captured by MARS and synthesized images sensed by DT MARS. Therefore, the generated synthesized crop images closely mimic real images in terms of realism, and they are employed to fine-tune object detectors such as YOLOv8. Extensive experiments demonstrated that our new DT/MARS-CycleGAN framework significantly boosts our MARS' crop object/row detector's performance, contributing to the field of robotic crop phenotyping.
Code (0)
등록된 구현이 없습니다.
Tasks
Image AugmentationObjectobject-detectionObject DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MARs: Multi-view Attention Regularizations for Patch-based Feature Recognition of Space Terrain
The visual detection and tracking of surface terrain is required for spacecraft to safely land on or navigate within close proximity to celestial objects. Current approaches rely on template matching with pre-gathered pa…
AttributeMetric LearningNavigateTemplate MatchingMars-Bench: A Benchmark for Evaluating Foundation Models for Mars Science Tasks
Foundation models have enabled rapid progress across many specialized domains by leveraging large-scale pre-training on unlabeled data, demonstrating strong generalization to a variety of downstream tasks. While such mod…
Object DetectionMARS-M: When Variance Reduction Meets Matrices
Matrix-based preconditioned optimizers, such as Muon, have recently been shown to be more efficient than scalar-based optimizers for training large-scale neural networks, including large language models (LLMs). Recent be…
Training-Free and Interpretable Hateful Video Detection via Multi-stage Adversarial Reasoning
Hateful videos pose serious risks by amplifying discrimination, inciting violence, and undermining online safety. Existing training-based hateful video detection methods are constrained by limited training data and lack …
Edge-detection applied to moving sand dunes on Mars
Here we discuss the application of an edge detection filter, the Sobel filter of GIMP, to the recently discovered motion of some sand dunes on Mars. The filter allows a good comparison of an image HiRISE of 2007 and an i…
Edge DetectionSand