Program-Guided Image Manipulators
Humans are capable of building holistic representations for images at various levels, from local objects, to pairwise relations, to global structures. The interpretation of structures involves reasoning over repetition and symmetry of the objects in the image. In this paper, we present the Program-Guided Image Manipulator (PG-IM), inducing neuro-symbolic program-like representations to represent and manipulate images. Given an image, PG-IM detects repeated patterns, induces symbolic programs, and manipulates the image using a neural network that is guided by the program. PG-IM learns from a single image, exploiting its internal statistics. Despite trained only on image inpainting, PG-IM is directly capable of extrapolation and regularity editing in a unified framework. Extensive experiments show that PG-IM achieves superior performance on all the tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Image InpaintingSimilar Papers 제목 키워드 기반
KiloBot: A Programming Language for Deploying Perception-Guided Industrial Manipulators at Scale
We would like industrial robots to handle unstructured environments with cameras and perception pipelines. In contrast to traditional industrial robots that replay offline-crafted trajectories, online behavior planning i…
Industrial RobotsMotion PlanningTask and Motion PlanningORCESTRA: VLM-driven Visual Robot programming in Mixed Reality
ORCESTRA is a mixed-reality system for programming robot digital twins through no-code waypoint teaching and language-guided control. In a passthrough mixed-reality workspace, users place robot twins on real surfaces, te…
Catch the Ball: Accurate High-Speed Motions for Mobile Manipulators via Inverse Dynamics Learning
Mobile manipulators consist of a mobile platform equipped with one or more robot arms and are of interest for a wide array of challenging tasks because of their extended workspace and dexterity. Typically, mobile manipul…
Trajectory Tracking with Reachability-Guided Quadratic Programming and Freeze-Resume
Many robotic systems must follow planned paths yet pause safely and resume when people or objects intervene. We present an output-space method for systems whose tracked output can be feedback-linearized to a double integ…
AERMANI-PLACE: Language Guided Object Placement with Aerial Manipulators
Object placement is a fundamental component of aerial manipulation tasks, yet existing systems typically require the desired placement position to be specified explicitly in metric coordinates. Such interfaces are not in…
Image Editing