Program synthesis performance constrained by non-linear spatial relations in Synthetic Visual Reasoning Test
Despite remarkable advances in automated visual recognition by machines, some visual tasks remain challenging for machines. Fleuret et al. (2011) introduced the Synthetic Visual Reasoning Test (SVRT) to highlight this point, which required classification of images consisting of randomly generated shapes based on hidden abstract rules using only a few examples. Ellis et al. (2015) demonstrated that a program synthesis approach could solve some of the SVRT problems with unsupervised, few-shot learning, whereas they remained challenging for several convolutional neural networks trained with thousands of examples. Here we re-considered the human and machine experiments, because they followed different protocols and yielded different statistics. We thus proposed a quantitative reintepretation of the data between the protocols, so that we could make fair comparison between human and machine performance. We improved the program synthesis classifier by correcting the image parsings, and compared the results to the performance of other machine agents and human subjects. We grouped the SVRT problems into different types by the two aspects of the core characteristics for classification: shape specification and location relation. We found that the program synthesis classifier could not solve problems involving shape distances, because it relied on symbolic computation which scales poorly with input dimension and adding distances into such computation would increase the dimension combinatorially with the number of shapes in an image. Therefore, although the program synthesis classifier is capable of abstract reasoning, its performance is highly constrained by the accessible information in image parsings.
Code (1)
Tasks
Few-Shot LearningProgram SynthesisUnsupervised Few-Shot LearningVisual ReasoningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Program Synthesis as Dependency Quantified Formula Modulo Theory
Given a specification $\varphi(X,Y)$ over inputs $X$ and output $Y$, defined over a background theory $\mathbb{T}$, the problem of program synthesis is to design a program $f$ such that $Y=f(X)$ satisfies the specificati…
Program SynthesisSystematic, Lyapunov-Based, Safe and Stabilizing Controller Synthesis for Constrained Nonlinear Systems
A controller synthesis method for state- and input-constrained nonlinear systems is presented that seeks continuous piecewise affine (CPA) Lyapunov-like functions and controllers simultaneously. Non-convex optimization p…
MultiMat: Multimodal Program Synthesis for Procedural Materials using Large Multimodal Models
Material node graphs are programs that generate the 2D channels of procedural materials, including geometry such as roughness and displacement maps, and reflectance such as albedo and conductivity maps. They are essentia…
Program SynthesisSynthesis of Static Test Environments for Observing Sequence-like Behaviors in Autonomous Systems
In this paper, we investigate formal test-case generation for high-level mission objectives, specifically reachability, of autonomous systems. We use Kripke structures to represent the high-level decision-making of the a…
Decision MakingSafe Large-Scale Robust Nonlinear MPC in Milliseconds via Reachability-Constrained System Level Synthesis on the GPU
We present GPU-SLS, a GPU-parallelized framework for safe, robust nonlinear model predictive control (MPC) that scales to high-dimensional uncertain robotic systems and long planning horizons. Our method jointly optimize…