D-Flow: Differentiating through Flows for Controlled Generation
Taming the generation outcome of state of the art Diffusion and Flow-Matching (FM) models without having to re-train a task-specific model unlocks a powerful tool for solving inverse problems, conditional generation, and controlled generation in general. In this work we introduce D-Flow, a simple framework for controlling the generation process by differentiating through the flow, optimizing for the source (noise) point. We motivate this framework by our key observation stating that for Diffusion/FM models trained with Gaussian probability paths, differentiating through the generation process projects gradient on the data manifold, implicitly injecting the prior into the optimization process. We validate our framework on linear and non-linear controlled generation problems including: image and audio inverse problems and conditional molecule generation reaching state of the art performance across all.
Code (2)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Normalizing Flows for Knockoff-free Controlled Feature Selection
Controlled feature selection aims to discover the features a response depends on while limiting the false discovery rate (FDR) to a predefined level. Recently, multiple deep-learning-based methods have been proposed to p…
Density Estimationfeature selectionvalidTraining Free Guided Flow Matching with Optimal Control
Controlled generation with pre-trained Diffusion and Flow Matching models has vast applications. One strategy for guiding ODE-based generative models is through optimizing a target loss $R(x_1)$ while staying close to th…
Image ManipulationProtein DesignCompos3D: Interactive Part-Based Composition for Creative Control in Generative 3D Models
While generative AI has unlocked new opportunities for 3D content creation, current workflows often rely on multiple regenerations, which provides limited control and unpredictable outcomes. We present Compos3D, a system…
OneFlow: Concurrent Mixed-Modal and Interleaved Generation with Edit Flows
We present OneFlow, the first non-autoregressive multimodal model that enables variable-length and concurrent mixed-modal generation. Unlike autoregressive models that enforce rigid causal ordering between text and image…
Image GenerationChannelFlow-Tools: A Configuration-Driven Pipeline for Generating Machine-Learning-Ready Datasets of 3D Obstructed Channel Flows
Data-driven surrogate models are increasingly used in computational fluid dynamics, and their reliability depends on the quality of the training data. These models are typically trained on fixed, pre-generated datasets. …