Flow Matching is a generative modeling technique that transforms a simple probability distribution into a complex target data distribution. It learns a continuous transformation, or flow, by training a time-dependent vector field. This vector field guides samples from a simple distribution, like a Gaussian, along smooth paths to match the target distribution. The key advantage of flow matching is its simple training objective, which directly regresses the vector field to match the velocity of predefined paths between distributions.
The mathematical formulation of flow matching defines a path between distributions. We interpolate between the source distribution p_0 and the target distribution p_1 using a parameter t that ranges from 0 to 1. The vector field u_theta is trained to match the velocity of these paths. The training objective is a simple regression loss that minimizes the difference between the predicted vector field and the true path velocity. This approach is more direct than other generative modeling techniques, as it directly learns the vector field that transforms one distribution into another.
The training process for flow matching is straightforward. First, we sample a random time t between 0 and 1. Then, we sample a point from the source distribution and compute its corresponding target point. We interpolate between these points to get x_t and compute the path velocity. The neural network is trained to predict this velocity vector at point x_t and time t. The loss function is simply the squared error between the predicted vector field and the true velocity. This process is repeated with many samples to train the network. As training progresses, the loss decreases, and the neural network learns to approximate the vector field that transforms the source distribution into the target distribution.
Once the flow matching model is trained, we can use it to generate new samples. The sampling process starts by drawing a sample from the source distribution, typically a standard Gaussian. Then, we solve the ordinary differential equation defined by the learned vector field. The ODE is given by dx/dt equals u_theta of x and t, where u_theta is our trained neural network. We start with the initial condition x at time 0 equals our source sample, and solve the ODE forward in time until t equals 1. The resulting point x at time 1 is our generated sample from the target distribution. Various numerical ODE solvers can be used, such as Euler's method or Runge-Kutta methods, with more sophisticated solvers generally providing more accurate results.
To summarize, flow matching offers several advantages over other generative modeling techniques. It has a simple training objective that directly learns the vector field without requiring adversarial training or complex likelihood computations. The sampling process is efficient and can be controlled through the choice of ODE solver. Flow matching has been successfully applied to various domains including image generation, molecular design, audio synthesis, and data augmentation. Its mathematical elegance and practical effectiveness make it a powerful addition to the generative modeling toolkit, combining the strengths of both diffusion models and normalizing flows.