Top row: the input video sequence of an explosion. Bottom row: the composite of the extracted explosion with another background image. Our method automatically extracted the explosion, which has considerable transparency, especially in the smoky regions of the 4th and 5th frames.

We propose a method to extract fluids from a video that is captured outside a special studio. Since such a video usually has a complex background and the fluids overlap with much transparency, it is a difficult, time-consuming task for a designer to extract them. Our goal is to develop an efficient method to solve the problem: we estimate the background of an input video, and then compute the foreground and alpha matte at each frame. Our method estimates the background by observing only pixels that have little motion at each frame. Given the estimated background, we estimate an initial alpha matte based on the color difference at every pixel between each frame and the estimated background. Since the initial alpha matte usually includes many artifacts, we employ the gradient-domain image processing approach to refine it: our method attenuates unrequired gradients adequately, and then integrate them to recover the refined alpha matte. The foreground, which explains about the color and texture pattern of the fluid itself, is also estimated in a similar manner. We demonstrate that our method enables to extract the fluids from a video, which were difficult to achieve using the previous methods.

We compare our method with previous methods. The trimap (c) is computed based on the color difference between the original (a) and the background (b) at each pixel. We use the trimap to compute the foreground color for Bayesian Matting (d). For Spectral Matting (e) and Shared Matting (f), we use the trimap as direct input to the algorithm.

Our software and some examples are here.

Makoto Okabe, Ken Anjyo, Rikio Onai, "Extracting Fluid from a Video for Efficient Post-Production", in Proc. of DigiPro 2012.