— RotoTexture – provides a less complicated way to artificially modify a surface (Image: Hui Fang)
Surface effects including "virtual make-up" can be automatically added to video clips using software developed by US researchers.
The software called RotoTexture provides a less complicated way to artificially modify a surface, including a person's face. Normally, a 3D model of a scene must be built in order for it to be modified in this way.
"Our approach is quite original because it only needs the minimum input: a raw video clip," explains Hui Fang, the computer-vision researcher who developed the system with colleague John Hart while at the University of Illinois at Urbana-Champaign, US, and who at now works at Google.
Videos show RotoTexture being used to alter the texture of a person's face(mov format, 19.2MB) and place an image on a piece of rippling, blank cloth(mov format, 6.5MB).
RotoTexture first estimates the orientation of a target surface based on its shading, Fang told New Scientist. Then, using this information, the software applies either an image or a textured pattern to the surface.
"We treat an image to paste as a large rubber band," explains Hui. "Then we try to paste that rubber band [onto a surface] such that the stretching within the rubber band is minimised. As a result, the image will appear to follow the undulation of the surface."
To apply a texture, the software segments a surface into small overlapping patches, each of which is applied in the correct order and with the correct orientation to make a seamless surface.
Features of the original surface are compared with the modified picture to ensure that the texturing is consistent between frames. But Fang concedes that better motion-tracking software could improve the process, pointing out that, currently, the applied texture sometimes "swims" slightly on the underlying surface.
Geoff Hodbod, founder of UK computer animation firm 3D Imaging, says RotoTexture would be attractive to many animators. "I could see this being used in children's TV to make people's faces look more interesting," he says. "It would also have great potential in science-fiction productions."
Currently, skilled artists must build a 3D wire-frame computer model of an object or surface in order for it to be re-textured, Hodbod explains. "[RotoTexture] looks like a prospective alternative," he says, "but it does slip about a bit, so I don't think it could immediately be used for a professional production." Providing this flaw can be fixed, Hodbod says the system could prove a big hit with animators.