University of Washington released a new way to get 3D scan facial features

European Conference on Computer Vision, held this year on September 6 -12 days in Zurich, Switzerland, the team from the University of Washington demonstrated a new way to get 3D scan facial features, you can avoid all this, but the device is very simple to use, No special apparatus.


3D rendering accurately reconstruct a person’s facial features, is one of the most difficult jobs 3D scanners have to face. Although a variety of technologies and software have been developed for many years, but highly non-rigid nature of the human face, as well as humanity itself able to distinguish very fine detail and ability geometric imperfections, making generates a high-frequency-quality rendering is very difficult. In addition, it is a very time consuming process and requires a lot of equipment. Subjects will be a long time locked in a studio, while dozens of cameras set up to collect the necessary data.


However, the European Conference on Computer Vision (european Conference on Computer Vision), held this year on September 6 -12 days in Zurich, Switzerland, the team from the University of Washington demonstrated a new way to get 3D scan facial features, which can be avoided everything, and the device is very simple to use, requires no special equipment.

“Premise is that we can get a person at different times, in different environments, with different expressions and postures taken a lot of photos. Fact, most people in life always taken a lot of photos and videos. Our proposal is use the same person all the available pictures and videos as a database to help in the virtual world to rebuild his / her face. “

Their approach is known as the general movement of the face scan (Total Moving FAce Scanning, TMFS), research team members include the University of Washington computer science and engineering graduate students Supasorn Suwajakorn and his mentor Assistant Professor Ira Kemelmacher-Shlizerman, professor Steven Seitz.

This TMFS method can be used for 3D modeling in a non-normal shooting conditions. Especially those with a large number of public image, video data celebrities such as Prince Charles, Arnold? Schwarzenegger, Tom? Hanks such people. This should be considered a large data usage right.

You can see from the picture below, the software combines a variety of lighting conditions, head posture, facial expressions and other data to generate a highly detailed and accurate 3D rendering.


This method is obviously different from conventional techniques. “Almost all the current 3D face tracking and video reconstruction methods are based on an assumption that the human face can be mixed performance by a linear combination of the shape.” While this approach can scientists measure to limit the number of parameters and, but it will results in low levels of 3D models, expression is limited, and can not capture some important details.

Use TMFS method, image video data of the target object more, then calculate the resulting 3D image will be more accurate and vivid. The shape of the face of a person may be slightly different at every moment, but they are substantially the shape (e.g., the eye, the length of the nose, the shape of the geometric distance between the whole and the like), tends to be consistent over time. Therefore, researchers can take advantage of all available images (photos and / or video frame) to capture the average shape and appearance under certain conditions, to reconstruct a 3D model of a person’s appearance and shape.

3D rendering itself is a complex algorithm. It is essential that a “standard conformance based on the picture, such an input video frame in the same grid Comparative effect of such a key depends on the capacity of each input frame of the grid can fit rendering.”


The method comprises rendering process TMFS two steps: first, the average shape (based on an image database) are deformed to match the movement of the same in the rendering of a 3D shape. Second, the resulting shape of each of the frames to be modified in accordance with the known details clues. This means that the non-rigid facial features (such as wrinkles) will be the first frame is captured and added to the reference, in the following some specific detail it will also be considered. All of this, obviously needs some serious calculations.

But the results do not lie. TMFS algorithm can successfully capture clear wrinkles like this very minute details and subtle changes in the expression on a small scale, which is unmatched by existing software. As scientists say:

“By paying attention to changes in facial expression of each frame (with respect to the mean shape), for example, the mouth open, closed and opened eyes, wrinkles appear and disappear, and other details of the eye region, and the like. This method, even if the magnitude of the object motion is The conditions are very large and reliable, even a simple outline can also provide high-quality results. “


Prospects TMFS technology is very broad, it can be used to develop video and photo-based 3D modeling software that allows for the portrait of 3D scanning and modeling easier, convenient and accurate. Also denied the use of expensive equipment to get high-quality results necessity.

Article By :