Project 3: Face Morphing

Student Name: Kelvin Huang

Part 1: Defining Correspondences

In this part, I manually select corresponding points on two images (Kelvin's and Jessi's faces) using interactive selection. These points represent key facial features, such as eyes, nose, and mouth. Once the points are defined, I calculate a midway shape by averaging the corresponding points from both images.

I then apply Delaunay triangulation to divide each image into triangular regions based on the selected points. The triangulation ensures that the morphing process is smooth by aligning facial features across both images. Triangulation is performed on Kelvin’s image, Jessi’s image, and the midway shape. The midway triangulation is used to warp and blend the two images consistently throughout the morphing process.

Part 2: Computing the "Mid-way Face"

In this part, I compute the mid-way face by blending two images (Kelvin's and Jessi's faces). The process involves three main steps:

1. Average Shape: I first calculate the average shape by averaging the positions of corresponding points from both faces. This gives us a set of keypoints representing the mid-way shape between the two faces.
2. Warping: Each triangle in the Delaunay triangulation is warped from both original images into the mid-way shape. An affine transformation matrix is computed for each triangle, and bilinear interpolation is used to map pixels from the original images into the new shape.
3. Averaging Colors: After warping, I compute the final mid-way face by averaging the pixel colors from the two warped images.

This results in a blended face that incorporates both geometrical and color features from the original images. Below, I display the original images (A and B) and the computed mid-way face.

Part 3: The Morph Sequence

In this part, I generate a smooth morphing sequence between two images (Kelvin's and Jessi's faces). The morphing process involves two key parameters: warp_frac and dissolve_frac, both of which vary from 0 to 1 over the sequence of 45 frames.

1. Warping the Shape: For each frame, the points in Kelvin's and Jessi's images are gradually shifted from the original positions to the final corresponding positions. The intermediate shapes are computed using the warp_frac parameter, which controls how much the shape warps between the two images.
2. Cross-Dissolving: After the shape is warped, the colors from both images are blended together using the dissolve_frac parameter. This cross-dissolving controls the appearance of the intermediate face by smoothly blending the pixel colors between the two images.
3. Frame Generation: For each of the 45 frames, I compute the warped and blended image and save them as intermediate frames in both video and GIF formats. Specific frames are saved as individual images to showcase key moments in the morphing process.

This method creates a seamless transition from one face to the other by controlling both the shape transformation and color blending at each stage of the morphing sequence.

If the GIF doesn't load properly, here is a backup video showcasing the morphing sequence:

Part 4: The "Mean Face" of a Population

In this part of the project, I used a dataset of annotated faces (the Danes dataset) to analyze and morph faces between different geometries.

I computed the average face shape by calculating the mean of the keypoints from all faces in the dataset. This average shape represents the geometric structure that all individual faces are warped into.

I computed them by dividing into two categories: male and female.

Then, I morph each of the faces in the dataset into the average shape. Some examples are shown below.

By averaging each of the warped images from two gender, I can obtain the average face of all Danes males and females.

I used the keypoint correspondences between Kelvin's face and the average Danes male face. Again, I applied Delaunay triangulation to divide the faces into triangles. An affine transformation was then computed for each triangle to warp Kelvin's face into the average Danes male face's shape. The warped image represents Kelvin's facial features, but transformed to the average Danes male face geometry.

Similarly, the average Danes male face was warped into Kelvin's geometry using affine transformations. The resulting image shows the average Danes male face, but with the overall geometry of Kelvin's face.

Part 5: Caricatures: Extrapolating from the Mean

In this part, I explore caricature generation by exaggerating or diminishing certain facial features using an extrapolation parameter, alpha. Alpha controls how much I exaggerate or contract the difference between the original face and the average face. Below are the results for values of alpha ranging from -1.0 to 2.0.

Extra: Bells and Whistles

For the Bells and Whistles of this project, I used a image of the average Chinese actress that I found online. I changed the gender of my face by morphing just the shape, just the appearance, and both. The results are displayed below.