{ Image Warping & Mosaicing }

In this project I try to perform image rectification and mosaicing using homographies.

Home Series

Home 1

Home 2

Home 3

Home 4

Bart Series

Bart 1

Bart 2

Bart 3

Bart 4

WeWork Series

WeWork 1

WeWork 2

WeWork 3

WeWork 4

Firstly, we need to define correspondnces between pictures (I used the given tool: https://cal-cs180.github.io/fa23/hw/proj3/tool.html). Then, using these keypoints, we want to recover homographies: Here's how I did it. We have: \( pts1 = \{(x_1, y_1), (x_2, y_2), \dots, (x_n, y_n)\} \), \( pts2 = \{(x'_1, y'_1), (x'_2, y'_2), \dots, (x'_n, y'_n)\} \). To compute a homography matrix \( H \), For each pair of points \( (x_i, y_i) \) from \( pts1 \) and \( (x'_i, y'_i) \) from \( pts2 \), we create: \[ M = \begin{bmatrix} x_1 & y_1 & 1 & 0 & 0 & 0 & -x_1 x'_1 & -y_1 x'_1 \\ 0 & 0 & 0 & x_1 & y_1 & 1 & -x_1 y'_1 & -y_1 y'_1 \\ \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots \\ x_n & y_n & 1 & 0 & 0 & 0 & -x_n x'_n & -y_n x'_n \\ 0 & 0 & 0 & x_n & y_n & 1 & -x_n y'_n & -y_n y'_n \end{bmatrix} \] and then flatten points from points 2 into a vector\[ \mathbf{res\_coords} = \begin{bmatrix} x'_1 \\ y'_1 \\ \vdots \\ x'_n \\ y'_n \end{bmatrix}\] Then, solve \( M \mathbf{h} = \mathbf{res\_coords} \), where \( \mathbf{h} \) is: \[ \mathbf{h} = (h_1, h_2, h_3, h_4, h_5, h_6, h_7, h_8) \]

To warp images, I create interpolation functions for each channel using RectBivariateSpline. Then I apply inverse homography matrix to the grid of result coords, for each channel interpolate the pixel values, and return the result. In my function, I add the desired shape for the output as an input.

For this part, I create a separate list of destination points that basically has all corners, and then define keypoints as corners of either portrait or poster.

Nature Morph Rectification

Original image

Warped

Poster Rectification

Original image

Warped

Initially I tried mosaicing two images. Here's what I did:

1. Compute homographies
2. Determine the bounding box of the panorama
3. Shift images so that all coordinates are positive
4. Warp images
5. Create masks
6. Blend warped images

Two-Image Mosaic

Four-Image Mosaics

Then, I tried doing the same thing with 4 pics, the only thing I changed was multiplying homographies

Home Series Mosaic

Bart Series Mosaic

WeWork Series Mosaic

Running the skeleton code on my pictures gives me this result:

Initial image

Skeleton code applied

In this part, I implemented ANMS to reduce the number of dots of potential keypoints. The result is below:

Initial keypoints

After ANMS

Here I extract feature descriptors (40x40 windows of 8x8 patches) and implement feature matching to find the best correspondences. More formally:

• match_descriptors: matches feature descriptors between two images by finding two euclidean distance of each pair and apply ratio test for reliable matches.
• extract_descriptors: extract a normalized descriptor for each point by downsampling images, also make it robust to brightness and contrast changes.

Feature matching between images

Two-Image Comparison

Here are the results with two pics:

Manual stitching

Automatic stitching

Four-Image Automatic Results

Here are the results with four pics:

Home Series - Automatic

Bart Series - Automatic

WeWork Series - Automatic

Manual vs Automatic Comparison

This is manual and automatic next to each other:

Home Series - Manual

Home Series - Automatic

Bart Series - Manual

Bart Series - Automatic

WeWork Series - Manual

WeWork Series - Automatic

I think the main problem here was bad pictures (shaky hands) and the fact that I tried to align to the first picture rather than the either second or third one. It was fun to learn how to manipulate pictures and see something that was hard to see initially like in one of the lectures with floor mosaic. Can't wait for the second part, where we wouldn't have to manually mark correspondences. For the part B it took me forever to center towards the third pictures, because I don't use the same homographies structure as I did before.