Down Sampling & Up Sampling Test Images

Since Friday I have been working towards applying the forward model mentioned in my previous post to a high resolution image, and then attempting to obtain the original high resolution image using the produced low resolution images and knowledge about the transformations that were applied to it.

Forward Model

In order to simplify my first attempt at applying the forward model and then reversing its affects I choose to implement the warping operation applied to the high resolution images to being a shifting affect only. The blurring operation was implemented using a Gaussian kernel with a σ = 0.5. Finally I chose to decimate the image using bilinear interpolation.

Applying the forward Model

The only parameter that was varied in order to produce the low resolution images was the amount that the original high resolution image was displaced before the blurring and decimation operations were performed.

The image below is the original high resolution image that I applied the forward model to.

The images below show some examples of the result of applying the forward model to the high resolution image.

Calculating the High Resolution Image

First Attempt at Reconstructing the original high resolution image

Second Attempt at Reconstructing the high resolution image

As suggested by Serge i have now inverted the original high resolution image before creating the low resolution frames, and then applying the super-resolution process to them, this has been done purely for cosmetic reasons so that the black lines that creep in at the borders as the image is shifted dont look out of place.

The image below shows the new high resolution image.

The image below shows 9 out of the 25 low resolution frames that were generated.

Since inverting the images i have noticed a slight bug in the matrix that is being used to decimate the high resolution images. This bug didn't show up when performing the super-resolution with an image with a white background. However along the right, and lower side of the reconstructed high resolution image a greyish strip can be seen. To solve this problem i believe i need to adjust the weights of the decimation matrix. But not wanting to rush this and break the whole program i will wait until i have more time before adjusting them.

Next Step(s)

• Fix bug in the decimation matrix
• Implement a function to calculate the error between the reconstructed image and the original image - probably in the form of mean square error
• Attempt to automatically determine the relative motions between the low resolution images, and then compare the new reconstruction error with the previous reconstruction error. Serge recommended implementing this using the optical flow algorithm.
  
• Look into implementing maximum likelihood estimation in order to reconstruct the high resolution image, rather than using the pseudo-inverse.
  
• Effects of noise on the system. Currently i have been using low resolution images that contain no noise, once maximum likelihood estimation is implemented a comparison as to which is more robust against the effects of noise will be performed.