Maximum Likelihood Estimation

For a change from working on the motion estimation side of the project, i decided to take a look at replacing the end of the super-resolution pipeline. The end of the pipeline is the part responsible for reversing the effects of the forward model. Up until know i have implemented this inversion using the pseudo-inverse function, whilst this is an effective approach for small images, and images which do not contain noise it will become infeasable to use this method once i start using larger images, and once i move ontoconsidering real world low resolution examples that contain noise.

The other motivation for implementing this part of the project now asides from the change from looking at motion estimation is that the maximum likelihood estimation approach is similar to work that i have been doing in my Neural networks class (ECE 173).



The X that satisfies the above equation is the X vector that is most likely to have been the original high resolution image. By finding the derivative of the above equation and setting it equal to zero we can then calculate the X vector that corresponds to the minimum of the function. The equation below shows the derivative:

Rearranging this formula and then applying the gradient descent method allows the X vector corresponding to the minimum error to the iteratively obtained.

Where

and


The graph below shows the error between the reconstruction and the original high resolution images against the number of iterations that were performed in order to acquire the reconstruction.

The reconstructed image can be seen below:

Motion Estimation

Well with CSE121 finally submitted last night i have been able to get back to work on the project. The next portion of the project i want to get done is the motion estimation part.

My first attempt at implementing motion estimation was to use the Lucas-Kanade optical flow algorithm. This provided promising results for displacements of only 1 or two pixels at high resolution so long as the decimated image wasn't 'too' decimated. For larger high resoltuon displacements however this method of estimating the motion between frames quickly broke down, increasing the factor by which the image is decimated resulted in the algorithm being accurate over smaller displacements. The factor by which an image has been decimated directly influences the amount of information that is c0ntained in the low resolution frames.

After trying Lucas-Kanade i searched around on the internet for more papers relating to motion estimation and came up with one which sounds like it might be a promising method, hopefully this paper will provide a method that works, otherwise i will just implement ransac, which for the moment seems like overkill as i am assuming that the transformations between images are pure translations.

The paper is titled: "Using Gradient Correlation for sub-pixel motion estimation of video sequences"

I have started implementing this algorithm but due to lack of time i have been unable to finish it before the lecture, after the lecture i will hopefully get the implementation finished and i will then improve this entry to include information on how the method works.

No Time

Well unfortunately this week I have had hardly anytime at all to spend on my project, been busy trying to get my CSE121 project done which just seems to take forever. As such i have been unable to do any more work on the motion estimation side of the project this week.

What time I have had ive spent playing around with the forward model that I have been using. As one of the next steps in the project I was aiming to test out the super-resolution approach on images that have been decimated using a different method, e.g. images I haven’t decimated my self using my own forward model.

In order to acquire the low resolution images I continued to use the same high resolution image that I used for the previous tests. In order to acquire the low resolution input frames I repeatedly shifted the images and then applied the matlab image resize command with the resize method set to bilinear.

Once these low resolution frames had been acquired I then attempted to reconstruct the original high resolution image, using my own forward model. I performed the reconstruction several times each time adjusting the forward model i was using slightly, i have included a selection of results that i obtained. I have also improved the function i have written to create the decimation matrix to enable me to perform decimation using nearest neighbour as well as bilinear interpolation.

Attempt 1

Forward Model Settings:

Gaussian Blur sig = 0.5, Decimation = Bilinear

Reconstructed Image:

Error = 0.1658

Conclusions: Looking at the error value and the reconstructed image it can clearly be seen that the forward model is incorrect, as the reconstructed image doesn't resemble the original image at all.

Attempt 2

Forward Model Settings:

Gaussian Blur sig = 0.25, Decimation = Bilinear

Reconstructed Image:

Error = 0.0289

Attempt 3

Forward Model Settings:

Gaussian Blur sig = 0 (no bluring), Decimation = Bilinear

Reconstructed Image:

Error = 0.0052

Conclusions: Low Error and the reconstructed image looks similar to the original high res image

Attempt 4

Forward Model Settings:

Gaussian Blur sig = 0.5, Decimation = nearest neighbour

Reconstructed Image:

Error = 0.5919

Attempt 5

Forward Model Settings:

Gaussian Blur sig = 0.25, Decimation = nearest neighbour

Reconstructed Image:

Error = 0.0221

Attempt 6

Forward Model Settings:

Gaussian Blur sig = 0 (no bluring), Decimation = nearest neighbour

Reconstructed Image:

Error = 0.0078