Image Enhancements

In my last blog, I have discussed the definition and importance of Image Processing, and Computer Vision can be in different fields. The collected use of these methods aims to mimic the way humans interpret images through collected transformation and analysis techniques. However, even for us humans, there are some cases where we find it difficult to extract information from images, especially for images with a lot of noise or unbalanced colors. This is where Image Enhancement comes in to transform images to be more interpretable. The famous blue and black or white and gold dress is one example where Image Enhancement could help us finally decide this dress’s color. In this blog, I will discuss Image Enhancement, why it is essential, and common techniques in Image Enhancement like Fourier Transform, White Balancing, and Histogram Manipulation.

Fig 1. Famous blue and black or white and gold dress.

As with any machine learning models, machine learning models that takes images as input relies on the interpretability of the images used to train them. Thus, using images with a lot of noise or is incomprehensible would lead to a model with poor performance. Image Enhancement is a process for improving the quality of an image that would help interpret information, providing better results for other image processing, computer vision, and machine learning models.

There are three Image Enhancement techniques I would like to talk about in this blog.

1. Fourier Transform
2. White Balancing
3. Histogram Manipulation

Fourier Transform is an operation that transforms a signal from a spatial domain into a frequency domain. Masking the periodic patterns in the Fourier space of our image would remove the noise in our image.

Fig 2. Noisy lunar image

In this example, notice that the vertical line in the right image represents the left image’s periodic patterns. Masking this vertical line would remove the noise in the left image.

Fig 3. Clean lunar image

White Balancing is a technique to correct the images’ color by adjusting the image’s pixels to balance its temperature. There are several methods for white balancing.

1. White Patch Algorithm is an approach to rescale or normalize each image channel to a high percentile value (e.g., 85th to 90th percentile) for that channel.
2. Gray-world Algorithm is an approach that assumes that, on average, pixels are gray. With this assumption, we can adjust each channel so that the mean value will be the same for all channels. However, a problem with this algorithm is when you have a mainly heterogeneous image.
3. Ground-truth Algorithm is an approach that manually selects the “true” white patch of an image. We can then normalize each channel of the image using the selected white patch.

Now let us the different white balancing methods on an old family photo shown below.

Fig 4. Old family photo

Comparing all the white balancing results of the old photo, the white patch using the 99th percentile returns the best result. Although it is almost identical to the skin patch’s max method ground truth algorithm, the 99th percentile white patch has a less reddish-brown color. The 95th and 90th percentile and ground truth with wall patch max method return a bright image that makes it difficult to see the facial features. The ground truth using the mean method and the gray world algorithm returns darker images, but the reddish-brown color is reduced. However, in the gray world algorithm, the reddish-brown color is replaced by a blue filter.

Fig 5. White balancing result of old family photo

Histogram Manipulation is a technique to make the distribution of the intensity values of the image evenly uniform. This technique is mostly used for images when an image is under or overexposed, where the pixel’s intensity values are skewed.

Fig 6. Dark painting image

The goal of histogram manipulation is to make the actual cumulative distribution linear. Interpolation is used to create a lookup table that maps the percentile value of the target CDF to the image’s intensity values. For example, the intensity value of 50 is in the 95th percentile. We then replace the image’s intensity value based on the 95th percentile intensity value of the target CDF, which is 244.

Fig 7. Balanced painting image

To enhance an RGB image using histogram manipulation, we need to perform interpolation for each channel and stack them together. In comparing the results of Linear Interpolation, Normal Gaussian, Gamma, and Beta distribution, we can see that the best result is using the Beta distribution. Even though all the approaches yield good results, only using the Beta distribution efficiently reduced the camera’s flash from the reflection. Although arguably, using linear interpolation and gaussian yields a more accurate color.

Fig 8. Different distributions for target CDF

With limited data in our training set, Image Enhancement would come in handy in the future to help up improve the accuracy of our machine learning models.