Image Segmentation Part 1 Thresholding, Otsu and HSV color space method

There will be cases where we need to isolate particular objects in our image. Based on my previous blog on Blob Detection and Connected Components , we can use the connected components’ region properties in our image, isolate particular objects using a pre-defined dimension. But what if the object of interest has no distinct size? Using only the region properties can be ineffective. This blog will discuss the Thresholding, Otsu, and HSV color space method to segment certain objects with this particular scenario.

Thresholding Method

The thresholding method is a technique to isolate objects in our image by setting a pixel intensity value threshold of what values we want to retain. From the figure below, I have binarized an orange bean bag’s image using a threshold value of 0.5. Pixel values greater than the threshold are converted to one; otherwise, they are converted to zero.

Fig 1. Thresholding Example of an orange bean bag

By using the Thresholding method, the tile patterns and reflections of the lights are removed. However, the technique failed to isolate the upper left shade in the image since it has the same level of pixel values as the bean bag. To separate the bean bag from other objects in this image, we can use Morphological operations and the connected components’ region properties to filter out unnecessary objects based on their area.

Fig 2. Morphed Image and Area of each object

However, this approach would be challenging to implement for hundreds of images since you need to predefine each objects’ dimensions and thresholds of interest and clean the different images.

Otsu’s Method

Otsu’s method automatically defines the threshold value of the image. The method assumes that the image has a background and a foreground where it minimizes the intra-class variance and maximizes the inter-class variance between the two. From the figure below, we can see the histogram of each pixel’s intensity values for the orange bean bag image.

Fig 3. Histogram of pixel intensity values (Orange bean bag image)

From the figure above, we can see that the image has a multi-modal distribution of pixel values. Otsu’s method assumes that these nodes are the background and the foreground.

Fig 4. Histogram of pixel intensity values with Inter-class variance

Otsu’s method refers to the maximum inter-class variance between the background and foreground as the threshold. Based on the figure above, we can see that the maximum variance is around 0.45. We can see how the Otsu’s method performed in the Orange bean bag image from the figure below.

Fig 5. Segmenting Orange bean bag using Otsu’s method

The Otsu’s method proved it was able to isolate the orange bean bag successfully. But what if we tried to segment an image containing multiple objects with different colors like the one in the figure below. Can Otsu’s method still isolate each object?

Fig 6. Multiple bean bags

In plotting the image’s distribution containing multiple bean bags, the Otsu’s method finds it difficult to segment the background from the foreground since it has more than just two modes. As a result, Otsu’s method failed to isolate the objects of interest since multiple objects of interest exist.

Fig 7. Pixel distribution and result of Otsu’s method for multiple bean bags

What if we only want to segment the blue bean bag? We can first binarize the image by converting pixels that majorly comprised of the Red or Green channel to zero while retaining the original values of pixels majorly consist of the Blue channel. After binarizing the image, we can then use Otsu’s method to isolate the blue bean bag, as shown in the figure below.

import numpy as np
from skimage.io import imread, imshow
from skimage.color import rgb2gray
from skimage.filters import threshold_otsu
bags = imread('bags.png')
bags_gray = rgb2gray(bags)
# Remove objects majorly comprised of the Red and Green channel blue = bags[:, :, 2] - bags_gray * 255
blue = np.where(blue > 0, blue, 0)
# Use Otsu's method to calculate for the threshold
thresh = threshold_otsu(blue)
blue_bin_otsu = blue > thresh
fig, ax = plt.subplots(1, 3, figsize=(17, 4))
ax[0].imshow(blue)
ax[0].set_title('Binarized major B channel contributed pixels') ax[0].set_axis_off()
ax[1].imshow(blue_bin_otsu)
ax[1].set_title('Otsu\'s method')
ax[1].set_axis_off()
bags_b = bags.copy()
bags_b[:, :, 0] = bags_b[:, :, 0] * blue_bin_otsu
bags_b[:, :, 1] = bags_b[:, :, 1] * blue_bin_otsu
bags_b[:, :, 2] = bags_b[:, :, 2] * blue_bin_otsu
ax[2].imshow(bags_b)
ax[2].set_title('RGB after segmentation')
ax[2].set_axis_off();

Fig 8. Otsu’s method segmenting the blue bean bag

Now that we have successfully segmented the blue bean bag, what if we are want to segment the orange bean bag instead? It would be challenging to use the RGB channel.

HSV Color Space

Handling color segmentation can be tricky especially considering different shades and hues. We can then use the HSV color space to isolate specific objects in our image. Now, let us try to segment the blue bean bag using the HSV color space.

from skimage.color import rgb2hsv
bags_hsv = rgb2hsv(bags)
lower_mask = bags_hsv[:,:,0] > 0.60
upper_mask = bags_hsv[:,:,0] < 0.70
mask = upper_mask*lower_mask
red = bags[:,:,0]*mask
green = bags[:,:,1]*mask
blue = bags[:,:,2]*mask
bags_masked = np.dstack((red,green,blue))
fig, (ax0, ax1, ax2) = plt.subplots(1, 3, figsize=(15, 3))
im = ax0.imshow(bags_hsv[:,:,0], cmap='hsv')
ax0.set_title('HSV')
ax0.set_axis_off()
plt.colorbar(im, ax=ax0)
ax1.imshow(mask)
ax1.set_title('Mask')
ax1.set_axis_off()
ax2.imshow(bags_masked)
ax2.set_title('Blue Bag')
ax2.set_axis_off()
plt.tight_layout();

Fig 9. HSV color space method segmenting the blue bean bag

Based on the figure above, the HSV color space method has successfully segmented the blue bean bag. Some unintended objects in the image are retained since they have the same HSV color space as the blue bean bag.

The HSV color space method has proven to be useful for the primary colors of light (RGB). But how does it compare to other colors like orange, green, and yellow, which have been the limitation of the Otsu’s method?

Fig 10. HSV color space method segmenting the Orange, Yellow and Green bean bag

The HSV color space method has successfully segment objects of interest with colors aside from the primary colors of light from the figure above.

The Otsu’s method and HSV color space method are potent tools for segmenting objects of interest for different colors. However, both methods have their limitations, where the Otsu’s method has difficulty in segmenting image for multiple objects of interest and colors outside the primary colors of light. In comparison, the HSV color space method can deal with these scenarios, although it fails to exclude unintended objects with the same HSV values level. In my next blog, I will discuss RG Chromaticity segmentation, which deals with the Otsu’s method and HSV color space method’s limitations.

Originally published at https://abeljoshuacruzada.wixsite.com on January 28, 2021.