Mastering Corner Detection: Harris Corner Detector in Python with OpenCV

by

💡 Problem Formulation: Corner detection is a fundamental step in many computer vision applications. It involves identifying points within an image that have significant variation in intensity in all directions. The Harris Corner Detector algorithm is a popular method for detecting these points. In this article, we’ll explore how to apply the Harris Corner Detector using Python and OpenCV, taking an image as our input and aiming to output an image with identified corners.

Method 1: Standard Harris Corner Detection

Detecting corners using the standard Harris Corner Detection involves applying OpenCV’s cv2.cornerHarris() function. It detects corners based on the windowed intensity gradient covariance matrix, considering the eigenvalues to determine the presence of a corner within the window. This method is remarkably effective for a wide range of applications.

Here’s an example:

import cv2
import numpy as np

img = cv2.imread('image.png')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
gray = np.float32(gray)
dst = cv2.cornerHarris(gray, 2, 3, 0.04)

# Result is dilated for marking the corners
dst = cv2.dilate(dst, None)

# Threshold for an optimal value
img[dst>0.01*dst.max()] = [0, 0, 255]

cv2.imshow('Corners', img)
cv2.waitKey(0)
cv2.destroyAllWindows()

The output of this code is an image with the identified corners marked in red.

This code snippet loads an image, converts it to grayscale, and processes it with the cornerHarris() function. After dilating the corner points to make them visible, thresholds are applied to select strong corners, which are then highlighted in red on the original image.

Method 2: Harris with Subpixel Accuracy

Harris corner detection can be refined to subpixel accuracy by employing the cv2.cornerSubPix() function, which iteratively refines the corner locations to achieve precision much higher than a single pixel. It is beneficial when you require highly precise corner localization.

Here’s an example:

import cv2
import numpy as np

img = cv2.imread('image.png')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
gray = np.float32(gray)
dst = cv2.cornerHarris(gray, 2, 3, 0.04)
dst = cv2.dilate(dst, None)

# Refining the corners to subpixel accuracy
ret, dst = cv2.threshold(dst, 0.01*dst.max(), 255, 0)
corners = np.int0(dst)
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 100, 0.001)
corners = cv2.cornerSubPix(gray, np.float32(corners), (5,5), (-1,-1), criteria)

cv2.imshow('Subpixel Corners', img)
cv2.waitKey(0)
cv2.destroyAllWindows()

The output of this code is an image with subpixel-accurate corners.

This snippet follows a similar approach to the standard method but refines the detected corner coordinates to achieve subpixel accuracy. The cornerSubPix() is used with a specific set of criteria to converge corner locations more precisely.

Method 3: Enhanced Harris with Non-Maximum Suppression

Improving the Harris corner detection method by integrating non-maximum suppression ensures that only the best corners are selected, removing redundant points that are too close to one another. This leads to a cleaner, more accurate set of corners, which is useful for feature-tracking algorithms.

Here’s an example:

import cv2
import numpy as np

def non_maximum_suppression(image, corners, threshold, min_distance):
    # Non-maximum suppression logic here (left as an exercise)

img = cv2.imread('image.png')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
gray = np.float32(gray)
dst = cv2.cornerHarris(gray, 2, 3, 0.04)
dst = cv2.dilate(dst, None)

# Non-maximum suppression
final_corners = non_maximum_suppression(img, dst, 0.01, 10)

cv2.imshow('NMS Corners', img)
cv2.waitKey(0)
cv2.destroyAllWindows()

The output is an image with a pruned set of corners following non-maximum suppression.

In this code, an additional function non_maximum_suppression() is created to apply non-maximum suppression logic to the corners found by cornerHarris(). The resulted corner points are then marked on the image.

Method 4: Harris Detection with Feature Strength Visualization

Instead of merely identifying the corners, visualizing the strength of corner features can help in understanding the relative importance of each corner. The strength can be represented as the intensity of the color used to mark the corner, usually the more intense the color, the stronger the corner.

Here’s an example:

import cv2
import numpy as np

img = cv2.imread('image.png')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
gray = np.float32(gray)
dst = cv2.cornerHarris(gray, 2, 3, 0.04)

# Normalize and convert to integers
dst_norm = np.empty(dst.shape, dtype=np.float32)
cv2.normalize(dst, dst_norm, alpha=0, beta=255, norm_type=cv2.NORM_MINMAX)
dst_int = np.uint8(dst_norm)

for i in range(dst_int.shape[0]):
    for j in range(dst_int.shape[1]):
        if int(dst_int[i,j]) > 127:
            cv2.circle(img, (j,i), 5, (int(dst_int[i,j]),0,0), 2)

cv2.imshow('Feature Strength Corners', img)
cv2.waitKey(0)
cv2.destroyAllWindows()

The output is an image where corners are marked with varying color intensities, corresponding to their feature strength.

The snippet demonstrates how to normalize the Harris response values for visualization purposes. Stronger corners are highlighted with greater intensity, providing visual cues for their significance.

Bonus One-Liner Method 5: Quick Harris Detection with OpenCV Functions

OpenCV provides the cv2.goodFeaturesToTrack() function which internally uses the Harris Detector or the Shi-Tomasi method. This one-liner approach is beneficial for quick implementations when you need to detect a fixed number of strong corners without fine-tuning parameters.

Here’s an example:

import cv2

img = cv2.imread('image.png')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
corners = cv2.goodFeaturesToTrack(gray, 25, 0.01, 10)
corners = np.int0(corners)

for corner in corners:
    x, y = corner.ravel()
    cv2.circle(img, (x,y), 3, 255, -1)

cv2.imshow('Quick Corners', img)
cv2.waitKey(0)
cv2.destroyAllWindows()

The output is an image with the top 25 corners marked clearly.

This one-liner code uses goodFeaturesToTrack() to quickly find and draw the strongest corners in an image, drastically reducing the complexity for achieving reliable results.

Summary/Discussion

  • Method 1: Standard Harris Detection. Straightforward and effective for many general corner detection tasks. However, it may produce too many candidate points and lacks subpixel accuracy.
  • Method 2: Harris with Subpixel Accuracy. Provides more accurate corner localization, which is beneficial in tasks like 3D reconstruction and certain tracking applications. More computationally intense than the standard method.
  • Method 3: Harris with Non-Maximum Suppression. Reduces redundant corners and yields cleaner results, especially important for feature-tracking. Requires additional implementation of non-maximum suppression logic.
  • Method 4: Harris Detection with Feature Strength Visualization. Assists in visual analysis of corner features, allowing the user to assess the strength of detected corners. Additional steps required for visualization may not be needed in all use cases.
  • Bonus One-Liner Method 5: Quick Harris Detection. Extremely efficient for rapid development, suitable for applications with fixed parameters where the detection of a certain number of strong corners suffices. Lacks customizability and fine-grained control over detection details.