5 Best Ways to Detect a Rectangle and Square in an Image Using OpenCV Python

by

💡 Problem Formulation: Detecting rectangles and squares in images is a common task in computer vision applications such as document scanning, object detection, and augmented reality. The input is an image file that may contain various shapes, and the desired output is the identification and marking of all the rectangles, distinguishing squares if necessary, within the image.

Method 1: Contour Approximation

This method uses the contour finding and approximation functionalities of OpenCV to detect polygons. By analyzing the length and angles of the approximated contour segments, one can differentiate between rectangles and squares. The function cv2.findContours() is key to outline shapes, while cv2.approxPolyDP() simplifies the count to four sides for potential rectangles.

Here’s an example:

import cv2

# Load image, grayscale, and Otsu's threshold
image = cv2.imread('image.jpg')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]

# Find contours
cnts, _ = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)

# Filter for rectangles and squares
for c in cnts:
    peri = cv2.arcLength(c, True)
    approx = cv2.approxPolyDP(c, 0.04 * peri, True)
    if len(approx) == 4:
        cv2.drawContours(image, [approx], -1, (0, 255, 0), 3)

# Display the image
cv2.imshow('Detected Rectangles and Squares', image)
cv2.waitKey(0)

The output is the original image with rectangles and squares outlined in green.

The example demonstrates contour detection where rectangular and square contours with four vertices are highlighted. The thresholded image simplifies the shapes, making it easier to detect contours, and the approximation helps to identify geometric shapes accurately.

Method 2: Hough Line Transform

The Hough Line Transform is a feature extraction technique used to isolate features that can be modeled as lines. By finding intersecting lines, you can infer the presence of rectangles or squares. OpenCV’s cv2.HoughLinesP() is used to detect these lines.

Here’s an example:

import cv2
import numpy as np

# Load image, grayscale, Canny edge detection
image = cv2.imread('image.jpg')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
edges = cv2.Canny(gray, 50, 150, apertureSize=3)

# Use HoughLinesP to detect lines
lines = cv2.HoughLinesP(edges, 1, np.pi/180, 100, minLineLength=100, maxLineGap=10)

# Draw lines on the image
for line in lines:
    x1, y1, x2, y2 = line[0]
    cv2.line(image, (x1, y1), (x2, y2), (0, 255, 0), 2)

# Display the image
cv2.imshow('Hough Lines', image)
cv2.waitKey(0)

The output is the original image with line segments marked in green, indicating potential edges of rectangles or squares.

In this code, Hough Line Transform is applied to detect lines on the edges of shapes. Any intersecting lines would signify the corners of rectangles or squares. The accuracy of line detection can be tweaked by adjusting the parameters for line length and gap.

Method 3: Edge Detection and Morphological Operations

Edge detection combined with morphological operations like dilation can help in identifying rectangles and squares. The edges are detected using Canny edge detection, and then dilated to form closed shapes which contours can be found for. The cv2.Canny() function is crucial in detecting edges, and cv2.dilate() helps in closing the lines.

Here’s an example:

import cv2
import numpy as np

# Load image, grayscale, and Canny edge detection
image = cv2.imread('image.jpg')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
edges = cv2.Canny(gray, 30, 100)

# Dilate to close gaps
dilated = cv2.dilate(edges, None, iterations=3)

# Find and draw contours
contours, _ = cv2.findContours(dilated.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cv2.drawContours(image, contours, -1, (0, 255, 0), 2)

# Display the image
cv2.imshow('Edges & Morphological Transform', image)
cv2.waitKey(0)

The output shows the image with shapes contoured in green lines.

In this approach, the combination of edge detection and morphological operations simplifies the image processing to isolate rectangles and squares efficiently. The dilate function helps ensure continuous boundaries for contours which are then easily identified.

Method 4: Machine Learning

For more complex images, a machine learning approach using an object detection model such as YOLO (You Only Look Once) or SSD (Single Shot Detector) can be trained to detect rectangles and squares. This method requires a dataset to train the model and OpenCV’s DNN module to run the inference.

Here’s an example:

# Pseudocode as this method requires a pre-trained model
import cv2

# Load pre-trained model and image
net = cv2.dnn.readNet('rectangle_and_square_detection.weights', 'config.cfg')
image = cv2.imread('image.jpg')

# Prepare image for the model and run the model
blob = cv2.dnn.blobFromImage(image, 1/255, (416, 416), (0, 0, 0), swapRB=True)
net.setInput(blob)
detections = net.forward()

# Process detections...

# Display the image with detected rectangles and squares

This method involves a more complex process and result interpretation, therefore a full example would exceed the scope of this article.

Machine learning methods are more robust to variations in the image and can handle more complex detection scenarios, which traditional image processing techniques struggle with.

Bonus One-Liner Method 5: Utilizing the Aspect Ratio

For a simple quick identification, the aspect ratio of contours can be checked to differentiate squares from rectangles after finding the contours. Squares have an aspect ratio close to one while rectangles do not.

Here’s an example:

# Example assumes `contours` have been found using one of the above methods
for cnt in contours:
    x, y, w, h = cv2.boundingRect(cnt)
    aspect_ratio = float(w)/h
    if aspect_ratio == 1:
        # It's a square; process accordingly

This line of code filters squares among detected contours by comparing their aspect ratios.

Applying the aspect ratio to differentiate between rectangles and squares post-contour detection is an efficient one-liner method when precision is not the top priority.

Summary/Discussion

  • Method 1: Contour Approximation. Effective for simple images. Struggles with overlapping shapes or noisy backgrounds.
  • Method 2: Hough Line Transform. Good for structured environments. May produce many false positives in cluttered images.
  • Method 3: Edge Detection and Morphological Operations. Useful for creating a clear distinction in images. Can be computational due to morphological operations.
  • Method 4: Machine Learning. Highly accurate and robust. Requires a dataset for training and can be resource-intensive.
  • Bonus Method 5: Aspect Ratio. Quick and easy check. Not reliable for differentiating rectangles with an aspect ratio close to one.