5 Best Ways to Get Emotions of Images Using Microsoft Emotion API in Python

by

💡 Problem Formulation: The challenge is to analyze images and detect emotions of the faces within them using Python and the Microsoft Emotion API. As an input, we have an image containing human faces, and the desired output is a list of identified emotions per face, such as happiness, sadness, surprise, etc.

Method 1: Using requests Library

This method involves using the ‘requests’ Python library to send an HTTP POST request directly to the Microsoft Emotion API with the image data. It requires setting up the API endpoint, subscription key, and content-type headers. The function should then parse the JSON response to retrieve the emotions.

Here’s an example:

import requests

api_endpoint = "https://YOUR_API_REGION.api.cognitive.microsoft.com/face/v1.0/detect"
subscription_key = "Your Subscription Key Here"
headers = {'Ocp-Apim-Subscription-Key': subscription_key, 'Content-Type': 'application/octet-stream'}
image_data = open('path_to_image.jpg', 'rb').read()

response = requests.post(api_endpoint, headers=headers, data=image_data)
emotions = response.json()
print(emotions)

Output:

[{'faceId': '1', 'faceRectangle': {...}, 'faceAttributes': {'emotion': {'anger': 0.0, 'contempt': 0.01, 'disgust': 0.0, ...}}}]

This code snippet first sets up the necessary information for the API call such as the endpoint and headers. Then, it opens the image and sends it as part of the POST request to the API. Finally, the script prints out the JSON response which contains the identified emotions.

Method 2: Using the Cognitive Face Python SDK

The Cognitive Face Python SDK is Microsoft’s official SDK for interacting with their Face API, which includes the emotion detection. This method abstracts the direct API interactions into simple function calls. Proper setup beforehand with your API key is required.

Here’s an example:

import cognitive_face as CF

KEY = 'Your Subscription Key Here'
CF.Key.set(KEY)
BASE_URL = 'https://YOUR_API_REGION.api.cognitive.microsoft.com/face/v1.0/'
CF.BaseUrl.set(BASE_URL)

img_url = "https://example.com/image.jpg"
attributes = 'emotion'
result = CF.face.detect(img_url, attributes=attributes)
print(result)

Output:

[{'faceId': '1', 'faceRectangle': {...}, 'faceAttributes': {'emotion': {'anger': 0.0, 'contempt': 0.01, 'disgust': 0.0, ...}}}]

This snippet uses the Cognitive Face Python SDK to simplify the process of accessing the Microsoft Face API. It sets up the API key and endpoint, specifies the image and desired attributes, and makes a call to obtain the emotions. The result is printed as a list of dictionaries containing the emotions detected.

Method 3: Asynchronously with asyncio and aiohttp

Asynchronous requests can significantly speed up the process when dealing with multiple images. This method incorporates Python’s ‘asyncio’ module together with ‘aiohttp’ to send non-blocking HTTP requests to the Microsoft Emotion API.

Here’s an example:

import aiohttp
import asyncio
import json

async def get_emotion(session, image_path):
    api_endpoint = "https://YOUR_API_REGION.api.cognitive.microsoft.com/face/v1.0/detect"
    subscription_key = "Your Subscription Key Here"
    headers = {'Ocp-Apim-Subscription-Key': subscription_key, 'Content-Type': 'application/octet-stream'}

    with open(image_path, "rb") as image:
        response = await session.post(api_endpoint, data=image, headers=headers)
        return await response.json()

async def main():
    async with aiohttp.ClientSession() as session:
        emotions = await get_emotion(session, 'path_to_image.jpg')
        print(emotions)

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

Output:

[{'faceId': '1', 'faceRectangle': {...}, 'faceAttributes': {'emotion': {'anger': 0.0, 'contempt': 0.01, 'disgust': 0.0, ...}}}]

By utilizing ‘asyncio’ and ‘aiohttp’, this method allows for concurrent API requests, which is especially useful when analyzing multiple images. The code defines an asynchronous function to post image data to the API and fetch emotions, then runs this task in an event loop.

Method 4: Handling Images in Memory with io.BytesIO

Images sometimes need to be processed in memory rather than saved to disk. This method uses Python’s ‘io’ module to handle image data as a byte stream with ‘BytesIO’, which can be sent directly to the Microsoft Emotion API.

Here’s an example:

import requests
import io

api_endpoint = "https://YOUR_API_REGION.api.cognitive.microsoft.com/face/v1.0/detect"
subscription_key = "Your Subscription Key Here"
headers = {'Ocp-Apim-Subscription-Key': subscription_key, 'Content-Type': 'application/octet-stream'}

# Assume 'image_data' is of type bytes representing an image.
image_stream = io.BytesIO(image_data)
response = requests.post(api_endpoint, headers=headers, data=image_stream)
emotions = response.json()
print(emotions)

Output:

[{'faceId': '1', 'faceRectangle': {...}, 'faceAttributes': {'emotion': {'anger': 0.0, 'contempt': 0.01, 'disgust': 0.0, ...}}}]

This code showcases how to prepare image data in memory for an API request. It utilizes ‘io.BytesIO’ to create a byte stream object from image data, which is then sent in the HTTP POST request. This approach is suitable when dealing with images as data streams instead of files.

Bonus One-Liner Method 5: Using the requests Library with Image URLs

For simplicity, the Microsoft Emotion API also accepts image URLs directly. This one-liner uses the ‘requests’ library to send a GET request with the image URL as a query parameter.

Here’s an example:

import requests

api_endpoint = "https://YOUR_API_REGION.api.cognitive.microsoft.com/face/v1.0/detect"
subscription_key = "Your Subscription Key Here"
headers = {'Ocp-Apim-Subscription-Key': subscription_key}

params = {
    'returnFaceId': 'true',
    'returnFaceLandmarks': 'false',
    'returnFaceAttributes': 'emotion',
}

response = requests.post(api_endpoint, params=params, json={"url": "https://example.com/image.jpg"}, headers=headers)
print(response.json())

Output:

[{'faceId': '1', 'faceRectangle': {...}, 'faceAttributes': {'emotion': {...}}}]

This one-liner demonstrates the ease of use provided by the ‘requests’ library to make a GET request to the API with an image URL. Remember to include the proper query parameters to specify that you want to receive emotional attributes in the response.

Summary/Discussion

  • Method 1: Using requests Library. Strength: Direct control over the HTTP request. Weakness: Manual setup of headers and request parameters.
  • Method 2: Using the Cognitive Face Python SDK. Strength: Simplified and abstracted functions. Weakness: Less flexible and dependent on the maintenance of the SDK.
  • Method 3: Asynchronously with asyncio and aiohttp. Strength: Great for handling multiple requests concurrently. Weakness: Complexity increases with the asynchronous programming model.
  • Method 4: Handling Images in Memory with io.BytesIO. Strength: Efficient handling of data streams. Weakness: Requires understanding of byte stream manipulation.
  • Method 5: Using the requests Library with Image URLs. Strength: Simple and quick for accessible online images. Weakness: Not suitable for local or protected images.