Python Read and Write to a Properties File

by

📝 Properties files are simple text files used for configuring parameters and settings. They store data in key-value pairs, so you need them to set up environment-specific configurations when creating software projects.

Example Property File

Here’s an example of a properties file for a web scraping application. This file contains key-value pairs defining various configurations required for the web scraping process.

# Web Scraping Application Properties

# URL to scrape
target_url = https://example.com

# Time in seconds to wait between requests to avoid server overload
request_interval = 2

# User agent to simulate browser behavior
user_agent = Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3

# Maximum number of retries for a failed request
max_retries = 3

# Timeout in seconds for web requests
request_timeout = 5

# Proxy settings (if required)
proxy_enabled = False
proxy_url = http://proxyserver:port

# Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
log_level = INFO

In this properties file:

  • target_url specifies the URL to be scraped.
  • request_interval ensures a delay between requests to prevent overloading the server.
  • user_agent defines the user agent string to be used for requests, mimicking a real browser.
  • max_retries sets the number of attempts for a failed request.
  • request_timeout is the time limit for each request.
  • proxy_enabled and proxy_url are used if accessing the target URL requires a proxy.
  • log_level sets the verbosity of log messages.

These settings are adjustable based on the specific needs of the web scraping application.

Reading a Properties File in Python

Python’s built-in functions make reading from a properties file straightforward.

def read_properties(file_path):
    properties = {}
    with open(file_path, 'r') as file:
        for line in file:
            if line.startswith('#') or not line.strip():
                continue  # Skip comments and blank lines
            key, value = line.strip().split('=', 1)
            properties[key] = value
    return properties

This function reads each line, skips comments and empty lines, and extracts key-value pairs.

When printed directly, the output will be in Python’s dictionary format. Here’s how the printed output of the properties dictionary would look:

{
    'target_url': 'https://example.com',
    'request_interval': '2',
    'user_agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3',
    'max_retries': '3',
    'request_timeout': '5',
    'proxy_enabled': 'False',
    'proxy_url': 'http://proxyserver:port',
    'log_level': 'INFO'
}

This format clearly shows each key-value pair as a Python dictionary, which is very useful for further processing or configuration within a Python script.

Modifying a Properties File

Altering a properties file involves reading the existing content, modifying it, and writing it back.

def modify_property(file_path, key, new_value):
    properties = read_properties(file_path)
    properties[key] = new_value
    with open(file_path, 'w') as file:
        for key, value in properties.items():
            file.write(f'{key}={value}\n')

Here, we read the properties, update the relevant key, and write back the modified content.

Putting It All Together

Here’s the complete Python script that combines everything: creating a mock properties file, reading the properties with the read_properties function, and then printing out the properties in a readable format.

This serves as a quick and easy example of how to handle properties files in Python for a web scraping application.

# Python Script to Read Properties from a File

def read_properties(file_path):
    """
    Reads a properties file and returns the properties as a dictionary.
    """
    properties = {}
    with open(file_path, 'r') as file:
        for line in file:
            line = line.strip()
            if line.startswith('#') or not line:
                continue  # Skip comments and blank lines
            key, value = [part.strip() for part in line.split('=', 1)]
            properties[key] = value
    return properties

# Example Properties File Content
properties_content = """
# Web Scraping Application Properties

# URL to scrape
target_url = https://example.com

# Time in seconds to wait between requests to avoid server overload
request_interval = 2

# User agent to simulate browser behavior
user_agent = Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3

# Maximum number of retries for a failed request
max_retries = 3

# Timeout in seconds for web requests
request_timeout = 5

# Proxy settings (if required)
proxy_enabled = False
proxy_url = http://proxyserver:port

# Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
log_level = INFO
"""

# Write the properties content to a temporary file
import tempfile

with tempfile.NamedTemporaryFile(delete=False, mode='w') as temp_file:
    temp_file.write(properties_content)
    temp_file_path = temp_file.name

# Read the properties from the file
properties = read_properties(temp_file_path)

# Print the properties
print("Properties read from the file:")
for key, value in properties.items():
    print(f"{key}: {value}")

# Optionally, you can delete the temporary file after reading
import os
os.remove(temp_file_path)

This script:

  1. Defines the read_properties function for reading properties files.
  2. Creates a mock properties file with relevant content for a web scraping application.
  3. Writes this content to a temporary file.
  4. Reads the properties from the file using read_properties.
  5. Prints the properties in a readable format.
  6. Deletes the temporary file after use.

You can run this script as is to see how it processes the properties file. The output will display the parsed key-value pairs from the properties content.