5 Best Ways to Implement Command Line File Downloader in Python

For developers and end-users alike, downloading files from the internet is a common task. In Python, achieving this from the command line enables the automation of this process, which is especially useful for scripted or batch operations. You might, for example, want to download a CSV file using a Python script that you can later process automatically. The desired output would be the file saved to your local disk.

Method 1: Using the urllib.request Module

The urllib.request module in Python is a powerful HTTP client that allows you to send HTTP requests and handle responses. It provides a high-level interface that makes it easy to fetch data across the web, including file downloads.

Here’s an example:

import urllib.request

def download_file(url, filename):
    urllib.request.urlretrieve(url, filename)

download_file('http://example.com/somefile.pdf', 'localfile.pdf')

Output:

The specified ‘somefile.pdf’ will be downloaded from the example website and saved locally as ‘localfile.pdf’.

This code snippet defines a function download_file() that accepts a URL and a local filename as arguments, using urlretrieve() from urllib.request to download the file from the web and store it with the specified local name.

Method 2: Using the requests Module

The requests module is a simple HTTP library for Python, which allows you to easily send HTTP/1.1 requests. It is famously ‘HTTP for Humans’. This module also supports file downloads with a clean and simple API.

Here’s an example:

import requests

def download_file(url, filename):
    response = requests.get(url)
    with open(filename, 'wb') as f:
        f.write(response.content)

download_file('http://example.com/somefile.pdf', 'localfile.pdf')

Output:

The ‘somefile.pdf’ will be fetched and saved as ‘localfile.pdf’ on your local disk.

The download_file() function initiates a GET request to the specified URL using requests.get(). It then saves the content of the response to a local file by opening the file in write-binary (‘wb’) mode.

Method 3: Using the wget Module

The wget module is a third-party library designed to mimic the GNU Wget functionality in Python. It’s typically used for non-interactive downloads, such as in scripts or automated processes.

Here’s an example:

import wget

url = 'http://example.com/somefile.pdf'
filename = wget.download(url)
print(f'Downloaded: {filename}')

Output:

Indicates the completion of the download and prints ‘Downloaded: somefile.pdf’.

The above example demonstrates the simplicity of using the wget module. Calling wget.download() with the URL will download the file to the current directory and return the filename.

Method 4: Using the http.client Module

Python’s http.client module provides low-level classes for HTTP services but can be used effectively for file downloading. While it offers more control, it is also more verbose.

Here’s an example:

import http.client
import os

def download_file(host, path, filename):
    conn = http.client.HTTPConnection(host)
    conn.request('GET', path)
    response = conn.getresponse()

    if response.status == 200:
        with open(filename, 'wb') as f:
            f.write(response.read())
        print(f'Download complete: {filename}')
    else:
        print('Error downloading file')
    
download_file('example.com', '/somefile.pdf', 'localfile.pdf')

Output:

Download complete: localfile.pdf (If successful)

In this case, we establish an HTTPConnection, send a GET request for the file, and upon getting a successful response (HTTP 200), we write the content to a local file. Error handling can be refined further based on specific requirements.

Bonus One-Liner Method 5: Using the os.system()

While not strictly a Python solution, the os.system() function can be used to execute a command in the underlying system’s shell. This could involve using system commands like wget or curl.

Here’s an example:

import os

os.system('wget http://example.com/somefile.pdf -O localfile.pdf')

Output:

This will execute the wget command in the shell, downloading ‘somefile.pdf’ and saving it as ‘localfile.pdf’.

This code will only work if you have wget installed on your system, and it adopts all the risks and idiosyncrasies of shell execution into your Python script.

Summary/Discussion

• Method 1: urllib.request. Straight-forward, part of the standard library. May not support advanced HTTP features.
• Method 2: requests. User-friendly and powerful. Requires an external library.
• Method 3: wget. Emulates GNU Wget. It is an external module that mimics shell commands.
• Method 4: http.client. Offers fine-grained control. Is slightly more complex and lower-level.
• Bonus Method 5: os.system(). Quick one-liner. Platform-dependent, less secure, requires shell tools.