[toc]
In this article, we will learn numerous ways to execute system commands in Python.
➥ Problem: Given an external command that can run on your operating system, how to call the command using a Python script?
➥ Example: Say you want to ping a remote server using your operating system’s ping command—all from within your Python program. How will you do it?
Before learning to execute system commands with Python, you must have an idea about what system commands are.
System Commands In Python
In Python, it is important to coordinate some features that work on various tasks of the system administration. These incorporate discovering the files, running some shell commands, doing some high-level document handling, and more. To do so, we need some approach that helps us to find an interface between the operating system and the Python interpreter.
Whether you are a developer or a system administrator, automation scripts that involve system commands are going to be a common part of your daily routine. These may include automating routine tasks like file backups or health checks. However, maintaining such shell scripts might become tedious and complex. This is where Python scripting comes in handy and opens up a whole new dimension to handle such tasks with ease. But for such scripts to work efficiently, we must learn to invoke external/shell commands in our Python scripts.
Now that we know what a system command is, let’s dive into the different methods to execute system commands with Python.
❖ The OS Module
Python is loaded with valuable tools within the Python standard library. The os module in Python is one of those powerful tools of the Python standard library. It is a standard utility module that helps you to communicate with the operating system.
➠ In other words, the os module in Python provides several functions to interact with the shell and the system directly. Let us have a look at a couple of methods to invoke system commands using the os module.
(1) os.system() – It returns the state code after the execution result is executed in the shell, with 0 that shows a fruitful execution.
(2) os.popen() – It is the immediate return of the execution result that returns a memory address and parses the information in the memory address with read().
os.system()
The os.system() function helps to immediately interact with the shell by passing commands and arguments to the system shell. It returns an exit code upon completion of the command, which means that an exit code of 0 denotes successful execution,re while anything other than 0 means that the execution was unsuccessful. On one hand, this is a convenient approach since the system() method can be used to run multiple commands at once using pipes and input/output redirection while on the other hand, you must manually handle escape characters such as spaces.
Note: You must import the os module in order to utilize it.
✨Example:
# Importing the os module
import os
# System command to Check Python Version
ver = "python --version"
# This method will return the exit status of the command
status = os.system(ver)
print('The returned Value is: ', status)
# System command to run Notepad
editor = 'notepad'
os.system(editor)Output:
The above example returns the Python version and opens Notepad on Windows using command line execution. It also returns the exit code (status = 0), which implies that the program executed the system command successfully.
os.popen()
The os.popen() will behave the same as os.system function. However, instead of returning the return code, it returns a file-like object which can be used to access the standard input/output for the process being executed. The popen command allows us to interact with the system and create a pipe to or from another command. It has four different variants, which are:
- popen
- popen2
- popen3
- popen4
os.popen() will do exactly the same thing as os.system apart from that, it’s anything but a document-like object that you can use to get to standard input or output for that process. On the off chance that you call the os.popen() it returns the output to Python as a string. The string contains numerous strings that have the newline character \n.
Syntax:
| os.popen(command[, mode[, bufsize]]) |
Here,
- command is what you’ll execute, and its output will be accessible through an open file.
- mode characterizes whether this output document is readable ‘r’ or writable ‘w’. To recover the exit code of the command executed, you should use the
exit()method for the document object. - bufsize advises
popenhow much data it can buffer. It accepts one of the following values:- 0 – un-buffered
- 1 – line buffered
- N – estimated buffer size, when
N > 0.
✨Example 1: Let us have a look at the following program which uses popen to print a string as output by invoking the shell command, echo:
import os
print(os.popen("echo Hello FINXTER!").read())Output:
Hello FINXTER!
✨Example 2: The following example illustrates how you can use the os.popen() command to create a new folder in windows using the mkdir command.
# Importing the os module
import os
# This method will store the output
p = os.popen('mkdir new_folder')
# Printing the read value
print(p.read())Output:
❖ The Subprocess Module
The subprocess module accompanies different techniques or functions to produce new processes, associate with their input or output and error pipes, then acquire their return codes.
Just like the os module, you have to import the subprocess module in order to utilize it.
subprocess.call()
The subprocess.call() method takes in the command line arguments. The arguments are passed as a list of strings or with the shell argument set to True. Thus the subprocess.call() function is used to return an exit code which can then be used in the script to determine if the command executed successfully or it returned an error. Any return code other than “0” means that there was an error in execution.
✨Example: Let us have a look at the following program using the call() function to check if the ping test is successful in the system :
import subprocess
return_code = subprocess.call(['ping', 'localhost'])
print("Output of call() : ", return_code)Output:
subprocess.Popen
subprocess.Popen takes into account the execution of a program as a child process. Since this is executed by the operating system as a different process, the outcomes are independent of the platform. The subprocess.Popen is a class and not simply a method. Consequently, when we call subprocess.Popen, we’re really calling the constructor of the class Popen.
Note: The Python documentation suggests the utilization of subprocess.Popen only in advanced cases, when other strategies such like subprocess.call can’t satisfy our necessities.
✨Example:
# Importing the subprocess module
import subprocess
# path will vary for you!
subprocess.Popen('C:\Program Files (x86)\Microsoft Office\Office14\EXCEL.EXE', shell=True)Output:
The above code will execute the command and start the MS EXCEL given that the shell is set to True.
subprocess.run()
In Python 3.5 and above, the run() function is the recommended method of invoking the subprocess module to run an external command.
It is similar to the call() function, however, unlike the call() function it does not return an exception if the underlying process returns an error code. The run() function is more flexible than the call() function and instead of returning a return code, it returns a CompletedProcess object once the code completes its execution.
import subprocess
return_code = subprocess.run(['ping', 'localhost'])
print("Output of run() : ", return_code)Output:
Pinging DESKTOP-PC [::1] with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Ping statistics for::1:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
Output of run() : CompletedProcess(args=['ping', 'localhost'], returncode=0)Conclusion
Thus from the above discussion, we can conclude that the following methods can be used to call an external command in Python :
subprocess.call()functionsubprocess.run()functionsubprocess.PopenClassos.system()functionos.popen()function
The official documentation of Python recommends using the subprocess module over the os module functions since the subprocess module provides several advantages over the os module like more flexible options, better error handling, and much more.
I hope you found this article helpful, and it helps you to understand how to invoke external commands in Python. Stay tuned and subscribe for more interesting discussions in the future.
Post Credits: SHUBHAM SAYON and RASHI AGARWAL
Python One-Liners Book: Master the Single Line First!
Python programmers will improve their computer science skills with these useful one-liners.
Python One-Liners will teach you how to read and write “one-liners”: concise statements of useful functionality packed into a single line of code. You’ll learn how to systematically unpack and understand any line of Python code, and write eloquent, powerfully compressed Python like an expert.
The book’s five chapters cover (1) tips and tricks, (2) regular expressions, (3) machine learning, (4) core data science topics, and (5) useful algorithms.
Detailed explanations of one-liners introduce key computer science concepts and boost your coding and analytical skills. You’ll learn about advanced Python features such as list comprehension, slicing, lambda functions, regular expressions, map and reduce functions, and slice assignments.
You’ll also learn how to:
- Leverage data structures to solve real-world problems, like using Boolean indexing to find cities with above-average pollution
- Use NumPy basics such as array, shape, axis, type, broadcasting, advanced indexing, slicing, sorting, searching, aggregating, and statistics
- Calculate basic statistics of multidimensional data arrays and the K-Means algorithms for unsupervised learning
- Create more advanced regular expressions using grouping and named groups, negative lookaheads, escaped characters, whitespaces, character sets (and negative characters sets), and greedy/nongreedy operators
- Understand a wide range of computer science topics, including anagrams, palindromes, supersets, permutations, factorials, prime numbers, Fibonacci numbers, obfuscation, searching, and algorithmic sorting
By the end of the book, you’ll know how to write Python at its most refined, and create concise, beautiful pieces of “Python art” in merely a single line.