To speed up my code, I just decided to (finally) dive into Python’s
async with statement. 🚀 In this article, you’ll find some of my learnings – let’s go! 👇
What Is Python Async With?
async with statement is a way to work with asynchronous context managers, which can be really useful when dealing with I/O-bound tasks, such as reading or writing files, making HTTP requests, or interacting with databases. These tasks normally block your program’s execution, involving waiting for external resources. But using
async with, you can perform multiple tasks concurrently! 🎉
import aiohttp import asyncio async def fetch(url): with aiohttp.ClientSession() as session: response = await session.get(url) content = await response.read() print(asyncio.run(fetch("https://example.com")))
But see the problem? This would block the event loop, making your app slower 😒.
The solution is using
async with alongside a context manager that supports it:
import aiohttp import asyncio async def fetch(url): async with aiohttp.ClientSession() as session: response = await session.get(url) content = await response.read() print(asyncio.run(fetch("https://example.com")))
async with, your code won’t block the event loop while working with context managers, making your program more efficient and responsive! 🌟
No worries if you didn’t quite get it yet. Keep reading! 👇👇👇
Python Async With Examples
async with statement is used when you want to run a certain operation concurrently and need to manage resources effectively, such as when dealing with I/O-bound tasks like fetching a web page 🌐.
Let’s jump into an example using an
asyncio-based library called
Here’s how you can make an HTTP GET request using an
async with statement and aiohttp’s
import aiohttp import asyncio async def fetch_page(session, url): async with session.get(url) as response: return await response.text() async def main(): async with aiohttp.ClientSession() as session: content = await fetch_page(session, 'https://example.com') print(content) loop = asyncio.get_event_loop() loop.run_until_complete(main())
In the example above👆, you use an
async with statement in the
fetch_page function to ensure that the response is properly acquired and released. You can see a similar pattern in the
main function, where the
aiohttp.ClientSession is managed using an
async with statement. This ensures that resources such as network connections are handled properly✅.
Now, let’s discuss some key entities in this example:
session: An instance of the
aiohttp.ClientSessionclass, used to manage HTTP requests📨.
url: A variable representing the URL of the web page you want to fetch🌐.
response: The HTTP response object returned by the server🔖.
clientsession: A class provided by the
aiohttplibrary to manage HTTP requests and responses🌉.
text(): A method provided by the
aiohttplibrary to read the response body as text📃.
async withstatement: A special construct to manage resources within an asynchronous context🔄.
Async With Await
In Python, the
await keyword is used with asynchronous functions, which are defined using the
async def syntax. Asynchronous functions, or coroutines, enable non-blocking execution and allow you to run multiple tasks concurrently without the need for threading or multiprocessing.
In the context of the
async with statement,
await is used to wait for the asynchronous context manager to complete its tasks. The
async with statement is used in conjunction with an asynchronous context manager, which is an object that defines
__aexit__() asynchronous methods. These methods are used to set up and tear down a context for a block of code that will be executed asynchronously.
Here’s an example of how
async with and
await are used together:
import aiohttp import asyncio async def fetch(url): async with aiohttp.ClientSession() as session: async with session.get(url) as response: return await response.text() async def main(): url = 'https://example.com/' html = await fetch(url) print(html) loop = asyncio.get_event_loop() loop.run_until_complete(main())
In this example, the
fetch function is an asynchronous function that retrieves the content of a given URL using the
async with statement is used to create an asynchronous context manager for the
await keyword is then used to wait for the response from the
session.get() call to be available, and to retrieve the text content of the response.
Async With Open
By using “
async with open“, you can open files in your asynchronous code without blocking the execution of other coroutines.
When working with async code, it’s crucial to avoid blocking operations. To tackle this, some libraries provide asynchronous equivalents of “
open“, allowing you to seamlessly read and write files in your asynchronous code.
import aiofiles async def read_file(file_path): async with aiofiles.open(file_path, 'r') as file: contents = await file.read() return contents
Here, we use the
aiofiles library, which provides an asynchronous file I/O implementation. With “
async with“, you can open the file, perform the desired operations (like reading or writing), and the file will automatically close when it’s no longer needed – all without blocking your other async tasks. Neat, huh? 🤓
Remember, it’s essential to use an asynchronous file I/O library, like
aiofiles, when working with
async with open. This ensures that your file operations won’t block the rest of your coroutines and keep your async code running smoothly. 💪🏼
Async With Yield
When working with Python’s async functions, you might wonder how to use the
yield keyword within an
async with statement. In this section, you’ll learn how to effectively combine these concepts for efficient and readable code.😊
💡 Recommended: Understanding Python’s
First, it’s essential to understand that you cannot use the standard
yield with async functions. Instead, you need to work with asynchronous generators, introduced in Python 3.6 and PEP 525. Asynchronous generators allow you to yield values concurrently using the
async def keyword and help you avoid blocking operations.🚀
To create an asynchronous generator, you can define a function with the
async def keyword and use the
yield statement inside it, like this:
import asyncio async def asyncgen(): yield 1 yield 2
To consume the generator, use
async for like this:
async def main(): async for i in asyncgen(): print(i) asyncio.run(main())
This code will create a generator that yields values asynchronously and print them using an async context manager. This approach allows you to wrapp an asynchronous generator in a friendly and readable manner.🎉
Now you know how to use the
yield keyword within an
async with statement in Python. It’s time to leverage the power of asynchronous generators in your code!🚀
Async With Return
async with statement is used to simplify your interactions with asynchronous context managers in your code, and yes, it can return a value as well! 😃
When working with
async with, the value returned by the
__enter__() method of an asynchronous context manager gets bound to a target variable. This helps you manage resources effectively in your async code.
async def fetch_data(url): async with aiohttp.ClientSession() as session: async with session.get(url) as response: data = await response.json() return data
In this example, the
response object is the value returned by the context manager’s
.__enter__() method. Once the
async with block is executed, the
response.json() method is awaited to deserialize the JSON data, which is then returned to the caller 👌.
Here are a few key takeaway points about returning values with
async withsimplifies handling of resources in asynchronous code.
- The value returned by the
.__enter__()method is automatically bound to a target variable within the
- Returning values from your asynchronous context managers easily integrates with your async code.
Advanced Usage of Async With
As you become more familiar with Python’s
async with statement, you’ll discover advanced techniques that can greatly enhance your code’s efficiency and readability. This section will cover four techniques:
- Async With Timeout,
- Async With Multiple,
- Async With Lock, and
- Async With Context Manager
Async With Timeout
When working with concurrency, it’s essential to manage timeouts effectively. In
async with statements, you can ensure a block of code doesn’t run forever by implementing an async timeout. This can help you handle scenarios where network requests or other I/O operations take too long to complete.
Here’s an example of how you can define an async with statement with a timeout:
import asyncio import aiohttp async def fetch_page(session, url): async with aiohttp.Timeout(10): async with session.get(url) as response: assert response.status == 200 return await response.read()
This code sets a 10-second timeout to fetch a web page using the
Async With Multiple
You can use multiple
async with statements simultaneously to work with different resources, like file access or database connections. Combining multiple
async with statements enables better resource management and cleaner code:
async def two_resources(resource_a, resource_b): async with aquire_resource_a(resource_a) as a, aquire_resource_b(resource_b) as b: await do_something(a, b)
This example acquires both resources asynchronously and then performs an operation using them.
Async With Lock
Concurrency can cause issues when multiple tasks access shared resources. To protect these resources and prevent race conditions, you can use
async with along with Python’s
import asyncio lock = asyncio.Lock() async def my_function(): async with lock: # Section of code that must be executed atomically.
This code snippet ensures that the section within the
async with block is executed atomically, protecting shared resources from being accessed concurrently.
Async With Context Manager
Creating your own context managers can make it easier to manage resources in asynchronous code. By defining an
async def __aenter__() and an
async def __aexit__() method in your class, you can use it within an
async with statement:
class AsyncContextManager: async def __aenter__(self): # Code to initialize resource return resource async def __aexit__(self, exc_type, exc, tb): # Code to release resource async def demo(): async with AsyncContextManager() as my_resource: # Code using my_resource
This custom context manager initializes and releases a resource within the
async with block, simplifying your asynchronous code and making it more Pythonic.✨
Error Handling and Exceptions
When working with Python’s Async With Statement, handling errors and exceptions properly is essential to ensure your asynchronous code runs smoothly. 🚀 Using try-except blocks and well-structured clauses can help you manage errors effectively.
Be aware of syntax errors, which may occur when using async with statements. To avoid SyntaxError, make sure your Python code is properly formatted and follows PEP 343’s guidelines. If you come across an error while performing an IO operation or dealing with external resources, it’s a good idea to handle it with an
except clause. 😎
In the case of exceptions, you might want to apply cleanup code to handle any necessary actions before your program closes or moves on to the next task. One way to do this is by wrapping your
async with statement in a try-except block, and then including a
finally clause for the cleanup code.
Here’s an example:
try: async with some_resource() as resource: # Perform your IO operation or other tasks here except YourException as e: # Handle the specific exception here finally: # Add cleanup code here
Remember, you need to handle exceptions explicitly in the parent coroutine if you want to prevent them from canceling the entire task. In the case of multiple exceptions or errors, using
asyncio.gather can help manage them effectively. 💪
While working as a researcher in distributed systems, Dr. Christian Mayer found his love for teaching computer science students.
To help students reach higher levels of Python success, he founded the programming education website Finxter.com that has taught exponential skills to millions of coders worldwide. He’s the author of the best-selling programming books Python One-Liners (NoStarch 2020), The Art of Clean Code (NoStarch 2022), and The Book of Dash (NoStarch 2022). Chris also coauthored the Coffee Break Python series of self-published books. He’s a computer science enthusiast, freelancer, and owner of one of the top 10 largest Python blogs worldwide.
His passions are writing, reading, and coding. But his greatest passion is to serve aspiring coders through Finxter and help them to boost their skills. You can join his free email academy here.