Did you check the news today, or receive an email newsletter from a company? Both modes of communication follow the publish-subscribe communication pattern. This article will show you how to implement your own PubSub system in Python using the flexible PyPubSub library. If you already know about the PubSub concept, feel free to move to the real meat: implementing the Pubsub system in Python!
Otherwise, stay with me to polish up your computer science fundamentals!
PubSub Concept
This type of communication is one way. Information flows from a publisher to the subscribers. The publisher creates information, the subscriber consumes information. Thus, computer scientists call this communication pattern publish and subscribe paradigm – pubsub.
When consuming news, you seek information about specific topics like sports or tech. You do not care too much about the source as long as it satisfies your needs. If you trust the information, understand it, and it is fresh, you keep using the news service.
A pubsub system is able to achieve this. You as a consumer register your information need. The system then delivers relevant information that satisfies your information need.
An example is Google News. Tell them your interested topics and your language preference. Google then pushes relevant news to your smartphone.
The strength of this communication paradigm is that it decouples sender and receiver. They do not even have to know each other. Have you ever received Google News article recommendations from sources you have never even heard from? Google’s pubsub system acts as a broker that takes care of organizing, finding, filtering, and delivering information.
For computers, this decoupling is also great. For instance, your smart home devices may subscribe to a weather service to react quickly. Your smart home device needs to know only one address – of the pubsub system. A huge advantage for systems with millions of participants such as the Internet of Things.
In the following, I’ll provide some Python pubsub systems, code on how to use them, and references for further reading. Check out the system that interests you most!
PyPubSub System Overview
Before we start, you should understand a few concepts of the PyPubSub library. We start with a short discussion on the different roles and features of PyPubSub.
Message Sender
In publish subscribe systems it’s all about the message. Senders send messages, listeners subscribe to messages. You send a pubsub message by calling pub.sendMessage().
Message Listener
Listens to the messages of a given topic. In PyPubSub, message listeners are callables, i.e., functions executed by the PyPubSub system on each message of a given topic.
Message Topic
Every message has a topic. You can define a hierarchy of topics with parent topics and child topics. A parent topic is more generic, a child topic is less generic.
Message Data
A message does not only consist of a topic but also of some data or payload. You can defind arbitrary data as keyword arguments in the sending method pub.sendMessage(topic, **data).
Message Delivery
All sent messages of a given topic are delivered to listeners registered for that topic or any parent topic. You can also listen to all topics by registering to the root of all topics—the parent of all—called ALL_TOPICS.
There are no guarantees regarding the ordering of message delivery. Messages can appear out of order but are delivered synchronously: before a message is delivered to the next listener, the previous listener must terminate its callable listener function. In other words, we don’t run multiple threads on each listener to deliver the messages in parallel. Instead, we deliver the messages one listener at a time.
Message Immutability
Listeners shouldn’t change the content of the message they receive. However, this is not checked by the PyPubSub system, so listeners could, in theory, do this!
Message Direction
The idea of publish subscribe is one-way: message senders send the message to all listeners of this message. However, you could pass a callback function into the message data to allow receivers of a message to respond to the message by calling the callback function.
PyPubSub Installation
This is one of the most popular Python pubsub modules. You can get it here.
Installation: Run the following code in your terminal, shell, or command line (operating system).
pip install PyPubSub
PyPubSub Create Your First App
A simple example of PyPubSub is given by the following app:
from pubsub import pub
def listener_alice(arg):
print('Alice receives news about', arg['headline'])
print(arg['news'])
print()
def listener_bob(arg):
print('Bob receives news about', arg['headline'])
print(arg['news'])
print()
# Register listeners
pub.subscribe(listener_alice, 'football')
pub.subscribe(listener_alice, 'chess')
pub.subscribe(listener_bob, 'football')
# Send messages to all listeners of topics
pub.sendMessage('football', arg={'headline': 'Ronaldo',
'news': 'Sold for $1M'})
pub.sendMessage('chess', arg={'headline': 'AI',
'news': 'AlphaZero beats grandmaster Carlsen'})
First, you create two listeners—that are callable functions that can be executed by the PyPubSub system:
listener_alice(arg)listener_bob(arg)
Both listeners receive data arg that is a dictionary with two keys: 'headline' and 'news'. This could be a push notification with a headline and the actual news string.
Second, you make sure that both listeners subscribe to one or more topic so that they get called as soon as a message regarding the topic is sent. Bob listens to the topic 'football', while Alice listens to two topics 'football' and 'chess'. To achieve this, you use the function pub.subscribe(topic, *data).
Third, you send two “news” messages:
- topic:
'football' - topic:
'chess'
The output is the following:
Alice receives news about Ronaldo Sold for $1M Bob receives news about Ronaldo Sold for $1M Alice receives news about AI AlphaZero beats grandmaster Carlsen
You can see that Alice receives the news messages about both topics 'football' and 'chess' while Bob only receives the news message about the topic 'football'.
This is the most basic use of the publish subscribe pattern in Python.