Have you ever wondered what happens in the system when you type https://app.finxter.com/ or https://google.com and press enter in your web browser? This is exactly what we will be covering in Python Networking.
- How the data flows from HTTP protocol to TCP/IP protocol stack. Then finally over the internet to fetch the data you requested.
- We discuss the TCP/IP or IP stack overview, followed by an overview of sockets.
- Then we set up a TCP/IP server and client using sockets.
- Finally, we set up a UDP/IP server and client again using sockets.
You can watch the accompanying video while going over the tutorial:
The TCP/IP stack is an internet protocol stack composed of multiple protocols from FTP, HTTP, ICMP, ARP etc. see Fig 1 below shows IP stack.
Let’s start with these layers from bottom to top and peel them one by one.
Network Access Layer
Also called Link layer or layer 2, it mostly comprises standard Ethernet frames called 802.3. The Ethernet driver of the network card handles these frames. It has protocols like ARP (Address Resolution Protocol) for local area network communication.
The Internet protocol layer or IP layer handles IP related addressing. Routing also called layer 3, handled here. The responsibility of this layer involves sending data from host to destination. IP routing is having the intermediate devices between two endpoints send the packets between them based on the IP header. The layer adds an IP header to the packet. The IP header can be either IPv4 or IPv6.
The transport layer is either TCP (Transmission Control Protocol) or UDP (User Datagram Protocol). TCP handles delivering the data between two nodes in a reliable and error-checked manner. TCP guarantees delivery as it is a connection oriented protocol. It uses acknowledgements for every packet received to achieve this. UDP is a connectionless oriented protocol and does not guarantee delivery. It is ideal for applications that prefer faster data delivery.
This layer has a number of application protocols like HTTP, FTP etc which make use of TCP or UDP to deliver data. Your browser makes use of these protocols to connect to any website you want to connect to.
Sockets in Networking
Network sockets help contact between processes on the same or distant machines. Sockets establish point-to-point channels of communication between client and server. Sockets use the capabilities of the operating system for communication. A socket is like a file descriptor in UNIX.
We can classify sockets as stream (TCP) or datagram (UDP).Based on IP addresses sockets are of type AF INET for IPv4 addresses or AF INET6 for IPv6 addresses. The socket module of python provides all the interfaces to write UDP and TCP clients and servers.
- Server: Represents an application waiting for a connection by a client
- Client: Represents an application that connects to the server
Some general socket methods used in setting up client and servers
To create a socket for e.g. use
Some other socket methods commonly used for e.g.
|Receive data from socket with given length|
|Receive data and the senders address|
|Receive data into a buffer|
|Receives data and address into a buffer|
|Send bytes of data to the destination|
|Sends data to a given address|
|Send all the data in the buffer to the socket|
|Close socket and release the memory|
TCP/IP Client and Server
Let’s start building a TCP client and server using the above socket api’s.
Below code sets up a TCP server.
import socket # TCP socket server. Waits for incoming connections from #clients. After connection receives the data and sends some data back # This address is your local host i.e. your own device where the program runs. As TCP is a connection-oriented protocol, after connection, it does not change SERVER_IP = "127.0.0.1" SERVER_PORT = 9998 # STEP 1 #Create a TCP socket server = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # STEP 2 # Bind to the server IP and Port server.bind((SERVER_IP, SERVER_PORT)) # STEP 3 # Listen to maximum number of incoming connections (here 10) server.listen(10) # STEP 4 # Accept incoming connection from client, its a blocking wait print("Waiting for connection:") socket_client, (host, port) = server.accept() # STEP 5 # Collect the received data from client received_data = socket_client.recv(1024) # printing out the received data print("Received data: ", received_data) # STEP 6 # Send some data back to the client. We use bytes() # when we need to send data over the network as we cannot send #string/chars directly. socket_client.send(bytes("Rxd data, thanks!.".encode('utf-8'))) server.close()
We import the socket module. We choose to run the server on the local host (same as where the client runs) for the demo, thus we use the address as 127.0.0.1. If you choose to run on a remote machine, then we must use the remote machine’s IP address.
These are the steps for TCP server
- Create a socket object for the server
socket.socket()with params for IPV4 (AF_INET) and TCP (stream).
- Next is
bind()allows you to associate a host and a port with a specific socket.
- Then we start to
listen()which takes a parameter for the max number of client connections we can accept.
accept()will wait for incoming connections and is a blocking call. It blocks until a response arrives.
- Finally use
send()to receive data from/to the client.
Below code for TCP client
import socket # TCP socket client. Connect to a server and send some data. SERVER_IP = "127.0.0.1" SERVER_PORT = 9998 # STEP 1 Create socket for TCP socket_client =socket.socket(socket.AF_INET, socket.SOCK_STREAM) # STEP 2 Connect to the server socket_client.connect((SERVER_IP, SERVER_PORT)) # Send data to the server socket_client.send(bytes("Hello from client!".encode('utf-8'))) # STEP 3 Receive some data if server sends msg = socket_client.recv(1024) print("From Server:", msg) # STEP 4 socket_client.close() # Close the connection
These are the steps for client
- Create a socket object for client
- Connect to the server which on this occasion is the local host with 127.0.0.1 and on port 9998. Please use the correct IP address and port if the server is remote. After connecting, you can now send data to the server.
- Receive some data from the server using
close()the client connection
Note: In this client and server we are only sending and receiving data once. Yet if you wish to send or receive large amounts of data then one needs a while loop for both client and server. You can also create and use threads in server and client.
UDP Client and Server
The UDP client and server are exactly like TCP, except that here we don’t wait for connections as UDP has data grams.
Below code for UDP server, the steps are self explanatory
import socket # UDP socket server. Receives the data and sends some # data back. UDP is a connectionless ("fire-and-forget") protocol. # With recvfrom() we know to whom you should send data back SERVER_IP = "127.0.0.1" # This is local host i.e your own machine SERVER_PORT = 6769 # STEP 1 Create a UDP socket server = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # STEP 2 Bind to the server IP and Port server.bind((SERVER_IP, SERVER_PORT)) # STEP 3 Collect the received data from client data, address = server.recvfrom(4096) print("Received data: ", data) # STEP 4 ,send back the data to the client. encode() for sending bytes instead of string received = server.sendto(bytes("I am the UDP server. Thanks for data...".encode('UTF-8')), address) server.close()
Notice the use of
SOCK_DGRAM. As UDP is connectionless we save the address of the client in a variable like address when we receive data.
Below code for UDP client
import socket # UDP socket client. Connect to a server and send some data. SERVER_IP = "127.0.0.1" SERVER_PORT = 6769 # STEP 1 Create socket for UDP socket_client = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # STEP 2 Send data to server address = (SERVER_IP, SERVER_PORT) socket_client.sendto(bytes("Hello from UDP client.!".encode('UTF-8')), address) # STEP 3 server_response, addr = socket_client.recvfrom(4096) print("Server response:", server_response) # STEP 4 Close the socket, terminates all communication socket_client.close()
We have to close the socket after its usage.
In this tutorial we learnt about python sockets for TCP and UDP.
An example usage of TCP is email or p2p application and UDP is video streaming. We reviewed sockets for networking. We implemented TCP and UDP servers and clients as real life use cases. You can also create your own chat applications using client and server. Hope this blog gives a head start for networking in python. You can further explore by setting up your own HTTP server and client and exchange data with no browser.
For further networking API’s see https://docs.python.org/3/library/socket.html