Have you ever wondered what happens in the system when you type https://app.finxter.com/ or https://google.com and press enter in your web browser? This is exactly what we will be covering in Python Networking.
- How the data flows from HTTP protocol to TCP/IP protocol stack. Then finally over the internet to fetch the data you requested.
- We discuss the TCP/IP or IP stack overview, followed by an overview of sockets.
- Then we set up a TCP/IP server and client using sockets.
- Finally, we set up a UDP/IP server and client again using sockets.
You can watch the accompanying video while going over the tutorial:
TCP/IP Stack
The TCP/IP stack is an internet protocol stack composed of multiple protocols from FTP, HTTP, ICMP, ARP etc. see Fig 1 below shows IP stack.
Let’s start with these layers from bottom to top and peel them one by one.
Network Access Layer
Also called Link layer or layer 2, it mostly comprises standard Ethernet frames called 802.3. The Ethernet driver of the network card handles these frames. It has protocols like ARP (Address Resolution Protocol) for local area network communication.
Internet Layer
The Internet protocol layer or IP layer handles IP related addressing. Routing also called layer 3, handled here. The responsibility of this layer involves sending data from host to destination. IP routing is having the intermediate devices between two endpoints send the packets between them based on the IP header. The layer adds an IP header to the packet. The IP header can be either IPv4 or IPv6.
Transport Layer
The transport layer is either TCP (Transmission Control Protocol) or UDP (User Datagram Protocol). TCP handles delivering the data between two nodes in a reliable and error-checked manner. TCP guarantees delivery as it is a connection oriented protocol. It uses acknowledgements for every packet received to achieve this. UDP is a connectionless oriented protocol and does not guarantee delivery. It is ideal for applications that prefer faster data delivery.
Application Layer
This layer has a number of application protocols like HTTP, FTP etc which make use of TCP or UDP to deliver data. Your browser makes use of these protocols to connect to any website you want to connect to.
Sockets in Networking
Network sockets help contact between processes on the same or distant machines. Sockets establish point-to-point channels of communication between client and server. Sockets use the capabilities of the operating system for communication. A socket is like a file descriptor in UNIX.
We can classify sockets as stream (TCP) or datagram (UDP).Based on IP addresses sockets are of type AF INET for IPv4 addresses or AF INET6 for IPv6 addresses. The socket module of python provides all the interfaces to write UDP and TCP clients and servers.
- Server: Represents an application waiting for a connection by a client
- Client: Represents an application that connects to the server
Some general socket methods used in setting up client and servers
To create a socket for e.g. use
socket.socket(socket.AF_INET,socket.SOCK_STREAM)
Some other socket methods commonly used for e.g.
socket.recv(buflen) | Receive data from socket with given length |
socket.recvfrom(buflen) | Receive data and the senders address |
socket.recv_into(buffer) | Receive data into a buffer |
socket.recvfrom_into(buffer) | Receives data and address into a buffer |
socket.send(bytes) | Send bytes of data to the destination |
socket.sendto(data, address) | Sends data to a given address |
socket.sendall(data) | Send all the data in the buffer to the socket |
socket.close() | Close socket and release the memory |
TCP/IP Client and Server
Let’s start building a TCP client and server using the above socket api’s.
TCP server
Below code sets up a TCP server.
import socket
# TCP socket server. Waits for incoming connections from #clients. After connection receives the data and sends some data back # This address is your local host i.e. your own device where the program runs. As TCP is a connection-oriented protocol, after connection, it does not change
SERVER_IP = "127.0.0.1"
SERVER_PORT = 9998
# STEP 1 #Create a TCP socket
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# STEP 2 # Bind to the server IP and Port
server.bind((SERVER_IP, SERVER_PORT))
# STEP 3 # Listen to maximum number of incoming connections (here 10)
server.listen(10)
# STEP 4 # Accept incoming connection from client, its a blocking wait print("Waiting for connection:")
socket_client, (host, port) = server.accept()
# STEP 5 # Collect the received data from client
received_data = socket_client.recv(1024) # printing out the received data
print("Received data: ", received_data)
# STEP 6
# Send some data back to the client. We use bytes()
# when we need to send data over the network as we cannot send #string/chars directly.
socket_client.send(bytes("Rxd data, thanks!.".encode('utf-8'))) server.close()We import the socket module. We choose to run the server on the local host (same as where the client runs) for the demo, thus we use the address as 127.0.0.1. If you choose to run on a remote machine, then we must use the remote machine’s IP address.
These are the steps for TCP server
- Create a socket object for the server
socket.socket()with params for IPV4 (AF_INET) and TCP (stream). - Next is
bind()allows you to associate a host and a port with a specific socket. - Then we start to
listen()which takes a parameter for the max number of client connections we can accept. accept()will wait for incoming connections and is a blocking call. It blocks until a response arrives.- Finally use
recv()andsend()to receive data from/to the client.
TCP Client
Below code for TCP client
import socket
# TCP socket client. Connect to a server and send some data.
SERVER_IP = "127.0.0.1" SERVER_PORT = 9998
# STEP 1 Create socket for TCP
socket_client =socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# STEP 2 Connect to the server
socket_client.connect((SERVER_IP, SERVER_PORT)) # Send data to the server
socket_client.send(bytes("Hello from client!".encode('utf-8')))
# STEP 3 Receive some data if server sends
msg = socket_client.recv(1024)
print("From Server:", msg)
# STEP 4
socket_client.close() # Close the connectionThese are the steps for client
- Create a socket object for client
socket.socket() - Connect to the server which on this occasion is the local host with 127.0.0.1 and on port 9998. Please use the correct IP address and port if the server is remote. After connecting, you can now send data to the server.
- Receive some data from the server using
recv() - Finally
close()the client connection
Note: In this client and server we are only sending and receiving data once. Yet if you wish to send or receive large amounts of data then one needs a while loop for both client and server. You can also create and use threads in server and client.
UDP Client and Server
The UDP client and server are exactly like TCP, except that here we don’t wait for connections as UDP has data grams.
UDP Server
Below code for UDP server, the steps are self explanatory
import socket
# UDP socket server. Receives the data and sends some
# data back. UDP is a connectionless ("fire-and-forget") protocol.
# With recvfrom() we know to whom you should send data back
SERVER_IP = "127.0.0.1" # This is local host i.e your own machine SERVER_PORT = 6769
# STEP 1 Create a UDP socket
server = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
# STEP 2 Bind to the server IP and Port
server.bind((SERVER_IP, SERVER_PORT))
# STEP 3 Collect the received data from client
data, address = server.recvfrom(4096)
print("Received data: ", data)
# STEP 4 ,send back the data to the client. encode() for sending bytes instead of string received = server.sendto(bytes("I am the UDP server. Thanks for data...".encode('UTF-8')), address) server.close()Notice the use of SOCK_DGRAM. As UDP is connectionless we save the address of the client in a variable like address when we receive data.
UDP Client
Below code for UDP client
import socket
# UDP socket client. Connect to a server and send some data.
SERVER_IP = "127.0.0.1"
SERVER_PORT = 6769
# STEP 1 Create socket for UDP
socket_client = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
# STEP 2 Send data to server
address = (SERVER_IP, SERVER_PORT)
socket_client.sendto(bytes("Hello from UDP client.!".encode('UTF-8')), address)
# STEP 3
server_response, addr = socket_client.recvfrom(4096)
print("Server response:", server_response)
# STEP 4 Close the socket, terminates all communication
socket_client.close()We have to close the socket after its usage.
Summary
In this tutorial we learnt about python sockets for TCP and UDP.
An example usage of TCP is email or p2p application and UDP is video streaming. We reviewed sockets for networking. We implemented TCP and UDP servers and clients as real life use cases. You can also create your own chat applications using client and server. Hope this blog gives a head start for networking in python. You can further explore by setting up your own HTTP server and client and exchange data with no browser.
For further networking API’s see https://docs.python.org/3/library/socket.html