Python Networking with Sockets

Rate this post

Have you ever wondered what happens in the system when you type https://app.finxter.com/ or https://google.com and press enter in your web browser? This is exactly what we will be covering in Python Networking.

  • How the data flows from HTTP protocol to TCP/IP protocol stack. Then finally over the internet to fetch the data you requested.
  • We discuss the TCP/IP or IP stack overview, followed by an overview of sockets.
  • Then we set up a TCP/IP server and client using sockets.
  • Finally, we set up a UDP/IP server and client again using sockets.

You can watch the accompanying video while going over the tutorial:

TCP/IP Stack

The TCP/IP stack is an internet protocol stack composed of multiple protocols from FTP, HTTP, ICMP, ARP etc. see Fig 1 below shows IP stack.

Let’s start with these layers from bottom to top and peel them one by one.

Network Access Layer

Also called Link layer or layer 2, it mostly comprises standard Ethernet frames called 802.3. The Ethernet driver of the network card handles these frames. It has protocols like ARP (Address Resolution Protocol) for local area network communication.

Internet Layer

Fig 1: IP stack [Image credit: https://www.w3.org]

The Internet protocol layer or IP layer handles IP related addressing. Routing also called layer 3, handled here. The responsibility of this layer involves sending data from host to destination. IP routing is having the intermediate devices between two endpoints send the packets between them based on the IP header. The layer adds an IP header to the packet. The IP header can be either IPv4 or IPv6.

Transport Layer

The transport layer is either TCP (Transmission Control Protocol) or UDP (User Datagram Protocol). TCP handles delivering the data between two nodes in a reliable and error-checked manner. TCP guarantees delivery as it is a connection oriented protocol. It uses acknowledgements for every packet received to achieve this. UDP is a connectionless oriented protocol and does not guarantee delivery. It is ideal for applications that prefer faster data delivery.

Application Layer

This layer has a number of application protocols like HTTP, FTP etc which make use of TCP or UDP to deliver data. Your browser makes use of these protocols to connect to any website you want to connect to.

Sockets in Networking

Network sockets help contact between processes on the same or distant machines. Sockets establish point-to-point channels of communication between client and server. Sockets use the capabilities of the operating system for communication. A socket is like a file descriptor in UNIX.

We can classify sockets as stream (TCP) or datagram (UDP).Based on IP addresses sockets are of type AF INET for IPv4 addresses or AF INET6 for IPv6 addresses. The socket module of python provides all the interfaces to write UDP and TCP clients and servers.

  • Server: Represents an application waiting for a connection by a client
  • Client:  Represents an application that connects to the server

Some general socket methods used in setting up client and servers

To create a socket for e.g. use

socket.socket(socket.AF_INET,socket.SOCK_STREAM)

Some other socket methods commonly used for e.g.

socket.recv(buflen)Receive data from socket with given length
socket.recvfrom(buflen)Receive data and the senders address
socket.recv_into(buffer)Receive data into a buffer
socket.recvfrom_into(buffer)Receives data and address into a buffer
socket.send(bytes)Send bytes of data to the destination
socket.sendto(data, address)Sends data to a given address
socket.sendall(data)Send all the data in the buffer to the socket
socket.close()Close socket and release the memory

TCP/IP Client and Server

Let’s start building a TCP client and server using the above socket api’s.

TCP server

Below code sets up a TCP server.

import socket

# TCP socket server. Waits for incoming connections from #clients. After connection receives the data and sends some data back # This address is your local host i.e. your own device where the program runs. As TCP is a connection-oriented protocol, after connection, it does not change
SERVER_IP = "127.0.0.1"
SERVER_PORT = 9998

# STEP 1 #Create a TCP socket
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# STEP 2 # Bind to the server IP and Port
server.bind((SERVER_IP, SERVER_PORT))
# STEP 3 # Listen to maximum number of incoming connections (here 10)
server.listen(10)

# STEP 4 # Accept incoming connection from client, its a blocking wait print("Waiting for connection:")
socket_client, (host, port) = server.accept()

# STEP 5 # Collect the received data from client
received_data = socket_client.recv(1024) # printing out the received data
print("Received data: ", received_data)

  # STEP 6
# Send some data back to the client. We use bytes()
# when we need to send data over the network as we cannot send #string/chars directly.
socket_client.send(bytes("Rxd data, thanks!.".encode('utf-8'))) server.close()

We import the socket module. We choose to run the server on the local host (same as where the client runs) for the demo, thus we use the address as 127.0.0.1. If you choose to run on a remote machine, then we must use the remote machine’s IP address.

These are the steps for TCP server

  • Create a socket object for the server socket.socket() with params for IPV4 (AF_INET) and TCP (stream).
  • Next is bind() allows you to associate a host and a port with a specific socket.
  • Then we start to listen() which takes a parameter for the max number of client connections we can accept.
  • accept() will wait for incoming connections and is a blocking call. It blocks until a response arrives.
  • Finally use recv() and send() to receive data from/to the client.

TCP Client

Below code for TCP client

import socket

# TCP socket client. Connect to a server and send some data.
SERVER_IP = "127.0.0.1" SERVER_PORT = 9998
# STEP 1   Create socket for TCP
socket_client =socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# STEP 2  Connect to the server
socket_client.connect((SERVER_IP, SERVER_PORT)) # Send data to the server
socket_client.send(bytes("Hello from client!".encode('utf-8')))

# STEP 3  Receive some data if server sends
msg = socket_client.recv(1024)
print("From Server:", msg)

# STEP 4
socket_client.close()    # Close the connection

These are the steps for client

  • Create a socket object for client socket.socket()
  • Connect to the server which on this occasion is the local host with 127.0.0.1 and on port 9998. Please use the correct IP address and port if the server is remote. After connecting, you can now send data to the server.
  • Receive some data from the server using recv()
  • Finally close() the client connection

Note: In this client and server we are only sending and receiving data once. Yet if you wish to send or receive large amounts of data then one needs a while loop for both client and server. You can also create and use threads in server and client.

UDP Client and Server

The UDP client and server are exactly like TCP, except that here we don’t wait for connections as UDP has data grams.

UDP Server

Below code for UDP server, the steps are self explanatory

import socket

# UDP socket server. Receives the data and sends some
# data back.  UDP is a connectionless ("fire-and-forget") protocol.
# With recvfrom() we know to whom you should send data back


SERVER_IP = "127.0.0.1"  # This is local host i.e your own machine SERVER_PORT = 6769

# STEP 1 Create a UDP socket
server = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
# STEP 2  Bind to the server IP and Port
server.bind((SERVER_IP, SERVER_PORT))
# STEP 3 Collect the received data from client
data, address = server.recvfrom(4096)
print("Received data: ", data)
# STEP 4 ,send back the data to the client. encode() for sending bytes instead of string received = server.sendto(bytes("I am the UDP server. Thanks for data...".encode('UTF-8')), address) server.close()

Notice the use of SOCK_DGRAM. As UDP is connectionless we save the address of the client in a variable like address when we receive data.

UDP Client

Below code for UDP client

import socket
# UDP socket client. Connect to a server and send some data.
SERVER_IP = "127.0.0.1"
SERVER_PORT = 6769

# STEP 1  Create socket for UDP
socket_client = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

# STEP 2  Send data to server
address = (SERVER_IP, SERVER_PORT)
socket_client.sendto(bytes("Hello from UDP client.!".encode('UTF-8')), address)

# STEP 3
server_response, addr = socket_client.recvfrom(4096)
print("Server response:", server_response)

# STEP 4   Close the socket, terminates all communication
socket_client.close()

We have to close the socket after its usage.

Summary

In this tutorial we learnt about python sockets for TCP and UDP.

An example usage of TCP is email or p2p application and UDP is video streaming. We reviewed sockets for networking. We implemented TCP and UDP servers and clients as real life use cases. You can also create your own chat applications using client and server. Hope this blog gives a head start for networking in python. You can further explore by setting up your own HTTP server and client and exchange data with no browser.

For further networking API’s see https://docs.python.org/3/library/socket.html