• Join Discord: https://discord.com/invite/Cn7jWhTckP
• Join GitHub: https://github.com/shandralor/PeerBrain

The post PeerBrain – A Decentralized P2P Social Brain Network App appeared first on Be on the Right Side of Change.

Most of you will not know me, so I’ll give you a quick introduction.

About Me

My name is Tom, and for the last 6 months, I’ve been teaching myself programming. Primarily Python with some web frameworks (Flask, FastAPI, Streamlit) thrown in for good measure. I’ve found I love the way you can create something new from an idea or even a stray thought. That fact is also what drove me to start learning programming in the first place.

Every one of us knows the feeling that a certain solution for a certain problem doesn’t quite cover all its bases. If only you could adapt or change a few things… That feeling, that urge to solve the issue, is what drives me. And as I grow more adept in this, I admit that the satisfaction gained from it keeps growing also.

It is the drive to learn that brought me to the Finxter blog and academy. I had been scouring the web, looking for learning platforms, tutorials, etc. After opening the site for the first time and having a quick read, the community aspect immediately appealed to me. The way it both has a thriving community while also being driven by it kind of puts it in its own category. I am quite certain that is why I decided to join the mailing list, looking back.

Fast forward to the start of 2023. Chris sent out the first of his 100 programming challenges for the new year. I jumped on the first challenge to create an interactive web app. I was learning Streamlit at the time, and my application made the shortlist. The email that contained that shortlist also first mentioned the idea for an application created by the community. I got hooked on that idea immediately. I shot off a quick email to Chris saying I would like to take part in this very much, whatever it turned out to be. Little did I know what I had gotten myself into .

The PeerBrain Vision – A Decentralized P2P Social Brain Network

It was a couple of weeks later when Finxter sent out the update email with the idea for Peerbrain/Brainwaves. A decentralized P2P application for sharing messages/tweets. An application without possible interference from governments or companies. I loved it!

The only problem was I had no idea in hell how to go about it. So I started researching existing applications or frameworks that were trying to achieve this. The Matrix project is the biggest one. I dove into the documentation and came out knowing two things. The guys who designed this are very good at what they do. The other thing was that I did not even understand half of what I would need if I was to try and use it. I also did not like using something that was already far along in development if I look back now. I wanted to be able to go through the entire process and see how every part of it worked.

After this, I decided that I could not find something that suited the needs I felt the project had. So I started designing and developing it myself. When brainstorming about how the application should work, I realized we had a problem. An application used by thousands or more people needs an always online component. There needs to be a sort of “base” that will make sure everyone that wants to connect to someone else can do so.

Finding Inspiration in Torrents

I found a lot of inspiration in how torrents work. The torrent tracker is a server that facilitates communication. It is an intermediate between the users (peers) who want to share files with one another. The actual sharing of those files is a P2P process. But the process would not be possible without the help of the trackers.

I transferred this concept to the current project. It meant we needed a server that would allow users to communicate with each other. As Python is my programming language of choice, I needed something that could do this. After some more research, I found the FastAPI framework. It fit all my requirements for a server, and I decided to give it a go. As I had never used it before, more research became my next goal. I am sure some of you will spot a trend by now.

After a few days of coding, and some emails back and forth between Chris and myself, I started to realize some things.

The first was that this project would be way more complex than I ever imagined. I didn’t and still don’t really feel comfortable trying to build it.  I do find that solving certain problems keeps giving me the motivation to continue. As long as this is the case, I will find the motivation to step over my impostor syndrome.

Another thing, and this one is more important, is that the P2P model would not work. Or at least not the part of storing the messages on local machines and not in the cloud. We live in a time when people log in somewhere with certain expectations. They want all their settings already in place, their contacts synced, and their data ready to consume. Storing everything locally would be a huge step back in their eyes. And I am quite sure that if we want this to succeed, we’ll need a lot of people! So it was back to the drawing board to change the application logic to consider this.

Where to Store Messages?

The first thing I am focusing on is the possibility of creating messages/tweets that your friends can read. The solution I found was that storing the messages locally to avoid tampering with them was not viable. Because of this, the messages themselves needed to be protected. If the messages exist on a server, we need to take this into account. This means making sure that even someone who controls the server could not read the contents. This means encryption and, more specifically, end-to-end encryption.

Let me tell you all, encryption is complicated.

I’m sure I am still only skimming the surface of it, but I found a working solution. I got inspiration from how both Apple and Owncloud encrypt user data. In short, the application will use both an RSA keypair and a symmetric key to ensure the security of the messages. The original idea was to only use one API server but this started to pose many problems almost immediately. Messages need to be always available. And new friends want to read messages created before they become friends with the user who created them.

So the solution for this was to split the functionality.

The main API server will store the encrypted messages and user objects. Within those objects, we define the user parameters and the friend relationships. It will handle authentication and user creation.

The second server will hold an object per user that contains the following: a username, a user password hash, their public key and their unencrypted symmetric key. This key is what gets used to encrypt any message the user creates.

Once a user has requested an encrypted copy of a friend’s symmetric key, this will also live within the object, to make future requests more efficient. Otherwise, the second server would need to re-encrypt that key every time a request comes through.

And it is at the end of this very long explanation that my question comes to you, dear Finxter community.

We, a few developers who have already joined, need your help. To develop this further, we need people who can test. And more important, people that break the application! We need to iron out the bugs and test it under load.

Clone the GitHub & Join Discord

That is why we would appreciate it if you clone the Github repo and try to run the client application. We need people to register an account, add friends and create and read messages.

GitHub: https://github.com/shandralor/PeerBrain

If you feel like this applies to you, please join the discord server! You can find all the resources there. You can also ask any question that you have.

I would like to thank you all for reading this and helping us out if you want to!

Sincerely,

Team Peerbrain

The post The PeerBrain Vision – A Decentralized P2P Social Brain Network appeared first on Be on the Right Side of Change.

• Part 1: The P2P Bootstrapping Problem – Starting My Journey To Help Build a P2P Social Network
• Part 2: BrainWaves P2P Social Network – How I Created a Basic Server

I’ve come to realize this might become a rather long series. The main reason for that is that it documents two things. This is the birth of an application and my personal journey in developing that application. I know parts 1 and 2 have been very wordy. This will change now. I promise that you will see a lot of code in this episode :-).

Database Code

So after that slight philosophical tidbit, it is time to dive into the actual database code. As I mentioned in the previous article, I chose to use Deta Space as the database provider. There are two reasons for this. The first is the ease of use and the second are its similarities to my favorite NoSQL database MongoDB.

Recommended: Please check my article on creating a shopping list in Streamlit on how to set it up. It takes only a few minutes.

For reference, the server directory with all the code to get the FastAPI server working looks as follows:

server
├── db.py
├── main.py
├── models.py
├── requirements.txt
├── .env

All the database code will live in the db.py file. For the Pydantic models, I’ll use models.py.

Database Functions

The database functions are roughly divided into three parts.

• We first need functionality for everything related to users.
• Next, we need to code that handles everything related to adding and managing friends.
• The third part applies to all the code for managing thoughts. Thoughts are Peerbrain’s equal of messages/tweets.

The file will also contain some helper functions to aid with managing the public keys of users. I’ll go into a lot more detail on this in the article about encryption.

To set up our db.py file we first need to import everything needed. As before, I’ll show the entire list and then explain what everything does once we write code that uses it.

"""This file will contain all the database logic for our server module. It will leverage the Deta Base NoSQL database api."""


from datetime import datetime
import math
from typing import Union
import os
import logging
from pprint import pprint #pylint: disable=unused-import
from uuid import uuid4
from deta import Deta
from dotenv import load_dotenv
from passlib.context import CryptContext

The #pylint comment you can see above is used to make sure pylint skips this import. I use pprint for displaying dictionaries in a readable way when testing. As I don’t use it anywhere in the actual code, pylint would start to fuss otherwise.

Tip: For those interested, pylint is a great tool to check your code for consistency, errors and, code style. It is static, so it can’t detect errors occurring at runtime. I like it even so.

After having imported everything, I first initialize the database. The load_dotenv() below, first will load all my environment variables from the .env file. 

load_dotenv()


#---DB INIT---#
DETA_KEY = os.getenv("DETA_KEY")
deta = Deta(DETA_KEY)
#---#
USERS = deta.Base("users")
THOUGHTS = deta.Base("thoughts")
KEYS = deta.Base("keys_db")

Once the variables are accessible, I can use the Deta API key to initialize Deta. Creating Bases in Deta is as easy as defining them with deta.Base. I can now call the variable names to perform CRUD operations when needed.

Generate Password Hash

The next part is very important. It will generate our password hash so the password is never readable. Even if someone has control of the database itself, they will not be able to use it. Cryptcontext itself is part of the passlib library. This library can hash passwords in multiple ways..

#---PW ENCRYPT INIT---#
pwd_context = CryptContext(schemes =["bcrypt"], deprecated="auto")
#---#
def gen_pw_hash(pw:str)->str:
    """Function that will use the CryptContext module to generate and return a hashed version of our password"""
   
    return pwd_context.hash(pw)

User Functions

The first function of the user functions is the easiest. It uses Deta’s fetch method to retrieve all objects from a certain Base, deta.users, in our case.

#---USER FUNCTIONS---#
def get_users() -> dict:
    """Function to return all users from our database"""
    try:
        return {user["username"]: user for user in USERS.fetch().items}
    except Exception as e:
        # Log the error or handle it appropriately
        print(f"Error fetching users: {e}")
        return {}

The fact that the function returns the found users as a dictionary makes them easy to use with FastAPI. As we contact a database in this function and all the others in this block, a try–except block is necessary. 

The next two functions are doing the same thing but with different parameters. They accept either a username or an email.

I am aware that these two could be combined into a single function with an if-statement. I still do prefer the two separate functions, as I find them easier to use. Another argument I will make is also that the email search function is primarily an end user function. I plan to use searching by username in the background as a helper function for other functionality.

def get_user_by_username(username:str)->Union[dict, None]:
    """Function that returns a User object if it is in the database. If not it returns a JSON object with the
    message no user exists for that username"""
   
    try:
        if (USERS.fetch({"username" : username}).items) == []:
            return {"Username" : "No user with username found"}
        else:
            return USERS.fetch({"username" : username}).items[0]
    except Exception as error_message:
        logging.exception(error_message)
        return None


def get_user_by_email(email:str)->Union[dict, None]:
    """Function that returns a User object if it is in the database. If not it returns a JSON object with the
    message no user exists for that email address"""
   
    try:
        if (USERS.fetch({"email" : email}).items) == []:
            return {"Email" : "No user with email found"}
        else:
            return USERS.fetch({"email" : email}).items[0]
    except Exception as error_message:
        logging.exception(error_message)
        return None

The functions above both take a parameter that they use to filter the fetch request to the Deta Base users.

If that filtering results in an empty list a proper message is returned. If the returned list is not empty, we use the .items method on the fetch object and return the first item of that list. In both cases, this will be the user object that contains the query string (email or username).

The entire sequence is run inside a try-except block as we are trying to contact a database.

Reset User Password

When working with user creation and databases, a function to reset a user’s password is required. The next function will take care of that.

def change_password(username, pw_to_hash):
    """Function that takes a username and a password in plaintext. It will then hash that password>
    After that it creates a dictionary and tries to match the username to users in the database. If
    successful it overwrites the previous password hash. If not it returns a JSON message stating no
    user could be found for the username provided."""
   
    hashed_pw = gen_pw_hash(pw_to_hash)
    update= {"hashed_pw": hashed_pw }
   
    try:
        user = get_user_by_username(username)
        user_key = user["key"]
        if not username in get_users():
            return {"Username" : "Not Found"}
        else:
           
            return USERS.update(update, user_key), f"User {username} password changed!"
    except Exception as error_message:
        logging.exception(error_message)
        return None  

This function will take a username and a new password. It will first hash that password and then create a dictionary. Updates to a Deta Base are always performed by calling the update method with a dictionary. As in the previous functions, we always check if the username in question exists before calling the update. Also, don’t forget the try-except block!

Create User

The last function is our most important one :-). You can’t perform any operations on user objects if you have no way to create them! Take a look below to check out how we’ll handle that.

def create_user(username:str, email:str, pw_to_hash:str)->None:
    """Function to create a new user. It takes three strings and inputs these into the new_user dictionary. The function then
    attempts to put this dictionary in the database"""


    new_user = {"username" : username,
                "key" : str(uuid4()),
                "hashed_pw" : gen_pw_hash(pw_to_hash),
                "email" : email,
                "friends" : [],
                "disabled" : False}
    try:
        return USERS.put(new_user)
    except Exception as error_message:
        logging.exception(error_message)
        return None

The user creation function will take a username, email, and password for now. It will probably become more complex in the future, but it serves our purposes for now. Like the Deta update method, creating a new item in the database requires a dictionary. Some of the necessary attributes for the dictionary are generated inside the function.

The key needs to be unique, so we use Python’s uuid4 module. The friend’s attribute will contain the usernames of other users but starts as an empty list. The disabled attribute, finally, is set to false. 

After finishing the initialization, creating the object is a matter of calling the Deta put method. I hear some of you thinking that we don’t do any checks if the username or email already exists in the database. You are right, but I will perform these checks on the endpoint receiving the post request for user creation.

Some Coding Thoughts and Learnings

One thing that never ceases to amaze me is the amount of documentation I like to add. I do this first in the form of docstrings as it helps me keep track of what function does what. I find it boring most of the time, but in the end, it helps a lot!

The other part of documenting that I like is type hints. I admit they sometimes confuse me still, but I can see the merit they have when an application keeps growing. 

We will handle the rest of the database function in the next article. See you there!

Participate in Building the Decentralized Social Brain Network

As before, I state that I am completely self-taught. This means I’ll make mistakes. If you spot them, please post them on Discord so I can remedy them . 

As always, feel free to ask me questions or pass suggestions! And check out the GitHub repository for participation!

GitHub: https://github.com/shandralor/PeerBrain

The post My Journey to Help Build a P2P Social Network – Database Code Structure appeared first on Be on the Right Side of Change.

The previous article was a theoretical explanation of how I envision this project. It is now time to start laying the groundwork!

I learn as I go…

As some of you might have guessed already, I’m a completely self-taught coder. Because of that, I’m sure many professionals might not agree with how I code.

I accept that and will welcome any constructive criticism. I have been learning non-stop since I started this project. I assume this will not slow down anytime soon. YouTube is my main source of knowledge as I learn best when seeing something done. I am, in other words, a visual learner. I found an article that explains it well here.

Articles on various sites are the other half of how I learn new concepts in coding. That is how I found Finxter :-).

So to sum it up, my code is far from perfect, and I will never claim it is. This is my take on trying to solve this puzzle. I actually look forward to alternative approaches!

You can open issues on my GitHub if you want to address something.

Now that we all know where we stand let us dive right in! How to build a server for our peer-to-peer social network app?

Flask vs FastAPI

In the previous article, I mentioned that I want to use FastAPI to build the relay server, as opposed to Flask. As I have done before and will do again, I asked ChatGPT about the differences between Flask and FastAPI.

 Flask vs FastAPI

Flask is based on the Werkzeug WSGI (Web Server Gateway Interface) toolkit, which is synchronous by default. However, Flask can still be used to build asynchronous applications. You will need to use a third-party library like gevent or asyncio. With these libraries, Flask can use coroutines and event loops to handle I/O operations asynchronously.

FastAPI, on the other hand, is designed to be fully asynchronous from the ground up. It uses the async/await syntax of Python to write asynchronous code. It is based on the ASGI (Asynchronous Server Gateway Interface) specification. FastAPI uses the Starlette framework as its foundation. the framework provides a high-performance event loop and asynchronous request handlers.

Both the speed and the asynchrony determined my choice for FastAPI.

Those of you familiar with Flask will know about its built-in development server. As FastAPI doesn’t have this, we’ll need to install a separate server.

Uvicorn Server

This is where I encountered my first small hiccup. I code on Windows (I know, sue me ), and I wanted to use Uvicorn. As this only runs on Linux, I needed to get it to function in WSL.

I’ll not go into all the details here, but I could write something about it if anyone has an interest in it. Let me know!

After getting Uvicorn to function as it should, we can continue. It is important to remember that the Python interpreter on WSL does not share anything with its Windows counterpart. This means that you either need two separate virtual environments or that you install pip packages for each OS.

Creating Basic FastAPI App

Once all this annoying prep work is done, creating a basic FastAPI app is very easy. We first import FastAPI as below:

from fastapi import FastAPI

All you need to do afterward is define the basic app and create an endpoint.

#---APP INIT---#
app = FastAPI()
 
@app.get("/")
async def root():
    return {"message": "Hello World"}

To get this to run, you need to navigate to the working directory of your FastAPI project via WSL. Afterward, you call the Uvicorn server. The command below assumes you called your Python file main.py!

uvicorn main:app --reload

I usually run the Uvicorn server in a separate terminal instance of WSL.

That way, I can leave it on and test any changes I make immediately. Later, when I’ll be working on the client also, I can split the terminal. You can then make API calls through the client terminal window. FastAPI’s response in the server WSL window is then visible immediately.

Receiving “Hello World” from Server

If you now navigate to 127.0.0.1:8000 you should get a JSON response with the "Hello World" we returned in the endpoint above. We will change this endpoints function later, but for now, it works to prove our API is working.

For the API server, I have the following layout in mind. It might change throughout the development process. I currently foresee two endpoints that do not require the user to be logged in with a JWT token. The first will be to get that token, and the second to register a new user. Everything else will require the user to be authenticated.

I stated earlier that I would change the root’s endpoint function. Its new role is now to allow a user to request a JWT token. The token is only granted after providing a correct combination of username and password. This requires a dedicated set of both helper functions and Pydantic models to work.

I will go into this in another article, as it requires much explaining :-). It was something I am still learning myself.

Endpoint Layout

The current layout of my endpoints at a high level is the following:

#---OPEN ENDPOINTS---#


#Root route to get token
@app.post("/", response_model=Token)


#Route to create a new user
@app.post("/api/v1/users")


#---AUTH ENDPOINTS---#


#Route to get all current users and their attributes(development only)
@app.get("/api/v1/users")


#Route to test if the token is valid, used while authenticating
@app.get("/api/v1/token-test")


#Route to get all thoughts/messages created by a certain user
@app.get("/api/v1/thoughts/{username}")


#Route to return all thoughts/messages containing the query string
@app.get("/api/v1/thoughts/{query_str}")


#Route to create a new message/thought
@app.post("/api/v1/thoughts")


#Route to return all info about the current user(like a user profile)
@app.get("/api/v1/me", response_model=User)

The current setup should allow for the barebones functionality of the application. At least from a server point of view. The routes above and/or their function are liable to change during development.  I do find it helps to have a visual reminder of what I am working toward. That is why I created the high-level outlay. As you might recall, I am a visual learner .

Database Considerations

I will dedicate the last part of this article to the database part. As we need to store both users, user credentials, and messages/tweets somewhere, a database is a must.

If you have read any of my previous articles, you will know I like Deta a lot.

Their NoSQL databases work great for development. They recently evolved into Deta Space. This change makes their ecosystem even more interesting for developers. The fact that they are free is also important for a single developer coding this app on his own time . Make sure to check them out!

The next article will focus on both the database code and the Pydantic models we will need to get our API to function.

Next Article: My Journey to Help Build a P2P Social Network – Database Code Structure

Join GitHub

As always, feel free to ask me questions or pass suggestions! And check out the GitHub repository for participation!

GitHub: https://github.com/shandralor/PeerBrain

The post BrainWaves P2P Social Network – How I Created a Basic Server appeared first on Be on the Right Side of Change.

When I saw the Finxter announcement about trying to create a decentralized P2P social network, I knew I had to take part.

My interest peaked as it had seldom been before. I love the idea of a social network that can function without interference from companies or governments. It feels like something the world needs.

On why someone did not build it yet, I don’t have a definitive answer. The only thing that occurs to me is that there is no real demand for it (yet). There might actually be some active pushback from major players in the field.

From Matrix to FastAPI

From a financial and monetization point of view, such an application is not very interesting. It would be very difficult to push ads on people as there would be no central control over the network. Which suits me fine :-).

To be honest, there are some standards that are very well developed and so far along that they are production worthy. I’m thinking of the Matrix API.

The problem, at least as far as I am concerned, is that it is difficult to get working. I spent a couple of hours trying to get through the documentation. In the end, I was not sure I understood it better than at the start. So complexity is a definite issue; I’ll have to keep that in mind.

After some emails going back and forth between Chris and myself, we arrived at the following conclusion.

The basic idea for an initial version of the app is a web-based, decentralized tweeting application. So far, so good. Now I needed to do (a lot of ) brainstorming on how to set this up.

As you might have gleaned from my previous articles, Python is my language of choice. I like HTML and CSS too, but not at the same level. So I continued my search for something that could both supply the functionality needed and still allow me to use Python.

The result of that search is the FastAPI framework.

This framework immediately piqued my interest immediately. I’ve been delving ever deeper into its uses, and I am convinced it will work well. The more I learn, the more I am reassured this could be the reliable server-side for the P2P social network we need!

The Role of Servers in P2P Systems

The next question now is: how I can go about doing that?

I discussed this with Chris some more. He posed the justified question of why we need a server for a P2P application.

The problem with P2P at the moment is that we are all still using IPv4. The problem with IPv4 is that we only have a limited amount of addresses available. To address this, there are certain workarounds used everywhere.

Recommended: Peer-to-Peer Communication Across Network Address Translators

NATs and rotating public IPs are the default way of “solving” this problem of scarcity. NATs and their private IP address ranges do pose problems.

When clients want to connect to each other, they need to traverse those NATs. Connecting to the outside from within the NAT is not a problem. Finding someone else behind their NAT is something else.

Added to this comes the fact that providers also rotate IP addresses. So, in the end, you are never certain where to find someone exactly .

Relay Servers in P2P Systems

This is where the “relay” server becomes important. It can be hosted on a server with a static IP, which means it will stay in one place, so to speak. By adding the option to our application to enter an IP or domain name, we will be able to connect to it. Once the connection is active and client details exchanged, we could, in theory, cut out the server.

Direct contact between the two clients would then become possible. It is an approach I will look into more in the future. But for now, I find it to still have too many uncertainties to get it to function reliably. A great article about this can be found here.

So, how to solve this problem?

Let us look back to what we wanted to build. The purpose of this project in its first iteration is the following. We want some version of a “decentralized P2P application that would allow some form of tweeting”.

A Word on Torrents

When I hear P2P, I can’t help but think of torrents. A vital part of torrents is the tracker. According to our good friend ChatGPT, a torrent tracker is:

Definition: A torrent tracker is a server that helps in the communication between peers using the BitTorrent file sharing protocol. It keeps track of the peers participating in a specific torrent “swarm” (a group of peers sharing a particular file. This facilitates the distribution of the file and allows peers to find each other.

It’s important to note that torrent trackers do not host any files themselves. Nor do they guarantee the availability of a particular file. They ease communication between peers and provide information about the location of seeds and peers.

This is the point where I am of two minds.

Implementing the application as a true P2P torrent equivalent could certainly work well. It would, however depend on the app running constantly. This is because it needs to serve its “tweets” or messages when requested by other interested clients. It would also need a constant network connection.

The other idea I am playing with is finding a way that would allow for the storing of messages on the relay server.

They would need to be encrypted, so it would be impossible for someone who had or gained control of the server to read them.

Encrypting Messages

For this to work, a cryptographic key must be exchanged when you “befriend” someone on the system. This key would allow for the decoding of those messages when you request them from the server. After decoding, you could then read them.

To me, the second approach feels more reliable to put in place. Especially when considering that it will allow for a proof of concept to be developed easier and faster.

Reliability is a big factor in that. Having decided this, I know what I have to do!

Stay Tuned

The second article in this series will focus on coding out both the basic server and CLI client. See you soon!

Join the decentralized P2P project on GitHub!

You can also check out the next tutorial on the Finxter blog about how I chose a server for the P2P project.

Recommended: BrainWaves P2P Social Network – How I Created a Basic Server

The post The P2P Bootstrapping Problem – Starting My Journey To Help Build a P2P Social Network appeared first on Be on the Right Side of Change.