How Many Publications Does One Need to Get a Ph.D. in Computer Science?

The following answer is based on my experience as a doctoral researcher in distributed systems. Computer science is a big field with vast differences in the quality and quantity requirements of your Ph.D. supervisor. Having said this, you’ll probably need somewhere between two to five publications to get a Ph.D. in computer science. I have … Read more

Transformer vs Autoencoder: Decoding Machine Learning Techniques

An autoencoder is a neural network that learns to compress and reconstruct unlabeled data. It has two parts: an encoder that processes the input, and a decoder that reproduces it. While the original transformer model was an autoencoder with both encoder and decoder, OpenAI’s GPT series uses only a decoder. In a way, transformers are … Read more

Transformer vs RNN: Women in Red Dresses (Attention Is All They Need?)

TL;DR: Transformers process input sequences in parallel, making them computationally efficient compared to RNNs which operate sequentially. Both handle sequential data like natural language, but Transformers don’t require data to be processed in order. They avoid recursion, capturing word relationships through multi-head attention and positional embeddings. However, traditional Transformers can only capture dependencies within their … Read more

Scalable Graph Partitioning for Distributed Graph Processing

I just realized that the link to my doctoral thesis doesn’t work, so I decided to host it on the Finxter blog as a backup. Find the thesis here: πŸ”— PDF Download link: Here’s the abstract: πŸ’‘ Abstract: Distributed graph processing systems such as Pregel, PowerGraph, or GraphX have gained popularity due to their … Read more

Top 10 LLM Training Datasets – It’s Money Laundering for Copyrighted Data!

I’ve read the expression of large language models (LLMs) being “Money Laundering for Copyrighted Data” on Simon Willison’s blog. In today’s article, I’ll show you which exact training data sets open-source LLMs use, so we can gain some more insights into this new alien technology and, hopefully, get smarter and more effective prompters. Let’s get … Read more

AI Scaling Laws – A Short Primer

The AI scaling laws could be the biggest finding in computer science since Moore’s Law was introduced. πŸ“ˆ In my opinion, these laws haven’t gotten the attention they deserve (yet), even though they could show a clear way to make considerable improvements in artificial intelligence. This could change every industry in the world, and it’s … Read more

Google’s RT-2 Enables Robots To Learn From YouTube Videos

In a game-changing development in robotics, Google DeepMind’s new artificial intelligence model, RT-2 (Robotics Transformer 2), seamlessly combines vision, language, and action to help robots understand and perform tasks with greater adaptability. The RT-2 is a Vision-Language-Action (VLA) model, unprecedented in its capacity to integrate text and images from the internet and use the acquired … Read more

6 Easiest Ways to Get Started with Llama2: Meta’s Open AI Model

With the unexpected release of its advanced language model, LLaMA 2, Meta has opened up new avenues for commercial and research applications. Unlike its predecessor, this second iteration of the model is (primarily) free and open-source. πŸ§‘β€πŸ’» Recommended: Llama 2: How Meta’s Free Open-Source LLM Beats GPT-4! The source code and data that forms the … Read more

Worldcoin – Who Pays for Universal Basic Income (UBI)?

The universal basic income (UBI) in Worldcoin isn’t directly funded by the Worldcoin Protocol but will eventually require a separate funding source, such as a share of profits from an AI Lab. There’s no guarantee or explicit sustainable protocol rule. However, the Worldcoin Whitepaper envisions the possibility to set a 1.5% annual inflation rate, essentially … Read more

Med-PaLM 2: Will This Google Research Help You Increase Your Healthspan?

Executive Summary Before we dive into Med-PaLM 2, let’s learn about the underlying technology: PaLM 2: πŸ‘‡ What Is PaLM 2? Google’s PaLM 2 is a next-generation language model with enhanced multilingual understanding, reasoning, and coding capabilities. The Advent of PaLM 2 We recently published an article about the fact that Google actually developed the … Read more