A couple of years ago, I watched a TED talk that changed my life. I had just finished my computer science master’s degree and was starting out as a fresh Ph.D. student in the department of distributed systems… … and I was overwhelmed. There are many computer science students reading the Finxter blog so I … Read more
The following answer is based on my experience as a doctoral researcher in distributed systems. Computer science is a big field with vast differences in the quality and quantity requirements of your Ph.D. supervisor. Having said this, you’ll probably need somewhere between two to five publications to get a Ph.D. in computer science. I have … Read more
An autoencoder is a neural network that learns to compress and reconstruct unlabeled data. It has two parts: an encoder that processes the input, and a decoder that reproduces it. While the original transformer model was an autoencoder with both encoder and decoder, OpenAI’s GPT series uses only a decoder. In a way, transformers are … Read more
TL;DR: Transformers process input sequences in parallel, making them computationally efficient compared to RNNs which operate sequentially. Both handle sequential data like natural language, but Transformers don’t require data to be processed in order. They avoid recursion, capturing word relationships through multi-head attention and positional embeddings. However, traditional Transformers can only capture dependencies within their … Read more
I just realized that the link to my doctoral thesis doesn’t work, so I decided to host it on the Finxter blog as a backup. Find the thesis here: 🔗 PDF Download link: https://blog.finxter.com/wp-content/uploads/2023/09/dissertation_christian_mayer_distributed_graph_processing_DIS-2019-03.pdf Here’s the abstract: 💡 Abstract: Distributed graph processing systems such as Pregel, PowerGraph, or GraphX have gained popularity due to their … Read more
I’ve read the expression of large language models (LLMs) being “Money Laundering for Copyrighted Data” on Simon Willison’s blog. In today’s article, I’ll show you which exact training data sets open-source LLMs use, so we can gain some more insights into this new alien technology and, hopefully, get smarter and more effective prompters. Let’s get … Read more
The AI scaling laws could be the biggest finding in computer science since Moore’s Law was introduced. 📈 In my opinion, these laws haven’t gotten the attention they deserve (yet), even though they could show a clear way to make considerable improvements in artificial intelligence. This could change every industry in the world, and it’s … Read more
I’ve written extensively on the advent of alien technology that is large language models (LLMs) and transformers. Magical files with numbers that can take thousands of tokens as inputs and generate thousands of tokens as outputs. If you’re like me, you may have wondered: How does a machine’s creativity compare to that of a human? … Read more
In a game-changing development in robotics, Google DeepMind’s new artificial intelligence model, RT-2 (Robotics Transformer 2), seamlessly combines vision, language, and action to help robots understand and perform tasks with greater adaptability. The RT-2 is a Vision-Language-Action (VLA) model, unprecedented in its capacity to integrate text and images from the internet and use the acquired … Read more
With the unexpected release of its advanced language model, LLaMA 2, Meta has opened up new avenues for commercial and research applications. Unlike its predecessor, this second iteration of the model is (primarily) free and open-source. 🧑💻 Recommended: Llama 2: How Meta’s Free Open-Source LLM Beats GPT-4! The source code and data that forms the … Read more
The universal basic income (UBI) in Worldcoin isn’t directly funded by the Worldcoin Protocol but will eventually require a separate funding source, such as a share of profits from an AI Lab. There’s no guarantee or explicit sustainable protocol rule. However, the Worldcoin Whitepaper envisions the possibility to set a 1.5% annual inflation rate, essentially … Read more
Executive Summary Before we dive into Med-PaLM 2, let’s learn about the underlying technology: PaLM 2: 👇 What Is PaLM 2? Google’s PaLM 2 is a next-generation language model with enhanced multilingual understanding, reasoning, and coding capabilities. The Advent of PaLM 2 We recently published an article about the fact that Google actually developed the … Read more