OpenAI’s GPT-3 paper was introduced May 28th, 2020, becoming a hot topic in the field of AI Natural Language Processing. Its 175 billion parameters transformer architecture made it very popular in both specialized and general news, thanks to the vast landscape of applications that some developers quickly showcased, some of which I listed on my introductory article:

How are state-of-the-art NLP models trained in different languages?

Let’s take GPT-3 as an example. According to the GPT-3 paper, it was pre-trained on massive datasets, including the Common Crawl dataset, which contains petabytes of data collected since 2008 and weights 60% of the total training mix for GPT-3, which also includes…

Back in 2016, DeepMind introduced AlphaGo, the first computer software able to defeat professional Go players, included the world champion. The games between Lee Sedol (18 Go world titles) and AlphaGo were even immortalized in a documentary, available now in Youtube.

Why was AlphaGo so relevant? Until then, computer programs were only able to play Go at an amateur level, as traditional Machine Learning methods such as search trees were simply not capable of evaluating all possible moves, board positions strengths, etc. …

Ethics has definitely become a trend in the field of Artificial Intelligence. It seems clear that AI faces a lot of challenges if we want it to have a positive impact for our society. Nevertheless, it is not the first time that researchers warn us about the risks of this kind of technology. Norbert Wiener, cybernetics pioneer, wrote this somehow prophetic piece back in his book God & Golem, inc, back in 1964:

It is relatively easy to promote good and to fight evil and good and evil are arranged against each other in two clear lines, and when those…

During my summer vacation, I ran across a CBInsights report called “AI trends to watch in 2020”. I was curious about what my colleagues and readers would think about the selected trends, so I launched a survey to see what they thought. I simply asked one question: “Based on your personal experience, which one is impacting your job/ industry the most?” and these were the results:

It has been already a year since I published a similar article on the same Gartner’s report for 2019 that you can find here. Which AI related technologies have been excluded from the report? Which ones should be, according to Gartner, focus areas for companies AI leaders?

First, a quick reminder of an important background to understand how Gartner’s Hype Cycles are presented. As Gartner explains in its research, its Hype Cycle covers a very broad spectrum of topics, so if a specific technology is not featured it does not necessarily imply that they are not important, quite the opposite…

Introduction

Let’s start with the basics. GPT-3 stands for Generative Pretrained Transformer version 3, and it is a sequence transduction model. Simply put, sequence transduction is a technique that transforms an input sequence to an output sequence.

GPT-3 is a language model, which means that, using sequence transduction, it can predict the likelihood of an output sequence given an input sequence. This can be used, for instance to predict which word makes the most sense given a text sequence.

A very simple example of how these models work is shown below:

There has been a lot of discussion during the last days around bias in the AI community, especially after Yann LeCun joined the conversation after this tweet:

PULSE, the algorithm that created this image, works by using Self-Supervised training to search a space of high-resolution artificial images generated using a GAN and identify ones that downscale to the low resolution image. A bias problem with the algorithm was quickly found: given downsampled (but still very recognizable) images of famous non-white people, the algorithm still upsampled them to produce…