After GPT-3, OpenAI returns with two models that combine text and images. Will DALL-E be the protagonist of 2021 in the field of AI?

Image for post
Image for post
Photo taken by David Pereira at Dali’s museum in Figueres.

While the community is still discussing one of 2020 AI big announcements, GPT-3, whose paper was published July 22nd, 2021 has just begun and we already have two impressive new neural networks from OpenAI: CLIP and DALL-E.

Both CLIP and DALL-E are multimodal neural networks, and their creators claim them to be “a step toward systems with deeper understanding of the world”.

Our experiences as humans are multimodal, meaning that we receive inputs from the world surrounding us in different formats (sound, image, odors, textures, etc.) …


The field of NLP has seen major advancements in the last years, but what does it mean for minority languages?

Image for post
Image for post
“File:Wubi86 keyboard layout.png” by Cangjie6 is licensed under CC BY-SA 4.0

OpenAI’s GPT-3 paper was introduced May 28th, 2020, becoming a hot topic in the field of AI Natural Language Processing. Its 175 billion parameters transformer architecture made it very popular in both specialized and general news, thanks to the vast landscape of applications that some developers quickly showcased, some of which I listed on my introductory article:

Let’s take GPT-3 as an example. According to the GPT-3 paper, it was pre-trained on massive datasets, including the Common Crawl dataset, which contains petabytes of data collected since 2008 and weights 60% of the total training mix for GPT-3, which also includes…


After several breakthroughs such as AlphaGo or AlphaZero, researchers from DeepMind have published their latest effort, MuZero. What is it all about?

Image for post
Image for post
“Artificial Intelligence & AI & Machine Learning” by mikemacmarketing is licensed under CC BY 2.0

Back in 2016, DeepMind introduced AlphaGo, the first computer software able to defeat professional Go players, included the world champion. The games between Lee Sedol (18 Go world titles) and AlphaGo were even immortalized in a documentary, available now in Youtube.

Why was AlphaGo so relevant? Until then, computer programs were only able to play Go at an amateur level, as traditional Machine Learning methods such as search trees were simply not capable of evaluating all possible moves, board positions strengths, etc. …


Norbert Wiener, one of cybernetics pioneers, envisioned AI ethics problems way ahead of us

Image for post
Image for post
“43081” by Tekniska museet is licensed with CC BY 2.0. To view a copy of this license, visit https://creativecommons.org/licenses/by/2.0/

Ethics has definitely become a trend in the field of Artificial Intelligence. It seems clear that AI faces a lot of challenges if we want it to have a positive impact for our society. Nevertheless, it is not the first time that researchers warn us about the risks of this kind of technology. Norbert Wiener, cybernetics pioneer, wrote this somehow prophetic piece back in his book God & Golem, inc, back in 1964:

It is relatively easy to promote good and to fight evil and good and evil are arranged against each other in two clear lines, and when those…


My readers selected AutoML as the trend that will impact their job/ industry in the short time the most. What is it and why should you care?

Image for post
Image for post
“Machine Learning & Artificial Intelligence” by mikemacmarketing, licensed under CC BY 2.0

During my summer vacation, I ran across a CBInsights report called “AI trends to watch in 2020”. I was curious about what my colleagues and readers would think about the selected trends, so I launched a survey to see what they thought. I simply asked one question: “Based on your personal experience, which one is impacting your job/ industry the most?” and these were the results:


Gartner’s 2020 Hype Cycle for Emerging Technologies is out, so it is a good moment to take a deep look at the report and reflect on our AI strategy as a company. You can find a brief summary of the complete report here.

Image for post
Image for post
https://www.gartner.com/smarterwithgartner/5-trends-drive-the-gartner-hype-cycle-for-emerging-technologies-2020/

It has been already a year since I published a similar article on the same Gartner’s report for 2019 that you can find here. Which AI related technologies have been excluded from the report? Which ones should be, according to Gartner, focus areas for companies AI leaders?

First, a quick reminder of an important background to understand how Gartner’s Hype Cycles are presented. As Gartner explains in its research, its Hype Cycle covers a very broad spectrum of topics, so if a specific technology is not featured it does not necessarily imply that they are not important, quite the opposite…


It has been almost impossible to avoid the GPT-3 hype in the last weeks. This article offers a quick introduction to its architecture, use cases already available, as well as some thoughts about its ethical and green IT implications.

Image for post
Image for post
Photo from https://unsplash.com/@franckinjapan

Let’s start with the basics. GPT-3 stands for Generative Pretrained Transformer version 3, and it is a sequence transduction model. Simply put, sequence transduction is a technique that transforms an input sequence to an output sequence.

GPT-3 is a language model, which means that, using sequence transduction, it can predict the likelihood of an output sequence given an input sequence. This can be used, for instance to predict which word makes the most sense given a text sequence.

A very simple example of how these models work is shown below:


While data bias is a very well-known cause for AI unfairness, it is definitely not the only one.

Image for post
Image for post
Obama upsampled to a white person by an AI, originally published as part of this tweet: https://twitter.com/Chicken3gg/status/1274314622447820801?s=20

There has been a lot of discussion during the last days around bias in the AI community, especially after Yann LeCun joined the conversation after this tweet:

Original tweet about the bias of PULSE, a new Photo Upsampling algorithm

PULSE, the algorithm that created this image, works by using Self-Supervised training to search a space of high-resolution artificial images generated using a GAN and identify ones that downscale to the low resolution image. A bias problem with the algorithm was quickly found: given downsampled (but still very recognizable) images of famous non-white people, the algorithm still upsampled them to produce…


The technical architecture for Gaia-X, the European effort to create the next generation of Data infrastructure, has been just published. Is it really the key for Europe’s cloud sovereignty?

Image for post
Image for post
“Computer Security — Protect Data — Computers” by perspec_photo88, licensed under CC BY-SA 2.0

Europe is making huge efforts and investments to create digital services that ensure transparency and interoperability, as well as privacy by design, with a strong focus on the ethical implications of the use of technology. In that context, a group of representatives from governments, business and science from Germany and France have proposed Gaia-X as a federated data infrastructure for Europe.

According to Gaia-X’s project webpage, Gaia-X is characterized by the…

David Pereira

Head of Data & Intelligence for Europe at everis, an NTT Data company. All opinions are my own. https://www.linkedin.com/in/dpereirapaz/

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store