Understanding Deep Learning through image generation.

Images 1 and 6 are synthetic: generated by AI from a time series of images.

2, 3, 4 and 5 are human creations. Mine, to be more precise. I draw them in an app called "Paper" (originally from FiftyThree, now from WeTransfer) with my fingers on the screen. I generated image 2 first and so on. Human, and time to unwind.

I shared with the AI a set of pieces with a similar style and aesthetic, and let OpenAI generate what would hypothetically have been the first of the series and the next one.

We humans would read into it a compositional logic, a chromatic logic, a spatial logic. For the AI it's a game of attributes, iterations and probabilistic distance. And even so, the result is simply wonderful.

𝙏𝙚𝙘𝙝𝙣𝙞𝙘𝙖𝙡𝙡𝙮: 𝚝𝚑𝚎 𝚖𝚘𝚍𝚎𝚕 𝚎𝚗𝚌𝚘𝚍𝚎𝚜 𝚎𝚊𝚌𝚑 𝚒𝚖𝚊𝚐𝚎 𝚒𝚗 𝚊 𝚑𝚒𝚐𝚑-𝚍𝚒𝚖𝚎𝚗𝚜𝚒𝚘𝚗𝚊𝚕 𝚕𝚊𝚝𝚎𝚗𝚝 𝚜𝚙𝚊𝚌𝚎, 𝚠𝚑𝚎𝚛𝚎 "𝚜𝚝𝚢𝚕𝚎" 𝚒𝚜 𝚛𝚎𝚙𝚛𝚎𝚜𝚎𝚗𝚝𝚎𝚍 𝚊𝚜 𝚊 𝚟𝚎𝚌𝚝𝚘𝚛. 𝙸𝚝 𝚊𝚟𝚎𝚛𝚊𝚐𝚎𝚜 𝚝𝚑𝚘𝚜𝚎 𝚟𝚎𝚌𝚝𝚘𝚛𝚜 𝚝𝚘 𝚒𝚗𝚏𝚎𝚛 𝚝𝚑𝚎 𝚍𝚒𝚛𝚎𝚌𝚝𝚒𝚘𝚗 𝚘𝚏 𝚝𝚑𝚎 𝚜𝚎𝚛𝚒𝚎𝚜 𝚊𝚗𝚍, 𝚝𝚑𝚛𝚘𝚞𝚐𝚑 𝚛𝚎𝚟𝚎𝚛𝚜𝚎 𝚍𝚒𝚏𝚏𝚞𝚜𝚒𝚘𝚗, 𝚒𝚝 𝚜𝚝𝚊𝚛𝚝𝚜 𝚏𝚛𝚘𝚖 𝚗𝚘𝚒𝚜𝚎 𝚊𝚗𝚍 𝚙𝚛𝚘𝚐𝚛𝚎𝚜𝚜𝚒𝚟𝚎𝚕𝚢 𝚍𝚎𝚗𝚘𝚒𝚜𝚎𝚜 𝚒𝚝 (𝚛𝚎𝚍𝚞𝚌𝚒𝚗𝚐 𝚝𝚑𝚎 𝚗𝚘𝚒𝚜𝚎) 𝚞𝚗𝚝𝚒𝚕 𝚒𝚝 𝚖𝚊𝚝𝚎𝚛𝚒𝚊𝚕𝚒𝚣𝚎𝚜 𝚊 𝚙𝚒𝚎𝚌𝚎 𝚌𝚘𝚑𝚎𝚛𝚎𝚗𝚝 𝚠𝚒𝚝𝚑 𝚝𝚑𝚊𝚝 𝚜𝚝𝚢𝚕𝚒𝚜𝚝𝚒𝚌 𝚝𝚛𝚊𝚓𝚎𝚌𝚝𝚘𝚛𝚢. 𝙸𝚝 𝚍𝚘𝚎𝚜 𝚗𝚘𝚝 𝚌𝚘𝚙𝚢: 𝚒𝚝 𝚒𝚗𝚝𝚎𝚛𝚙𝚘𝚕𝚊𝚝𝚎𝚜.

In other words, the model encodes the images in a latent space, or a mathematical map of thousands of dimensions where visual features are represented as semantic vectors. By averaging these vectors, the AI identifies the stylistic trajectory of the series. It doesn't copy images: it interpolates values.

And meanwhile, I don't lose my capacity for wonder at the nuances, and at how an AI is able to generate the first and the sixth of a human series of four images.

Images 1, 2, 3 and 4. Human creations

Latent Space: Mapping creativity mathematically

Imagine you had to describe each of your drawings with, let's say, 512 sliders. One controls "how circular vs. organic it is", another "chromatic saturation", another "density of concentric layers", another "presence of navy blue"… and so on up to 512. No slider has a name a human would recognize —they are dimensions the algorithmic model has discovered on its own (deep learning) during training—, but between them all they capture the essence of any image.

That 512-dimensional space is the latent space. The number, actually, is quite a bit larger: in models like Stable Diffusion, the space where generation happens has around 16,000 dimensions, and the associated semantic spaces —the ones that encode "style" or "concept"— are around 768 or 1,024.

The observable universe contains around 10⁸⁰ atoms. With just 512 binary sliders, the latent space already has more possible combinations than there are atoms in the universe, raised to almost twice that power.

In larger closed models, the figures go up to several thousand. The idea of "512 sliders" is a pedagogical simplification. What matters is to retain that we are talking about thousands of axes, not hundreds, and that each one is a dimension learned automatically, with no human name. Inferences.

Every image in the world is a point in there. Similar images live close by; radically different images, far away.

What is powerful about the latent space is that inside it you can do arithmetic with meaning. The classic example from word embeddings: king − man + woman ≈ queen. With images it works the same way: smiling portrait − neutral portrait = "smile vector", which you can then add to any other portrait to make it smile.

The six drawings are six points in that space. They form a small cloud. The AI can calculate the center of that cloud (the "average style"), its dominant direction (where the series is heading), or a point interpolated between any two of them. That is what it does to infer "the first one" (image 1) and "the next one" (image 6): it locates a plausible position inside your stylistic cloud. It doesn't look for the exact point; it samples a plausible position. That is why, if we asked it to generate "the seventh image" ten times, it would give us ten different images, all coherent with my style. It is a stochastic process, not a deterministic one. Does the idea of non-deterministic models click now?

The latent space is a compressed map of the visual world where geometry means something. Moving in one direction amounts to transforming the image in a coherent way.

Image 1 and Image 6 in the series

Diffusion: Stabilizing creation out of noise

Diffusion is the generation mechanism. And its logic is counterintuitive but beautiful.

During training, the AI does the opposite of generating. It takes a real image and adds Gaussian noise to it little by little, step by step, until it turns it into pure static —that white noise of a badly tuned television—. And at each step, it learns to predict what noise has just been added. It repeats this millions of times with millions of images. It learns to destroy images in a controlled way. It learns to tell what's superfluous for an image to make sense again.

When the moment to generate arrives, the AI reverses the process. It starts from pure noise —literally random values drawn from a Gaussian bell curve— and, step by step, subtracts the noise that "it" itself predicts. Little by little it cleans up that noise. At each step it removes a part of the excess until a coherent image starts to appear. With each iteration the image is a little less dirty, a little more defined. After fifty or a hundred steps, a coherent image emerges from the chaos. And all of this happens in seconds, maybe minutes.

A technical nuance: the noise it starts from doesn't live in the semantic space —the one of "style, concept, composition"—, but in the autoencoder's latent space, a compressed representation of the image. The semantic space acts differently: as a conditioning vector that guides the process. At each of the ~50 steps, the model receives two inputs —the current noise and that condition— and predicts which portion of the noise is excess in order to move closer to an image coherent with the style requested. The random noise is the raw material; the embedding of your series is the force field that sculpts it.

The most precise metaphor we can use is that of the sculptor Michelangelo. Michelangelo di Lodovico Buonarroti Simoni used to say that the David was already inside the block of marble, and that he only removed the excess.

Diffusion is exactly that. The image is already latent in the noise.

The model only removes, step by step, what does not belong. With one addition that Michelangelo did not have: the "David" that emerges depends on which stylistic condition is whispered to the chisel at each blow.

These days the chisel or the brush can be physical or semantic. Starting from words or from images or from audio or from text, etc. The chisel or the brush are the variable. And they can also be a constant. Creativity still exists. And what we call art is an emotional consideration.

Are images 1 and 6 art?