
Convolutional Neural Networks (CNNs) have revolutionized the field of machine learning in recent years, particularly in image classification. Various applications and advances in the understanding and explainability of these models have also allowed us to identify their flaws: issues with spatiality or so-called "adversarial" attacks.

Picasso – Guernica (source: Wikiart)
For example, we can use the analogy of the faces in Guernica by Picasso to illustrate the inability of CNN models to understand the spatial relationships between objects present in images.
If we were to translate what a CNN interprets when presented with an image of a face into plain English, it would look something like this:
"If the image contains:
– One or more eyes,
– A nose,
– A mouth,
– Eyebrows...
Then it's a face!"
At first glance, these criteria seem sufficient to correctly detect a real human face. However, if our image contains the features of a face, but with a somewhat unusual arrangement (4 eyes, 2 mouths, a nose on the left, vertical eyebrows...), should we really let the algorithm recognize the image as a face?

We could also discuss adversarial attacks which are the bane of these models. These are perturbations that are almost invisible to the naked eye, but which can radically deceive the network.

To overcome these problems, Geoffrey Hinton and his team proposed an improvement to this type of model in 2017 by introducing capsules to replace neurons in a standard neural network.
Capsules enhance neurons by retaining an information vector rather than a simple quantity (scalar). This vector retains much more detail: pose, texture, locality, and more.
By using a process similar to the classic CNN, the capsule network can thus learn more complex patterns.
To return to our face detection example, such a network can learn the spatial relationship between the eyes, nose, and mouth, and could even, in theory, understand the pose (the orientation of the face). Thus, the transcription in English could look like this:
"If the image contains:
– Two eyes, to the left or right of another eye,above the nose
– A nose, oriented vertically, in the center, below the eyes
– A mouth, horizontally oriented, below the nose and eyes
– A pair of eyebrows horizontally oriented, above the eyes …
Then it's a face!"
Depending on the information retained in the capsule vector, the network can in theory be sensitive or insensitive to features that are interpretable from this information.
Since then, many papers have been published on the subject proposing improvements, adaptations for various use cases (other than 2D), as well as the sensitivity of capsules to "adversarial" attacks.
We recommend reading this article available on medium.com for more details on their history and how they work: https://medium.com/ai³-theory-practice-business/understanding-hintons-capsule-networks-part-i-intuition-b4b559d1159b
Lorem ipsum dolor sit amet consectetur. Ipsum penatibus viverra feugiat amet varius pharetra porta risus lectus. Turpis enim massa non tincidunt in faucibus.