Overfitting is a term used in the fields of artificial intelligence, big data, smart data and digital transformation. It describes a common problem that occurs when developing models for data analysis or machine learning.
Imagine that an artificial intelligence is supposed to recognise whether a picture shows a cat with the help of many photos. During overfitting, the AI remembers too many details from the training images, for example the sofa in the background or the colour of the ceiling - in other words, unimportant details that only appear in the training data. As a result, the model works excellently on the trained images, but on new, unfamiliar photos it suddenly no longer recognises cats reliably because it is paying attention to the wrong cues.
Overfitting often happens when a model is too complex or too little training data is used. The aim is therefore to learn the right patterns without getting lost in unimportant details.
Overfitting can be recognised when the results are very good on known data but suddenly much worse on new data. To avoid overfitting, experts use various methods, for example more training data or certain techniques that simplify the model.