Automatic Classification of Fresco Fragments: A Machine and Deep Learning Study

The reconstruction of destroyed frescoes is a complex task: very small fragments, irregular shapes, color alterations and missing pieces are only some of the possible problems that we have to deal with. Surely, an important preliminary step involves the separation of mixed fragments. In fact, in a real scenario, such as a church destroyed by an earthquake, it is likely that pieces of different frescoes, which were close on the same wall, end up mixed together, making their reconstruction more complex. Their separation may be especially difficult if there are many of them and if there are no (or very old) reference images of the original frescoes. A possible way to separate the fragments is to treat this problem as a stylistic classification task, in which we have only parts of an artwork instead of a complete one. In this work, we tested various machine and deep learning solutions on the DAFNE dataset (to date the largest open access collection of artificially fragmented fresco images). The experiments showed promising results, with good performances in both binary and multi-class classification.