17. October 2022 · Reading Time: 4 Min · By Jan Pötzscher, Mark Reinke

The “Art Browser” prototype: AI-supported access to art

The “Art Browser” prototype developed by exdatis and implemented together with the Staatliche Kunstsammlungen Dresden (SKD) can be described as the next evolutionary step in searching works of art. With this pilot project, we are demonstrating ways in which artificial intelligence can be used to make SKD exhibits searchable and experienceable for an even wider audience.

Fluid navigation through exhibits

The application opens up a new dimension in viewing artworks. Based on the image data, the AI recognises characteristics of the exhibits such as colours and materials used or stylistic features. This allows connections to be made between the exhibits on a wide variety of levels. In short, the artificial intelligence can now read in images.

Viewers move within the diversity of the exhibits like on a map and embark on an intuitive journey through the works of art.

Internationalisation with the help of AI

To foster connection with international researchers and to promote scientific exchange and cooperation in the field of art is one of the SKD’s concerns.

Therefore, it was a basic requirement to create a language-independent access to the artworks. This is achieved by using both image files and text in different languages instead of a classical keyword search.

If a viewer uploads a new image, the AI recognises the motif depicted, characteristics of the technique used as well as special features of the implementation, such as a particularly impulsive line pattern, and can thus present similar exhibits. Or a user enters what interests her. This input is compared with the collection images and then the images that come closest to the description are displayed.

Thanks to our multi-modal AI model, these results can be achieved even if an image collection is not catalogued and enriched with metadata. In addition, metadata from content-enriched holdings can be used to further filter and group the data.

Artificial intelligence for future viability and data currency

The great potential lies in keeping the data, which is constantly changing due to ongoing research, up-to-date in a fully automated way. A “learning” AI-based system offers precisely this interesting perspective to show cross-connections between drawings and discover new references in the process.

We have illustrated how this intelligent search feels in practice in a video using 16th century Italian drawings from the Kupferstich-Kabinett: