Do you ever wonder which objects are in the paintings in the Rijksmuseum? 'Birds' is often the vague description on the information accompanying a painting about – yes, you guessed it – birds, possibly followed by the name of the painter. A smart computer program ensures that experts can add their knowledge to paintings.
'Eagle and other birds', is the description of a work that printmaker Pieter Schenk made between 1675 and 1711. The curator probably had no idea which six other birds are depicted in the artwork. This kind of metadata about paintings and prints is often limited. This also applies to information in descriptions of vases, clothing, sculptures and other art objects. The Rijksmuseum has therefore developed a special tool together with various scientists.
The Accurator is aimed at improving the description process for digital collections. At a theme day for bird watchers in October 2015, several birds were identified using the Accurator. Reason for the Rijksmuseum to collaborate with Modemuze, a platform for lovers of fashion and costumes in the Netherlands, to organize a similar day for specialists and lovers of fashion and costumes.
Scientist Lora Aroyo, assistant professor of Intelligent Information Systems, and involved in the project, explains.
What is the Accurator? “The Accurator is an online tool that allows you to tag objects. You can even tag specific parts of an object, such as details about a particular bird's wings. This way we get more and more reliable data about objects. Lovers of fashion and birds still had to come to Amsterdam for the tests, but in the end everyone should be able to help make an inventory of the collection from their lazy chair. We are working on the latest technical details, hundreds of people must be able to work with the tool at the same time and that requires some technology.”
How do you prevent incorrect data from being provided? “Algorithms are used to determine how reliable the input of the specialists is. If one person keeps making contributions that the rest don't agree with, alarm bells start ringing. There are maybe two thousand objects that have a bird on them, but only a part of them has been digitized. And we don't release the entire digital selection immediately, so several people work on a painting or other object at the same time. The algorithm is also getting smarter. Details of bird characteristics, such as the color of the feathers, are combined. A swan looks different from a sparrow.”
The Accurator's first test was in October 2015. What was the result? “The birding day was great. Due to the size of the room, we were limited to 40 participants, but I am impressed with their knowledge. They had piles of books with them. Discussion took place in small groups. Which bird is this, is it the winter version or the summer version. We noticed that everyone had their own specialization, for example one of the participants knew a lot about the color differences per gender. They didn't want to stop at the end of the day, earlier for more difficult pictures and a bigger challenge.”
Naturalis, the Institute for Sound and Vision and the Koninklijke Bibliotheek have also joined the bird project. What was the reason for this? “As citizens, we want more and more information. We want to know which painters were inspired by a particular bird. However, we also want to know in which nature programs that bird appears, which books have been written about it and whether the skeleton of that bird is present in Naturalis. So you have to tag, link and bundle data.”
Speaking of tagging, each curator has their own system. Was the library's metadata comparable to that of the Rijksmuseum? “We have learned a lot from it. Each party had its own formats, a different way of tagging, its own process for structuring data and a different language. That had to come together in the Accurator. It concerns four types of cultural heritage. A library, an art museum, a science museum and an image archive. At the library, for example, the photos were barely tagged, but the chunks of text were searchable. Ideally, each party uses the same form for annotations. You enter a keyword (for example 'mus') into a database, and then you get hits from all places where information about the requested topic can be found."
After the project with the bird watchers, the Accurator was also used for the stitch by stitch project. How did this go? “Again forty specialists and enthusiasts have been invited, this time to describe fashion-related objects from the collections of the Rijksmuseum and the Modemuze museums. Some items of clothing and accessories are familiar to everyone, while others require an expert eye. What exactly do we see? What kind of sleeve is this? What materials and techniques were used to make this garment? 306 paintings were identified that day and nearly 2000 tags were added. Those tags are about accessories, lace, costumes, for example.”
How does the project proceed now? “Collection managers can use the Accurator. Based on their own collection, they can invite specialists to map out details. They will soon be able to help identify objects from the comfort of their own home. There are many sub-collections that you can use the Accurator for. For example, think of flowers, ships, castles, and other animal species. But they can also use it to organize an exhibition around a very specific topic (for example, Chinese dresses from a certain era) and to borrow objects from other museums.”