Students help Naturalis recognise butterflies with deep learning

Monitoring biodiversity is a challenge, and for Naturalis it is mainly in the area of data. The biodiversity centre relies on observations by volunteers for their research into the distribution and movement of, for example, butterfly species. These volunteers take photographs of butterflies, which can be indexed by experts according to species. In this way Naturalis gets good data without having to go into the field itself. But what if artificial intelligence (AI) could convert that data directly into useful insights?

Citizen Science

ontributing to scientific research without being a researcher is a common practice. It is called citizen science; volunteers share data from archaeological finds or flora and fauna they encounter in nature. This data, including coordinates and date, is shared via a platform or app. Image Recognition Models (IRM) provide identification through the apps Iobs (iOS) and ObsIdentify (Android). But identifying what is in the photo can be difficult if the image quality is poor. With butterfly species, which were the focus of this project, there are also cases where two species cannot be distinguished from each other. In addition, observations are not made everywhere, which creates gaps in the data.

Species Distribution Models

Species distribution models can offer a solution to this problem. Based on geographical and climate-related data, such a model can predict the likelihood of a species occurring at location X. This could provide volunteers with an immediate identification of their observation and improve data quality for Naturalis. Moreover, it would allow predictions to be made about population distribution (e.g., based on climate change), and thus fill in the gaps in observations. Developing such a model with AI was the task of Fontys Hogeschool ICT students Max de Goede, Lars van Driel, Pol Roskam and Jochem Wienk based on deep learning AI.

Deep learning for better insights

The students developed the tooling to process and merge data for this model. A 'data pipeline' that can be extended in the future. Two datasets were available for this, says student Max de Goede: "The first set consisted of observations of butterfly species by volunteers via observation.org. Images, including species, coordinates and date. The second set of data was collected by Naturalis itself with geo-factors, such as altitude, climate, and other variables."

With this, the students developed a neural network, which extracted output from the data with deep learning that makes predictions possible. The application of deep learning is therefore the innovation. Through a series of steps, it teaches the system to draw conclusions from the data itself, without intervention. Max: "What is delivered has two dimensions. The first is a model, which you should see as an algorithm that can predict how probable it is that a species of butterfly will be present at location X. The second is the ability to produce a map of the area. The second is the ability to draw a map that shows how certain the model is that a butterfly species can be present at various locations." In their paper, the students indicate that this is a start, further collaboration with biologists can help train the system to function even more precisely.

Learning and innovating in practice

"It is very instructive to work with a partner on a real research question. I don't know anything about butterflies, so you really speak a different language when you're working with a biologist. There was also a data scientist involved, who we learned a lot from." The prototype of the SDM could be used as support in the observation apps, Max explains: "Using the data on the living conditions, the app can indicate during an observation whether it is species A or species B (when these are distinctive), because these are decisive for the different species." Not all butterfly species can be recognised with the algorithm. The model is a proof of concept, which is limited to five species. Naturalis can expand this concept to make more predictions based on their observations. Max is eager to see this happen: "I hope that this project will be continued, as there is still much more that can be done with it. The technique is growing and developing enormously, so further development is the next step."