Developing an AI-powered app to identify invasive bugs

A team member using a 3D imaging system to take images of stink bugs from many angles to train the AI model inside the app. This requires several hundred images per species.

By Andrea Wild

Australia’s national science agency, CSIRO, is using artificial intelligence (AI) to develop an app that will help keep brown marmorated stink bugs out of Australia, an invasive species with the potential to wipe out more than 300 different species of plants if it made it past quarantine.

The app, being developed for the Department of Agriculture, Water and the Environment (DAWE), is based on a prototype co-funded by Microsoft to identify seeds of noxious weedy daisies, using AI to identify stink bug species based on thousands of specimens held in CSIRO’s National Research Collections Australia.

DAWE is now trialling the app in its quarantine stations.

CSIRO Chief Executive Dr Larry Marshall said collaborative partnerships were a powerful way to turn new technologies into real solutions to the nation’s greatest challenges, like protecting our native flora.

“Australia’s growing AI capability can be among the best in the world, but it doesn’t mean anything until we translate it into solutions that make life better for everyone, like ensuring our increasingly interconnected world doesn’t jeopardize our biosecurity,” Dr Marshall said. “This app will help our biosecurity officers tell invasive species apart from our own native species.”

CSIRO taxonomist Dr Alexander Schmidt-Lebuhn said the app demonstrated the practical applications of having a rich insect database.

“We’re taking detailed digital images of the stink bugs in our insect collection, including using a 3D imaging system to take photographs from many angles,” Dr Schmidt-Lebuhn said.

“Using a smartphone camera to zoom in or out and look at the bug from different angles, the AI model in the app identifies the species and shows how likely it is to be correct.

“The app also has species profiles with example images and species information. Users can record a photo of the bug, its identification and the geographic coordinates and local time to help build out the database and inform biosecurity responses.”

A brown marmorated stink bug specimen.

Microsoft Australia National Technology Officer, Lee Hickin, said: “Since establishing our partnership with CSIRO in June 2020 our focus has been on supporting CSIRO in their scientific and research work. By leveraging AI tools and image classification technology, CSIRO has been able to rapidly build the stink bug detection models needed to help confidently identify the brown marmorated stink bug. All whilst developing new skills in automated machine learning and cloud automation tools.”

While Australia has around 600 named native stink bug species, as well as several thousand more undescribed species, the brown marmorated stink bug is native to China and has spread to many countries around the world, where it is a threat to crops such as apples, stone fruits, hazelnuts and grains.

It breeds in large groups in well-lit areas, such as car plants, then pregnant females hibernate in dark places, such as cars awaiting export. This can cause infestations in new countries where they lack specialised natural enemies to keep their populations in check.

The research team behind the app hopes to expand the work in two different directions.

“We want to add AI models for more types of biosecurity threats, beyond stink bugs, and we also hope to involve the public in biosecurity work so that people can identify and report pests and weeds,” Dr Schmidt-Lebuhn said.

CSIRO

CSIRO