ΑΙhub.org
 

Accelerating laboratory automation through robot skill learning

by
26 October 2022



share this:
autonomous robotic scraping setup

Transforming materials discovery plays a pivotal role in addressing global challenges. The applications of new materials could range from clean energy storage, to sustainable polymers and packaging for consumer products towards a more circular economy, to drugs and therapeutics. Stemming from the COVID-19 pandemic, where scientists had to halt experiments due to stringent social distancing measures or accelerate their efforts towards quickly producing a vaccine, there has recently been an increased interest in using robotics and automation in laboratory environments. The challenge here is that laboratories have been designed by and for humans and thus the available glassware, tools and equipment pose difficult problems for traditional automation methods that are inherently open loop and not adaptable. Learning-based methods that rely on autonomous trial and error are increasingly being used to achieve robotic tasks that could not be previously addressed with automation. For laboratory robotics this is particularly beneficial for scenarios where humans would naturally adapt their movements for example in the presence of molecules that exhibit hardness, are hygroscopic or exhibit different properties. As a result, methods that rely on learning through interaction are crucial to success in such domains.

In robotics, reinforcement learning has become a compelling tool for robots to autonomously acquire behaviours and skills, by training an agent to interact with its environment and learn useful behaviours towards this goal. In this work, we introduce model-free reinforcement learning to the laboratory task of sample scraping. While there exist different laboratory skills that would benefit from this approach, autonomous sample scraping is fundamental in a large number of materials discovery workflows. This stems from the need to recover as much of the sample as possible since synthesising molecules is costly in both time and money.

Inspired by how human chemists carry out the task, we defined scraping as follows: the goal is to reach the target position at the bottom of a glass vial, while maintaining contact with the vial wall for material removal. We learn a scraping policy for a Franka Emika Panda robotic arm that relies on proprioceptive and force observations. We also explored curriculum learning [1], where the key idea is to learn first on simple examples before moving to more difficult problems. Here, we increased the challenge of the scraping task by moving the starting pose to outside the vial, such that the task now covers both insertion and scraping. By using a curriculum, we could successfully learn the more challenging insertion and scraping task, while the method without a curriculum fails to learn. We also demonstrate our method on a real robotic platform, illustrated in Figure 1 using tools and glassware that are typically used by human chemists. We trained our policy directly on hardware using a plastic vial, and then tested the model on a glass vial. We adopted this method to minimise potential glass breakage during training exploration.

autonomous robotic scraping setupFigure 1: An overview of the autonomous robotic scraping setup.

Vials on a bench with varying levels of scrapageFigure 2: Qualitative results of autonomous sample scraping, which illustrates (a) the initial vial condition, (b) midway through (powder partially removed) and (c) the powder scraped off.

For this task, a vial with powder on its inner walls was used and the robot was able to scrape off the majority of the powder, as illustrated in Figure 2. As we carried out the experiments on an open bench, we used food-grade material (flour) as the material inside the vial. Our work was shown to also generalise to different laboratory equipment, where the task was successful with different sized vials and scrapers of different length. This experiment even provided insight into optimising laboratory scraping for human scientists, particularly on the tool choice and how this could affect task success. We envisage that with an increased deployment of autonomous robotic scientists we could provide bi-directional knowledge to human scientists on optimisation of manual laboratory tasks, which would potentially also increase human throughput.

This work is a first step towards deploying the next generation of robotic scientists within a material discovery lab as part of the EU-funded ADAM project. In our earlier paper [2] that was published in ICRA 2022 and was an Outstanding Automation Paper finalist, we introduced an architecture for heterogeneous robotic chemists. This work will contribute towards this vision of having autonomous robotic systems carrying out laboratory experiments, albeit with more intelligence.

References

[1] S. Narvekar, B. Peng, M. Leonetti, J. Sinapov, M. E. Taylor, and P. Stone, “Curriculum learning for reinforcement learning domains: A framework and survey,” J. Mach. Learn. Res., vol. 21, pp. 181:1–181:50, 2020.
[2] H. Fakhruldeen, G. Pizzuto, J. Glawucki, and A. I. Cooper, “Archemist: Autonomous robotic chemistry system architecture,” IEEE International Conference on Robotics and Automation, 2022. [Read on arXiv here.]




Gabriella Pizzuto is a senior researcher at the University of Liverpool
Gabriella Pizzuto is a senior researcher at the University of Liverpool




            AIhub is supported by:


Related posts :



#AAAI2023 tweet round-up from the first two days

Find out what attendees have been up to at AAAI in Washington DC.
08 February 2023, by

Fully autonomous real-world reinforcement learning with applications to mobile manipulation

A system that learns to clean up a room directly with a real robot via continual learning.
07 February 2023, by

Radical AI podcast: featuring Deep Dhillon

In this episode Jess and Dylan chat to Deep Dhillon about the ins and outs of ChatGPT.
06 February 2023, by

What’s coming up at #AAAI2023?

Find out about the talks, workshops, tutorials, and other events scheduled at AAAI this year, taking place from 7 - 14 February.
03 February 2023, by

Science communication for AI researchers: our tutorial at #AAAI2023

Find out about our in-person hands-on course at AAAI.
02 February 2023, by

Riemannian score-based generative modelling

The winners of a NeurIPS 2022 best paper award write about their work on generative modelling.
01 February 2023, by





©2021 - Association for the Understanding of Artificial Intelligence


 












©2021 - ROBOTS Association