Automated handling of biological objects with a flexible gripper for biodiversity research

1 Introduction

The global decline of insect populations constitutes a profound environmental crisis, extensively documented across many scientific studies [1], [2], [3], [4], [5]. They highlight a major biodiversity issue that affects ecosystems worldwide because significant decreases in insect abundance and diversity are impacting critical natural processes, including pollination, nutrient cycling, and food webs that sustain both wildlife and humans. The loss of insects is thus not merely a conservation problem but a serious threat to ecological stability and biodiversity, with direct implications for the sustainability of agricultural productivity and thus food security [6], [7], [8]. All this explains why biodiversity decline is rated as one of the greatest threats to the world by the World Economic Forum [9]. It also highlights the need to develop new approaches to monitoring insect biodiversity. This is all the more important because our understanding of insect biodiversity is currently poor, with more than 80 % of all terrestrial arthropod species still unknown to science [10] and an even larger proportion unidentifiable to all but a few taxonomic experts. What is needed is the development of effective monitoring methods that allow for quickly identifying changes and causes of population change so that countermeasures can be implemented in a timely manner.

To evaluate insect bulk samples (see Figure 1a) and automate insect biodiversity research, we propose the pipeline shown in Figure 2. Specimens preserved in ethanol, either from new samples or museum collections, are first processed with the DiversityScannerr [11] or the Entomoscope [12]. The DiversityScanner is an automatic imaging, sorting, and classification device for insects, which is currently only suitable for specimens with a maximum length of 3 mm. The DiversityScanner localizes insects spread out in a Petri dish, takes an image of an individual specimen, and subsequently sorts it into a 96-well microplate. Larger specimens can be imaged and classified with the Entomoscope [12]. The Entomoscope is a photomicroscope developed for imaging and classifying insects preserved in ethanol. While the Entomoscope can process larger specimens cheaply and frugally and does not require extensive training, the DiversityScanner is even faster and allows for automated, high-throughput specimen handling with minimal user intervention. Both systems are suitable for classifying specimens based on images using artificial intelligence (AI). Three example images are shown in Figure 1b–d. Currently, the AI-based classification covers 14 different classes (mainly at the “family” level in the Linnean order) of the most commonly found specimens in Malaise trap samples [11], [12]. It is also important that the systems are open-source with part lists and building instructions available to the public, thus allowing for adaptations to specific use cases. In designing both systems, we emphasized the use of mostly 3D-printed or commercially available parts that are available online and inexpensive. Such a “frugal” approach to insect biomonitoring is crucial because it enables the collection and analysis of biodiversity in regions where financial resources are scarce [13]. Frugal methods for insect biomonitoring are urgently needed because none of the global targets for protecting nature are currently met while humanity’s well-being is intricately linked to healthy biodiversity. Unfortunately, the funding and infrastructure for high-cost scientific methods are lacking, especially in biodiverse tropical areas. Developing cost-effective, simplified methods for insect biomonitoring is thus critical so that data collection capabilities can be dramatically increased to enhance our ability to understand ecosystems and implement conservation measures. Currently, DiversityScanner and Entomoscope can already be used to sort, image, and classify bulk insect samples at a lower cost than DNA sequencing. The goal is to eventually classify most specimens into species using machine learning. The first step in this direction is sorting specimens into families or orders. This process already provides abundance and biomass information at this taxonomic level. To classify to species-level, we combine imaging with DNA barcoding [14], [15]. Each image can be tagged with a species identifier, and the tagged images can be used to train species-level AI models. Using this approach, the number of specimens that must be sequenced reduces over time because most specimens belong to species covered by AI models for image-based identification. New species will continue to be discovered, but they will eventually be documented well enough for inclusion in AI identification models.

Figure 1:

A Malaise trap bulk sample (a) and example images of three different insects from the dataset (b)–(d). (a) Insect bulk sample. (b) Diptera phoridae. (c) Diptera acalyptratae. (d) Diptera chironomidae.

Figure 2:

In the automated research pipeline for insects, specimens preserved in ethanol undergo initial processing using the DiversityScanner or Entomoscope. This step yields bulk samples, which are subsequently imaged, sorted, and classified to a specific degree. This process generates information on the abundance and biomass of identified species. DNA barcoding is conducted for unknown species, offering ground truth data for new datasets. These datasets are then utilized to train Convolutional Neural Networks (CNNs).

The automated handling of insect specimens in ethanol is a major challenge because specimens of numerous sizes and shapes have to be covered. Currently, the DiversityScanner is limited to specimens with a maximum body length of 3 mm. New methods are needed for handling larger specimens, to further automize the evaluation of insect bulk samples. The difficulty here lies in the different sizes and shapes of the various specimens, as well as in their extreme fragility. For handling small-sized specimens with the DiversityScanner pipetting has shown good results. For the DiversityScanner 4 K [16], the shortened, conical pipette tip that was originally used in the DiversityScanner is replaced with a glass tube with an inner diameter of 3.7 mm. Preliminary tests show that pipetting can be applied for specimens up to 6 mm, using glass tubes with larger diameters. However, larger specimens would require even larger pipette openings, but the ethanol tends to flow out, so a different transport principle must be applied. To the best of our knowledge, no suitable automated handling tool exists to manipulate insect specimens larger than 6 mm in ethanol. A hydraulic soft micro-gripper design presented by Kordmahale et al. [17] seemed promising, but it is way too small to handle specimens larger than 6 mm. Therefore, we decided to develop a new mini-gripper based on the same principle for larger specimens.

2 Concept and methods

A new pneumatic mini-gripper is designed and tested to handle insects in ethanol with a size of 6 mm–15 mm. The gripper uses a Polydimethylsiloxane (PDMS) membrane with three “fingers” positioned in an equilateral triangle. When the pressure on the inside of the gripper is increased, the membrane creates a convex shape, resulting in an opening of the gripper (see Figure 3 left). Decreasing the pressure creates a concave shape, resulting in its closing (see Figure 3 center). The membrane of the mini-gripper has a diameter of 45 mm and a thickness of 1 mm. This allows for high deformability while not being prone to tears. The fingers are arranged in an equilateral triangle with a distance of 12 mm, while being 15 mm long and having a diameter of about 3 mm. These values proved to be a good compromise between gripping width, grasping force, and stability. To cast the mini-gripper-membrane, a three-part 3D printed mold is designed, that can be printed on any FDM 3D printer. The premixed PDMS is filled into the mold with a disposable syringe. As casting material SYLGARD 184 (The Dow Chemical Company, Midland, USA) is chosen. A higher stiffness of 43 Shore A compared to the material recommended in Kordmahale et al. [17] (35 Shore A [17]) is necessary to provide the mini-gripper with higher stability. A 3D-printed connector connects the membrane to the tubing, which connects it to the control pump (see Figure 3 center and right). A suitable linear pump is developed to drive the mini-gripper (see Figure 4, (left)). A linear spindle drive stepper motor realizes the linear movement of an O-ring sealed piston in an aluminum tube. All components are screwed to a 3D-printed frame see Figure 4 (left). To verify the functionality of the design, a test bench (see Figure 4 right) is set up. The mini-gripper can be moved in the test bench vertically and horizontally by linear drives with stepper motors. Two Petri dishes are mounted underneath as pick-up and deposit containers for testing. Using the new mini-gripper, three insects with body lengths of about 4 mm, 8 mm, and 15 mm are first to be transported 50 times from the pick-up Petri dish to the target Petri dish to a predefined position. If the intermediate success rate is above 50 %, 50 additional repetitions are performed. After each repetition, a photo of the target Petri dish with the deposited insect is taken to determine the variance in deposition accuracy by creating a stacked image of all the photos taken. Tests are also conducted to determine if the orientation of the insect affects whether the mini-gripper can pick it up. Therefore, a static collection test is performed, where samples are picked up and held but not transported. A sum of 40 repetitions is executed. Every ten repetitions, the test specimens are rotated in 45° increments. Furthermore, tests are run to determine the smallest specimen size the mini-gripper can collect. For this purpose, specimens of decreasing size in steps of approximately 1 mm are selected, starting with 7.5 mm. Each insect is lifted and transported 20 times.

Figure 3:

The mini-gripper with three fingers in its opened position (left), in its closed position (center), and while holding an insect (right).

Figure 4:

Test bench for testing the mini-gripper (right) with an aluminum frame (1), a linear axis for moving the mini-gripper horizontally (2), a z-axis for moving the mini-gripper vertically (3), the mini-gripper itself (4), Petri dishes for picking up and placing the specimen (5), a motor control unit (6), and a linear pump (7). The pump (left) comprises a spindle-driven linear actuator (i) and an O-ring sealed piston inside an aluminum cylinder with a tube connector (ii). All components are mounted on a 3D-printed frame (iii).

3 Results

The mini-gripper successfully picked up specimens of 15 mm in 100 % of the cases for 100 repetitions. A slightly lower success rate of 95 % (repetitions: 100) was achieved for the 8 mm specimen. The testing series for the 4 mm specimen was canceled after 50 runs failed to collect specimens. The stacked images used to determine the repeatability of insect placement are shown in Figure 5. The circles where the specimens were placed are shown in blue. Note that only the bodies and not the wings were considered. For the 15 mm specimen (Figure 5 left), this circle had a diameter of approximately 25 mm, and for the 8 mm specimen (Figure 5 right) approximately 23 mm. Tests to determine the influence of rotation (0°, 45°, 90° and 125°) on the grasping of an insect showed for the 15 mm sample that there was no influence for positioning. The success rate was always 100 %. Slight changes can be observed with the 8 mm sample. The highest success rate (100 %) was achieved for 45° rotation and the lowest (70 %) for 90° rotation, followed by 0° (80 %) and 135° (90 %). The test series to find the lower size limit for the mini-gripper shows that the success rate decreases sharply with decreasing specimen size. While a success rate of 90 % is still achieved with a 7.5 mm specimen, it already drops to 70 % for a 6.5 mm specimen and is only 5 % and 0 % for a 5.5 mm specimen and a 4.5 mm specimen, respectively.

Figure 5:

Variance of the deposit locations for the mini-gripper with a specimen of 15 mm (left) and a specimen of 8 mm (right). All insects are deposited within a circle of approximately 25 mm for the 15 mm specimen and in a circle of approximately 23 mm for the 8 mm specimen.

4 Conclusion and discussion

A new type of mini-gripper is presented, suitable for handling ethanol-preserved insects between 7.5 mm and 15 mm in length. Especially for larger specimens (>8 mm), a very high success rate of over 95 % can be achieved. It is shown that the mini-gripper can securely grip and transport the insects regardless of their positioning. Below 7.5 mm the insects tend to slip past the three fingers of the mini-gripper. Increasing the pump volume and/or reducing the membrane thickness could reduce this limitation, making the mini-gripper suitable for even smaller specimens. Additionally, reducing the tube diameter could effectively reduce the volume of the pneumatic system which could also result in a better responsiveness of the mini-gripper. The mini-gripper is a pivotal advancement, facilitating the automated processing of a larger proportion of bulk biodiversity samples with the DiversityScanner. Two strategies can be adopted to realize this enhancement. One option involves configuring DiversityScanners for distinct size categories, necessitating the prior sorting of samples into defined size categories. We are currently developing an efficient, time-sensitive, and cost-effective system for this purpose, ensuring the fragile specimens remain intact throughout the process. Alternatively, a larger-scale DiversityScanner equipped with multiple handling tools tailored to specimens of varying sizes could be implemented. While this approach eliminates the need for pre-sorting, it introduces greater complexity and necessitates additional components. Each approach presents unique advantages; for instance, the latter obviates pre-sorting requirements but entails a more intricate system setup. Either way, the need for automated systems to study insect biodiversity cannot be overstated. Rapid deployment is imperative to quantify the scope of the ongoing mass extinction accurately and, ideally, institute mitigation measures. The DiversityScanner with the newly developed mini-gripper is currently being used as part of an EU project, “FORSAID” (FORest Surveillance with Artificial Intelligence and Digital technologies), to detect invasive insect species in Europe.