Foundations and case studies on the scalable intelligence in AIoT domains

3.1 Perspectives on evolving technologies in the edge

The opportunities of doing AI in the edge is the target for Liu et al. [3]. They emphasize the challenges of computation and other resources on the edge. Their perspectives are to implement economical techniques for deep learning feasibility in low-capacity environments.

McMahan et al. [4] deliver an interesting viewpoint for federated learning (FL). The whole concept provides a safe way to reinforce the AI without passing data over the open Internet. Their results are a new solution, which they claim to reduce the need for transmitting the data and do ML on the edge in scale.

Computer vision is one of the key areas of AIoT. A survey arranged by Kittley-Davies et al. [5] point out that the visual feedback of the stages of pattern recognition is important, but difficult to achieve.

In their research, Lee and Nirjon are looking for a deep learning solution to edge computing [6]. The focus of this article is on dataset adaptation, and learning process by concentrating on feed-forward execution. The result is an effective adaptation process with efficiency.

Xiong and Chen point out the challenges on AIoT development [7]. Focusing on existing technologies, including 5G, their approach is presented with two use cases, which are considered common to edge computing. Those are streaming video analysis and industrial IoT (IIoT). The authors are in search of developing a cloud native edge computing system.

In their article, Lu and Zheng concentrate on the impact of 6G next-generation information systems [8]. They are looking forward to seeing the development of new intelligent solutions and use cases for secure interaction with systems. According to the authors, the 6G concept is in a major role as the enabling technology for different levels of connectivity and communication development.

The problem of complexity and management of IoT networks is addressed by Rafique et al. in their research on software defined networking (SDN) and edge computing [9]. Their concerns are in the different levels of vulnerabilities of the IoT systems, which could be managed by focusing on the SDN-Edge research and development.

In regard to autonomous systems, Jahan et al. [10] refer to both physical and virtual robots. Their target is in learning of security modeling in those environments. As a result, they emphasize the need for focusing the research more on possible threats and vulnerabilities on those application areas.

3.2 Viewpoints on applications

An interesting AIoT scale solution is addressed in an article concerning garbage classification. In their study, Song et al. use an applicable dataset [11]. The results reveal high accuracy for the new algorithm they developed.

A step toward low-end systems and AI is taken in a survey by Wang et al. [12]. Their aim is to implement deep learning with microcontroller units (MCUs). This kind of computation requires balancing with small memory sizes – especially considering random access memory (RAM) – and reduced computing power. With their experiments they have shown that the deep neural networks can be reduced to fit into constrained environments with acceptable performance.

A more focused approach is taken by Gao et al. [13] in a survey on the transition towards evolving intelligent robotics. That includes the next steps in collaboration between humans and robots. Also the interesting development of robot operating systems [14] is considered as a platform for more intelligent operations.

An important use case for realtime edge intelligence is unmanned aerial vehicles (UAVs). They can be extended to other areas as well, namely, underwater and terrain/subterrain operating systems. Xue et al. [15] emphasize the safety and reliability of positioning. Their solution is to use imagery collected from different sources including the UAV’s own camera. With the application of deep learning, they can detect spoofing.

Human–machine interaction (HMI) is revisited by Dong et al. [16]. They emphasize the energy supply in their research as well as the growing capacity of small-scale systems in the edge. The main conclusions they have made are the improvement of energy harvesting for increasing demand coming from more and more intricate sensors, actuators, computation, and storage solutions providing sophisticated AI for supporting decision-making.

Debauche et al. [17] introduce a new architecture for AIoT services deployment. Their approach is on application of state-of-the-art solutions: microservices, containers, AI in a customized perspective, and persistence of data with AI models. As continuation of the work by Debauche, they present also an interesting use case that is dealt with in the article about AIoT and real-time poultry monitoring. The research is carried out by Debauche et al. [18]. Their solution collects data every minute by a sensor network. The data is then applied to monitoring and prediction of conditions of poultry. An interesting finding is the way they implement a specific AI algorithm – gated recurrent unit – for environmental analysis.

Autonomous vehicles, and more specifically maritime applications, are under investigation by Chan [19]. They focus on background subtraction algorithms to provide good practices for this kind of special use cases of AIoT. As a result, they show its potential with benchmarks.

A practical approach to AIoT is taken by Zhang and Tao in their application to real-time monitoring of tunnel construction [20]. The telemetrics of operating tools in those conditions are collected and stored locally. Then the data is being applied to ML and to random forest in their case. The results show rapid responses providing real-time predictive control of construction equipment.

To the problem of environmentally sustainable systems, Yang et al. [21] propose the application of AI and more specifically reinforcement learning (RL). According to their studies, RL applied in this field of decision-making will deliver more intelligence to decision-making.

In consideration of the applicability of the IIoT Malik et al. [22] conclude in their review that applications will become ubiquitous. The foundation of their claim is in the availability enabling technologies.

Tanque [23] approach the fundamental building blocks of providing AI on IoT applications. Their research supports the usage of advanced technologies in edge-type environments to develop complete solutions from low-end data collection to AI.

4.1 Case study 1 – Experiences on setting up a scalable AI/ML with single-board commodity development devices

The first case is about setting up an AIoT system with a target as an intelligent home environment in a rural area. The purpose of this system is to show the optional independence of the data-to-knowledge process from the cloud services. Even though the cloud is considered as an important part of these kinds of systems, the network outage should not be a show stopper. This is provided by the local operations at the edge.

Power shortage may be addressed as well by using batteries and generators. Same ideas could be applied in an industrial environment as well.

The sensors and respective data collection described here is exemplary, not meant to represent fully digitalized living conditions. It is possible that with the selected components, one might approach this environment by compiling a digital twin, with a disclaimer that not everything is being controlled, though.

The communication technology to the Internet is based on 4G with speed varying from download speed between 10 and 50 Mbps and upload speed between 1–15 Mbps. Speed depends on the load based on nearby highway, and the net activity of neighbors. In this context, the cloud-only approach is not the primary choice, given that the 4G connections can also be occasionally down. These facts motivate us to examine on-premises AIoT services, with possible cloud options, naturally.

Since this case discusses about home dependent data, it is important that following conditions are met:

1. Privacy: No data nor AI models in any form may leak from the premises without consent,

2. Security: If data or AI is transmitted, it should be kept in encrypted format.

3. Reliability: The systems should keep data and AI persistent, even though individual components might crash or shut down.

1. Computer development boards and other equipment:

   • –   1 Raspberry Pi 4/8GB with Samsung 970 EVOPlus 500 GB SSD [24] memory device.

   • –   1 Coral Google Edge TPU ML Accelerator [25] connected to the previous Raspberry.

   • –   6 Raspberry Pi 4/4GB with Samsung 970 EVOPlus 500 GB SSD and 256 GB microSD memory devices: 4 are connected as a cluster; 1 as an in-house, 1 as storage building data collection stations.

   • –   1 Raspberry Pi 3 with a 256 GB microSD memory card: collecting streaming video with Raspberry Pi official NoIR camera V.2 and also data from one Ruuvi multisensor [26].

   • –   2 Nvidia [27] Xavier 16 GB with 256 GB microSD memory cards: one system runs a container with Timescaledb, and another container with GraphQL Application programming interface (API) between data provider, persistence, and consumers. Second Xavier 16 GB is for data science and ML development and operations.

   • –   Netatmo [28] weather system contains a base station, 3 additional integrated sensor stations, an anemometer, and a rain gauge.

   • –   4 Arlo [29] wireless 4K cameras and 2 wireless video doorbells for surveillance of the surroundings.

   • –   7 RuuviTag [26] Bluetooth Low Energy (BLE) multisensors (temperature, humidity, pressure, accelerometer, and telemetrics of the devices) measuring conditions in the refrigerator, freezer, sauna, rooms not covered with Netatmo stations, and 3 storage spaces.

   • –   2 DJI Tello [30] lightweight drones with cameras and accessible APIs.

   • –   All devices are connected to local Intranet – wired or wireless depending on their location and equipment.

2. Software tools and components

   • –   Operating systems for Raspberry Pis are 64-bit Raspbian versions [31]. For Nvidia devices, the operating systems are 64-bit Ubuntu versions embedded into the JetPack architecture of Nvidia [32].

   • –   All services are packaged into containers accessed from Docker repository, or built locally. Helm is applied for running the containers in Kubernetes [33].

   • –   In the cluster the containers are managed with Rancher K3S [34]. The idea here is to enable scaling out with additional different computers, but optionally to the cloud as well.

   • –   Tensorflow, Tensorflow Lite, and Tensorflow RT for deploying AI/ML [35].

   • –   Timescaledb [36] as an industrial-scale time series database is implemented for close to real-time response times for data persistence. It is also executed in containers.

   • –   GraphQL [37] running in its own container was selected because of its flexibility to query and mutation variations. API for dynamically changing operations with database is the target here.

1. Computer vision for showing the possibilities of development boards in practical applications. The two camera systems have different approaches: Arlo is the manufacturer of ready-to-use entities with an option to upload videos to the cloud service. The local storage is also provided. The second system is Raspberry Pi Camera concept, where users are required to compile the devices and software by themselves. The process is presented in Figure 2.

2. Local sensor data collection with AI operations as a target. The different tags containing sensors and providing appropriate APIs were put in place within the range of BLE communication with Raspberry Pis.

Figure 1

Example of edge system components.

Figure 2

Image detection and respective ML process with participating devices and software tools.

As a result, only demonstrations were made to learn the behavior and compatibility of software and hardware components. The different APIs were experimented, and it became clear that by splitting the activities between data collection and further processing can be done, but it needs a lot of attention and time.

It became also clear that there are APIs available for many intelligent home appliances. The camera systems from Arlo and the DJI drones provide their own connections with the options to communicate with their sensors and actuators. To set up an industrial-scale surveillance system with this equipment would require a lot more reliability and functionality of the systems.

The setup of the experimental system described earlier required many iterations. The data collection and storage processes are quite straightforward per se: sensors connected to nearby Raspberry Pis communicating with a relational database over a standard API technology with the support of basic operating system timing functionality.

On the other hand, a lot of the work was invested in detecting compatible software components and to make them work in practice.

Especially challenging is to make cloud-proven systems, like high availability Kubernetes to operate smoothly on the edge. Same applies to AI-related tasks, because of the novelty of all systems involved.

4.2 Case study 2 – Water quality measurement station at a mining site

Water quality monitoring at mining sites is an excellent example of industrial IoT (IIoT) application. Water quality monitoring is required to prevent environmental emissions. IIoT technology provides a feasible way to obtain data from a sparsely distributed measurement station network.

Mining industry consumes lots of water in the refining of ore. Used process water is stored into large tailings ponds. Water is purified and then recycled back into the process (95%) and outside the mining site as effluent (5%). The quality of water is monitored, because the performance of the process depends largely on the cleanness of the recycled water, and emissions of harmful substances to nature with effluent must be prevented.

Water quality is monitored continuously at monitoring stations, which are located by the tailings ponds. Monitored parameters include electrical conductivity of water (salt concentrations), turbidity (suspended solids), pH, and flow of water. The data from the measurements are usually transferred via wireless IoT systems.

Sometimes the mining sites are located in remote areas where the availability of cellular networks and electricity is limited. These limitations set the basic requirements for the IoT edge system (i.e., the monitoring system). They usually are powered with solar cells or/and wind turbines. A radio network, e.g., LoRa, is used for local data transfer. The system operates in harsh environmental conditions – in arctic areas, the temperatures in winter are often below − 3 5 ∘ C .

A measurement station is shown in Figure 3. The system consists of a monitoring well module, measurement sensors, and a LoRa transmitter. The sensors are hard-wired to the transmitter via CAN bus. The system is powered with a 100 W solar panel and a wind turbine (not shown in Figure 3). The transmitter is located in an insulated cabinet. The cabinet is heated with a 50 W cabinet heater, which is just enough to keep the temperature inside above 0 ∘ C in freezing ( − 3 5 ∘ C ) conditions.

Figure 3

Water quality measurement station.

From a measurement station, data is sent to the cloud (InFlux DB) via LoRa/Cellular router utilizing Publish-Subscribe protocols as described in Figure 4. The data can also be distributed to the production system of the mining site via the router, and the customer (mining company) has access to data stored in the database.

Figure 4

Water quality station’s IoT framework.

The quality of the data is quite critical, and the sensors are prone to so-called fouling. Foul consists of salts and dirt deposited on the surfaces of the sensors, and it interferes with the measurements. Sensors must be cleaned frequently, and there must be an indication of how much the fouling causes interference in the measurement. An ML model is used to detect the degree of fouling. This model is based on the detection of sensor signal variance and cross-correlation of all sensors with each other.

The ML model runs partially in the cloud and in the sensors as an embedded ML application. The embedded part of the ML model detects the signal variance index, while the cross-correlation of all signals is run in the cloud. Also the signal variance part could be run in the cloud, but running the model in the sensor instead of the cloud provided us a proof-of-concept of low-level embedded edge intelligence.

Referring to our RQs given earlier in the text, we may ask how the choice between cloud and edge computing should be made.

Part of the ML model requires information from all sensors, and there can be tens of monitoring stations at a mining site. Cross-correlations of all sensors against each other indicate which sensors begin to foul and should be maintained. The sensors have enough processing power to run this kind of model, but it does not make sense to send all the data to the sensors. Thus, computing the model in the cloud is justified. Alternatively, there could be an edge computer dedicated to this purpose.

On the other hand, the signal variance model does not need data from other sensors. Such a model also benefits from higher time resolution than the cross-correlation model. Thus, when running this part of the model in the sensors, we have the benefits of higher accuracy, because a high-time resolution model can be run in the processor of the sensor while keeping the data transmission rate low. This, in turn, saves the energy needed for the data transmission. Consideration of energy may sound trivial, but in arctic conditions, this matter is actually quite critical.

In this case, we have relied on both cloud and edge computing. The case is quite simple, but the benefits were well proven. With the ML model, the operators at the mining site were able to detect the sensors and stations that were in the need of maintenance. Because the access to the stations is difficult and distances long, a considerable amount of work was saved and the reliability of the water quality measurements was increased significantly.

4.3 Estimates of unit prices

The following price ranges deliver the scale of costs, since there is variability on both availability of components and their market prices. Given the rough price ranges on the fall 2022, the estimates are following:

1. Raspberry Pi: Prices vary from low end model 2 W Zero starting at 30 € to version 4/8 GB at 150 € .

2. Nvidia: From Jetson Nano 2GB at 50 € to Jetson AGX Orin at 2,000 € ,

3. SSD drives: 512 GB at 70 € .

4. Google Coral TPU accelerator: At 145 € .

5. Netatmo: Full Weather Station at 380 € .

6. Arlo: 4 devices at 4K Ultra cameras and the station 950 € .

7. RuuviTag: 40 € /tag.

8. DJI Tello: 100 € .

9. The Lora-IoT base stations: at 15 € /customized circuit board.

10. Weatherproof cabinets and other materials at 1,000 € .

11. Sensors: 1,500 € .