Skip to main content
Chapter
Licensed
Unlicensed Requires Authentication

3 Application of convolutional neural networks for visual control of intelligent robotic systems

  • , and
Become an author with De Gruyter Brill
Explore this Subject How to publish with us
Soft Computing in Smart Manufacturing
This chapter is in the book Soft Computing in Smart Manufacturing

Abstract

Intelligent mobile robots are foreseen as one of the possible solutions to efficiently performing transportation and manipulation tasks in intelligent manufacturing systems (IMS) of Industry 4.0. In the last few decades, deep learning models have been recognized as a promising technique to enable the intelligent behavior of mobile robots for performing such tasks. For the particular problems of object detection and classification, a class of deep learning models, namely Convolutional Neural Networks (CNN), is the most widely used. This chapter presents an application of Region-based CNN (R-CNN) for advanced object identification tasks by using transfer learning. The proposed learning approach is further used for the improvement of Image- Based Visual Servoing (IBVS) algorithm used to control an intelligent mobile robot. The proposed algorithms are implemented in the MATLAB software package, and both simulation and the experimental verification of the proposed concept are performed on intelligent mobile robot, DOMINO (Deep learning Omnidirectional Mobile robot with INtelligent cOntrol). Four different CNN models are trained for object detection and classification, and the most suitable CNN model is ResNet-18, with the best recorded mean Average Precision (mAP) of 77%. Achieved experimental results show the applicability of CNN for accurate detection and classification of different manufacturing entities and the IBVS algorithm for efficient mobile robot control within IMS.

Abstract

Intelligent mobile robots are foreseen as one of the possible solutions to efficiently performing transportation and manipulation tasks in intelligent manufacturing systems (IMS) of Industry 4.0. In the last few decades, deep learning models have been recognized as a promising technique to enable the intelligent behavior of mobile robots for performing such tasks. For the particular problems of object detection and classification, a class of deep learning models, namely Convolutional Neural Networks (CNN), is the most widely used. This chapter presents an application of Region-based CNN (R-CNN) for advanced object identification tasks by using transfer learning. The proposed learning approach is further used for the improvement of Image- Based Visual Servoing (IBVS) algorithm used to control an intelligent mobile robot. The proposed algorithms are implemented in the MATLAB software package, and both simulation and the experimental verification of the proposed concept are performed on intelligent mobile robot, DOMINO (Deep learning Omnidirectional Mobile robot with INtelligent cOntrol). Four different CNN models are trained for object detection and classification, and the most suitable CNN model is ResNet-18, with the best recorded mean Average Precision (mAP) of 77%. Achieved experimental results show the applicability of CNN for accurate detection and classification of different manufacturing entities and the IBVS algorithm for efficient mobile robot control within IMS.

Chapters in this book

  1. Frontmatter I
  2. Preface V
  3. About the editors IX
  4. Contents XI
  5. List of contributing authors XIII
  6. 1 Control, monitoring, modeling, and optimization of manufacturing and machining technologies 1
  7. 2 Trustworthy human–machine assistance by dynamic process security monitoring in industrial environments 55
  8. 3 Application of convolutional neural networks for visual control of intelligent robotic systems 83
  9. 4 Digital twins in smart manufacturing 113
  10. 5 A novel implementation of an open CNC PC-based and service-oriented architecture-based monitoring system for STEP-NC – a case study 135
  11. 6 Smart system based on STEP-NC for machine vision inspection (3SMVI) 155
  12. 7 Soft computing in advanced cutting processes 181
  13. 8 Modeling and optimization of AWJM process on the processing of Banana fiber-reinforced polymer composites using Taguchi-JAYA method 253
  14. Index 271
https://doi.org/10.1515/9783110693225-003

Chapters in this book

  1. Frontmatter I
  2. Preface V
  3. About the editors IX
  4. Contents XI
  5. List of contributing authors XIII
  6. 1 Control, monitoring, modeling, and optimization of manufacturing and machining technologies 1
  7. 2 Trustworthy human–machine assistance by dynamic process security monitoring in industrial environments 55
  8. 3 Application of convolutional neural networks for visual control of intelligent robotic systems 83
  9. 4 Digital twins in smart manufacturing 113
  10. 5 A novel implementation of an open CNC PC-based and service-oriented architecture-based monitoring system for STEP-NC – a case study 135
  11. 6 Smart system based on STEP-NC for machine vision inspection (3SMVI) 155
  12. 7 Soft computing in advanced cutting processes 181
  13. 8 Modeling and optimization of AWJM process on the processing of Banana fiber-reinforced polymer composites using Taguchi-JAYA method 253
  14. Index 271
Downloaded on 19.4.2026 from https://www.degruyterbrill.com/document/doi/10.1515/9783110693225-003/html
Scroll to top button