Skip to main content
Journal of Cloud Computing

Advances, Systems and Applications

Download PDF

The product quality inspection scheme based on software-defined edge intelligent controller in industrial internet of things

Journal of Cloud Computing volume 12, Article number: 113 (2023) Cite this article

Abstract

The Industrial Internet of Things (IIoT) enables the improvement of the productivity and intelligent level of factory. The procedure of product quality inspection has generally adopted machine intelligence algorithms instead of manual operation to improve efficiency. In this paper, we propose a product quality inspection system scheme based on software-defined edge intelligent controller (SD-EIC). By adopting the software definition and resource virtualization technologies, the hardware platform of SD-EIC is designed to support the real-time control tasks and non-real-time edge computing tasks at the same time. To this end, we propose the scheme and architecture of product quality inspection system based on SD-EIC. Multiple virtual controllers and virtual edge computing nodes are constructed on a set of SD-EIC hardware platform to realize the integrated deployment of the real-time control for terminal devices and the AI model reasoning of product defect recognition algorithm based on machine vision respectively. In addition, the management and control scheme of product quality inspection system based on industrial information model is proposed. By constructing the semantic-based digital twin information model of terminal device, the flexible adjustment and parameter configuration of terminal device are realized to meet the demands of flexible production and manufacturing. The proposed product quality inspection system solution can effectively improve the utilization of hardware resources and the efficiency of product quality inspection, and reduce the overall deployment cost of the system. It can flexibly adapt to product diversity and different industrial scenarios.

Introduction

With the development of Industrial Internet of Things (IIoT) and Industry 4.0 technologies, the intelligent transformation and upgrading of manufacturing industry is promoting and driving the boundary between the physical world and the cyber world more and more blurred [1, 2]. This requires the further in-depth integration and intelligent coordination of operation technology (OT) and information technology (IT) to form an efficient, real-time, flexible and safe industrial automation system [3]. So the intelligent level and production efficiency of industrial manufacturing will continue to improve. It is an important goal for the development of IIoT and smart manufacturing technologies to replace traditional manual operation and improve work efficiency and product quality through the automatic and intelligent equipments and technologies [4].

In the process of industrial production and manufacturing, product quality inspection is an important part, which directly affects the quality of products. The traditional quality inspection method is manual judgment, which relies on the experience of quality inspectors to inspect product defects by visual observation. Due to the limited ability and experience of quality inspectors, they cannot achieve high inspection accuracy. The missed rate and false detection rate is high, and the efficiency is low. Moreover, it is difficult to detect the subtle defects.

In the new smart manufacturing system, the artificial intelligence (AI) technologies such as machine vision and deep learning are applied in the field of autonomous product quality, which has gradually become the mainstream methods inspection [5]. They collect the appearance image information of products through industrial camera or high-definition camera. Then these images are analyzed and understood with the image recognition algorithms to automatically distinguish defective products. The automatic machine vision technology can greatly improve the efficiency and accuracy of product quality inspection [6].

There are many intelligent methods have been proposed and applied in the production and manufacturing scenarios. These advanced algorithms usually need to be deployed on cloud platforms or servers with high requirements on computing power and network. This may cause time delay, high cost and energy consumption in practical applications with massive data to be processed [7]. In addition, the existing schemes can detect a single type of product. The equipment is highly specialized and can only detect specific products and specific appearance problems in specific scenarios. The inspection algorithm cannot be self-learning, and cannot continuously improve the accuracy with the increase of detection products [8].

Edge computing promotes the integration of IT and OT. It enables traditional industrial control to interoperate with AI algorithm model by performing these tasks on a single hardware platform [9, 10]. This improves the efficiency of operating equipments and developing industrial applications. On the one hand, the high reliability and real-time performance of OT applications can ensure. On the other hand, the advanced IT methods can also be integrated. Therefore, IIoT technologies based on edge computing will accelerate the realization process of industrial intelligence [11].

In view of the disadvantages of the existing product quality inspection schemes and the highlights of edge computing, this paper proposes a product quality inspection scheme based on software-defined edge intelligent controller (SD-EIC). The scheme uses edge computing, edge intelligent controller, machine vision and AI technologies to realize the automatic quality inspection of industrial products. On the whole, the scheme adopts the deployment mode of AI model training in the cloud and AI model reasoning at the edge. This makes full use of the powerful computing power of cloud and the high real-time of edge intelligent controller to improve the efficiency of quality inspection system.

In the proposed scheme, we adopt the SD-EIC to support real-time control tasks and non-real-time edge computing tasks at the same time. Through the integrated deployment of machine vision recognition model and industrial intelligent control function, it can not only realize the model reasoning of product defect recognition algorithm based on machine vision, but also realize the real-time control of servo motor and robot arm on the same hardware platform. It improves the real-time performance of overall inspection system effectively. Meanwhile, it also improves the utilization rate of hardware resources and reduces the number of devices and the overall deployment cost of the system. The architecture of quality inspection system is simplified. Furthermore, the scheme has a self-learning mechanism of product defect recognition model based on cloud-edge cooperation to improve the robustness and accuracy of model. In addition, the management and control scheme of product quality inspection system based on industrial information model is designed. By constructing the semantic based digital twin information model of terminal device in the SD-EIC, the flexible adjustment and parameter configuration of terminal device can be realized on the edge side. It effectively solves the problem that the traditional quality inspection system can only detect a single product, and meets the inspection demands of multiple appearance quality problems for multiple products in multiple industrial scenarios.

The main contributions of this paper are shown as follows.

  • Based on software-definition mechanism, the hardware platform architecture of SD-EIC is proposed to implement the integrated deployment of real-time industrial control tasks and non-real-time edge computing tasks on a set of hardware infrastructure.

  • The product quality inspection system scheme based on SD-EIC is designed to provide an automated product defect inspection solution with self-learning mechanism based on cloud-edge collaboration.

  • A management and control scheme based on industrial information model is proposed to flexibly adapt to different product defect inspection scenarios and meet the demands of flexible production and manufacturing.

The rest of this paper is organized as follows. Section II reviews the related works on product quality inspection scheme. Section III presents the hardware platform of SD-EIC. Section IV proposes the product quality inspection system scheme based on SD-EIC. Section V proposes the flexible management and control scheme of product quality inspection system based on industrial information model. Section VI summarize the characteristics of proposed scheme. Section VII draws a conclusion.

Related works

Currently, the IoT, edge computing, machine vision, and AI technologies have been applied in industrial applications to enable the automation and intelligence [12]. The automated product quality inspection has become a hot topic in recent years. The machine vision methods have been widely adopted in quality inspection scenarios [13]. According to different deployment architectures, these schemes can be divided into two categories: cloud computing-based quality inspection scheme and edge computing-based quality inspection scheme.

For the cloud computing-based quality inspection scheme, the industrial camera is used to collect appearance image data of product, which is uploaded to cloud center. The image data is analyzed and recognized by using the powerful computing power of cloud and the deployed defect recognition algorithm [7]. Then the recognition results are fed back to the manufacturing execution system (MES). When a defective product is found, the MES sends a control command to the motion controller, which controls the robot arm to perform corresponding operations on the defective product.

Although this method can make full use of the powerful computing power of cloud for AI tasks and realize automatic product quality inspection based on machine vision, it needs to upload the image data to the cloud for recognition. The time delay of the whole processes is relatively large, which is difficult to meet the industrial scenes with high real-time requirements. Moreover, this method needs to consume large bandwidth resources and high network cost.

For the edge computing-based quality inspection scheme, the appearance image of product is analyzed and recognized locally by deploying the edge computing server in the industrial field and deploying the defect recognition algorithm to the edge computing server. When a defective product is found, the edge computing server feeds back the recognition result to the motion controller, which controls the robot arm to perform corresponding operations on the defective product [14].

This method completes the recognition and control tasks on the edge side, which reduces the transmission delay and network bandwidth cost [15]. However, due to the decentralized deployment mode, the edge computing server and motion controller need to be deployed respectively. This results in the high deployment cost and operation cost and the low resource utilization. In addition, the current product quality inspection system based on edge computing can only inspect a single product, which can only be used to detect specific appearance problems for specific products in specific scenes. The scalability and adaptability is poor.

In above schemes, edge computing and cloud computing have been adopted to deploy the product quality inspection system. On this foundation, we design the product quality inspection scheme and architecture based on SD-EIC with terminal-edge-cloud collaboration framework. SD-EIC is applied to implements the integrated deployment of product defect recognition algorithm and real-time control algorithm at the edge side. The cloud platform mainly support the model training and self-learning mechanism of product defect recognition algorithm. The proposed product quality inspection scheme simplifies system architecture, reduces the deployment cost, and supports the flexible adaptation to multiple industrial scenarios.

Hardware platform of SD-EIC

Different from traditional PLC, SD-EIC is a new industrial controller which adopts the new hardware architecture can support both real-time industrial control tasks and non-real-time edge computing tasks in the way of software definition [16]. It adopts industrial fanless design and can adapt to wide temperature range. It has the advantages of small size, low cost and low power consumption, and can meet the demands of different complex industrial scenarios.

SD-EIC adopts the hybrid heterogeneous computing architecture based on multiprocessor, and combines virtualization technology to build virtual controller and virtual edge computing node with software definition mechanism. SD-EIC can run real-time and non-real-time operating system on the same hardware platform synchronously, so as to meet the diversified real-time control and edge computing tasks in various applications. It integrates edge computing into the industrial controller, so that the controller has the abilities to load and execute AI algorithms including machine vision, image processing, deep learning and pattern recognition on the edge side. It can not only realize real-time control functions such as motion control and logic control, but also realize edge computing functions such as localized image recognition, product defect inspection and intelligent decision-making [17, 18].

As shown in Fig. 1, we propose the hardware platform architecture of SD-EIC. The hardware resources of SD-EIC include computing resources, storage resources, communication resources and peripheral input/output (IO) resources. The core computing unit adopts the hybrid heterogeneous architecture of multi-core CPU + NPU + FPGA. According to different task processing requirements, it schedules different computing resources to realize real-time control, edge computing and other functions in a way of software definition. Among them, real-time motion control tasks will call FPGA and CPU resources to meet the requirements of low delay, and edge AI computing and industrial machine vision recognition tasks will call NPU resources to meet the requirements of high complexity and low latency processing. The storage resources include DDR, flash, hard disk, etc. The communication resources include high-speed real-time network, industrial fieldbus, Ethernet, etc. The peripheral IO resources include AI/AO, DI/DO and other interfaces. All these resources constitute the hardware foundation of SD-EIC.

Fig. 1
figure 1

Hardware platform architecture of SD-EIC

Full size image

In addition, the SD-EIC also provides a variety of communication interfaces to facilitate the access of various devices, including Gigabit Ethernet port, RS-232/422/485 serial port, 5G/4G/WiFi/LTE wireless transmission port, USB port, VGA and HDMI port, etc. The SD-EIC can support various mainstream fieldbus protocols, such as ProfiBus-DP, ProfiNet, EtherNet, Powerlink, and custom protocols. It reserves PCIe communication interface to support the expansion of more industrial communication protocols [19, 20]. The whole machine adopts modular design, and the main resources and expansion resources can be flexibly expanded. In addition to supporting conventional data acquisition functions, it also supports high-frequency data sampling of field devices and sensors through high-frequency data acquisition modules.

Based on the hardware resources of SD-EIC, the hypervisor layer is deployed to realize the virtualization mapping and scheduling management of hardware resource. It adopts the key technologies of virtual CPU kernel scheduling, HyperCalls, memory address mapping management, virtual IO handler and configurator. According to the resource allocation requirements of the controller and edge computing node, two types of virtual machines are constructed, namely virtual controller and virtual edge computing node. The mapping and scheduling relationships between virtual resources in the virtual machine and physical hardware resources are established. Furthermore, the real-time and non-real-time operating systems are run on the two types of virtual machines respectively.

In a set of SD-EIC hardware platform, multiple virtual controllers and multiple virtual edge computing nodes can be constructed synchronously in the way of software definition. They respectively call different hardware resources and perform physical isolation. This can ensure that the virtual controller is not affected by the virtual edge computing node to ensure the reliable execution of control tasks. They share a set of hardware infrastructure and resources. The shared memory method is applied to realize the rapid data interaction between real-time control applications and non-real-time edge computing applications, and improve the communication efficiency between different processing units.

  • Virtual controller: According to the different control requirements, virtual controllers with differentiated computing power can be constructed by adopting the resource virtualization technologies to flexibly configure hardware resources. In the virtual controller, industrial real-time operating system (RTOS) is run. Through deploying motion control algorithm, the real-time control functions for robot arm, servo motor and other objects can be realized.

  • Virtual edge computing node: Based on NPU computing resources, virtual edge computing nodes with differentiated computing power are constructed by adopting the resource virtualization technologies. The non-real-time operating system Linux is run on it. By loading and deploying the product defect recognition algorithm based on machine vision, the automatic inspection of product defects can be realized on the edge side.

Based on the virtual controller and virtual edge computing node, the SD-EIC realizes the integrated deployment of machine vision recognition and intelligent control. This mode replaces the traditional deployment scheme based on distributed devices, which need to deploy edge server, motion controller and programmable logic controller (PLC) to achieve the above functions. The integrated implementation of machine vision recognition and industrial control can effectively reduce the communication delay between recognition module and control module, and decrease the number of devices. And the architecture of quality inspection system is simplified. Moreover, by using resource virtualization technologies, multiple virtual controllers and multiple virtual edge computing nodes can be constructed on the one SD-EIC hardware platform, which can support multiple work stations to execute product quality inspection tasks synchronously at the same time. The scheme improves the utilization of hardware resources and the efficiency of product quality inspection, and reduces the overall deployment cost of the system. In addition, the SD-EIC has various communication interfaces and supports many industrial communication protocols. It can be compatible with the interconnection and intercommunication of various devices from different manufacturers, and flexibly adapt to a variety of different industrial scenarios.

In addition, SD-EIC can collaborate with the cloud computing center. It can deploy the virtualization management platform to support lightweight microservices and container deployment. It can also harmoniously adapt to the current mainstream cloud native technology to achieve cloud-edge collaboration.

The product quality inspection system based on SD-EIC

System architecture

As shown in Fig. 2, we propose the architecture of product quality inspection system based on SD-EIC. On the whole, the system is a terminal-edge-cloud collaboration architecture, and the main functional modules are described as follows.

Fig. 2
figure 2

The architecture of product quality inspection system based on SD-EIC

Full size image

  • Terminal

The terminal layer mainly includes high-speed industrial camera, servo motor, robot arm and other sensors and actuators. They are installed on the production line of the factory shop floor. The industrial camera is responsible for collecting the appearance image information of products. The servo motor and robot arm are responsible for receiving and executing the control commands issued by the SD-EIC. According to the result of product defect inspection, the normal products will flow into the next production process, and the defective product will be taken out and placed in the designated position.

  • Edge

The edge layer is mainly a SD-EIC, which can be installed in the machine room of the workshop or near the production line. In practical production, the SD-EIC usually adopts the deployment mode of active / standby redundancy mechanism to ensure high reliability and availability. When the main controller fails and cannot work, the terminal device can be taken over by the backup controller seamlessly. It mainly includes the following functions.

  1. (1)

    The product defect recognition algorithms based on machine vision are deployed in the virtual edge computing node of SD-EIC, which receives the product appearance image data collected by the industrial camera, and processes, analyzes and recognizes it to realize the automatic inspection of product defects.

  2. (2)

    The motion control algorithms are deployed in the virtual controller of SD-EIC, which generates the control commands according to the product defect recognition results, and controls the servo motor and robot arm to perform the corresponding actions.

  3. (3)

    By constructing the information models of terminal devices, the management and parameter adjustment of terminal devices are implemented in the SD-EIC, so as to adapt to the industrial products with different types and different sizes from different scenes. It can meet the requirements of flexible manufacturing.

  4. (4)

    The SD-EIC sends the image information of defective products to the industrial cloud platform periodically for self-learning of product defect recognition model. The industrial cloud platform regularly updates the latest trained product defect recognition model to SD-EIC to continuously improve the accuracy of defect recognition.

  5. (5)

    The qualified rate, defective rate and other statistical data of products are counted regularly in SD-EIC. On the one hand, they are visually displayed in the local human machine interface (HMI) display screen for viewing at any time. On the other hand, the statistical results are also provided to the industrial cloud platform, and used by the upper MES, ERP and other industrial systems.

  • Cloud

The cloud layer is mainly the industrial cloud platform. Relying on the powerful computing and storage capacities of cloud computing, the model training tasks of product defect recognition algorithm based on machine vision is deployed and implemented. And the images of defective products are continuously collected to increase the training sample database. Through periodically relearning of model, its robustness and accuracy will improve continuously.

In this product quality inspection system, the SD-EIC provides a variety of communication interfaces. So the terminal devices can access SD-EIC through high-speed Ethernet, fieldbus, 5G/WiFi and other industrial communication protocols. The SD-EIC can access the industrial cloud platform through the core network.

The workflow of product quality inspection system

In our proposed scheme, the detailed workflow of product quality inspection system is shown in Fig. 3.

Fig. 3
figure 3

The detailed workflow of product quality inspection system

Full size image
  1. (1)

    The industrial camera collects the image information of the product to be inspected, and sends it to the virtual edge computing node in the SD-EIC.

  2. (2)

    The virtual edge computing node recognizes the received image information through the defect recognition algorithm to determine whether the product is qualified.

  3. (3)

    If it is a qualified product, the virtual controller in the SD-EIC will generate control commands to control the servo motor and robot arm to transmit the product to the next process normally.

  4. (4)

    If it is a defective product, the virtual controller will generate control commands to control the servo motor and robot arm to take out the defective product to the specified position and give an alarm. Meanwhile, the image data of defective product will be sent to the industrial cloud platform for regular self-learning of product defect recognition model.

  5. (5)

    The system regularly counts the qualified rate and defective rate of products, and visually displays them in the local HMI display screen. In addition, these statistical data will be sent to the industrial cloud platform to support the upper industrial applications.

Self-learning mechanism of product defect recognition model based on cloud-edge collaboration

In the process of product quality inspection, when a defective product is found on the edge side, the SD-EIC sends the image data of the defective product to the industrial cloud platform as a new sample of defective product image database. In this way, the data volume of defective product image data set continues to increase. The cloud platform regularly retrains the product defect recognition model based on new data set to improve the robustness and accuracy. Furthermore, the cloud platform sends the latest model to the SD-EIC for model updating. In this way, the model self-learning mechanism based on cloud-edge collaboration can be implemented. The self-learning processes of product defect recognition model are shown in Fig. 4.

Fig. 4
figure 4

The self-learning processes of product defect recognition model

Full size image

Flexible management and control scheme of product quality inspection system based on industrial information model

In order to improve the application scope of proposed product quality inspection system and meet the inspection requirements of multiple appearance quality problems of multiple products in multiple industrial scenarios, we propose a management and control scheme based on industrial information model [21]. The digital industrial information models of terminal devices (including industrial camera, robot arm, and servo motor) are constructed in the SD-EIC to form the one-to-one digital twins [22]. When a new device is connected, the corresponding attribute data needs to be collected and the classes need to be configured according to the content defined in the industrial information model. The synchronization mechanism between digital twin and physical device is also implemented to respond to changes of device attributes or states, which is based on periodic or event triggered mechanism. It is used to realize the flexible adjustment and configuration of terminal devices on the edge side [23, 24]. This scheme can support the demands of flexible production and manufacturing, and realize an intelligent, flexible and adaptive industrial product quality inspection solution.

The information models of terminal devices adopt the digital twin description modeling technologies based on semantic. The identification (ID), attribute, class and other relevant information of the terminal device are defined into a standard digital abstraction and semantic description model according to the rules and basic framework, which is shown in Fig. 5. Moreover, the resource description framework (RDF) language is used to formally express the model and transform it into a language expression form that can be understood by machine.

Fig. 5
figure 5

The basic framework of information model

Full size image

In the basic framework of device information model, the ID is the unique mark of the device identity. The attribute is the description for the basic information of device, including static attribute and dynamic attribute. Static attribute is the description for the static characteristic and relationship of device, such as name, size, shape, etc. Dynamic attribute is the description for the dynamic characteristic and relationship of device, such as speed, working state, etc. The class is the description for the operations or methods or behaviors performed on the device.

The information model of industrial camera is shown in Fig. 6. ID number is the identity of camera. The static attribute includes resolution ratio, maximum frame rate, communication interface, etc. They describe the basic and static information of industrial camera. The dynamic attribute includes control command, working state, camera angle, etc. They define the dynamically adjustable parameters of the camera, so as to support image acquisition for appearance problems of different products, different angles and different granularity. The information model also defines camera operation class which can perform control operations on the camera.

Fig. 6
figure 6

The information model of industrial camera

Full size image

The information model of robot arm is shown in Fig. 7. ID number is the identity of robot arm. The static attribute includes manufacturer name, maximum arm length, communication interface, etc. They describe the basic and static information of robot arm. The dynamic attribute includes control command, clamping angle, clamping mouth size, etc. They define the dynamically adjustable parameters of robot arm. For different application scenarios, products of different sizes can be clamped by adjusting these parameters. The information model also defines operation control class which can perform control operations on the robot arm. The servo drive class defines multiple categories of servo drive interfaces to support servos and robot arms from different manufacturers.

Fig. 7
figure 7

The information model of robot arm

Full size image

The information model of servo motor is shown in Fig. 8. ID number is the identity of servo motor. The static attribute includes manufacturer name, size, communication interface, etc. They describe the basic and static information of servo motor. The dynamic attribute includes control command, speed, rotation direction, etc. They define the dynamically adjustable parameters of servo motor, so as to meet different servo service requirements. The information model also defines operation control class which can perform control operations on the servo motor. Spindle drive class defines multiple categories of servo spindle drive interfaces to support servos from different manufacturers.

Fig. 8
figure 8

The information model of servo motor

Full size image

Based on above information model, RDF language is used to formally express the model contents. The RDF file of model is generated and stored in the virtual controller of SD-EIC. The virtual controller can dynamically load and parse RDF file to obtain model contents and parameters. The information model is used as the input for control algorithm model. The parameters of the information model instance can be modified, so that the implementation effect of control algorithm can be adjusted flexibly and conveniently.

The information models of terminal devices are the digital twins corresponding to the physical devices one by one. The modification of attributes and states in the information model can be transformed into the control and adjustment of physical device in real time. Therefore, according to the inspection requirements of various appearance quality problems of various products in different industrial scenes, the virtual controller in the SD-EIC can generate corresponding control commands based on the instances of information models to realize the flexible adjustment and parameter configuration of terminal devices. For example, the shooting angle and focal length of the camera can be reasonably adjusted according to the inspection requirements. According to the size of product, the clamping angle and the size of clamping mouth of the robot arm can be adjusted and controlled flexibly. In this way, the proposed quality inspection system solution can efficiently detect multiple types of products on one or more production lines, and the application scope of system is expanded.

Performance analysis

In practical production, the proposed product quality inspection system scheme based on SD-EIC can be applied in many industrial fields, for example, quality inspection of printed circuit board (PCB), welding inspection, surface scratches and smoothness inspection, etc. The advantages of this scheme mainly include the following aspects.

1) Improving utilization of basic hardware resources

In our scheme, the SD-EIC adopts resource virtualization technology. Multiple virtual controllers and multiple virtual edge computing nodes can be constructed synchronously in the way of software definition on the one SD-EIC hardware platform. They share a set of hardware infrastructure and resources. And the one SD-EIC hardware platform can support multiple work stations to execute product quality inspection tasks synchronously at the same time. This greatly improves the utilization rate of hardware resources.

2) Simplifying the inspection system architecture

The SD-EIC realizes the integrated deployment of product defect recognition based on machine vision and intelligent control tasks in virtual controller and virtual edge computing node. It replaces the traditional deployment scheme based on distributed devices, which need to synchronously deploy edge server, motion controller and PLC to achieve the functions of product defect recognition and intelligent control. It greatly reduces the number of devices, simplifies the inspection system architecture, and facilitates the deployment and maintenance of system.

3) Decreasing system communication delay

On the one hand, our scheme adopts terminal-edge-cloud collaboration architecture. The product defect recognition algorithm model based on machine vision can be deployed at the edge side. The product appearance data collected by the camera can be analyzed and processed at edge computing side without sending it to the cloud. It not only reduces the network transmission, but also decreases the system communication delay. On the other hand, the SD-EIC adopts the shared memory method to realize the rapid data interaction between product defect recognition module and intelligent control module. It improves the communication efficiency between different processing units.

4) Reducing the cost of the overall solution

The proposed defect inspection scheme applied SD-EIC in the edge side to implement the real-time control and computing functions, which need to synchronously deploy edge server, motion controller and PLC to achieve them in traditional deployment scheme. The one SD-EIC hardware platform can support multiple work stations to execute product quality inspection tasks synchronously. In addition, the same set of hardware infrastructure can support the defect inspection for multiple appearance quality problems of multiple products by the management and control scheme based on industrial information model. For a manufacturing enterprise, our scheme will greatly reduce the cost of overall system.

5) Flexibly adapting to the product quality inspection requirements of multiple industrial scenarios

With the proposed product quality inspection system architecture based on SD-EIC, we design a management and control scheme based on industrial information model. The information models of terminal devices are constructed to digitally abstract and describe device information in the form of digital twins. The information model is used as the input for control algorithm model. The virtual controller in the SD-EIC can generate corresponding control commands based on the information model instances. Moreover, in a set of SD-EIC hardware platform, multiple virtual controllers and multiple virtual edge computing nodes can be constructed synchronously in the way of software definition. Therefore, by adjusting and configuring the parameters in the information model instance, the proposed scheme can to detect multiple types of products on the same or different production lines, and support multiple product quality inspection scenarios. It can satisfy the inspection demands of multiple appearance quality problems for multiple products in multiple industrial scenarios.

Conclusion

In this paper, we have proposed the scheme and system architecture of product quality inspection based on SD-EIC to enable the applications of smart manufacturing and IIoT and implement the automatic inspection and recognition of the appearance quality defect of industrial products without manual participation. It improves the work efficiency of product quality inspection. In our scheme, the SD-EIC supports real-time control and non-real-time edge computing functions synchronously. On the same hardware platform, it not only implements the AI model reasoning of product defect recognition algorithm, but also implements the real-time control of servo motor and robot arm. It effectively reduces the communication delay between recognition module and control module. Moreover, a set of SD-EIC hardware can virtualize multiple virtual controllers and multiple virtual edge computing nodes, so that it can simultaneously serve the quality inspection tasks of multiple work stations and various types of products. The method can replace the traditional edge server and PLC, and improve the utilization of hardware resources. Through the self-learning mechanism of product defect recognition model based on cloud-edge cooperation, the robustness and accuracy of the model can be improved. In our proposed scheme, the semantic based digital twin information model of terminal device is constructed in the SD-EIC to realize the flexible adjustment and parameter configuration of terminal device. It can meet the inspection demands of multiple appearance quality defects of multiple products in multiple industrial scenarios, and effectively support the demands of flexible production and manufacturing. In addition, the proposed scheme can also be extended to other fields of product quality inspection based on sound, ultrasonic and infrared. A new product quality inspection system can be quickly constructed by replacing the data acquisition device, the defect recognition algorithm in the SD-EIC and the model training method in the industrial cloud platform.

Availability of data and materials

Not applicable.

References

  1. Xu H, Yu W, Griffith D, Golmie N (2018) A survey on industrial internet of things: A cyber-physical systems perspective. IEEE Access 6:78,238–78,259. https://doi.org/10.1109/ACCESS.2018.2884906

  2. Qiu T, Chen N, Zhang S (2022) Robustness Optimization for IoT Topology. Springer. https://doi.org/10.1007/978-981-16-9609-1

    Article  Google Scholar 

  3. Bajic B, Rikalovic A, Suzic N, Piuri V (2021) Industry 4.0 implementation challenges and opportunities: A managerial perspective. IEEE Syst J 15(1):546–559. https://doi.org/10.1109/JSYST.2020.3023041

  4. Tao F, Cheng J, Qi Q (2018) Iihub: An industrial internet-of-things hub toward smart manufacturing based on cyber-physical system. IEEE Trans Ind Inform 14(5):2271–2280. https://doi.org/10.1109/TII.2017.2759178

    Article  Google Scholar 

  5. Ha H, Jeong J (2021) Cnn-based defect inspection for injection molding using edge computing and industrial IoT systems. Appl Sci 11:6378. https://doi.org/10.3390/app11146378

    Article  Google Scholar 

  6. Jiang W, Ning X, Xu Y (2018) Review on big data fusion methods of quality inspection for consumer goods. In: 2018 5th IEEE International Conference on Cyber Security and Cloud Computing (CSCloud)/2018 4th IEEE International Conference on Edge Computing and Scalable Cloud (EdgeCom), pp 95–102. https://doi.org/10.1109/CSCloud/EdgeCom.2018.00025

  7. Wu Y, Wang J, Chen YQ (2021) An efficient defect detection system for printed circuit boards with edge-cloud fusion computing. In: 2021 3rd International Conference on Industrial Artificial Intelligence. Shenyang, IAI 2021. pp 1–6

  8. Hu P, He C, Zhu Y (2022) The scheme and system architecture of product quality inspection based on software-defined edge intelligent controller (SD-EIC) in industrial internet of things. In: 2022 IEEE International Conference on Smart Internet of Things (SmartIoT), pp 58–64. https://doi.org/10.1109/SmartIoT55134.2022.00019

  9. Rong G, Xu Y, Tong X, Fan H (2021) An edge-cloud collaborative computing platform for building aiot applications efficiently. J Cloud Comput 10(1):1–14

  10. Almutairi J, Aldossary M (2021) A novel approach for IoT tasks offloading in edge-cloud environments. J Cloud Comput 10(1):1–19

  11. Shi W, Cao J, Zhang Q, Li Y, Xu L (2016) Edge computing: Vision and challenges. IEEE Internet Things J 3(5):637–646. https://doi.org/10.1109/JIOT.2016.2579198

    Article  Google Scholar 

  12. Hu P, He C, Sun Y (2022) Software-defined industrial internet of things (SD-IIoT) oriented for industry 4.0. In: Advances in Artificial Intelligence and Security. Springer, Qinghai, p 575–584

  13. Mei S, Yang H, Yin Z (2018) An unsupervised-learning-based approach for automated defect inspection on textured surfaces. IEEE Trans Instrum Meas 67(6):1266–1277. https://doi.org/10.1109/TIM.2018.2795178

    Article  Google Scholar 

  14. Li L, Ota K, Dong M (2018) Deep learning for smart industry: Efficient manufacture inspection system with fog computing. IEEE Trans Ind Inform 14(10):4665–4673. https://doi.org/10.1109/TII.2018.2842821

    Article  Google Scholar 

  15. Jacqueline S, Jochen B, Thorbjörn B, Gunter B, Jochen D, (2020) Predictive model-based quality inspection using machine learning and edge cloud computing. Adv Eng Inform 45:101101. https://doi.org/10.1016/j.aei.2020.101101

  16. Mei H (2017) Understanding “software-defined” from an OS perspective: technical challenges and research issues. Sci China (Inf Sci) 60(12):271–273

  17. Hu P, He C (2020) Edge computing-based solution and framework for software-defined industrial intelligent control in industrial internet of things. In: Cyberspace Data and Intelligence, and Cyber-Living, Syndrome, and Health. Springer, Beijing, p 142–153

  18. Hu P, Chen W, He C, Li Y, Ning H (2020) Software-defined edge computing (SDEC): Principle, open IoT system architecture, applications, and challenges. IEEE Internet Things J 7(7):5934–5945. https://doi.org/10.1109/JIOT.2019.2954528

    Article  Google Scholar 

  19. Qiu T, Zhang L, Chen N, Zhang S, Liu W, Wu DO (2022) Born this way: A self-organizing evolution scheme with motif for internet of things robustness. IEEE/ACM Trans Netw 30(6):2644–2657. https://doi.org/10.1109/TNET.2022.3178408

    Article  Google Scholar 

  20. Chen N, Qiu T, Lu Z, Wu DO (2022) An adaptive robustness evolution algorithm with self-competition and its 3d deployment for internet of things. IEEE/ACM Trans Networking 30(1):368–381. https://doi.org/10.1109/TNET.2021.3113916

    Article  Google Scholar 

  21. Schmied S, Gro?mann D, Mathias SG, Klaus Mueller R (2020) An approach for an industrial information model management. In: 2020 IEEE Conference on Industrial Cyberphysical Systems (ICPS), vol 1, pp 402–405. https://doi.org/10.1109/ICPS48405.2020.9274751

  22. Li Y, Cheng T, Pan L (2021) Research and application of information model of industrial robot welding system based on opc ua. In: 2021 IEEE 2nd International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering (ICBAIE), pp 1084–1087. https://doi.org/10.1109/ICBAIE52039.2021.9389867

  23. Hu P, Ning H, Chen L, Daneshmand M (2019) An open internet of things system architecture based on software-defined device. IEEE Internet Things J 6(2):2583–2592. https://doi.org/10.1109/JIOT.2018.2872028

    Article  Google Scholar 

  24. Wan J, Tang S, Shu Z, Li D, Wang S, Imran M, Vasilakos AV (2016) Software-defined industrial internet of things in the context of industry 4.0. IEEE Sensors J 16(20):7373–7380. https://doi.org/10.1109/JSEN.2016.2565621

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers for their insightful comments and suggestions on improving this paper.

Funding

Not applicable.

Author information

Authors and Affiliations

  1. Central Research Institute, HollySys Group Co., Ltd., Beijing, China

    Pengfei Hu, Yiming Zhu & Tianhui Li

  2. Beijing HollySys Co., Ltd., Beijing, China

    Pengfei Hu & Tianhui Li

  3. HollySys Group Co., Ltd., Beijing, China

    Chunming He

Authors
  1. Pengfei Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chunming He
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yiming Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tianhui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Pengfei Hu proposed and designed the architecture and scheme. Chunming He and Yiming Zhu designed the hardware platform. Tianhui Li analyzed the system performance. Pengfei Hu wrote the manuscript. Chunming He, Yiming Zhu and Tianhui Li reviewed and polished the manuscript. All authors read and approved the final manuscript.

Authors’ information

Pengfei Hu received the B.E. degree from the Zhengzhou University of Aeronautics, Zhengzhou, China, in 2012, and the Ph.D. degree from the University of Science and Technology Beijing, Beijing, China, in 2018. He is currently a Senior Research Fellow and Technical Director of Research Institute of HollySys Group Co., Ltd., Beijing, China. His research interests include Industrial Internet of Things and edge computing. He has served as the Workshop Chair of ATC 2019, a TPC member of SmartWorld 2019 and CyberDI 2019, and the Publicity Chair of CyberDI 2019.

Chunming He received the B.E., M.E., and Ph.D. degrees from the Beijing Institute of Technology, Beijing, China, in 1990, 1993, and 1996, respectively. He is a Vice President of HollySys Group Co., Ltd. At present, his research is mainly focused on system architecture, application scenarios, and implementation path of the automation system intelligent upgrading. He has been engaged in the research and development of high-speed railway and subway train control system for a long time, and been in charge of the research and development of a large number of high-speed railway train control system products in China. As a major member, he participated in the compilation of a number of CTCS technical standards/specifications for China’s high-speed railway train control system.

Yiming Zhu received the B.E. degree from the Beijing Institute of Technology, Beijing, China, in 1992, and the M.E. degree from the Huazhong University of Science and Technology, Wuhan, China, in 2005. He is a chief engineer of HollySys Group Co., Ltd. At present, his research is mainly focused on intelligent control system and cyber-physical system.

Tianhui Li received the B.E. and M.E. degrees from the Beijing Institute of Technology, Beijing, China. He is a manager of Research Institute of HollySys Group Co., Ltd. His research is mainly focused on intelligent control system and smart manufacture system.

Corresponding author

Correspondence to Pengfei Hu.

Ethics declarations

Ethics approval and consent to participate

The research has consent for ethical approval and consent to participate.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, P., He, C., Zhu, Y. et al. The product quality inspection scheme based on software-defined edge intelligent controller in industrial internet of things. J Cloud Comp 12, 113 (2023). https://doi.org/10.1186/s13677-023-00487-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13677-023-00487-7

Keywords