Digitization of Gauge Inspection Control Procedure

Gauge inspection tools are usually used for the visual and geometrical tolerances verification, using paper tutorials for guidance through the inspection steps and paper to register the results of the performed inspection. This practice may be misleading to the quality control management, not providing an effective and error-prone quality control operation. This paper proposes a cost-effective solution for supporting the realization of gauge inspection control procedures by using emergent technologies, namely Human-Machine interfaces and virtual reality, to guide more efficiently the operators during the execution of their tasks, and using Internet of Things (IoT) and cloud technologies to dynamically collect and store the inspection results. The proposed solution was deployed in an industrial production facility, with the achieved results showing its efficiency and robustness, as well as a very positive feedback from the operators. This solution also contributes for a faster and more efficient execution of inspection tests, particularly reducing the setup time required to change the product reference being inspected.

I. INTRODUCTION Industry 4.0 takes a pioneering role in the digital transformation of traditional manufacturing factories. Each industrial revolution inferred efforts to increase the manufacturing quality, scalability, and efficiency in the manufacturing of goods. From steam-powered plants to computerized production facilities, the fourth iteration of this process is an evolution to make manufacturing increasingly intelligent, thus, more adaptable, automated, and flexible [1]. Industry 4.0 relies on Cyber-Physical Systems (CPS) to seamlessly connect cyber and physical counterparts, i.e. the virtual space with the physical world [2], being organized in a distributed network and interacting to achieve a common goal [3]. In particular, CPS acts as a backbone to realize Industry 4.0 compliant applications, integrating disruptive digital technologies, e.g., Artificial Intelligence (AI), Internet of Things (IoT) and Cloud Computing, to improve decision-making processes in a decentralized manner [1], and considering data as the central element [4]- [6].
In industrial environments, quality control assumes a crucial role to enable an independent and efficient evaluation of the products and processes, ensuring the compliance with predefined quality objectives. This accomplishment is achieved with monitoring measurement technologies [7], ranging from more visual and manual procedures to more automatic ones.
To guarantee the product quality, one inspection technique usually applied in manufacturing industry is to use gauge tools to verify the correctness of manufactured parts according to a straightforward process that uses plug gauges to find out if it conforms with the tolerances specified in its guidelines [8]. Traditionally, the gauge inspection control procedures are used by operators to account for unconformities in produced products. For this purpose, the operators use paper spreadsheets that contain the instructions to perform each procedure and guide them through the inspection process. The paper usage to record these process data can be misleading since the data recorded may be wrong or incomplete, needing a future treatment to register more efficiently in a digital means. Another problem caused by the use of paper is the difficulty of learning the inspection process by having multiple different guidelines that contain various pieces of information needed to perform those tasks.
Having this in mind, this paper describes the development of an immersive personal software assistant that digitizes the quality control operation and provides guidance to the operator performing the gauge inspection control operations. The developed solution uses emerging digital technologies, e.g., cloud computing, portable devices, and Virtual Reality (VR), to provide operators with a user-friendly Human-Machine Interface (HMI), enabling them to perform the inspection task in a faster and more efficient manner. The proposed personal assistant also provides the managers and quality control department with valuable data about the operators' performance and complete reports regarding the detected failures.
The developed solution was deployed in an industrial case study related to a steel stamping factory that produces components for automotive industry through a cold stamping process. The preliminary experimental results show that the deployed solution can enable operators to perform the inspection task more efficiently while creating a faster learning experience for new employees.
The rest of the paper is organized as follows: Section II describes the related work and Section III describes the industrial case study related to the gauge inspection control procedures. Section IV presents the software architecture of the proposed digital personal assistant, as well as its develop-ment and deployment in the factory plant. Section V discusses the experimental results and particularly the usability results. Finally, Section VI rounds up the paper with the conclusions and presents the future work.

II. RELATED WORK
Quality control inspection methods aim to evaluate products by creating metrics and measurements to monitor the process quality [7], essential to identify the product defects and calibrate the machines disposed along the production line. Conventional quality control practices use visual inspection and geometry inspection. The visual inspection consists of verifying visible damages present in produced parts, being performed according to a pre-defined frequency or sampling process. The geometry inspection technique uses gauge tools to verify the manufactured parts' correctness, finding out if it conforms with the tolerances specified in its guidelines [8]. These traditional inspection processes present several problems, namely the loss of time and occurrence of errors when the inspection procedure is not known, and the manual collection of data related to detected defects.
The realization of quality control inspection can be significantly improved by using emergent ICT technologies, e.g., HMI interfaces, VR technologies, and automatic inspection tools. In fact, in some situations, the improvement of inspection tasks uses automatic systems that increase the speed and reliability of inspection processes by reducing the human intervention. As examples, a 3D scanner is used to check automatically the geometric tolerance of parts [9], and multiple cameras are used to detect and measure unconformities in irregular-shaped objects before the laser cutting operation [10]. However, implementing these automatic systems is not the proper solution for all processes since they usually require an high investment and/or present a high implementation complexity that is strongly dependent on the inspection process.
In other situations, where the quality control inspection need to be performed manually by operators, the use of ICT technologies can contribute to develop digital and immersive solutions that improve the quality of executed operations and reduce the time consuming of these operations, minimizing the errors. Particularly, the inspection effectiveness can be improved by using in Multi-touch User Interfaces (MTUI) that relies on 3D graphics to visualize operation instructions. HMI principles are applied to enable efficient interactions between humans and machines without potential errors [11], and in industrial contexts, the use of HMI allows the operator to identify and intercede effectively and quickly on irregularities within a process before they turn into a real emergency [12], [13]. By using general and 3D specific principles, e. g., consistency, recoverability, maximization of workspace and direct manipulation, studies on MTUI usage for 3D-related applications concludes that most experiment subjects find the handheld device suitable for a 3D graphics application [14], [15]. They showed that it is possible to translate 3D computer graphics applications into the 3D mobile application using MTUIs instead of mouse input for dragging, rotating and zooming [15].
The use of VR technologies can also improve the user immersion, interaction and involvement with their tasks [16] [17]. A VR immersive experience may provide persons with a virtual environment with the feeling and experience of being in the actual environment, thus improving their perceptions. Applying VR technologies in industry, and particularly in inspection tasks, can provide an entirely new panorama on preparing operators and processes. With the use of virtual models, the operators engage in training, e.g., perceiving what steps they should take to replace a damaged component or disassemble the machine components [18]. As example, a VRcontrolled robot was developed in an immersive interactive, safe and non-expensive environment where the operator can be trained [19]. The achieved results show that the immersive experience succeeds in increasing the efficiency of the simulation processes with a cost-effective solution. However, it was also noticed that the intensive usage of an Head-Mounted-Device (HMD) might cause sickness, which clearly shows the limitation of using VR solutions based on the current HMD technology in industrial scenarios that require the use of these devices during a long period of time.
Although very promising, the developed solutions using emergent ICT technologies are highly experimental and mainly tested in a laboratory environments with only specific parts being used, and usually not considering the industrial requirements. Another drawback for the before-mentioned approaches is their expensive implementation costs, a disadvantage for small and medium-sized companies that aim to make the transitioning to use more digital geometry inspection methods.

III. DESCRIPTION OF THE CASE STUDY
The case study considered in this work is related to the use of gauge tools to perform the quality control of parts produced in a steel cold stamping factory plant. In such process, the operator uses a pre-configured testing bench, as illustrated in Figure 1, to inspect the geometry compliance of a particular produced part. These testing benches, one for each part model, provide gauge tools for the component-specific inspection, instruction and component conformity guides, and expiration dates for the tools' effectiveness. The guidance through the visual and geometrical inspection process is provided to the operators via a tutorial paper, as shown in Figure 2, that provides step-by-step information on how to proceed with the inspection, and showing its frequency of execution and the exercised method. Once the operator finishes the inspection, the results are recorded in the paper spreadsheet, that also offers insights on the non-conformities found during the inspection. The recorded information is sent to quality control management, which will use the data to register metrics and better understand the production problems. The main problem with the actual approach is the execution time and the number of errors that can exist due to the operators' difficulties in memorizing and execute correctly the entire inspection sequence for the different part references. This is more evident in case of changing the part reference, particularly when the operator starts the inspection procedure and requires time to adjust the inspection routine. Additionally, the use of paper to record data can be misleading once the data recorded may be wrong or incomplete, needing the future treatment to register more efficiently in a digital means. The traditional method also creates a loss of time when converting the inspection report into a digital support. Changing the test bench reference also implies changing all the paper guides used, creating a more time-consuming task.
The objective is to use emergent ICT technologies to digitize this inspection process, mainly contributing to improve the inspection speed and the operator's perceived comfort, using a 3D environment to enhance the parts inspection's learnability to ease the operator's work. Additionally, it is important to have a reporting functionality that automatically collects information from the inspection process, without depending on operators, including the photos of visual non-conformities. The collected data should be accessed in real-time by external systems, e.g., management personnel that can visualize and analyze the results using proper metrics, and better understand non-conformities, contributing to identify earlier failure trends and also proceed with machine calibration.

IV. DIGITIZING THE GAUGE INSPECTION PROCESS
An immersive personal software assistant was designed and developed for the digitization of the gauge control quality process, taking into consideration the industrial requirements.

A. Software Architecture
The designed software architecture comprises two main parts, the client-side and the server-side, as illustrated in Figure 3, providing maintainability and reusability of the software functionalities. The interface to the server-side can be performed through HyperText Transfer Protocol (HTTP) requests, allowing the connection of any authorized solution, e.g., Mobile Apps, Web Apps, and IoT applications.
. Decoupling the client-side application responsibilities from the server-side allows to reuse the back-end in a diversity of applications. This separation of responsibilities also improves the client-side, enabling the developer to use more suitable techniques to provide the user with a better experience. One example is the software pattern stale while revalidate (HTTP RFC 5861) that is applied to all client-side applications. This pattern improves the general user experience by loading the server-side content faster and reacts to data changes in an "almost" real-time way, having a delay of 5000ms between each data revalidation event.
The proposed solution also relies on cloud services for deploying applications and storing data. Many industrial companies may not have IT personnel or infrastructure to support the data security, high-quality storage and information distribution, but by using a cloud solution, these problems are handled by a third-party system. In the designed solution, the cloud system handles the server's deployment and security by adding a Transport Layer Security (TLS) to cryptograph the communication between the client and server, which prevents attacks that aim to store information in the payload of transport operations (i.e., Man in the Middle threats).
The cloud is also responsible for storing the system data, namely two types of data: i) relational text data related to the inspection procedures and achieved results, test benches, and user accounts, and ii) binary data related to images and 3D models binary. A different kind of database is needed for each case, with the relational text database handling data for SQL queries from the server and the binary database (file storage) facilitating the access by the client-side system.

B. System Development
The described software system architecture was developed by implementing three main applications: Server, Web, and Mobile. Each one of these software pieces works together through the network to provide the required information that supports the execution of a reliable inspection process.
1) Server Application: The server application was implemented with NodeJS by using ExpressJS to create the endpoints to handle HTTP requests through the RESTful API. Requests are sent to controllers, which handle the use case specific operations and return the expected HTTP response. The server application aims to persist, process, and transform data from the client-side, and also communicate with other cloud instances, e.g., databases, to handle the persistence of data. For this purpose, the server uses a Postgres relational database to store the text-based information, e.g., accounts, instructions' descriptions and inspection results, and the Google Cloud Storage to store binary data, e.g., images and 3D animations. Although being compatible with other databases, e.g., SQL Server, Azure blob storage and MySQL, the option to use Postgres and Google Cloud Storage was due to their simplicity for integration without additional costs.
2) User Interface: The User Interface (UI) was prototyped using Figma and following the HMI good practices to achieve a consistent and friendly interface. The interface design process was realized for both the Web and Mobile applications and aimed to create screens that provide users with an easy access to all offered functionalities. With the UI prototype as a guideline, the visual elements were implemented by using JSX (the markup language used by ReactJS and React Native) and Cascading Style Sheet (CSS).
3) Web Application: The web application's primary use objective is to provide the quality control management with a UI capable of creating digital manuals and visualizing the results of the performed inspections. The web application is accessible by any device with a browser. The ReactJS library was used to create a Single Page Application (SPA) that dynamically re-render the current web page elements with new data instead of loading an entirely new web page. To handle the HTTP request data and implement a cache invalidation strategy, Axios and SWR were used, updating data from the back-end "almost" in real-time.
4) 3D Render Engine: The web application exports a route for rendering the 3D models by receiving the file requested as an URL parameter to display the 3D animations. The render engine seeks for the specified file in the Binary Database, and the ThreeJS is used to load the GLTF binary data and display the 3D models, textures, and animations for the specified database response. This technique enables part of the web application to work as a render engine, being accessible by the mobile application using a web view.

5) Mobile Application:
The primary objective of the mobile application is to guide operators in their tasks by displaying 3D animations for each inspection instruction. React Native and Expo are used to implement a multi-platform app that instructs and records the quality control operations activities. The main deployment target is Android tablets that are used by operators during the quality control process. Like the web application, the mobile app also uses Axios and SWR to handle HTTP requests and implement the cache invalidation strategy.
6) Rendering 3D in the Mobile Application: The mobile app implements a web view component to present the 3D animation and interactions, by requesting the web application render engine route with the instruction-specific file. This implementation allows a better performance using ThreeJS and WebGL than the native binding provided by Expo-three (note that Expo-three is a binding that enables ThreeJS to run natively by providing a WebGL interface for native OpenGL).

C. Digitization of the Test Bench
An important pieces in this software personal assistant is the gauge testing methods that should be expressed in a digital manner. The digitization process of these methods involves four steps: Data arrangement, Model Optimization, Model Texture and Model Animation.
1) Data Arrangement: related to obtain the data for a specific part, based on the operation instruction guide, operation control guide, and test bench 3D models, which are provided by the test bench manufacturer. The images provided by the instruction guides are extracted into labeled files that will be used by the mobile app as an alternative to the 3D instructions, enabling operators to choose their preferred approach. Finally, the 3D model, usually provided in the STEP format, can be open in a CAD software package and exported into STL format, keeping each mesh separated in a file.
2) Model Optimization: related to import multiple STL files into Blender software package, being divided into groups for each original piece component. Also, by the nature of the CAD project, the STL file contains a very detailed model that may not be useful and will directly impact the performance of the 3D rendering. After importing and removing all unnecessary meshes and faces, the "decimate" modifier is used in complex mashes to reduce the performance impact. When the optimization process is concluded, the file should lower its size and complexity significantly when exported as GLTF binary. As a result, the files can be loaded faster through the network and be supported by less powerful devices, increasing the solution's cost-effectiveness.
3) Model Texture: Textures can be applied to the model using "Principled BSDF," a Blender shader node that combines multiple texture elements into a single setup. Parameters like base color, metallic, and roughness may be added to the materials to achieve a more similar look to the real-life equivalent. Although the image-based texture can also be used as materials, the final file will be substantially heavier and very similar when displayed in a low-resolution display. 4) Model Animation: Animations are essential to promote the user-friendly information delivered during the inspection process. The final blender file with texture and optimization should be duplicated and renamed for each instruction it represents. After understanding the instruction-specific process, it may be replicated to the Blender Keyframe functionality to create the animation. When the animation is complete, any unnecessary mesh should be removed to optimize the model even more, and the final instruction-specific animation should then be exported as GLTF binary. This process may be repeated until all the inspection instructions are fully animated.
The web application may also be used to create the inspection manual, by filing the form using the operation instruction guide and the operation control guide content. Each instruction may be created by uploading the required image and the 3D animation, and the resulting form is submitted to be reflected in the mobile application when fetching the available manuals.

V. EXPERIMENTAL TESTING
The developed software application for the gauge inspection quality control was deployed in the factory plant and was tested during the production operation (see Figure 4).

A. Deployment and Testing in Practice
The server was deployed in a Cloud environment, internally accessible via Internet. User accounts were created for the operators and management personnel, each one possessing different levels of permission to access data. The mobile software application was installed in a Samsung Galaxy Tab A6, with processor Octa-core 1.6GHz Cortex-A53. The tablet was fixed to a vertical stand near the test benches, as illustrated in Figure 4, maximizing the operator's accessibility to the device during the inspection operation.
Before starting the inspection, operators select the part model to be inspected by scrolling in a list of entries or by reading a QR Code present in the test bench (see Figure  5.a). During the inspection, two types of instructions can be visualized: i) a visual inspection that shows photos of visual unconformities that can be present in the part, and ii) geometric inspection that uses 3D animations to guide the operator through the inspection operation (see Figure 5.b).
The mobile application was connected to an online crash analytics service to monitor the software failure during its usage in the production environment. The solution resembled robustness by not presenting any failures during the experimentation period, and the operators didn't perceived crashes or failures. The software application may very dependable in the network connection to load images and 3D models, making the loading time different according to the device's position and connection speed in the industrial facility. Although, no criticisms about the network speed were reported by operators using the application in the production line.
The results from the performed inspections are available in the web application's dashboard menu, as illustrated in 5.c, displaying the inspection performance and analysis responses

B. Usability Analysis
The operators' overall experience during the usage of the software application was evaluated by using the standardized system usability scale (SUS). Eleven operators tested the mobile software application during one week. The collected results for each SUS question are summarized in Table I (note that odd-numbered questions should be maximized and the even-numbered questions should be minimized). Observing the achieved results, it is possible to notice an average score of 88.40, which corresponds to a very good usability product [20]. Particularly, these results show that the users recognize that the software solution is easy to use, integrates well the functionalities and is an add-value during the execution of their inspection tasks.
The feedback from the company was also positive, reviewing the application as "a great added value¨for the parts inspection operation, being very well perceived by the quality control management and operators performing the tasks. This application may also improve the analysis consistency registering the accurate inspection time, allowing the correlation with the correct failed inspection step, and removing the human involvement in the reporting process. Another long term impact of this solution is to reduce the operator's time to set up the inspection environment, particularly in case of changing the part reference, acting as a driver to implement the SMED (Single Minute Exchange of Die) lean technique.

VI. CONCLUSIONS
The quality control is a fundamental part of production processes that uses inspection methods to verify the conformity of parts with the established norms. In case of visual and geometrical tolerances verification, the gauge inspection tools uses paper manuals for guiding operators through the inspection steps and paper spreadsheets to register the results of these inspection tasks. This practice may be misleading to the quality control management and does not properly support the operators guidance during the execution of their tasks.
This paper proposes a digital personnel assistant that supports the guidance of operators during the execution of the inspection process while automatically records the data related to the performed inspections. The developed application supports the non-immersive VR in a Desktop VR experience using tablets to visualize the 3D scenes while performing realworld actions, without supporting sensory output.
The developed software application was deployed in an industrial factory plant, with the achieved results during the testing operation in the production environment showing that the users' perception of the application usability was very positive. The software robustness during the testing was also noticed, without any crash or error reported. Additionally, the proposed software approach contributes for the long-term time optimization by removing unnecessary tasks related to the data collection with more automated procedures. Furthermore, the platform's cost-effectiveness can be attractive for small-sized industrial companies that seek the digital transformation of their processes.
Future work will be devoted to apply artificial vision algorithms to automatically verify the correctness of performed inspection operations.