Improving the Mobile Robots Indoor Localization System by Combining SLAM with Fiducial Markers

—Autonomous mobile robots applications require a robust navigation system, which ensures the proper movement of the robot while performing their tasks. The key challenge in the navigation system is related to the indoor localization. Simultaneous Localization and Mapping (SLAM) techniques combined with Adaptive Monte Carlo Localization (AMCL) are widely used to localize robots. However, this approach is susceptible to errors, especially in dynamic environments and in presence of obstacles and objects. This paper presents an approach to improve the estimation of the indoor pose of a wheeled mobile robot in an environment. To this end, the proposed localization system integrates the AMCL algorithm with the position updates and corrections based on the artiﬁcial vision detection of ﬁducial markers scattered throughout the environment to reduce the errors accumulated by the AMCL position estimation. The proposed approach is based on Robot Operating System (ROS), and tested and validated in a simulation environment. As a result, an improvement in the trajectory performed by the robot was identiﬁed using the SLAM system combined with traditional AMCL corrected with the detection, by artiﬁcial vision, of ﬁducial markers.


I. INTRODUCTION
Mobile robots are widespread in several areas, such as industrial automation, agriculture, medical care, autonomous driving, product deliveries, planetary exploration, smart warehouses, personal services, construction, reconnaissance, entertainment, emergency rescue operations, patrolling and transportation [1]. Being one of the fastest-growing scientific fields today, mobile robotics has considerable impact not only on research but also on the economy. Even with the Covid-19 pandemic scenario, the robotics market is expected to move around U$23 billion in 2021 and growing to U$54 billion in 2023 [2], pointing to a considerable expansion of autonomous mobile robots applications.
Characterized as intelligent systems, mobile robots have the ability to move autonomously, without the human interference [3], making decisions based on the information collected from their sensors (e.g., LiDAR, sonar and cameras), which allows them to help humans with heavy or timeconsuming tasks [4]. However, for a mobile robot to be able Email:ebertonha@utfpr.edu.br This work has been supported by FCT -Fundação para a Ciência e Tecnologia within the Project Scope UIDB/05757/2020 and PhD Grant UI/BD/151286/2021. 978-1-6654-0761-8/21/$31.00 ©2021 IEEE to operate effectively in an environment it is necessary to implement efficient solutions for factors such as locomotion, perception with sensors, cognition, and navigation [5].
For mobile robots to perform autonomous operations, their localization system require to know the position and orientation of the robot in a given coordinate system. Considering the high demand for indoor applications, correlated with the complexity and dynamism of these environments as well as the inaccuracy of the GPS signal, locating a mobile robot in an indoor environment represents an arduous and challenging task. Alternative perception systems, e.g., systems based on electromagnetic received signal strength indication (RSSI) such as WiFi and Bluetooth are susceptible to high noise levels and are currently not robust enough for precise indoor localization. On the other hand, systems with time-of-flight based signals, such as POZYX, have a good position result, however they do not obtain a satisfactory orientation [6]. Another adopted method for indoor location is the Simultaneous Localization and Mapping (SLAM), which was introduced to enable robots to generate maps of their surroundings based on LiDAR information, which is used for robot position estimation as well as the path planning between different points in the environment [7]. The intrinsic error due to noise measurements or inaccurate LiDAR readings can be minimized by using the Adaptive Monte Carlo Localization (AMCL) algorithm. The AMCL algorithm estimates the position of the robot applied to a two-dimensional occupancy map through a particle filter, where the particles swarm is used to describe the possible position of the robot in a known map. Particles are randomly added during resampling to slow down the convergence rate of the algorithm preventing localization errors related to robot kidnapping problem [8].
Even when performing SLAM with the AMCL algorithm, the robot's position estimation can be subject to errors. This discrepancy between the real and estimated position increases when the scenario has dynamic obstacles, which ends up constantly changing the laser sensor readings, or hallways that are very long and very similar in their extension, making difficult to determine the actual position of the robot in the environment [9]. This paper presents an approach to achieve the localization of a wheeled mobile robot during its indoor navigation in more reliable manner. SLAM techniques will be integrated with the position tracking of fiducial markers distributed in an environment. The fiducial markers will serve as landmarks since the distance between them and a camera (which will be positioned on the robot) can be determined with artificial vision techniques. Whenever a marker is detected, and since its actual position in the environment is already known, it will update the robot's position data by resetting the AMCL's particle filter, eliminating the existing discrepancies in the localization estimation. The achieved results showed that the robot's localization is more reliable than the simple use of SLAM techniques based on LiDAR and odometry.
The rest of this paper is organized as follows. Section II briefly reviews the state-of-the-art on SLAM techniques and fiducial markers used for the position tracking. Section III describes the proposed architecture for the indoor localization system, and the Section IV presents the system implementation. Section V analyses the achieved results, and finally, Section VI presents the conclusions and future work.

II. RELATED WORK
Considering the importance of SLAM techniques in the navigation of mobile robotics, this section briefly discusses their associated concepts and issues, as well as the concepts related to the detection and tracking of fiducial markers by RGB cameras.

A. Simultaneous Localization And Mapping
As stated earlier, the mobile robot navigation involves solving the problems of mapping, path planning, obstacle avoidance and localization which are covered by SLAM techniques [10]. Mapping consists of creating a representation of the environment that can be used by the robot as a form of recognition. There are different types of map representations, e.g., the topological map and the metric map, where can be highlighted the occupancy grid representation [4]. The map is divided into two distinct types of cells arranged in grids, which are based on an occupancy threshold. The cells may represent regions where the navigation is not possible, considered as occupied, and the others reflect free space that indicates regions where non-collision is guaranteed [11]. Well-known algorithms such as Gmapping and Hector SLAM are widely used in the creation of grid occupancy maps, applied in SLAM approaches so that a mobile robot can be located by estimating its position using LiDAR data.
The SLAM localization is essential for the navigation of a robot to be autonomous, requiring robust calculations and approximations to determine the robot's position in the environment, in order to know how to act during its movement for the reliable execution of tasks [12]. However, the localization is subject to many factors that contribute to its lack of accuracy, e.g., encoder/odometry and mapping errors, location uncertainty based on scan matching, loss of previous position information, sensor inaccuracy and environmental conditions. For these reasons algorithms such as Kalman filter, Markov localization and particle filters like AMCL are used to improve SLAM localization feature [13].
Even using some of the mentioned techniques, it is not possible to guarantee a robust indoor robot localization for dynamic and complex scenarios. Although the particle filtering converges on the most likely position of the robot in the environment, this technique has difficulties especially in global localization, where the particle convergence speed is slow when there is no initial position information of the robot, or in the kidnapped robot problem. The particle convergence is even slower or error-prone in dynamic environments or with simple and repetitive geographic features [14].
The techniques associated with SLAM have problems, e.g., the localization uncertainty, and difficulty in identifying the corresponding position obtained from samples of sensor data in order to be able to associate this data with a known position on the map. In addition, problems related to time complexity due to limited processing to estimate the position and desynchronism with high-speed robots can compromise a robot mission in an indoor environment [7].

B. Fiducial markers
Fiducial markers are elements used as reference points or measurements RGB data, and they are usually characterized as artificial marks of known size and shape. There are different open-source packages of fiducial markers that have integration with ROS (Robot Operating System), which facilitates their use in robotics applications, e.g., ARTag, ARToolkit, AprilTag, ArUco, STag, and others [15].
The fiducial markers are widely used in mobile robotics localization applications. By positioning them in an environment as landmarks, it is possible to accurately determine their relative position and orientation between them and a camera positioned on the mobile robot, information that can be used to significantly increase the accuracy of the localization system [9]. However, since artificial vision strategies are applied for tracking the markers, the use of the presented packages requires a high computational power, which can interfere with the efficiency of the localization system in the navigation of a robot (note that the data must be processed in real time).
The high accuracy in the position tracking of fiducial markers, as well as their versatility, allows to apply them in different mobile robotics scenarios. For example, fiducial markers are used as a complementary localization system for a Unmanned Aerial Vehicle (UAV) in power distribution line inspections [16], in simulations for target tracking for UAVs [17], and localize mobile robots in a smart warehouse system with fiducial markers positioned on top of the robots and the camera positioned on the warehouse ceiling [18]. Similarity, visual markers are captured by a ceiling-mounted camera to position virtual LiDAR sensors on small robots [19]. Fiducial markers are also applied as a method to improve the localization of a turtlebot robot in an indoor environment, combining the tracking position of the tags in walls with SLAM techniques and the Kalman filter algorithm, achieving an autonomous navigation system [20].
Aiming to overcome the problems identified with the use of the SLAM techniques, the presented work attempts to improve the SLAM localization by combining the AMCL algorithm with the visual tracking of fiducial markers positioned on the environment ceiling. The markers provide global reference points possible to be integrated into the AMCL position estimation, ensuring more robust localization results and allowing a way to obtain the initial position information by detecting the tags and contributing to mitigate the localization uncertainty, data correspondence, and kidnapped robot problems. This allows to offer a solution to global localization and abducted robot localization problems, and to integrate the localization system for the autonomous navigation of wheeled mobile robots in indoor environments.
III. INDOOR LOCALIZATION SYSTEM ARCHITECTURE As previously referred, this work proposes to improve the estimation of the robot's indoor localization (position and orientation) by using the generated map and LiDAR data, that are combined with the odometry and corrected with the tracking of fiducial markers.
For this purpose, the fiducial markers are distributed in the environment, fixed on the ceiling, being their positions and orientations properly stored in a database. When the artificial vision system detects a marker, the robot's position in the localization system is corrected. The Algorithm 1 describes the proposed indoor localization system considering the detection of the markers. When a marker enters the field of view of a camera positioned on the robot, the artificial vision system gets the ID i of the fiducial marker. Then, the algorithm identifies the P g , i.e., the global position and orientation (x g , y g , z g , θ g ) of the marker i checking in the database. The next procedure gets the position and orientation (x m , y m , z m , θ m ) of the marker in relation to the camera (P m ).
With the procedure that correlates the global position of the marker P g , and the position in relation to the camera obtained by the artificial vision system P m , it is possible to determine the global position and orientation of the camera in the environment. Once the camera is fixed, through the existing offset between the camera and the robot, the robot's position (P r ) is determined.
The application of this method, allows to update the position estimator algorithm based on the location obtained by the fiducial marker detection, removing the accumulated intrinsic errors. The flowchart in Fig. 1 represents the architecture of the indoor localization system, in it the odometry and LiDAR sensor data are used by the position estimator algorithm and for creating the environment map. The map generation is a one-time process, once a map exists there is no need to create a new one. The map data is also used by the position estimator algorithm. Whenever a tag is identified in the environment by the artificial vision system through the images captured by the camera, Algorithm 1 computes the position of the robot based on and performs the correction of the position estimation algorithm. It is worth mentioning that it is not necessary to always have a marker being detected, they can be positioned in the environment so as to be detected periodically, keeping the localization being done by the position estimator when there are no markers, and use them only to eventually reduce the errors generated by the estimation.

IV. IMPLEMENTATION OF THE LOCALIZATION SYSTEM
The validation of the proposed localization system were performed in a simulation case study environment. For this purpose, the V-REP simulator was used, which is integrated with ROS. V-REP provides several useful features, such as physical and dynamics restrictions, to simulate robots applications. On the other hand, ROS was used for the implementation and integration of the localization system algorithms, since ROS enables the simple integration with numerous robotic systems, as well as providing packages that support applications such as SLAM, the AMCL algorithm, and ARTag to localize the fiducial markers.
In the simulation environment, it was used a wheeled mobile robot model similar to a Roomba vacuum cleaner, as illustrated in Fig. 2. It is equipped with a LiDAR sensor, which will be responsible for acquiring the distances between the robot and the walls and obstacles of the environment, and an RGB camera positioned upwards, in order to obtain the necessary images for the identification and tracking of the fiducial markers positioned on the ceiling. The fiducial markers chosen to be used belong to the ARTag package (ar track alvar) and are known as AR tags.
The selected environment for the localization tests was an university hallway, which a real scale version was designed in the V-REP simulator. Fig. 3 shows a floor plan of the environment, with a 26x20 m size, and with the blue blocks representing cabinets present in the real environment, and the red blocks represent the distribution of the fiducial markers.   A total of 20 AR tags were positioned on the ceiling of the environment (which is 2.5 m high). Each marker has a dimension of 20x20 cm. The positioning was done in a way that makes it possible to use the markers as landmarks for different routes between points in the environment, to perform the robot navigation during the localization tests. The real environment was also considered for the tags arrangement once it was planned to perform future tests with a real robot, so places with elements that could hinder the tags detection, such as lamps, doors, pillars, among others, were avoided during the positioning. Since there will not always be a tag in the camera's field of view, the localization in these places is done only by the AMCL algorithm.
Due to the large computational requirements to run the simulation in V-REP simultaneously with the localization system, it was decided to use a distributed computing strategy. For this purpose, two computers were connected in the same ROS network, as illustrated in Fig. 4. One of them acts as the ROS Master, and is also responsible for running the V-REP simulator, where the environment information will be collected through the LiDAR sensors, the camera, and the robot's odometry information, as well receiving the velocity information published to perform the navigation of the mobile robot. The second computer is responsible for processing the received data, running the SLAM algorithm, tracking the tags, path planning, sending the velocity messages for the robot, requesting the robot's destination point.
For the operation of the localization system, the construction of a map of the environment is needed. For this purpose, the ROS Gmapping package was used, which allows to use laser-based SLAM strategies, creating a grid occupancy map from the robot's odometry data and a laser sensor, in this case the LiDAR. The map only needs to be created once and is used to assist in the localization of the robot.
The images collected by the RGB camera are processed by the ar track alvar package, which detects the tags and determines the distance and orientation between them and the camera, which is positioned above the robot. Then an implemented python algorithm based on Algorithm 1 is used to determine the global position of the robot. A ROS service is used afterwards to reset the particle filter of the AMCL algorithm with the calculated position of the robot. In this way, the existing intrinsic error in the position estimation made by the AMCL is eliminated, ensuring that the new estimations are more accurate. When there are no position updates being performed by the tag detection, the AMCL algorithm uses the data received from the LiDAR sensor readings and the robot's odometry, in conjunction with the map data generated by Gmapping through a ROS service called map server, to perform the particle filtering, estimating the localization of the robot in the environment.
For the tests, a Dijkstra algorithm has been implemented for the path planning of the robot in the environment, while the navigation is performed with the help of the ROS package move base, which is responsible for controlling the robot by sending a speed message and obstacle avoidance, guiding the robot to the goal based on the position information received from the AMCL algorithm.
Based on the information from the LiDAR sensor and using the map server, cost maps are created. In the global cost maps, tolerance regions are created in relation to the existing occupation areas of the original map (walls, obstacles, etc.), preventing the collisions with these regions. A local cost map is also created, indicating tolerance regions based on the original map and in obstacles detected by the LiDAR readings. The detection of new obstacles not only updates the local cost map, but also causes the re-planning of the robot's route to avoid collisions with the obstacle.

V. RESULTS AND DISCUSSIONS
Using an implemented python algorithm, the mobile robot was teleguided through the simulation environment to create the map using the ROS Gmapping package. The map is necessary to be used as reference data for the indoor localization system. Fig. 5 shows the obtained grid occupancy, as well as the two different chosen trajectories for the robot's navigation tests. Four different scenarios were defined to perform the localization tests, where different parameters were incorporated into the AMCL algorithm, as detailed below: • Scenario 1: Odometry data, laser sensor (LiDAR) readings and known initial position.
• Scenario 2: Odometry data and laser sensor readings. In this case, the initial position of the robot is not informed to the AMCL algorithm (kidnapped robot problem).
• Scenario 3: Odometry data, laser sensor readings, position updates based on AR tags detection and known initial position.
• Scenario 4: Odometry data, laser sensor readings and position updates based on AR tags detection. The initial position of the robot is not informed. The results obtained for the first scenario in the two different routes, in terms of trajectory performed by the robot, are illustrated in Fig. 6, with the red arrows representing the robot's real position and orientation and the green ones the position estimated by the algorithm. In this scenario, since the robot's initial position was known, the AMCL algorithm was able to perform the convergence of its particle filter very close to the robot's actual position, keeping the estimation of the location consistent especially in the first trajectory test along the path between the first two waypoints. However, after some time, a more significant error began to exist between the actual and estimated positions.
Despite some discrepancies, the robot was able to navigate the desired path. However, in a real environment, the errors could be much larger, since the robot odometry used by the AMCL algorithm could be subject to errors due to numerous  factors, e.g., wheel misalignment, divergent wheel diameters, uneven terrain and skidding. Similarly, the LiDAR sensor readings could also be subject to errors from the sensor itself, or due to changes in the environment, such as moving obstacles. These factors would lead to a greater accumulation of errors in position estimation.
In the second scenario, the robot did not know its initial position, and although the AMCL initially converged close to the robot's actual position based on the LiDAR sensor readings, there was a large discrepancy in the angle estimation. This made the robot to get lost during the navigation in the first trajectory, moving far away from the destination point, before the AMCL is able to perform a particle convergence that best matches the actual position of the robot. In the second test, the AMCL was able to converge its estimates with a higher precision relative to the robot's actual position, ensuring that it could reach the final goal, but the discrepancies remained high for the most part of the course until the particle filtering could be better performed based on the LiDAR sensor reading and the map. Fig. 7 shows the position samples during the tests.  For the third scenario, the initial position of the robot is reported to the AMCL algorithm. The position updates to remove the intrinsic errors accumulated by the algorithm based on the tag detection allowed the position estimates to remain closer to the actual robot positions than in the first scenario in both trajectories. In Fig. 8 the samples of the actual and estimated positions throughout the robot's navigation are displayed, as well as the times at which the tag-based updates were performed. It is quite noticeable that when position updates occur due to the detection of fiducial markers, the existing error between the actual and estimated position is reduced.
(b) Trajectory 2. The effectiveness of using the tag detection for improving the robot's indoor localization is even more evident in the fourth scenario. Despite representing a case of the kidnapped robot problem, the tag detection allowed obtaining an accurate initial position for the AMCL, which prevented the robot from getting lost as occurred in the second scenario. The other position updates made by the system kept the estimated and actual positions close together throughout the trajectories, eliminating any errors that arose in regions without tags. The results of the localization system are shown in Fig. 9.  The integration of the AR tags detection with the AMCL algorithm proved to be a more reliable way to obtain the localization information to a mobile robot in an indoor environment and to achieve the autonomous robot navigation, than just use the AMCL algorithm with the LiDAR and odometry data. The proposed method allows the robot to accurately determine its position and orientation even in situations where the robot is kidnapped, and be helpful in decreasing the localization errors.

VI. CONCLUSIONS AND FUTURE WORK
The present work addressed the problem of mobile robot indoor localization. The integration of the probabilistic algorithm AMCL with the position tracking of AR tags, proved to be efficient to obtain an accurate localization system in an indoor environment, being more effective than the more traditional approach of using the AMCL algorithm, especially in cases of the abducted robot problem, since it provides a way to obtain the actual position of the robot in the environment based on the tag identification. Future work will be dedicated to validating the localization system in the real environment, to ensure that the system can be robust enough to perform autonomous navigation of a mobile robot amidst dynamic obstacles, more complex routes, and high uncertainty in the data collected by the sensors.