Towards bridging the sim to real gap for robotic manipulation through the development of a realistic simulation using Unity, MuJoCo and ROS2

Abstract

[EN] The dissimilar behaviour that occurs in a simulation and in the real world with seemingly the same controllers and physical features is a very real problem that current researchers are trying to minimise. This problem is known as the "Reality Gap" problem. Since the first use of simulations the problem has been present, as it must be remembered that there is currently not such thing as a perfect representation of the world. There are numerous factors that must be taken into account and be modelled in order to have the perfect simulation, and multiple that cannot be. Many current approaches focus on reducing this gap either by modifying the parameters of the simulation to fit as good as possible to the real world, or by generating more robust controllers that can adapt to some amount of discrepancies. Either way, even the quantification of the gap can be complex, and it should be adapted or limited to the task in hand. In this master thesis, the MuJoCo physics engine has been used to develop a simulation in Unity. In order to achieve the integration in the ROS2 (Robot Operating System) ecosystem, most precisely with MoveIt2, the proper software has been developed and implemented as well. Then, in order to quantify the fitness of the simulation, the interactions of a robotic arm and gripper with an aluminium piece in a manipulation task has been studied, both in simulation and in reality. These analyses have been portrayed in a metric, which has the objective of comparing the behaviour of said piece in simulation and reality and quantify the dissimilarities. The simulation parameters provided in this work show promising results, accurately representing a very close behaviour in simulation compared to reality. The metric also provides a distance-like coefficient as a result, thus opening the gates to parameter optimisation techniques in order to further reduce the Reality Gap.