Daniela's PhD

Calibration of LarCCell

LarCCell is a collaborative cell that serves as a case of study for this project. The ultimate goal is that it becomes a tridimensional space where it is safe for robots and humans to cooperate and perform common tasks. To achieve this goal, it is necessary to have a great abundance of sensors to monitor the space and to have redundancy to foresee occlusions. The current setup of LarCCell is shown in the following image.

To create a controlled environment and to have a "ground truth" to compare our real results, a simulation of our collaborative cell was developed to mimic the real cell that is installed in our laboratory. The following video shows a simulation of our current setup.

The calibration of Larcc was done both in the simulated and real environment. The simulation serves as a base, because the position of the sensors is precisely known and the calibrated position can be compared with the actual position of the sensors, and we can evaluate whether our system is behaving the way it was supposed to. For this, we needed four datasets: a train and test dataset for real world, and the same for simulation. The train dataset is used to do the actual calibration, and the test dataset serves to run the evaluation metrics: the calibrated transformations are applied to the test dataset collections, that have the sensors in the same position as the train dataset, but different collections and calibration pattern positions.

As mentioned in previous posts, to calibrate Larcc it was necessary to add the depth camera to ATOM framework. The automatic labeling of this camera was developed with a tetra-directional propagation algorithm. The propagation happens from a seed points that is initially manually given, and then it is updated from frame to frame by assuming that the centroid of the chessboard in the depth image is still inside the chessboard in the next frame. Overall this works well but sometimes it is necessary to manually readjust its position. The following image shows an example of a labeled depth image. This labeling is divided in limit points (purple points) and inside points (yellow points).

The following images show RGB and LiDAR labels in the real larcc.

Manual Labeling

We noticed that there were being some problems with the LiDAR and depth labeling in the real data because nearby objects often confuse the automatic algorithms. Even small errors can have a huge impact on the calibration accuracy, so these labeling issues had to be addressed. For this, we developed a dataset reviewer, that loads the dataset and allows to manually correct both LiDAR and depth labelling to obtain more accurate results. The following video shows an example of the utilization of this dataset reviewer.

Calibration

The calibration framework was overall maintained except for the added depth camera. To calibrate the depth camera the philosophy was similar to the LiDAR: the inside label points were used to calculate the ortogonal error between the pattern and the depth projection and the limit labels were used to calculate the longitudinal errors. The main difference between these methodologies is that LiDARs already give us tridimensional points, and depth give us (x_pix, y_pix) and a distance value. So these values need to be converted using the pinhole camera model and by knowing Z (the value of the pixel in a specific coordinates) and we can obtain the real X,Y,Z coordinates.

So far, all the calibration tests performed in real world datasets retrieved from different bagfiles showed promising results with sub-pixel error for RGB cameras and milimetric error for LiDARs and depth cameras.

The following video shows a demonstration of the calibrate collaborative cell in real world.

The following images show the difference between calibrated and uncalibrated data.

January to April

Year 2

Calibration of LarCCell

Manual Labeling

Calibration

Other problems solved