Multi-State Constraint Kalman Filter
From February to August 2019 I wrote my master’s thesis at D-ARIA. The thesis revolved around the expansion of a Visual-Inertial Odometry algorithm for 3D position estimation. A stereocamera based photometric Multi-State Constraint Kalman Filter was derived and implemeted.

Resources

First of all, you can download the thesis as a pdf.

Also, watch this video summary of what the thesis is about:


Fancy Abstract

Position estimation is a fundamental component of an agent’s autonomy. Any mobile actor, ranging from humans, over autonomous cars to augmented reality devices, would cease to function without approximate knowledge of its current position. Therefore, extensive research has been devoted to position estimation algorithms. High precision estimators use an array of sensors and require a large amount of computational resources. But small mobile devices and robots are severely payload-limited, regularly operate in environments where external sensors are not feasible and offer only limited computing power.

Visual-inertial odometry is a class of algorithms commonly used when faced with these restrictions. Inertial information is fused with the data of one or multiple cameras making the algorithms independent of external sensors. By only estimating the relative change of the vehicle per time-step, the algorithms lose some of their accuracy but also greatly reduce the computational overhead. One of the most resource-friendly algorithms in this class is the stereo-camera based Multi-State Constraint Kalman Filter, which maintains competitive estimation results. This filter uses feature points to correct the predicted movement of the inertial sensor. A variation on a mono-camera based Multi-State Constraint Kalman Filter instead uses pixel-patches of photometric measurements as a basis for the cost function.

This thesis presents the derivation and implementation of a stereo-camera based photometric Multi-State Constraint Kalman Filter. Additionally, the feature position estimation of the stereo Multi-State Constraint Kalman Filter is reformulated, constraining the feature position by the first camera measurement - the anchor frame. Both implementations are based on the opensource implementation of the Multi-State Constraint Kalman Filter pipeline by Kumar Robotics. Both implementations are extensively evaluated using 14.5 hours of sensor recordings from open-source datasets. The anchor-frame based filter shows a 2.2% reduction in CPU load with a relative root mean square error (RMSE) increase from 0.19% to 0.29%. The stereophotometric approach shows a lack of robustness in most recordings, as it is dependent on high-quality feature selection and has a small error tolerance. The implementation returns an accuracy of 1.1% relative RMSE when operating within the error tolerances, showing promise for increased robustness in future implementations.