Point-Line LIVO Using Patch-Based Gradient Optimization for Degenerate Scenes

Simultaneous localization and mapping based on 3-D light detection and ranging (LiDAR) tends to degenerate in structural-less environments, leading to a distinct reduction in localization accuracy and mapping precision. This article proposes a point-line LiDAR-visual-inertial odometry (PL-LIVO) based on the system implementation of FAST-LIVO for robust localization in LiDAR-degenerate scenes. The key idea is integrating both points and lines into the proposed direct visual odometry subsystem (PL-DVO). By minimizing the patch-based gradient residuals for state optimization, PL-DVO provides additional constraints complementary to LiDAR. Furthermore, a LiDAR map assisted visual features depth extraction (LM-VDE) method is proposed to recover 3-D positions of visual features by mapping them onto the 3-D planes of the LiDAR map. This method is independent of the single scan's density and notable for superior generalization across various LiDAR sensors. Extensive experiments on both public datasets and our datasets demonstrate that PL-LIVO ensures robust localization and outperforms other state-of-the-art systems in LiDAR degenerate scenes.

Point-Line LIVO comprises a backbone LiDAR-inertial odometry (blue and gray components) and a direct visual odometry subsystem PL-DVO (red component). Our proposed LM-VDE (green component) method is employed to extract all visual features depth. The data of IMU, LiDAR, and camera are tightly coupled at the observation level.

Initially, the input data from LiDAR and camera are fed into the Preprocessing module for data alignment and feature extraction, resulting in an aligned unit for the following steps. Concurrently, the IMU data is gathered for state propagation. Then, the aligned unit is utilized to construct observation residuals in the Data Association module. In detail, the point-to-plane residual is calculated for each LiDAR scan point, while the patch-based gradient residual is computed for each visual feature within the camera FoV. In particular, the lines patch pattern is shaped as a rectangle aligned with the line direction. All residuals are weighted by different factors and subsequently employed to update the state in an error-state Kalman filter framework. Furthermore, the Feature Generation module recovers 3-D coordinates of all extracted 2-D visual features by our proposed LM-VDE method.