Initially when I joined Robotics Research Center, IIIT Hyderabad, I worked on the project of making Husky autonomous using visual perception. I worked on this project for nearly a month and made the bot capable of avoiding obstacles and follow waypoints in indoors and outdoors. The video presented below shows a short video taken in indoors.
In order to start off with this project I tried creating occupancy grids from stereo images. Firstly, I tried this out on KITTI dataset to get a hold of it. I started by doing Semi-Global Block Matching (SGBM) on KITTI images to get stereo disparities. Also the SGBM parameters were tuned to get good disparity values. And further I extended this script to get PCL pointclouds. After downsampling the pointcloud to a size so that it becomes computationally less expensive, I did planar segmentation of the ground and removed it, following which a Euclidean Cluster Extraction was applied to get the obstacle information. These were projected down to a plane to form an occupancy grid.
This method was computationally expensive and it would be difficult to plan at high frequencies. Thus, moved on to creating occupancy grids based on height thresholds. One such package which was helpful in creating was octomap server. The created occupancy grid was used by an RRTstar planner to plan path towards the goal locations that are specified. In case of outdoors a global planner was present based on which the local planner planned its path. Odometry was obtained using various methods and was finally fused to get a reliable state estimate. We used ORBSLAM2 on stereo images, husky's inbuilt odometry estimate based on wheel encoder and finally fused this information along with IMU to get a good estimate of the pose.