A group of researchers from Zhejiang University has proposed an interesting approach for image instance segmentation called DeepSnake that achieves state-of-the-art results on the popular Cityscapes dataset.
Image segmentation is traditionally addressed as pixel-wise classification and most of the state-of-the-art methods in semantic, instance and panoptic segmentation rely on neural networks to perform this task. Another approach that does not rely on pixel segmentation is the so-called snake algorithm which is in fact, a contour-based method that tries to fit an object boundary around an object and perform segmentation in this way. However, existing methods based on this approach rely on handcrafted features and objective functions.
The proposed method – DeepSnake, uses deep neural networks together with contour deformation in order to provide real-time instance segmentation. Researchers proposed a two-stage architecture where contour proposals are generated in the first stage, and during the second stage, contour deformation provides the final output. They introduced circular convolutions to learn features on the contours of the object which are considered as graphs. In their setting, after “rough” contours were detected in the contour proposal stage, the graph representing the contour is regressed at each vertex to deform the contour to the exact object boundary.
In the paper, researchers explain the detailed architecture of DeepSnake and how it can be applied to real-time instance segmentation. They conducted a number of experiments using wide-known datasets such as Cityscapes, Kins and Sbd. The evaluations showed that the method is superior to existing state-of-the-art methods on the three datasets while having real-time performance.
The implementation of the method was open-sourced and it is available on Github. More in detail about DeepSnakes can be read in the paper published on arxiv.