A Graph-Based Approach for Simultaneous Semantic and Instance Segmentation of Plant 3D Point Clouds

Katia Mirande (1, 2, 3) , Christophe Godin (2, 3) , Marie Tisserand (3) , Julie Charlaix (3) , Fabrice Besnard (3) , Franck Hétroy-Wheeler (1)

1 UNISTRA - Université de Strasbourg
2 MOSAIC - Simulation et Analyse de la morphogenèse in siliCo
3 RDP - Reproduction et développement des plantes

Accurate simultaneous semantic and instance segmentation of a plant 3D point cloud is critical for automatic plant phenotyping. Classically, each organ of the plant is detected based on the local geometry of the point cloud, but the consistency of the global structure of the plant is rarely assessed. We propose a two-level, graph-based approach for the automatic, fast and accurate segmentation of a plant into each of its organs with structural guarantees. We compute local geometric and spectral features on a neighbourhood graph of the points to distinguish between linear organs (main stem, branches, petioles) and two-dimensional ones (leaf blades) and even 3-dimensional ones (apices). Then a quotient graph connecting each detected macroscopic organ to its neighbours is used both to refine the labelling of the organs and to check the overall consistency of the segmentation. A refinement loop allows to correct segmentation defects. The method is assessed on both virtual and real 3D point-cloud data sets of Chenopodium album (wild spinach) and Solanum lycopersicum (tomato plant).

Mots clés: instance segmentation; semantic segmentation; Fiedler vector; quotient graph; phenotyping

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]

Katia Mirande, Christophe Godin, Marie Tisserand, Julie Charlaix, Fabrice Besnard, et al.. A Graph-Based Approach for Simultaneous Semantic and Instance Segmentation of Plant 3D Point Clouds. 2022. ⟨hal-03746572v1⟩