Automatic liver segmentation in abdominal CT images using combined 2.5D and 3D segmentation networks with high-score shape prior for radiotherapy treatment planning

Liver segmentation is a prerequisite for measuring hepatic volume in liver transplantation, modeling of the liver anatomy in hepatic surgery planning, and contouring in radiotherapy treatment planning. The main challenges of liver segmentation are the appearance similarity of liver and surrounding stomach, heart, and spleen in 2D images and are the large shape variations of liver in 3D volume. Therefore, we propose a deep learning-based liver segmentation method by using global context of three orthogonal planes to localize the liver in whole abdomen and by using local context of targeted liver bounding volume and high-score shape prior to delineate the liver without leakage to the surrounding structures. To localize the liver within the whole abdomen and exclude outliers through the global context, three 2D segmentation networks are learned on each axial, coronal, and sagittal planes. To consider the shape information obtained from the 2D segmentation network in the next 3D segmentation network, the high-score shape prior is generated by a weighted fusion of three score maps. To correct the fine details of the liver in the targeted liver bounding volume and to be less affected by shape variation, the 3D segmentation network is learned based on 3D U-Net with highscore shape prior. Experimental results show that the DSC of the proposed segmentation network with high-score shape prior (LiverNet-WS) was 94.3%, which is 5.4% higher than LiverNet without high-score shape prior. The proposed method accurately localized the liver within the whole abdomen by using global contexts of three orthogonal planes. Moreover, segmentation accuracy improved fine details considering local context and avoided over-segmentation considering high-score shape prior.