(P 233) MULTI-CENTER EXTERNAL VALIDATION OF AN AUTOMATED MODEL DIFFERENTIATING ALT FROM LIPOMA USING RADIOMICS AND DEEP-LEARNING BASED SEGMENTATION

Objective: Lipomatous tumors are the most frequently encountered soft tissue tumors (STT), ranging from benign lipomas to malignant liposarcomas. Atypical lipomatous tumors (ALTs) are considered borderline tumors due to their locally aggressive growth patterns, tendency for recurrence, and risk for progression to dedifferentiated liposarcoma (DDLS) which can metastasize. As prognosis and treatment differs between ALTs and lipomas, it is vital to differentiate between these two phenotypes. Distinction is typically achieved through investigation of MDM2 amplification, which serves as a diagnostic marker for ALT, through Fluorescence In Situ Hybridization (FISH) testing on needle biopsy material. However, biopsies impose burdens on the patient and are susceptible to sampling errors.
Alternatively, Magnetic Resonance Imaging (MRI) gathered in routine patient care could provide insight in the differentiation of lipomas and ALT. Previously, we have proposed to differentiate between lipomas and ALTs with quantitative imaging features using machine learning (radiomics) to remove the need for invasive biopsies in the diagnostic process [1]. However, before introducing radiomics into clinical practice, external validation of our radiomics models is crucial to ascertain its generalizability and clinical applicability. Moreover, radiomics requires manual segmentation of the region of interest (ROI) to extract imaging features which requires expert knowledge, is time-consuming, and prone to interobserver variability, and thus unfeasible in the radiologist’s clinical workflow. Automatic segmentation techniques using deep learning offers a potential solution by streamlining the process.
This study aims to externally validate our previously described radiomics model to differentiate between lipomas and ALTs on MRI in a large multi-center cohort [1]. Additionally, we extend our model with automatic and minimally interactive segmentation methods to increase the clinical feasibility [2].

Methods: Three different study cohorts were created. For training, we used the original training cohort previously described by Vos et al [1], which includes 48 Lipoma and 48 ALT patients from the Netherlands (Erasmus MC) and is publicly available [5]. This dataset includes manual segmentations made either by a medical MSc student or a medical PhD student under supervision of a musculoskeletal radiologist. For validation, two datasets were used. An external dataset was formed by data from the United States (UC Davis, MD Anderson, UCSF, and UC Irvine), and the United Kingdom (Royal Marsden). A prospective dataset contained prospectively collected data from the Erasmus MC. For both cohorts, patient characteristics, tumor type, MDM2 amplification status, and T1-weighted MRI sequences were collected.
Our previously trained automatic segmentation method was used to automatically segment all lipoma and ALT lesions on the plain T1-weighted MRI scans of the validation cohorts [2]. Subsequently, segmentations were visually scored by a medical PhD student under the supervision of a musculoskeletal radiologist. Based on the percentage of volume overlap of the automatic segmentation with the tumor, segmentations were scored “Sufficient” (>75% overlap ) or "Insufficient” ( < 75% overlap) . When scored “Insufficient”, our interactive segmentation method was used instead [2]. Finally, in patients where the interactive segmentation also failed to provide an accurate segmentation, segmentations were manually adjusted (Figure 2).
To predict between MDM2 amplification status, we used the model as described by Vos et al. [1] based on the workflow for optimal radiomics classification (WORC) radiomics method [4]. WORC extracts imaging features from the tumor region such as intensity, shape, and texture features. Next, WORC searches, in a fully automated way, among a large pool of machine learning algorithms, including feature selection, dimensionality reduction, and classification algorithms, to find a combination which maximizes the prediction accuracy while maintaining robustness (Figure 1). Performance was evaluated using a 100 times random-split cross-validation on the training dataset, and for the validation datasets using 10000 times bootstrap resampling.

Results: The two validation datasets in total included 444 tumors (181 ALT, 263 lipoma), of which 86 (24 ALT, 62 lipoma) in the prospective validation cohort and 358 (157 ALT, 201 lipoma) in the external validation cohort. The mean age was 58 years and 50% of patients was male. In the two validation cohorts, 347 automatic segmentations were scored sufficient and 97 insufficient. The insufficient automatic segmentations were interactively segmented, resulting in 84 sufficient and 13 insufficient segmentations. These 13 insufficient interactive segmentations were manually adjusted.
The receiver operating characteristic curves of the model are illustrated in Figure 3; the metrics are reported in Table 1. The radiomics model based on T1-weighted MRI sequences resulted in a mean area under the curve (AUC) of 0.83, sensitivity of 0.71, and specificity of 0.78 in the cross-validation in the training cohort. The validation of the radiomics model in the external validation cohort resulted in a mean AUC of 0.80, sensitivity of 0.76, and specificity of 0.72. Lastly, validation in the prospective cohort resulted in a mean AUC of 0.91, sensitivity of 0.71, and specificity of 0.88.

Conclusion: Our radiomics model can accurately differentiate between lipomas and ALT based on quantitative imaging features on MRI in a large, multi-center external cohort, and in prospective validation. Furthermore, our automatic and minimally interactive segmentation methods provided adequate segmentation for use in radiomics, as the need for manual segmentation was minimal. Therefore, our radiomics and segmentation pipeline provides an accurate, non-invasive, and clinically feasible method for differentiating between lipomas and ALTs, possibly limiting the need for biopsies.

References:
1. Vos M et al. Radiomics approach to distinguish between well differentiated liposarcomas and lipomas on MRI. Br J Surg. 2019 Dec;106(13):1800-1809.
2. Spaanderman D.J. et al. Minimally Interactive Segmentation of Soft-Tissue Tumors on CT and MRI using Deep Learning. 2023, Submitted.
3. Kikinis R, Pieper SD, Vosburgh K (2014) 3D Slicer: a platform for subject-specific image analysis, visualization, and clinical support. Intraoperative Imaging Image-Guided Therapy, Ferenc A. Jolesz, Editor 3(19):277–289 ISBN: 978-1-4614-7656-6 (Print) 978-1-4614-7657-3 (Online).
4. Starmans, M.P.A. et al. Reproducible radiomics through automated machine learning validated on twelve clinical applications. Medical Image Analysis. 2022 Aug; https://doi.org/10.48550/arXiv.2108.08618.
5. Starmans, M.P.A. et al. The WORC database: MRI and CT scans, segmentations, and clinical labels for 930 patients from six radiomics studies. Data in Brief. 2021 Aug; https://doi.org/10.1101/2021.08.19.21262238