Comparison of different machine learning methods for the classification of indeterminate adrenal lesions incidentally diagnosed in contrast enhanced CT
DOI:
https://doi.org/10.37004/sefm/2024.25.2.001Keywords:
Radiomics, machine learning, LIFEx, adenoma, incidentaloma, suprarenalAbstract
Purpose: A comparison of different machine learning models to discriminate adrenal incidentalomas by CT studies was performed.
Methods: Sixty-two different features were obtained from a sample of 61 incidentalomas using the free license software LIFEx and 19 radiomic studies were performed with different models and feature selection methods to obtain the most efficient determination of possible malignancy. For all of them, four cross-validation methods were evaluated. Adenoma contouring was performed in duplicate by different radiologists evaluating all models in both groups.
Results: ROC AUC between 0.42 (0.09-0.81) and 0.92 (0.63-1.00), and accuracy of the models between 0.63 (0.43-0.79) and 0.94 (0.82-1.00). The best-performing model was the balanced logistic regression applied to the 14 features with an intraclass coefficient greater than 0.9, with which accuracy of 0.94 (0.74-1.00), ROC AUC of 0.917 (0.63-1.00), benign recall of 0.92 (0.65-1.00) and malignant recall of 1.00 (0.71-1.00) were obtained.
Conclusions: The evaluation and validation of different models has allowed us to obtain an efficient radiomic model for the discrimination of adrenal incidentalomas
References
1. Saruta T, Suzuki H, Shibata H. Adrenal incidentaloma. Nippon Naibunpi Gakkai zasshi 1993; 69(5): 509–519. https://doi.org/10.1507/endocrine1927.69.5_509
2. Mayo-Smith WW, Song JH, Boland GL, et al. Management of Incidental Adrenal Masses: A White Paper of the ACR Incidental Findings Committee. J. Am. Coll. Radiol. 2017; 14(8): 1038–1044. http://dx.doi.org/10.1016/j.jacr.2017.05.001
3. Terzolo M, Stigliano A, Chiodini I, et al. AME position statement on adrenal incidentaloma. Eur. J. Endocrinol. 2011; 164(6): 851–870. https://doi.org/10.1530/EJE-10-1147
4. Cawood TJ, Hunt PJ, O’Shea D, Cole D, Soule S. Recommended evaluation of adrenal incidentalomas is costly, has high false-positive rates and confers a risk of fatal cancer that is similar to the risk of the adrenal lesion becoming malignant; time for a rethink? Eur. J. Endocrinol. 2009; 161(4): 513–527. https://doi.org/10.1530/EJE-09-0234
5. Welch HG, Black WC. Overdiagnosis in cancer. J. Natl. Cancer Inst. 2010; 102(9): 605–613. https://doi.org/10.1093/jnci/djq099
6. Gillies RJ, Kinahan PE, Hricak H. Radiomics: Images are more than pictures, they are data. Radiology 2016; 278(2): 563–577. https://doi.org/10.1148/radiol.2015151169
7. Larue RTHM, Defraene G, De Ruysscher D, Lambin P, Van Elmpt W. Quantitative radiomics studies for tissue characterization: A review of technology and methodological procedures. Br. J. Radiol. 2017; 90(1070). https://doi.org/10.1259/bjr.20160665
8. Ligero M, Jordi-Ollero O, Bernatowicz K, et al. Minimizing acquisition-related radiomics variability by image resampling and batch effect correction to allow for large-scale data analysis. Eur. Radiol. 2021; 31(3): 1460–1470. https://doi.org/10.1007/s00330-020-07174-0
9. Nioche C, Orlhac F, Boughdad S, et al. Lifex: A freeware for radiomic feature calculation in multimodality imaging to accelerate advances in the characterization of tumor heterogeneity. Cancer Res. 2018; 78(16): 4786–4789. https://doi.org/10.1158/0008-5472.CAN-18-0125
10. Nioche C, Orlhac F, Buvat I. Texture — User Guide Local Image Features Extraction. https://www.lifexsoft.org/images/phocagallery/documentation/ProtocolTexture/UserGuide/TextureUserGuide.pdf (2023).
11. Carrasco JL, Jover L. Estimating the Generalized Concordance Correlation Coefficient through Variance Components. Biometrics 2003; 59(4): 849–858. https://doi.org/10.1111/j.0006-341X.2003.00099.x
12. Lahey MA, Downey RG, Saal FE. Intraclass correlations: There’s more there than meets the eye. Psychol. Bull. 1983; 93(3): 586–595. https://doi.org/10.1037/0033-2909.93.3.586
13. Raju VNG, Lakshmi KP, Jain VM, Kalidindi A, Padma V. Study the Influence of Normalization/Transformation process on the Accuracy of Supervised Classification. Proc. 3rd Int. Conf. Smart Syst. Inven. Technol. ICSSIT 2020 2020; (Icssit): 729–735. https://doi.org/10.1109/ICSSIT48917.2020.9214160
14. Peeken JC, Shouman MA, Kroenke M, et al. A CT-based radiomics model to detect prostate cancer lymph node metastases in PSMA radioguided surgery patients. Eur. J. Nucl. Med. Mol. Imaging 2020; 47(13): 2968–2977. https://doi.org/10.1007/s00259-020-04864-1
15. Zhang H, Li Z, Shahriar H, Tao L, Bhattacharya P, Qian Y. Improving prediction accuracy for logistic regression on imbalanced datasets. Proc. - Int. Comput. Softw. Appl. Conf. 2019; 1: 918–919. https://doi.org/10.1109/COMPSAC.2019.00140
16. Zhou CY, Chen YQ. Improving nearest neighbor classification with cam weighted distance. Pattern Recognit. 2006; 39(4): 635–645. https://doi.org/10.1016/j.patcog.2005.09.004
17. Quinlan JR. Induction of decision trees. Mach. Learn. 1986; 1(1): 81–106. https://doi.org/10.1007/bf00116251
18. Jin Z, Shang J, Zhu Q, Ling C, Xie W, Qiang B. RFRSF: Employee Turnover Prediction Based on Random Forests and Survival Analysis. Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics) 2020; 12343 LNCS: 503–515. https://doi.org/10.1007/978-3-030-62008-0_35
19. de Jesus FM, Yin Y, Mantzorou-Kyriaki E, et al. Machine learning in the differentiation of follicular lymphoma from diffuse large B-cell lymphoma with radiomic [18F]FDG PET/CT features. Eur. J. Nucl. Med. Mol. Imaging 2022; 49(5): 1535–1543. https://doi.org/10.1007/s00259-021-05626-3
20. Masters T. Multilayer Feedforward Networks. Pract. Neural Netw. Recipies C++ 1993; 77–116.
21. Seiffert C, Khoshgoftaar TM, Van Hulse J, Napolitano A. RUSBoost: A hybrid approach to alleviating class imbalance. IEEE Trans. Syst. Man, Cybern. Part ASystems Humans 2010; 40(1): 185–197. https://doi.org/10.1109/TSMCA.2009.2029559
22. Martínez-García JM, Suárez-Araujo CP, Báez PG. SNEOM: A sanger network based extended over-sampling method. Application to imbalanced biomedical datasets. Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics) 2012; 7666 LNCS(PART 4): 584–592. https://doi.org/10.1007/978-3-642-34478-7_71
23. Sharma N V., Yadav NS. An optimal intrusion detection system using recursive feature elimination and ensemble of classifiers. Microprocess. Microsyst. 2021; 85(June): 104293. https://doi.org/10.1016/j.micpro.2021.104293
24. Hasan KA, Hasan MAM. Prediction of Clinical Risk Factors of Diabetes Using Multiple Machine Learning Techniques Resolving Class Imbalance. ICCIT 2020 - 23rd Int. Conf. Comput. Inf. Technol. Proc. 2020; (December). https://doi.org/10.1109/ICCIT51783.2020.9392694
25. Ding H, Feng PM, Chen W, Lin H. Identification of bacteriophage virion proteins by the ANOVA feature selection and analysis. Mol. Biosyst. 2014; 10(8): 2229–2235. https://doi.org/10.1039/c4mb00316k
26. Berrar D. Cross-validation. Encycl. Bioinforma. Comput. Biol. ABC Bioinforma. 2018; 1–3(January 2018): 542–545. https://doi.org/10.1016/B978-0-12-809633-8.20349-X
27. Risk C, James PMA. Optimal Cross-Validation Strategies for Selection of Spatial Interpolation Models for the Canadian Forest Fire Weather Index System. Earth Sp. Sci. 2022; 9(2): 1–17. https://doi.org/10.1029/2021EA002019
28. Liu X, Lu J, Zhang G, et al. A machine learning approach yields a multiparameter prognostic marker in liver cancer. Cancer Immunol. Res. 2021; 9(3): 337–347. https://doi.org/10.1158/2326-6066.CIR-20-0616
29. Harrigan MP, Sultan MM, Hernández CX, et al. MSMBuilder: Statistical Models for Biomolecular Dynamics. Biophys. J. 2017; 112(1): 10–15. https://doi.org/10.1016/j.bpj.2016.10.042
30. Raschka S. Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning. 2018; http://arxiv.org/abs/1811.12808
31. Crimì F, Quaia E, Cabrelle G, et al. Diagnostic Accuracy of CT Texture Analysis in Adrenal Masses: A Systematic Review. Int. J. Mol. Sci. 2022; 23(2). https://doi.org/10.3390/ijms23020637
32. Elmohr MM, Fuentes D, Habra MA, et al. Machine learning-based texture analysis for differentiation of large adrenal cortical tumours on CT. Clin. Radiol. 2019; 74(10): 818.e1-818.e7. http://dx.doi.org/10.1016/j.crad.2019.06.021
33. Lisa M. Ho, Ehsan Samei, Maciej A. Mazurowski, Yuese Zheng, Brian C. Allen, Rendon C. Nelson and DM. Can Texture Analysis Be Used to Distinguish Benign From Malignant Adrenal Nodules on Unenhanced CT, Contrast-Enhanced CT, or In- Phase and Opposed-Phase MRI? Am. J. Roentgenol. 2019; 212:3(March): 554–561. https://doi.org/10.2214/AJR.18.20097
34. Sun PAN, Wang D, Mok VCT, Shi LIN. Comparison of Feature Selection Methods and Machine Learning Classifiers for Radiomics Analysis in Glioma Grading. 2019; 7.
35. Ni M, Wang L, Yu H, et al. Radiomics Approaches for Predicting Liver Fibrosis With Nonenhanced T 1 -Weighted Imaging : Comparison of Different Radiomics Models. 2021; https://doi.org/10.1002/jmri.27391
36. Prinzi F, Currieri T, Gaglio S, Vitabile S. Shallow and deep learning classifiers in medical image analysis. Eur. Radiol. Exp. 2024; 2. https://doi.org/10.1186/s41747-024-00428-2
37. Naseri H, Skamene S, Tolba M, et al. Radiomics ‑ based machine learning models to distinguish between metastatic and healthy bone using lesion ‑ center ‑ based geometric regions of interest. Sci. Rep. 2022; 1–13. https://doi.org/10.1038/s41598-022-13379-8
38. Decoux A, Duron L, Habert P, et al. Comparative performances of machine learning algorithms in radiomics and impacting factors. Sci. Rep. 2023; 1–10. https://doi.org/10.1038/s41598-023-39738-7
39. Parmar C, Grossmann P, Bussink J, Lambin P, Aerts HJWL. Machine Learning methods for Quantitative Radiomic Biomarkers. Nat. Publ. Gr. n.d.; 1–11. https://doi.org/10.1038/srep13087
40. Shao S, Zheng N, Mao N, et al. A triple-classi fi cation radiomics model for the differentiation of pleomorphic adenoma, Warthin tumour , and malignant salivary gland tumours on the basis of diffusion-weighted imaging. Clin. Radiol. 2021; 76(6): 472.e11-472.e18. http://dx.doi.org/10.1016/j.crad.2020.10.019
41. Qi S, Zuo Y, Chang R, Huang K, Liu J, Zhang Z. Using CT radiomic features based on machine learning models to subtype adrenal adenoma. 2023; 1–12. https://doi.org/10.1186/s12885-023-10562-6
42. Zheng Y, Liu X, Zhong Y, Lv F, Yang H. A Preliminary Study for Distinguish Hormone-Secreting Functional Adrenocortical Adenoma Subtypes Using Multiparametric CT Radiomics-Based Machine Learning Model and Nomogram. Front. Oncol. 2020; 10(September): 1–11. https://doi.org/10.3389/fonc.2020.570502
43. Winkelmann MT, Gassenmaier S, Walter SS, et al. Differentiation of adrenal adenomas from adrenal metastases in single-phased staging dual-energy CT and radiomics. Diagnostic Interv. Radiol. 2022; 28(3): 208–216. https://doi.org/10.5152/dir.2022.21691
