Objective To determine the agreement and reliability of fully automated coronary artery calcium (CAC) scoring in a lung cancer screening population. Materials and Methods 1793 low-dose chest CT scans were analyzed (non-contrast-enhanced, non-gated). To establish the reference standard for CAC, first automated calcium scoring was performed using a preliminary version of a method employing coronary calcium atlas and machine learning approach. Thereafter, each scan was inspected by one of four trained raters. When needed, the raters corrected initially automaticity-identified results. In addition, an independent observer subsequently inspected manually corrected results and discarded scans with gross segmentation errors. Subsequently, fully automatic coronary calcium scoring was performed. Agatston score, CAC volume and number of calcifications were computed. Agreement was determined by calculating proportion of agreement and examining Bland-Altman plots. Reliability was determined by calculating linearly weighted kappa (κ) for Agatston strata and intraclass correlation coefficient (ICC) for continuous values. Results 44 (2.5%) scans were excluded due to metal artifacts or gross segmentation errors. In the remaining 1749 scans, median Agatston score was 39.6 (P25–P75:0–345.9), median volume score was 60.4 mm3 (P25–P75:0–361.4) and median number of calcifications was 2 (P25–P75:0–4) for the automated scores. The κ demonstrated very good reliability (0.85) for Agatston risk categories between the automated and reference scores. The Bland-Altman plots showed underestimation of calcium score values by automated quantification. Median difference was 2.5 (p25–p75:0.0–53.2) for Agatston score, 7.6 (p25–p75:0.0–94.4) for CAC volume and 1 (p25–p75:0–5) for number of calcifications. The ICC was very good for Agatston score (0.90), very good for calcium volume (0.88) and good for number of calcifications (0.64). Discussion Fully automated coronary calcium scoring in a lung cancer screening setting is feasible with acceptable reliability and agreement despite an underestimation of the amount of calcium when compared to reference scores.
References
[1]
Hansson GK (2005) Inflammation, atherosclerosis, and coronary artery disease. N Engl J Med 352: 1685–1695. doi: 10.1056/nejmra043430
[2]
Unverdorben M, von Holt K, Winkelmann BR (2009) Smoking and atherosclerotic cardiovascular disease: part II: role of cigarette smoking in cardiovascular disease development. Biomark Med 3: 617–653. doi: 10.2217/bmm.09.51
[3]
Wexler L, Brundage B, Crouse J, Detrano R, Fuster V, et al. (1996) Coronary artery calcification: pathophysiology, epidemiology, imaging methods, and clinical implications. A statement for health professionals from the American Heart Association. Writing Group. Circulation 94: 1175–1192. doi: 10.1161/01.cir.94.5.1175
[4]
Yeboah J, McClelland RL, Polonsky TS, Burke GL, Sibley CT, et al. (2012) Comparison of novel risk markers for improvement in cardiovascular risk assessment in intermediate-risk individuals. JAMA 308: 788–795. doi: 10.1001/jama.2012.9624
[5]
Nasir K, Rubin J, Blaha MJ, Shaw LJ, Blankstein R, et al. (2012) Interplay of coronary artery calcification and traditional risk factors for the prediction of all-cause mortality in asymptomatic individuals. Circ Cardiovasc Imaging 5: 467–473. doi: 10.1161/circimaging.111.964528
[6]
Ma S, Liu A, Carr J, Post W, Kronmal R (2010) Statistical modeling of Agatston score in multi-ethnic study of atherosclerosis (MESA). PLoS One 5: e12036. doi: 10.1371/journal.pone.0012036
[7]
McEvoy JW, Blaha MJ, Nasir K, Blumenthal RS, Jones SR (2012) Potential use of coronary artery calcium progression to guide the management of patients at risk for coronary artery disease events. Curr Treat Options Cardiovasc Med 14: 69–80. doi: 10.1007/s11936-011-0154-5
[8]
van den Bergh KA, Essink-Bot ML, Borsboom GJ, Th Scholten E, Prokop M, et al. (2010) Short-term health-related quality of life consequences in a lung cancer CT screening trial (NELSON). Br J Cancer 102: 27–34. doi: 10.1183/09031936.00123410
[9]
National Lung Screening Trial Research Team, Aberle DR, Adams AM, Berg CD, Black WC, et al (2011) Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med 365: 395–409. doi: 10.1056/nejmoa1102873
[10]
Mets OM, Vliegenthart R, Gondrie MJ, Viergever MA, Oudkerk M, et al. (2013) Lung cancer screening CT-based prediction of cardiovascular events. JACC Cardiovasc Imaging 6: 899–907. doi: 10.1016/j.jcmg.2013.02.008
[11]
Xie X, Zhao Y, de Bock GH, de Jong PA, Mali WP, et al. (2013) Validation and Prognosis of Coronary Artery Calcium Scoring inNon-Triggered Thoracic Computed Tomography: Systematic Review and Meta-Analysis. Circ Cardiovasc Imaging.
[12]
Jacobs PC, Gondrie MJ, van der Graaf Y, de Koning HJ, Isgum I, et al. (2012) Coronary artery calcium can predict all-cause mortality and cardiovascular events on low-dose CT screening for lung cancer. AJR Am J Roentgenol 198: 505–511. doi: 10.2214/ajr.10.5577
[13]
Wu MT, Yang P, Huang YL, Chen JS, Chuo CC, et al. (2008) Coronary arterial calcification on low-dose ungated MDCT for lung cancer screening: concordance study with dedicated cardiac CT. AJR Am J Roentgenol 190: 923–928. doi: 10.2214/ajr.07.2974
[14]
Budoff MJ, Nasir K, Kinney GL, Hokanson JE, Barr RG, et al. (2011) Coronary artery and thoracic calcium on noncontrast thoracic CT scans: comparison of ungated and gated examinations in patients from the COPD Gene cohort. J Cardiovasc Comput Tomogr 5: 113–118. doi: 10.1016/j.jcct.2010.11.002
[15]
Kim SM, Chung MJ, Lee KS, Choe YH, Yi CA, et al. (2008) Coronary calcium screening using low-dose lung cancer screening: effectiveness of MDCT with retrospective reconstruction. AJR Am J Roentgenol 190: 917–922. doi: 10.2214/ajr.07.2979
[16]
Jacobs PC, Isgum I, Gondrie MJ, Mali WP, van Ginneken B, et al. (2010) Coronary artery calcification scoring in low-dose ungated CT screening for lung cancer: interscan agreement. AJR Am J Roentgenol 194: 1244–1249. doi: 10.2214/ajr.09.3047
[17]
Isgum I, Prokop M, Niemeijer M, Viergever MA, van Ginneken B (2012) Automatic coronary calcium scoring in low-dose chest computed tomography. IEEE Trans Med Imaging 31: 2322–2334. doi: 10.1109/tmi.2012.2216889
[18]
Rutten A, Isgum I, Prokop M (2011) Calcium scoring with prospectively ECG-triggered CT: using overlapping datasets generated with MPR decreases inter-scan variability. Eur J Radiol 80: 83–88. doi: 10.1016/j.ejrad.2010.06.009
[19]
Isgum I, Prokop M, Jacobs PC, Gondrie MJ, Mali WPTM, et al. (2010) Automatic coronary calcium scoring in low-dose non-ECG-synchronized thoracic CT scans. in SPIE MedImag: vol. 7624: 76240M–76240M-76248. doi: 10.1117/12.840514
[20]
Rumberger JA, Brundage BH, Rader DJ, Kondos G (1999) Electron beam computed tomographic coronary calcium scanning: a review and guidelines for use in asymptomatic persons. Mayo Clin Proc 74: 243–252. doi: 10.4065/74.3.243
[21]
Buckens CF, de Jong PA, Mol C, Bakker E, Stallman HP, et al. (2013) Intra and interobserver reliability and agreement of semiquantitative vertebral fracture assessment on chest computed tomography. PLoS One 8: e71204. doi: 10.1371/journal.pone.0071204
[22]
Kottner J, Audige L, Brorson S, Donner A, Gajewski BJ, et al. (2011) Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. J Clin Epidemiol 64: 96–106. doi: 10.1016/j.jclinepi.2010.03.002
[23]
Hokanson JE, MacKenzie T, Kinney G, Snell-Bergeon JK, Dabelea D, et al. (2004) Evaluating changes in coronary artery calcium: an analytic method that accounts for interscan variability. AJR Am J Roentgenol 182: 1327–1332. doi: 10.2214/ajr.182.5.1821327
[24]
Sevrukov AB, Bland JM, Kondos GT (2005) Serial electron beam CT measurements of coronary artery calcium: Has your patient’s calcium score actually changed? AJR Am J Roentgenol 185: 1546–1553. doi: 10.2214/ajr.04.1589
[25]
Bland JM (2005) The half-normal distribution method for measurement error: two case studies. Unpublished talk available on http://www-users.york.ac.uk/~mb55/talks/？halfnor.pdf.
[26]
Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33: 159–174. doi: 10.2307/2529310
[27]
Wender R, Fontham ET, Barrera E Jr, Colditz GA, Church TR, et al. (2013) American Cancer Society lung cancer screening guidelines. CA Cancer J Clin 63: 107–117. doi: 10.3322/caac.21172
[28]
Polonsky TS, McClelland RL, Jorgensen NW, Bild DE, Burke GL, et al. (2010) Coronary artery calcium score and risk classification for coronary heart disease prediction. JAMA 303: 1610–1616. doi: 10.1001/jama.2010.461
[29]
Shaw LJ, Raggi P, Schisterman E, Berman DS, Callister TQ (2003) Prognostic value of cardiac risk factors and coronary artery calcium screening for all-cause mortality. Radiology 228: 826–833. doi: 10.1148/radiol.2283021006
[30]
Mets OM, Vliegenthart R, Gondrie MJ, Viergever MA, Oudkerk M, et al. (2013) Lung Cancer Screening CT-Based Prediction of Cardiovascular Events. JACC Cardiovasc Imaging.
[31]
Callister TQ, Cooil B, Raya SP, Lippolis NJ, Russo DJ, et al. (1998) Coronary artery disease: improved reproducibility of calcium scoring with an electron-beam CT volumetric method. Radiology 208: 807–814.
[32]
Schmermund A, Voigtlander T (2011) Predictive ability of coronary artery calcium and CRP. Lancet 378: 641–643. doi: 10.1016/s0140-6736(11)61129-x
[33]
Liu YC, Sun Z, Tsay PK, Chan T, Hsieh IC, et al. (2013) Significance of coronary calcification for prediction of coronary artery disease and cardiac events based on 64-slice coronary computed tomography angiography. Biomed Res Int 2013: 472347. doi: 10.1155/2013/472347
[34]
Bielak LF, Kaufmann RB, Moll PP, McCollough CH, Schwartz RS, et al. (1994) Small lesions in the heart identified at electron beam CT: calcification or noise? Radiology 192: 631–636.
[35]
Achenbach S, Ropers D, Holle J, Muschiol G, Daniel WG, et al. (2000) In-plane coronary arterial motion velocity: measurement with electron-beam CT. Radiology 216: 457–463. doi: 10.1148/radiology.216.2.r00au19457
[36]
Shemesh J, Henschke CI, Farooqi A, Yip R, Yankelevitz DF, et al. (2006) Frequency of coronary artery calcification on low-dose computed tomography screening for lung cancer. Clin Imaging 30: 181–185. doi: 10.1016/j.clinimag.2005.11.002
[37]
Mets OM, de Jong PA, Prokop M (2012) Computed tomographic screening for lung cancer: an opportunity to evaluate other diseases. JAMA 308: 1433–1434. doi: 10.1001/jama.2012.12656
