Interobserver variability and positive predictive value for ultrasonographic BI-RADS categories requiring pathohistological evaluation

Objective: The objective of this study was an analysis of interobserver variability and positive predictive value (PPV) for BI-RADS categories requiring pathohistological evaluation: 4A, 4B, 4C, and 5. Material and Methods: Interobserver variability for each of descriptors as well as PPV for final BI-RADS categories requiring pathohistological evaluation was measured in a retrospective study which included 30 ultrasonographic reports, with pathohistological verification, randomly selected from ultrasonographic reports from Institute for Oncology and Radiology of Serbia where about 1,100 breast cancers are verified every year. Ten observers, seven gynecologists, and three radiologists, independently rated each ultrasonographic report according to the fourth edition of BI-RADS atlas. Interobserver variability was measured with k coefficient. Results: There was most conformity for a category of orientation (k = 0.79). Substantial degree of conformity was also present for both boundary (k = 0.71) and shape (k = 0.65) categories. Moderate degree of conformity was achieved for posterior features (k = 0.54) and margins (k = 0.41) descriptors, while there was poor conformity in echogenicity (k = 0.38). In case of a final score, common conformity for all BI-RADS 4A, 4B, 4C, and 5 categories was (k = 0.51); it was the greatest for category 5 (k = 0.50), and it was less for categories 4C (k = 0.37), 4B (k = 0.32), and 4A (k = 0.29). Conclusions: Interobserver conformity for ultrasonographic descriptors and final evaluation of BI-RADS 4A, 4B, 4C, and 5 categories is good. PPV implies that not only division into categories 4 and 5, but also classification into categories 4 and subcategories 4A, 4B, and 4C are justified and clinically applicable.

BI-RADS; Interobserver variability; PPV.

[1] Raza S., Chikarmane S.A., Neilsen S.S., Zorn L.M., Birdwell R.L.: “BI-RADS 3, 4, and 5 lesions: value of US in management-- follow-up and outcome”. Radiology, 2008, 248, 773. doi: 10.1148/radiol. 2483071786. Epub 2008 Jul 22.

[2] Rahbar G., Sie A.C., Hansen G.C., Prince J.S., Melany M.L., Reynolds H.E., et al.: “Benign versus malignant solid breast masses: US differentiation”. Radiology 1999, 213, 889.

[3] Stavros A.T., Thickman D., Rapp C.L., Dennis M.A.,Parker S.H., Sisney G.A.: “Solid breast nodules: use of sonography to distinguish between benign and malignant lesions”. Radiology, 1995, 196, 123.

[4] Skaane P., Engedal K.: “Analysis of sonographic feature in the differentiation of fibroadenoma and invasive ductal carcinoma”. AJR Am. J. Roentgenol., 1998, 170, 109.

[5] Milosević Z., Karapandzić V.P., Jovićević A., Vuković V., Vucićević J., Pavlović T., Borojević N.: “Diagnosis of breast cancer in women age 40 and younger: mammography and breast ultrasound”. Acta Chir Iugosl., 2009, 56, 77. [Article in Serbian]

[6] Bird R.E., Wallace T.W., Yankaskas B.C.: “Analysis of cancers missed at screening mammography”. Radiology, 1992, 184, 613.

[7] Goergen S.K., Evans J., Cohen G.P., MacMillan J.H.: “Characteristics of breast carcinomas missed by screening radiologists”. Radiology, 1997, 204, 131.

[8] Mann B.D., Giuliano A.E., Bassett L.W., Barber M.S., Hallauer W., Morton D.L.: “Delayed diagnosis of breast cancer as a result of normal mammograms”. Arch. Surg., 1983, 118, 23.

[9] Wallis M.G., Walsh M.T., Lee J.R.: “A review of false negative mammography in a symptomatic population”. Clin. Radiol., 1991, 44, 13.

[10] Cahill C.J., Boulter P.S., Gibbs N.M., Price J.L.: “Features of mammographically negative breast tumors”. Br. J. Surg., 1981, 68, 882.

[11] Huynh P.T., Jarolimek A.M., Daye S.: “The false-negative mammogram”. Radiographics, 1998, 18, 1137.

[12] Siegal E.C., Angelakis E.J., Hartman A.: “Can peer review contribute to earlier detection of breast cancer? A quality initiative to learn from false-negative mammograms”. Breast J., 2008, 14, 330.

[13] Buchberger W., Niehoff A., Obrist P., DeKoekkoek-Doll P., Dunser M.: “Clinically and mammographically occult breast lesions: detection and classification with high-resolution sonog-raphy”. Semin. Ultrasound CT MR, 2000, 21, 325.

[14] Crystal P., Strano S.D., Shcharynski S., Koretz M.J.: “Using sonography to screen women with mammographically dense breasts”. AJR Am. J. Roentgenol., 2003, 181, 177.

[15] Gordon P.B., Goldenberg S.L.: “Malignant breast masses detected only by ultrasound. A retrospective review”. Cancer, 1995, 76, 626.

[16] Kaplan S.S.: “Clinical utility of bilateral whole-breast US in the evaluation of women with dense breast tissue”. Radiology, 2001, 221, 641.

[17] Leconte I, Feger C, Galant C, Berlière M., Berg B.V., D'Hoore W., Maldague B.: “Mammography and subsequent whole-breast sonog-raphy of nonpalpable breast cancers: the importance of radiologic breast density”. AJR Am. J. Roentgenol., 2003, 180, 1675.

[18] Berg W.A., Blume J.D., Cormack J.B., Mendelson E.B., Lehrer D., Böhm-Vélez M., et al.: “Combined screening with ultrasound and mammography vs mammography alone in women at elevated risk of breast cancer”. JAMA, 2008, 299, 2151. doi: 10.1001/jama.299. 18.2151.

[19] Kerlikowske K., Grady D., Rubin S.M., Sandrock C., Ernster V.L.: “Efficacy of screening mammography: a meta-analysis”. JAMA, 1995, 273, 149.

[20] D'Orsi C.J., Bassett L.W., Berg W.A., et al.: “BI-RADS: mammogra-phy”. 4th ed. In: D'Orsi C.J., Mendelson E.B., Ikeda D.M., et al., (eds). Breast imaging reporting and data system: ACR BI-RADS – breast im-aging atlas. Reston (VA): American College of Radiology, 2003.

[21] Mendelson E.B., Baum J.K., Berg W.A., Merritt C.R., Rubin E.: “Ul-trasound”. In: Breast imaging reporting and data system (BI-RADS). 4th ed. Reston, Va: American College of Radiology, 2003.

[22] Ikeda D.M., Hylton M.N., Kuhl C.K., et al.: “Magnetic resonance imaging”. 1st ed. In: D'Orsi C.J., Mendelson E.B., Ikeda D.M., et al., (eds). Breast imaging reporting and data system: ACR BI-RADS – breast im-aging atlas. Reston (VA): American College of Radiology, 2003.

[23] Levy L., Suissa M., Chiche J.F., Teman G., Martin B.: “BIRADS ul-trasonography”. Eur. J. Radiol., 2007, 61, 202.

[24] Lazarus E, .Mainiero M.B., Schepps B., Koelliker S.L., Livingston L. S.: “BI-RADS lexicon for US and mammography: interobserver variability and positive predictive value”. Radiology, 2006, 239, 385. Epub 2006 Mar 28.

[25] Park C.S., Lee J.H., Yim H.W., Kang B.J., Kim H.S., Jung J.I., et al.: Observer agreement using the ACR Breast Imaging Reporting and Data System (BI-RADS)-ultrasound, First Edition (2003)”. Korean J. Radiol., 2007, 8, 397.

[26] Pichler E.: “Ultrazvučni atlas dojke: diferencijalna dijagnoza i interventne tehnike”. Zagreb: Skolska knjiga, 2005.