Inter-observer agreement and accuracy in classifying radiographs for pneumoconiosis among Asian physicians taking AIR Pneumo certification examination

Naw Awn J-P, Agus Dwi Susanto, Erlang Samoedro, Muchtaruddin Mansyur, Sutarat Tungsagunwattana, Saijai Lertrojanapunya, Ponglada Subhannachart, Somkiat Siriruttanapruk, Narongpon Dumavibhat, Eduardo Algranti, John E. Parker, Kurt G. Hering, Hitomi Kanayama, Taro Tamura, Yukinori Kusaka, Narufumi Suganuma

Research output: Contribution to journalArticlepeer-review

Abstract

This study examined inter-observer agreement and diagnostic accuracy in classifying radiographs for pneumoconiosis among Asian physicians taking the AIR Pneumo examination. We compared agreement and diagnostic accuracy for parenchymal and pleural lesions across residing countries, specialty training, and work experience using data on 93 physicians. Physicians demonstrated fair to good agreement with kappa values 0.30 (95% CI: 0.20-0.40), 0.29 (95% CI: 0.23-0.36), 0.59 (95% CI: 0.52-0.67), and 0.65 (95% CI: 0.55-0.74) in classifying pleural plaques, small opacity shapes, small opacity profusion, and large opacities, respectively. Kappa values among Asian countries ranging from 0.25 to 0.55 (pleural plaques), 0.47 to 0.73 (small opacity profusion), and 0.55 to 0.69 (large opacity size). The median Youden's J index (interquartile range) for classifying pleural plaque, small opacity, and large opacity was 61.1 (25.5), 76.8 (29.3), and 88.9 (23.3), respectively. Radiologists and recent graduates showed superior performance than other groups regarding agreement and accuracy in classifying all types of lesions. In conclusion, Asian physicians taking the AIR Pneumo examination were better at classifying parenchymal lesions than pleural plaques using the ILO classification. The degree of agreement and accuracy was different among countries and was associated with background specialty training.

Original languageEnglish
Pages (from-to)459-469
Number of pages11
JournalIndustrial Health
Volume60
Issue number5
DOIs
Publication statusPublished - 1 Oct 2022

Keywords

  • AIR Pneumo
  • Chest radiograph
  • Diagnostic accuracy
  • Occupational health
  • Pneumoconiosis
  • Reader agreement

Fingerprint

Dive into the research topics of 'Inter-observer agreement and accuracy in classifying radiographs for pneumoconiosis among Asian physicians taking AIR Pneumo certification examination'. Together they form a unique fingerprint.

Cite this