Generalized Obuchowski-Rockette Methodology for Analysis of Radiologic Diagnostic Imaging Studies

PI: Stephen L Hillis

Co-investigators: Brian J Smith, Kevin Schartz

Funding: National Institute of Biomedical Imaging and Bioengineering (NIBIB) of the National Institutes of Health under Award Number R01EB025174

Dates: 09-01-2018 – 05-31-2022

Project Summary

Limitations in current statistical analysis methodology for radiologic diagnostic imaging studies greatly restrict a researcher’s choices of study designs and types of analyses. Therefore there is a critical need to create a more encompassing multi-reader diagnostic-study analysis methodology that does not limit the ability of researchers to answer research questions because of strict study-design constraints and few analysis options.

Our long-term goal is to develop statistical methodology and software appropriate for diagnostic radiological imaging studies that accounts for both patient and reader variability. Our objective here is to further develop Obuchowski-Rockette (OR) methodology (and related Dorfman-Berbaum-Metz) software in a way that allows for many more types of analyses and study designs. Specifically, we will pursue four specific aims:

(1) Develop a generalized OR methodology that accommodates the same types of designs and analysis models that can currently be used with conventional analysis-of-variance (ANOVA) and linear regression methodology.

(2) Provide illustrative examples showing how the generalized OR methodology can be applied to situations where the present OR methodology is not applicable. These include unbalanced designs, regression analyses, multivariate analyses, ANCOVA analyses, partially-paired-reader data, and missing-cases data.

(3) Empirically validate the OR methodology for each example in Aim 2 through simulations.

(4) Develop R, MATLAB and SAS software code for implementing the generalized OR methodology.

Expected outcomes include a generalized OR methodology that has the same analysis and design options as conventional linear regression and ANOVA methodology, in contrast to present OR methodology. The methodology will apply to any type of reader performance outcome, including ROC, FROC, region-of-interest (ROI) and lo cation ROC (LROC), with conclusions generalizing to both cases and readers. Other expected outcomes include illustrative examples of applications of the methodology that were not previously possible, R,MATLAB and SAS code for implementing the methodology, and a thorough validation of the methodology.

This research is significant because it will liberate the diagnostic-radiology researcher from the strict analysis and design limitations of the present methodology, thus enhancing the researcher’s ability to assess the usefulness of imaging modalities and the effects of reader and case attributes on reader performance in an efficient  and meaningful manner. It will be innovative because it will enable new NIH-pertinent horizons that presently are unattainable because of limited analysis methodology. Presently researchers in diagnostic radiologyare greatly constrained in the research questions that they can investigate by the stringent limitations of the OR (and related DBM) analysis methods. The proposed methodology, by expanding the scope of analyses, will greatly expand the possible scope of diagnostic radiology research questions that can be investigated.