Multi-Reader Sample Size Program for Diagnostic Studies

Overview (updated 12-18-18): 

The Multi-Reader Sample Size Sample Size Program for Diagnostic Studies performs sample size computations for diagnostic reader-performance studies. Such studies are commonly used in radiology, where radiologists evaluate cases based on corresponding images resulting from an imaging modality (such as mammography for breast cancer) with respect to confidence of disease. In such studies a typical reader-performance measure is the area under the ROC curve. In the most common study design, the factorial design, each reader reads all cases under each modality (or test). 

The program includes options for specifying five different study designs, nonequivalence and noninferiority hypotheses, and three different inference situations. The program works for any type of reader performance outcome. The program requires parameter estimates that are either conjectured or obtained from pilot data. Pilot data estimates can be obtained from analyses of data sets that use the methods proposed by Obuchowski & Rockette and by Dorfman, Berbaum and Metz, and updated by Hillis and colleagues. The software provides various combinations of the numbers of readers and cases needed for a specified power, or alternatively provides the power corresponding to specified numbers of readers and cases.

A demonstration of the program and discussion of its underlying statistical methodology in provided in the Journal of Medical Imaging article “Multireader sample size program for diagnostic studies: demonstration and methodology” by Hillis and Schartz (2018).  A briefer introduction is given in the User Manual.  See below for download links for both of these documents.

This software was supported by the National Institute of Biomedical Imaging and Bioengineering (NIBIB) of the National Institutes of Health under Award Numbers R01EB013667 and R01EB019967. 

Questions? Contact Steve Hillis (steve-hillis@uiowa.edu).  To receive notification of updates for the power and sample size program, send an email to steve-hillis@uiowa.edu with “Register for sample-size software updates” in the subject line.

Brief Instructions

The program will run under Windows, macOS, and Linux. However, you must have a Java run-time environment (JRE) installed. The process of obtaining the JRE varies with operating system. To install the program, simple download the “Executable” file below. This is an executable jar file. Users may also wish to download the “Instructions” file below which provides a description of obtaining the current JRE for Windows. To run the program, simply double-click the executable jar file. The name of the jar file will be of the form pssYYYYMMDD.jar where YYYYMMDD is the date of the current version of the program (e.g., pss20171206.jar). The user manual is contained within the jar file and may be accessed using the Help menu in the program (it is also available as a separate download below).

Note to macOS and Linux users: The program has only been tested using the JRE provided by Oracle. Use of other JREs is possible but not supported.

Downloads