Assessing mass media reporting of disease-related genetic discoveries: Development of an instrument and initial findings

To develop an instrument to assess the content and balance of media stories about genetic discoveries relevant to human diseases and assess its ability to detect variability. Methods: Consumer focus groups to help develop the instrument, and surveys of scientists and journalists to evaluate the items. Ratings by at least two readers assessed reliability. Variability was measured by comparing scores of 47 stories emanating from two discoveries. Results: Reader agreement on content items was good (kappa = 0.74). Consumers, scientists, and journalists were in substantial agreement about what content items should be included. Some items deemed essential by journalists appeared in very few stories. One-third of stories exaggerated the benefits of the discovery and one-third presented a balanced selection of expert opinion. Coverage of specific items varied from 11.5 percent to 84.6 percent. The instrument is reliable; contains items deemed important by consumers, scientists, and journalists; and detects variability.