Stony Brook University


Grant Number: 1U01AR057948-01
Project Title: Ecological validity of PROMIS instruments
PI Name, Title and Contact Information:
  1. Arthur Stone, PhD, Principal Investigator, Distinguished Professor & Vice Chair, Department of Psychiatry and Behavioral Science, Stony Brook University
  2. Joan Broderick, PhD, Principal Investigator, Research Associate Professor, Department of Psychiatry and Behavioral Science, Stony Brook University
Other Key Personnel: Joan Broderick, PhD, Principal Investigator
Joseph Schwartz, PhD, Co-Investigator
David Cella, PhD, Co-Investigator
Web Site:  www.hsc.stonybrook.edu/som/psychiatry

Abstract:

Patient Reported Outcomes (PROs) are a necessary component of medical research and practice. The NIH-sponsored Patient-Reported Outcome Measurement Information System (PROMIS) has developed item banks and scales for several domains of symptoms and functioning, and these assessments are ready for validation. Ecological validity is an emerging and innovative approach to measurement validation. It is based on the premise that assessment scale scores using reporting periods of several days or more should have a high level of correspondence with ratings of that experience that are made on a daily or more frequent basis for the same reporting period. Good correspondence indicates that the recall measure is not seriously compromised by memory deficits or other recall biases. This application proposes three major aims:

AIM 1: to provide evidence of ecological and known–group validity of the PROMIS scales

AIM 2: to provide evidence of the responsiveness of the scales to change across time

AIM 3: to test a comprehensive theoretical model of 7–day recall in PROs.

Study participants will complete weekly computer–assisted (CAT) PROMIS scales of the several domains and will monitor symptoms daily (to provide ecologically valid information) across a 4–week protocol. Five samples have been selected to provide variability in mean levels of the PROMIS scales (e.g., general population vs. osteoarthritis) and to provide patterns of change in symptoms over the 28–days (e.g., symptom shifts following chemotherapy or surgery). Second, the ability of PROMIS scales to track expected patterns of within–group change that correspond to changes captured by daily ratings will comprise a strong test of PROMIS scales’ responsiveness to change. A comprehensive theoretical model of factors (cognitive heuristics) that influence recall ratings will also be tested. This research will provide a unique and important form of validation of PROMIS scales. The ability to probe the "meaning" of recall ratings will also make important contributions to the science of self–report and inform clinical interpretation of PROs.

GOALS:

  1. We propose to conduct an ecological validation study of PROMIS instruments for the five primary domains. The PROMIS scale scores will be compared with data captured with daily assessments over the 7–day reporting period used by the PROMIS assessments. This study will evaluate how accurately the PROMIS scale scores represent patient experience reported on a daily basis, which is less susceptible to recall and other biases. A high degree of correspondence between aggregated daily ratings and the PROMIS scale scores will demonstrate validity. Within this aim we will also test known-groups validity based on the ability of the PROMIS instruments to detect expected between-group differences.
  2. Another essential aspect of validity is the ability of PROMIS instruments to detect change across assessment points (responsiveness). We will determine the degree to which PROMIS instruments replicate actual patterns of change as measured by daily ratings. The one–month study will allow us to compare changes over four PROMIS 7–day recall ratings to changes in daily ratings for the same 7–day periods. Five different subject samples will be used to generate predictable variations in the amount of change (none to substantial) to provide a strong test of responsiveness. For example, it is expected that there will be little change across a month for the general population and osteoarthritis patients, whereas patients who are undergoing a surgical procedure or a chemotherapy infusion will show marked changes from before to after the medical procedure.
  3. Several factors have been hypothesized to affect the quality of 7–day recall PRO data, including cognitive heuristics. To date these factors have been investigated in a piecemeal manner, resulting in an incomplete understanding of their effects. This protocol will allow us to examine the predictive ability of a comprehensive model of recall in PROs. Prior research has suggested that 7–day recall PROs share approximately 50% of the variance with daily ratings. However, the nature of the unshared variance has not been studied. A cognitive interview at the close of the study will provide another source of information about how participants complete the 7–day PROMIS instruments. Understanding what information contributes to (and possibly biases) 7–day recall scores will advance the science of PRO research. Such insights could enhance our clinical interpretation of PRO data and provide an empirical basis for future efforts to improve and refine PRO measurement.

Patient Population:

We have proposed studying five groups of individuals. They are:

  1. healthy community individuals;
  2. patients with osteoarthritis drawn from the community;
  3. patients with either Premenstrual Syndrome or Premenstrual Dysphoric Disorder;
  4. patients attending an ambulatory surgery center for repair of either an inguinal or ventral hernia; and,
  5. female patients on a regimen of chemotherapy for breast cancer.

Disease Groups:

See above.

For questions concerning Stony Brook University’s role in the PROMIS network, please contact Arthur Stone, PhD, by e-mail at arthur.stone@sunysb.edu. For more information about the PROMIS network, go to http://www.nihpromis.org.