|
|
|
|
|
|
|
From July 2006 to March 2007, data were collected from the U.S. general population and multiple disease populations. The sampling plan was developed for collecting responses to the candidate items from the targeted PROMIS domains and was designed to accommodate multiple objectives: (1) create item calibrations for each domain; (2) estimate profile scores for various disease populations; (3) create linking metrics to legacy questionnaires (e.g., SF-36); (4) confirm the factor structure of the domains; and (5) conduct item and bank analyses. Because of the large total number of items (> 1000), it was not possible for participants to respond to the entire pool. Based on an estimate of 4 questions per minute, the number of items administered to any respondent was limited to about 150 (37 minutes).
Two data collection designs (“full bank” and “block administration”) were used. Some individuals were administered full banks of 56 items for a subset of the PROMIS Calibration Studies domains while others were administered 14 blocks of 7 items selected from the 14 item banks (3 physical functioning banks, anxiety, depression, anger, substance abuse, fatigue impact, fatigue experience, social-role performance, social-role satisfaction, pain interference, pain quality, pain behavior). In addition to the PROMIS items and appropriate “legacy” items (items from widely used fixed length measures such as the SF-36) completed by those administered full banks of items, participants were administered approximately 21 auxiliary items consisting of global health rating items and sociodemographic variables including age, income, number of hospitalizations, disability days, whether they take prescribed medicines, body mass index, gender, race/ethnicity, whether the person was married or living with someone, educational attainment, and whether the person was working full-time, a full time student, or not working full time or going to school full-time. There were also a series of questions about the presence and degree of limitations related to 25 chronic medical conditions: hypertension, angina, CAD, heart failure, heart attack, stroke or TIA, liver disease, kidney disease, arthritis or rheumatism, osteoarthritis, migraines, asthma, COPD, diabetes, cancer, depression, anxiety, alcohol or drug problems, sleep disorder, HIV/AIDS, spinal cord injury, multiple sclerosis, Parkinson’s disease, epilepsy, and amyotrophic lateral sclerosis.
The full-bank administration allows for evaluation of dimensionality and calibration within item banks (domains). The block administration permits an evaluation of associations between domains. The ability to calibrate items on a common metric (item linking) is possible because of the administration of blocks of items from the full bank to general population and clinical samples. Each item was administered to at least 900 respondents from the general population and 500 respondents with a chronic medical condition.
Data were collected primarily by YouGovPolimetrix (www.polimetrix.com, also see www.pollingpoint.com), a polling firm based in Palo Alto, CA. YouGovPolimetrix operates PollingPoint.com, a centralized portal that allows interested individuals to provide their views about public policy and other current issues. The respondents for a typical YouGovPolimetrix Internet survey are selected from the PollingPoint panel, a panel of over one million respondents who have provided YouGovPolimetrix with their names, physical addresses, email addresses, and other information, and who regularly participate in online surveys. YouGovPolimetrix uses a sample matching procedure to select a representative sample when a listing of the population is available but it is difficult to draw a random sample from the population frame. The sample matching methodology starts with a listing of all respondents in the desired or target population. Next, a random sample of the desired size is selected from the population listing (the “target sample”). Third, for each element of the target sample, the closest match is selected from the PollingPoint panel. This method has been shown to give accurate results in a wide variety of applications, even for groups significantly underrepresented on the Internet. Its validity depends upon the panel of available respondents being sufficiently large and diverse, not upon Internet usage or other types of behavior. For PROMIS,
|
|
|
we specified targets in terms of gender (50% female), age (20% in each of 5 age groups: 18-29, 30-44, 45-59, 60-74, 75+), race/ethnicity (10% black and Hispanic), and education (10% less than high school graduate).
Panelists have been recruited by a variety of methods, such as e-random digit dialing, invitations via web newsletters, and Internet poll-based recruitment where panelists have opted to participate in a survey advertised on the World Wide Web. The topics of these surveys vary greatly – from politics to polls on popular entertainment. All of the PollingPoint panelists have provided their e-mail so that they may receive survey invitations to participate in future surveys. Panel members receive minimal compensation.
This methodology was developed by Stanford Political Science Professor Douglas Rivers and was recently evaluated using the 2005 California Special Election. The matched samples performed as well or better than conventional random-digit-dial (RDD) telephone surveys, correctly predicting the outcomes for all of the propositions and having an average absolute error of 3 percent (consistent with normal sampling error).
In general, YouGovPolimetrix has reported a base response rate of 67% for general population surveys (similar to the general population component of the PROMIS Calibration Studies data collection) and rates at or exceeding 80% with more specialized, clinical samples such as the ones in PROMIS. They also utilize sample matching methodologies for selection of representative samples from non-randomly selected pools of respondents, which is ideally suited for Web access panels.
|
|
The PROMIS Calibration Studies sample included 21,133 respondents, with 1,532 recruited from primary research sites associated with PROMIS network sites and the vast majority (19,601) from YouGovPolimetrix’s panel sample. YouGovPolimetrix sample data was collected using their website on a secure server. The PROMIS network site data was collection using the PROMIS Assessment System.
The Statistical Coordinating Center based at CORE received de-identified datasets from YouGovPolimetrix. Each of the 7,005 individuals (6,676 from YouGovPolimetrix, 236 from UNC, and 93 from Stanford) assigned to administration of full item banks was administered a pair of the 14 item banks. The 7-item blocks for each of the 14 banks were administered to 14,128 individuals (6,245 general population, 7,883 clinical samples). The clinical samples included persons with heart disease (n = 1,156), cancer (n = 1,754), rheumatoid arthritis (n = 557), osteoarthritis (n = 918), psychiatric illness (n = 1,193), chronic obstructive pulmonary disease (n = 1,214), spinal cord injury (n = 531), and other conditions (n = 560).
The clinical population supplied by YouGovPolimetrix was identified through a pre-survey of 250,000 YouGovPolimetrix panel members. These respondents were asked to complete the PROMIS clinical form. Persons who self-reported currently being diagnosed with a given condition were eligible for inclusion in the Calibration Studies clinical sample associated with that condition. It should be noted that the general population sample included people with and without diseases (general population respondents were also administered the clinical form but their responses did not impact their participation in the general population sample).
Overall Sample
The overall sample (n = 21,133) was 52% female. The median age was approximately 50 years with 12% 18-29, 12% 30-39, 16% 40-49, 32% 50-64, and 28% 65 and greater years old. Eighty-two percent were white, 9% Black, 8% multi-racial, and 1% other (Asian/Pacific Islanders and Native Americans). Nine percent of the sample was Hispanic or Latino. Educational attainment ranged from less than high school (3%) to advanced degree (19%) with 24% with a college degree, 39% some college, and 16% a high school diploma. The overall sample was used primarily for calibrating item parameters.
Scale Setting Sub-Sample
The distribution of gender, age, race, and education in the 2000 census was: 52% female; 22% 18-29, 32% 30-44, 24% 45-59, 14% 60-74 and 8% 75 and greater years old; 74% White, 11% Black, 11% Hispanic/Latino, and 4% Other; and 51% more than high school. A subset of the PROMIS general population sample (includes respondents administered full bank and item blocks) was created with the purpose of representing the marginal distributions of race/ethnicity (White, Black/Hispanic/Other) and education (High School or less versus more than high school) in the census. The distribution of characteristics for the PROMIS scale setting sub-sample (n = 5,239) was: 57% female; 15% 18-29, 22% 30-44, 28% 45-59, 22% 60-74 and 13% 75 and greater years old; 74% White, 10% Black, 11% Hispanic/Latino, and 4% Other; 51% had more than a high school education. The scale setting sub-sample was primarily used to establish U.S. population norms.
Additional Chronic Pain Sample
In order to increase the number of participants experiencing pain in the calibration sample, individuals with chronic pain were recruited through the American Chronic Pain Association (ACPA). An invitation to complete the PROMIS pain survey was posted on the ACPA website. To be eligible, participants had to be 21 years of age or older and have at least one chronic pain condition for at least 3 months prior to participating in the survey. Those who met eligibility criteria were asked to complete an online informed consent form. After obtaining informed consent, participants immediately began the survey. The survey was posted on the website of the ACPA from September 2007 to March 2008.
There were 967 participants who responded to 47 Pain Interference, 42 pain behavior, and 41 pain quality items, and one global average pain intensity item through online administration. The average age was 48.2 years (SD = 11.1). Eighty-one percent of the respondents were female, and 91% were Caucasian, 1.5% were African-American, and 5% were of Hispanic origin. Eighty-one percent of the participants had an education equal to or greater than high school. The data was combined with Calibration Studies full-bank test data to calculate item calibrations.
Sleep and Wake Disturbance Sample
The sample for the Sleep and Wake Disturbance banks included administration of 128 Sleep Wake Functioning items to 1,993 individuals from YouGovPolimetrix (1,259 from general population, and 734 with self-identified sleep problem) and 259 individuals with sleep disorders from Pittsburgh clinical sites. The overall sample (n = 2, 252) was 43.8% female. The median age was 52 years old with 20.7% 65 and older and 79.3% less than 65 years old. Eighty-two percent were White, 12.6% Black, 2.7% Native American or Alaskan, 0.4% Native Hawaiian or Pacific Islander, and 5.9% other. Ten percent of the sample was Hispanic or Latino. Educational attainment ranged from high school or less (13.6%), some college (38.6%), college degree (27.9%) to advanced degree (19.9%). Item response data from the overall sample (2, 252 individuals) were used for item calibration.
|
|
|
|
|
|
|
|
|
|
|
|
 |
|
|
|
|