Introduction

Large-scale studies that pair genetic samples with participants’ medical data have become an important staple of biomedical research. As understanding of gene–disease relationships grows, genotypic data collected by researchers may hold increasing clinical or personal value for research participants. Decreasing costs of exome and genome sequencing and the growth of population- and clinic-based biobanks mean that some clinically relevant genomic variants are likely to be identified in large numbers of research participants.1

Studies have shown that genetic research participants are interested in receiving, and may expect some, individual research results (IRRs) in return for participating.2,3,4,5,6,7,8,9,10,11 A public survey about a possible large-cohort study found that 90% of participants wanted to learn their IRRs regardless of the utility of the findings; 75% said they would be less likely to participate in a study that did not return IRRs.8 Discussion about IRRs has shifted from the question of whether IRRs should be returned to deliberations regarding which results should be returned, when, and in what manner. Evidence of this shift can be found in emerging guidelines that suggest that researchers and clinicians may have an ethical obligation to return at least some results.12,13,14 Although these guidelines specify some criteria about which IRRs to return, considerable discretion is left to researchers.12,14 For example, the 2010 National Heart, Lung, and Blood Institute guidelines state that individual genetic results should be offered to study participants if the finding has important well-established health implications, the finding is actionable, the test is analytically valid, and the study participant has opted to receive the information during an informed consent process.14 Under this National Heart, Lung, and Blood Institute guideline, one study found that researchers using whole-exome sequencing could unearth between 3,955 and 12,579 findings per participant that could qualify for disclosure.15 Another study simulated the return of incidental findings by first classifying 2,016 genes linked to Mendelian disorders into “bins” based on their clinical utility and validity. Using their binning algorithm, the researchers estimated that a person undergoing whole-exome or whole-genome sequencing could expect zero to two results with clear utility, which the authors felt should be returned without eliciting patient preference, and up to 10 additional results of unclear utility, for which the decision to report should involve the patient.16 Similarly, Dorschner et al.17 screened 1,000 adults for 114 variants associated with medically actionable conditions and estimated that 1.2% of patients of African descent and 3.4% of patients of European descent would carry a pathogenic or likely pathogenic variant. Initial clinical recommendations by the American College of Medical Genetics and Genomics (ACMG) advised that clinical laboratories performing whole-exome or whole-genome sequencing report findings from 57 genes related to 30 serious actionable conditions, regardless of patient age or preference, and estimated that 1% of reports would include one of these variants.18 It is not clear whether any of these recommendations could or should be used to decide what researchers should divulge; current financial, regulatory, staff, and logistic burdens would make returning 4,000 findings to a single participant impossible. These examples illustrate that there are multiple, widely varying resources that a researcher might use, appropriately or inappropriately, to prioritize what to return.19,20 Considerable inputs will be required to help researchers make these decisions, while balancing policies, participant preferences, the financial and logistic burdens of returning IRRs, and the integrity of their research.

This study contributes to the ongoing discussion about returning IRRs by measuring the policy preferences of a critical stakeholder in genetic research—participants. This research used a conjoint experiment to assess public preferences for eight principles related to the return of research results in a hypothetical large-cohort study and to examine whether the priorities for a results policy differ between respondents who would or would not be willing to participate in the study. Additional questions measured general opinions about the study and the return of research results.

Materials and Methods

To inform the design of the survey, 10 focus groups conducted in three US cities explored the types of research results that people valued, the importance of clinical validity and utility, and preferences for communicating IRRs. The focus group study, reported on elsewhere,10 collected opinions on several current issues in the debate related to IRRs, including the following: (i) whether to return IRRs,21,22,23,24,25 (ii) the evidence required to support return of a given result,12,14,26 (iii) the criteria defining which results to return,12,14 (iv) the financial and clinical support needed to provide results,10,14 and (v) whether returning IRRs would influence participant enrollment and retention.8 The focus group study found several attributes of IRRs that might influence research participants’ interest in the information. These included whether the result is associated with a common disease, a serious disease, or a condition for which treatment is available; whether the result is associated with a large change in risk; and whether the risk estimate is well validated or might change over time. The availability of detailed result reports and study staff to explain them were also seen as important. Many expected that results would be provided at no cost to research participants. These were not the only factors that appeared to influence whether someone would want an IRR, but they were among the most important factors observed.

Building on this work, an online survey was developed. The conjoint experiment that is the focus of this study was part of a larger survey instrument designed to measure public attitudes about the design of a national cohort study proposed by the National Human Genome Research Institute27 that would follow 500,000 Americans over time to study genetic and environmental causes of disease.28 The broader survey focused on several aspects of biobank participation, including study design preferences, privacy concerns, broad consent, and the return of IRRs.

The survey was conducted online in English and Spanish, according to respondent preference. Sample selection and survey administration was managed by Knowledge Networks (KN, which has since become the KnowledgePanel, run by the company GfK).29 During the field study period of 29 April–12 May 2011, potential respondents aged 18 years and older were randomly selected from KN’s Web-enabled master panel of ~35,000 US residents. For households without Internet access, KN provides a laptop computer and an Internet connection to complete the surveys. Black non-Hispanics and Hispanics were oversampled. Hispanics were sampled from KN’s KnowledgePanel Latino panel, which comprises a representative sample of US adults who identify with Hispanic culture and values. The Hispanic sample was stratified to include equal numbers of individuals who chose to take the survey in English or in Spanish.

Survey respondents viewed English or Spanish versions of a 3-minute video describing the proposed study. Respondents who could not view the video saw a written script of the video, accompanied by a diagram of study components. Hyperlinks to the study description and definitions were available throughout the survey (Supplementary Table S1 online). Respondents were then asked about their support for, and willingness to participate in, the study as well as their interest in receiving IRRs. The survey was qualified by the Johns Hopkins University Institutional Review Board as exempt (application NA_00040539).

Because of the length of the survey, half of the respondents were randomly selected to participate in the conjoint experiment discussed here; the other half participated in a conjoint experiment measuring preferences for overall study design. No significant demographic differences were observed between the two groups.

Conjoint experiment

Respondents completed a conjoint experiment that used 12 tasks to measure the relative importance of eight principles ( Table 1 ) of a policy to return IRRs in the hypothetical cohort study. The hallmark of conjoint analysis is that it forces participants to make a series of trade-offs, in this case between different principles of a policy to return IRRs. This technique provides important information about which principles of a results policy will determine whether it is viewed favorably and quantifies how each attribute is valued by the respondents by forcing them to reveal their preferences through a structured series of choice experiments. The eight IRR policy principles, which were presented to participants as they appear in Table 1 , were selected based on our previous survey and focus group research and by reviewing the relevant scientific literature, policy statements, and guidelines addressing the return of IRRs.8,10,12,14,21,22,23

Table 1 Eight principles of a policy to return individual results
Full size table

Respondents were told, “Imagine study planners have decided to offer to return some individual research results to interested participants. Over the course of a 10-year study like this, there may be hundreds or thousands of results for each participant. Therefore study planners will need to figure out which individual results to return and how to return them.” Then, in a series of 12 choice tasks (exemplified in Figure 1 ), the policy principles were divided into two mutually exclusive groups, and respondents were asked to choose their preferred policy.

Figure 1
figure 1

Example of a task from among 12 tasks that each participant completed in the conjoint exercise on a policy to guide the return of individual research results.

Full size image

To identify which of the eight factors were most valued and whether preferences differed between those who would and would not participate in the proposed study, we constructed paired-comparison choice tasks using an orthogonal array. A total of 256 profiles combining subsets of the eight principles that could guide return of IRRs can be derived. Policy profiles for the conjoint exercises were created using a 28 array from the SAS catalog30; the principles were separated into two subsets—policy/plan A and policy/plan B.31,32,33 Consistent with good research practices,34 the experimental design characteristics were verified, and the resulting experiment was perfectly orthogonal. In each of the 12 tasks, respondents were asked to choose the policy (A or B) they would prefer if they were a participant in the study. No further explanation or justification was required, and the tasks were forced choices.

An initial pilot study (9–11 February 2011; n = 68) evaluated survey length, logic, skip patterns, and wording. Five cognitive interviews evaluated respondent comprehension of the conjoint experiment. A second pilot (18–21 March 2011; n = 72) looked for an order effect because the conjoint exercise could precede or follow a battery of questions that might influence response. No ordering effect was observed.

Statistical analysis

Multivariate regression used McFadden’s conditional logistic regressions.35 The dependent variable was considered the respondents’ choice of plan, and the independent variables were eight dichotomous variables representing whether each of the principles was included in the plan or not. The conditional logistic regression does not allow for respondent characteristics, such as age or willingness to participate, to be entered into the regression because each respondent acts as his/her own control in the regression, which measures differences in individual responses to experimentally controlled variations in the IRR policies.

Results focus on the value that respondents placed on each factor as represented by the parameter estimate from the conditional logistic regression. Overall differences between respondents who would and would not participate in the cohort study were estimated through stratified regression, and differences in values of each principle were tested using t-tests and the Wald test.

Results

A 57% survey response rate yielded 1,523 respondents who participated in the conjoint exercise, and the 1,515 respondents (99.4%) providing complete data on the conjoint exercise served as the basis of the analyses below. Both weighted and unweighted demographic distributions of the complete sample and those completing the conjoint analysis were comparable to those of the US 2010 Census (Supplementary Table S1 online). Two in three respondents were able to watch the introductory video; the remainder read the study description. No differences were noted between those who watched the video and read the description with respect to the findings below.

Immediately after viewing or reading the study description, 56% said that they would definitely (16%) or probably (40%) participate in the study, and 44% said that they would probably not (30%) or definitely not (14%) participate (see Supplementary Table S1 online). Furthermore, 78% would be interested in receiving their IRRs if they participated in the study, 9% would not be interested, and 13% were not sure. Given a choice, 57% said they would want all of their IRRS, 32% would choose to receive only the results for “diseases that can be prevented or treated,” and 12% “would not want or need any individual research results.” Slightly more than half (56%) agreed “it would be fair if I only received the results of findings for diseases that can be prevented or treated”; 62% of those likely to take part in the study agreed with this statement.

Determinants of preferences for a return-of-results policy

Summarizing the views of all respondents who completed the conjoint analysis, all eight principles of a potential policy to return IRRs significantly and positively influenced preference for the policy ( Figure 2 ). The most attractive attribute tested was “providing results at no cost to research participants.” The remaining seven attributes can be divided into two categories: types of results to return, and provisions for returning them. After returning results at no cost, the next most attractive principles reflected the types of results that should be returned—those related to treatable and serious diseases, and results in which the genotype–phenotype association has been confirmed. Although respondents viewed the return of IRRs about common diseases and major changes in risk positively, these characteristics were significantly less important. Respondents valued a detailed result report more than having study staff available to discuss results.

Figure 2
figure 2

Overall results of conjoint analysis to determine the significance and relative importance of eight policy attributes related to the return of individual research results (IRR) ( N = 1,515). CI, confidence interval; OR, odds ratio.

Full size image

We also examined the conjoint analysis results separately for respondents who would or would not be willing to take part in the hypothetical study ( Figure 3 ). Examining the priorities of people most likely to take part in a large biobank could identify attributes of a results policy that would promote their participation. Similarly, identifying preferences of those not willing to participate could reveal principles that might underlie, in part, their decision.

Figure 3
figure 3

Comparison of the significance and relative importance of eight policy attributes related to the return of individual research results (IRR) between those likely ( n = 861) and unlikely ( n = 654) to take part in a hypothetical large genomic research study ( N = 1,523). CI, confidence interval.

Full size image

Overall, the relative ranking of the principles was similar in those willing (n = 861) and unwilling (n = 654) to participate ( Figure 3 ). In both groups, providing results at no cost was the most attractive attribute (odds ratio (OR) of finding the policy acceptable = 1.82 in those willing to participate; OR = 1.89 in those unwilling; P = 0.25). Returning results for treatable diseases (OR = 1.69 and 1.59, respectively; P = 0.09) was the second most important attribute tested. In both groups, respondents placed a significantly higher value on receiving reports with detailed interpretations than having study staff available to discuss results.

The magnitude of preferences for three principles differed between those who were willing and those unwilling to participate. Respondents who would be unwilling to participate placed a higher value on results related to common diseases (unwilling respondents OR = 1.15, willing respondents OR = 1.06; P = 0.02). By contrast, respondents who would take part in the cohort study placed a significantly higher value on policies that share both results for serious diseases (willing OR = 1.69 vs. unwilling OR = 1.58; P = 0.03) and results showing major changes in risk (ORs = 1.17, 0.97 respectively; P < 0.001) compared with those who were not willing to participate ( Figure 3 ). Moreover, people unlikely to participate in the study viewed the return of results showing large changes slightly negatively (OR = 0.97), although they did not see returning these results as a significant aspect of policy (P = 0.35).

Considering that receiving results at no cost was the most attractive attribute among all respondents, we examined whether this preference was influenced by respondents’ ability to pay and then split the sample by the median income of the study group. Receiving results at no cost ranked high in both income groups, but this was the most influential attribute in the lower-income group and the third most important factor in the higher-income group. The difference in the magnitude of importance in the two groups was not significant (P = 0.17).

Economic effects that returning results would have on the proposed study

Two additional survey questions explored attitudes about the effect that the additional costs of returning results could have on the proposed study. Despite widespread interest in results, nearly 4 in 10 respondents (38%) agreed with the statement “returning IRRs would make the study too expensive.” A second question asked directly about one potential trade-off of providing results. Respondents were told, “It might cost the study a great deal of money to return IRRs. To be able to afford returning IRRs, the study would have to enroll fewer people. A smaller study would take longer to make scientific discoveries. In thinking about this trade-off, which study design would you prefer?” Putting the cost in terms of slowing scientific discovery, 24% would prefer a study that returns no individual results but that takes 10 years to make its findings, and 60% would prefer a 12-year study returning “a small number of IRRs about diseases that people could prevent.” A 20-year study that returns all individual results was preferred by 16% of respondents.

Limitations

This research is based on respondents’ reactions to a large, proposed biobank funded by the National Institutes of Health. Responses about what individuals prefer in this hypothetical study may not correspond perfectly to actual future preferences or preferences in other studies.10 Second, although all eight policy principles tested were regarded as significant positive aspects of a policy on the return of results, indicating that influential meaningful principles were selected for study, there are clearly other principles of an IRR policy that were not measured but that could be important in a results disclosure policy. These include policies on privacy protection, data sharing, data access, and inclusion of results into the medical record. Finally, the attribute describing large changes in risk did not distinguish between large increases and large decreases in disease risk. Thus, we cannot discern whether respondents’ preferences were motivated by a desire for information about increased risk or reduced risks.

Discussion

This is the first study to use conjoint analysis to elicit public preferences for returning IRRs. By forcing the participants to make trade-offs on their preferences for various attributes of a results policy, the data accurately prioritized public preferences regarding the principles of a results policy. In addition, this methodology enabled respondents to participate in an exercise that simulated some of the real-world policy challenges facing the research community. Consistent with several other studies, these data show high willingness to participate in large-cohort genetic research, as well as a strong interest in receiving IRRs.8,10,11,36

Our data provide insight about which individual results are most valued by the public. Overall, all eight principles of a return-of-results policy tested were significant positive influences in the selection of policies to disclose research results. In research, it appears that individual results that are returned should be delivered at no cost to participants if possible. The issue of cost was particularly important among participants in the middle- and lower-income groups; issues of justice or fairness might arise in studies that ask individuals with lower incomes to bear some costs. The overall importance of receiving results for free may reflect a common public opinion that receiving some IRRs is a form of compensation for study participation.7,8 Nevertheless, when presented with a trade-off that researchers might make to accommodate the costs of returning results (lengthening the time to scientific discovery), a large majority preferred to minimize the adverse effects on research—84% would prefer to receive a small number of actionable results (60%) or no results at all (24%) in order to avoid significantly slowing research progress. The public may be largely willing to accept few or no results if the impact of the costs is explained.

When prioritizing which results to return, these data show that the public places the greatest emphasis on three categories of results: results for serious health conditions, results for diseases that can be treated or prevented, and well-validated results whose relationship to disease has been clearly established. These features were significantly more important than returning results for common diseases or those showing major changes in disease risk. For example, 62% of those who would participate in the proposed study felt it would be fair if they “only received the results of findings for diseases that can be prevented or treated.” Although participants might like to learn about a wide range of results given the opportunity, it seems many people feel that receiving a set of actionable valid results would be fair and acceptable. Returning even a limited number of actionable results at no cost to participants could satisfy some of the public’s most important priorities.

Some recommendations for the return of IRRs12,13,14 have emphasized returning findings that confer high risks of disease. Learning about high magnitudes of risk was a positive attribute of an IRR policy, but it was significantly less important than learning about valid, actionable results. This echoes the observation in our focus groups that participants would be equally interested in variants conferring low, high, or even average risks of disease. Many people may be interested in finding out where they stand in relation to others, as well as in finding out which diseases they are at high risk for developing.

The public priorities expressed here are fairly consistent with emerging recommendations that a few, actionable results be returned at no charge to participants.12,13,14 Although there has been much interest regarding the potential tension between participants, who might want to receive all of their IRRs, and the research community, who want to limit their workload in this area, it appears that the views of the public and those of the research community are not far apart. Concordance of researcher and public opinions about what results take priority is encouraging. However, the two groups may diverge somewhat when defining which results are actionable or have utility. The biomedical community’s definition of clinical utility may be relatively narrow compared with the public’s definition because many results with no proven clinical use may hold strong personal utility to a research participant.7,10,11,36,37,38,39 Because actionable results are important to people, the potential difference in how researchers and participants interpret the term “actionable”7,38 should be considered when IRR policies are drafted, to avoid misunderstandings.

Respondents who said they would not take part in the study were significantly less likely to support policies to return results on serious diseases and slightly preferred policies that would not return results showing major changes in disease risks. These observations suggest that there may be an element of fear or apprehension about the types of information that could be delivered, although we do not have definitive information to support this hypothesis. The concentration of such attitudes among people not likely to take part in genetic research raises questions about whether this apprehension is related to mistrust of the research enterprise, or whether fear of bad news in the form of IRRs might discourage participation.

Not everyone wants to receive all or any IRRs. Some people will be more interested than others in a given result. This empirical finding supports other suggestions that studies returning an IRR must allow participants to choose whether or not to receive it.12,13,14 This finding also supports the ACMG’s recent revised guideline on clinical incidental findings, which now states that patients undergoing clinical sequencing should be allowed to opt out of any of the ACMG’s prescribed set of incidental findings.40 It also appears that, in the public’s opinion, it is significantly less important for a research study to provide staff to explain results than it is to provide a detailed result report. It may be that as more information becomes available online, participants will seek their own sources to interpret their IRRs. Whether research participants can get satisfactory, accurate, and comprehensible information and what they go on to do with it are still issues that warrant consideration.

Although this study focuses on IRRs in a proposed research study, the findings may also be relevant to the clinical community as they consider delivery of incidental findings. The routine practice for the delivery of such findings has not yet been established. For example, the ACMG recommendation18 that patients undergoing genomic sequencing receive some incidental findings may become the standard, but the details of how best to accomplish this are far from settled. Our findings on some of the priorities of the public for receiving research results might be one of many places of departure for those examining how to address incidental findings in the clinic. For example, our observation that it is significantly more important to people to receive a detailed result report than to have access to staff to explain results suggests that it may be worth examining the use of different report formats to lessen burdens on clinical staff working with genomic test data. Additional research in clinical settings would help to compare and hone the effectiveness of such reports.

This study has identified eight aspects of a potential IRR policy that are of significant importance to the general public. Developing clear policies for returning IRRs that address each of these eight aspects may provide transparency to research participants, foster positive researcher–participant relationships, provide clear guidance to researchers about their obligations for returning results, and aid study planners in their efforts to successfully recruit and retain participants. It may be as important to explain a study’s IRR policy with respect to these eight aspects as it is to create the public’s “ideal policy.” Nonetheless, modest return policies that can deliver even a small number of well-understood clinically actionable results may be viewed very positively by most participants.

Disclosure

The authors declare no conflict of interest.