You’ll never eat lunch in this town again [1].

High-quality clinical practice guidelines and expert consensus recommendations are important tools in clinical decision making. Several hematopoietic cell transplant organizations and scientific bodies have published such documents [2,3,4] (https://bsbmtct.org/indications-table/). Consequently, readers of the Journal may have been shocked to read the recent article in JAMA Internal Medicine by Kim et al. criticizing the quality of transplant guidelines from the American Society for Transplantation and Cellular Therapy (ASCTC) [5]. Mon dieu; is nothing sacred?

In reality, no one should be surprised by a critical review. In 2018, we published an article in the Journal: Is there expert consensus on consensus? in which we discussed the development of expert consensus guidelines. Such guidelines are based on respectable judgments theory that argues that views of a group have greater validity and reliability than the judgment of an individual and that systematic and explicit structured methods for developing judgments are more valuable than informal accuracy and communicability methods. We also discussed diverse opinions on the value of consensus ranging from: Consensus means that lots of people say collectively what nobody believes individually (Abba Eban), Historically, the claim of consensus has been the first refuge of scoundrels; it is a way to avoid debate by claiming that the matter is already settled (Dr. Michael Crichton) and Consensus is the quickest road to mediocrity (Mayer Rus). We also discussed several definitions of experts: … somebody from out of town with slides; A man who tells you a simple thing in a confused way in such a fashion as to make you feel the confusion is your fault (William Castle) and …a man who has made all the mistakes which can be made in a very narrow field (Niels Bohr).

In a well-known, often replicated wine tasting experiment, a panel of experts and novices was asked to sample and comment on an oak barrel aged chardonnay. Descriptors used by experts included hints of pineapple, papaya, mango vanilla, spice and butter. Novice descriptors were more prosaic: not bad, a bit acidic, I like it etc. The panel then reviewed a wine identified as a cabernet sauvignon but which was the same chardonnay with tasteless red dye No. 2 added. Expert descriptors switched to: leather, tobacco, dark fruit flavors, plums, apricots and black pepper. In contrast, several novices, but not expert, commented the cabernet tasted very much like the chardonnay. There are several lessons from this experiment: (1) the danger experts may miss the forest from the trees. We recently discussed the role of heuristics in this process elsewhere in the Journal (Being certain even when you’re wrong: heuristics and thin slicing in haematopoietic cell transplantation) [6]; (2) the influence of prior knowledge, even errors or deceptions on descriptors (Bayesian wine tasting?) and (3) the importance of panelist selection on internal and external validity.

Which brings us back to the recent critical review of the ASCTC guidelines by Kim et al. The authors reviewed randomized controlled trials (RCTs) supporting 70 ASCTC indications for allotransplants and 41 for autotransplants covering 15 cancers, predominately leukaemias and lymphomas. Recommendations were categorized by the ASCTC authors as S (standard-of-care), defined as: where [the] indication is well-defined and supported by evidence or C (clinical evidence available, standard-of-care) defined as: standard-of-care, clinical evidence available, where large clinical trials and observational studies are unavailable but have been shown to be effective therapy.

The descriptors used in the ASCTC and other guidelines such as those of the European Group for Blood and Marrow Transplantation (EBMT), the British Society of Blood and Marrow Transplantation confuse and/or mis-use terms such as indications, guidelines, recommendations and standard-of-care. These terms are not synonymous. In medicine an indication is a valid reason for a procedure, different from a guideline or recommendation, a guide to decision making. A bigger concern is use of the term standard-of-care. A standard-of-care is not an indication, guideline or recommendation but a duty determined by a given set of circumstances that present in a particular patient, with a specific condition, at a definite time and place. In other words, a standard-of-care is sensitive to time, place and person [7]. However, for example, the ASCTC guidelines state: the guidelines should not be used to determine whether HCT should be pursued as a treatment for an individual patient. Moreover, there may be several standards-of-care in someone. For example, in a 50-year-old with acute myeloid leukaemia in first complete remission with a FLT3 mutation, a transplant and post-remission chemotherapy with or without midostaurin are both standards-of-care. In a similar 80-year-old, supportive care is also a standard-of-care.

There are several other problems with several transplant indications, guidelines and recommendations that limit internal and external validity such as: (1) not using well-established criteria for panelist selection or panel size; (2) no systematic data review; (3) no formal ranking of quality-of-evidence; (4) no description of methods to adjudicate discordances; (5) no description of degree of discordance between panelists and others.

Back to the Kim et al. analyses of the ASCTC indications and guidelines. These authors found only 4 RCTs for allotransplants and 24 RCTs for autotransplants corresponding to 14 of the S and 7 of the C recommendations. Overall, only 4 of 70 (6%) standard-of-care recommendations for allotransplants and only 17 of 41 (41%) standard-of-care recommendations for autotransplants were supported by data from RCTs. Moreover, median number of subjects in the allotransplant RCTs for S recommendations was <100 and for autotransplants, about 75. The authors concluded: Taken together our results demonstrate that there has been widespread adoption of HCT, especially allogeneic transplant, based on low levels of evidence. (Bolding ours.)

The fundamental limitation to the ASCTC and most transplant indications, guidelines and recommendations is lack of high-quality data from RCTs. We admit data from such trials have limitations that we discuss elsewhere in the Journal and in the current statistics series edited with Prof. Mei-Jie Zhang [8] (https://www.nature.com/collections/ejhigdbeeh). Regardless, results of RCTs are our best quality-of-evidence for or against an intervention.

There are many reasons why there are few transplant-related RCTs. These trials face challenges of recruitment, equipoise and ethics. Detecting an incremental benefit of receiving a transplant typically needs many subjects putting adequately powered trials beyond the reach of single centers and necessitating multi-center and sometimes international collaboration. Physician- and potential recipient-related motivations explain in part reluctance to participate in RCTs. Even when there is approval by Ethics Committees and Institutional Review Boards, transplant-orientated physicians experience considerable anxiety and ethical concerns recruiting subjects to the control arm of RCTs even when there are no convincing data of safety and/or efficacy of the investigational intervention. We recently discussed the relationship of this attitude to what social scientists term cult-like behavior in the Journal [9]. Moreover, potential trial subjects with no therapy preference perceive that chance (randomization) does not relieve their uncertainty choosing between transplant and an alternative intervention. Those with a therapy preference may decline enrollment to avoid being randomized to an alternative therapy. Because of these considerations, the quality of direct evidence supporting a transplant becomes a less important consideration in decision making and physicians use other data such as indirect evidence, anticipated desired and undesired effects of the therapy, cost and feasibility of the therapy option.

These considerations also operate in developing indications, guidelines and recommendations. Lacking high-quality data from RCTs experts believe a recommendation for or against a transplant should be based on factors other than direct evidence. As experts, they think they have an implicit and comprehensive mastery of scientific and practical data and that by using this they will reach what they consider the most appropriate decision. By comparing individual arguments in what they perceive to be an equitable way experts think consensus can be reached and that this consensus is the best decision possible.

However, what is the evidence this reasoning is correct and acquiring direct evidence from RCTs would not modify the direction of the decision? As we discuss many transplant-related guidelines, recommendations and indications result from biased reasoning and interventions considered a standard-of-care do not stand the test of evidence from RCTs. We need look no further than the fiasco of high-dose therapy and autotransplants for advanced breast cancer to underscore this criticism (reviewed in [10]).

It is often easy to criticize but what are possible solutions? These include better trial design and endpoints to make RCTs more attractive [11], better decision models [12,13,14,15,16] intelligent use of high-quality observational databases such as those of the CIBMTR and EBMT, high-quality outcomes research, scenario planning and the increasing use of real world data to derive real world evidence (https://www.smestrategy.net/blog/what-is-senario-planning-and-how-to-use-it) [17]. To reverse the crisis of confidence in the ethics of RCTs, Freedman coined the concept of clinical equipoise, where the requirement is satisfied if there is genuine uncertainty within the expert medical community about the preferred treatment [18]. Our position is that there is clinical equipoise for many transplant-related decisions in hematologic cancers and that RCTs remain essential for proving safety and efficacy of transplants compared with therapy alternatives.

Defining a clear, concise research question is essential in RCT planning. Hematologic cancers are increasingly molecularly defined resulting in smaller and smaller subsets of potential subjects that increases the difficulty in doing RCTs. Furthermore, implementation of RCTs against the headwinds of enthusiasm for new therapies of molecularly characterized diseases requires an innovative trial infrastructure. Several approaches are proposed as alternative designs to increase numbers of RCTs by diminishing ethical concerns and the influence of subject preference including partially randomized subject preference trials (RPPT). The aim of this design is to assign potential subjects with a preference to that therapy whilst randomizing subjects with no preference [19]. As people become more knowledgeable about their disease and about therapy options, RPPTs seem a reliable alternative to RCTs resulting in quicker enrollment of a more representative population and improved external validity without compromising internal validity.

In her book, You’ll never eat lunch in this town again, author Julia Phillips described what she termed the boys club amongst Hollywood producers. (1) The book, perhaps like our Editorial, was considered scandalous, vindictive, career ending, a suicide note. However, as David Thompson writing in The Independent noted: No one ever claimed Phillips had got Hollywood wrong in her book in which case, you have to give a little more credence to the theory that Hollywood is prepared to let the club be run by raving egotists, indictable rascals, desperate addicts of one thing or several others, betrayers, connivers, and hypocrites.. So long as they are guys. Or transplant experts like us?