Introduction

Changes in interventions, either new or modifications of those already in use, can provide better outcomes for both patients and clinicians. However, a key element in ensuring the quality of dental care lies in the clinician's ability to evaluate the robustness of clinical, scientific and public health-related research publications, to assess the validity of claims made by industry and apply all this information in their practice to improve their care of patients.

Historically, new developments in practice have arisen through research published in established journals that manage some form of peer-review process. Having accessed the information, either through reading the relevant journal or through word of mouth, clinicians would subsequently decide whether the intervention offered the potential to benefit the patients they cared for. Irrespective of the route through which they learned about developments, competence in critical appraisal is central, incorporating: the abilities to access information on interventions; to assess their benefits as well as any associated risks and problems issues, for example, related to the conditions under which they should be applied; and most importantly, to translate the findings into relevant outcomes in practice.

There has been an exponential growth in the number of research articles published since the turn of this century, with 22,115 papers on the subject of ‘dentistry' published in 2021 alone.1 However, Bassini et al.2 have reported that poor-quality research is a major challenge in oral health research. Furthermore, many different types of bias may limit the validity of biomedical research.1

Faced by the multiplicity of potential information sources on health, disease and its management now available, and a multitude of routes for dissemination, the ability and confidence of care providers to ensure the validity of research findings and product data, and assess the implications for treatment and oral health, are of growing importance. The problem is not limited just to healthcare. Løvlie et al.3 highlighted the need for training in journalism to help ensure readers understood evidence and uncertainty.

Accelerated by advances in technology, research (formerly the domain of universities) is now conducted through a myriad of arrangements, including ‘in-house' studies. Additionally, the mechanisms through which research is disseminated and claims are communicated have changed. For example, social media now plays an important role as a route through which clinicians, and, equally importantly, the public, gather information. “The Advertising Standards Agency has also recognised this by developing advice for the oral health sector which included the guidance that ‘marketers are required to hold robust supporting evidence before making objective claims'.”4 Even by 2004, eight in ten internet users have looked for health information online, of which over 50% concerned medical treatments or procedures.5

While adherence to good scientific practices is improving overall, there is still a need ‘to change current transparency practices in dental research'.6 The processes through which product claims are derived often remain opaque. Together, these changes all highlight the need for skills in assessment and critical appraisal, and underline that this is an essential skill in ensuring that healthcare is evidence-informed.

Critical appraisal seeks to provide answers to two key issues that clinicians face when presented with information: firstly, how valid is the information received; and secondly, how relevant are the findings, and consequently the implications for the care of their patients. It is crucial, therefore, that clinicians are conversant with advances in potential interventions ‘and critically assess its relevance to treatment planning, advice and treatment provision'7which the General Dental Council expand in detail:

‘It is understood that the “degree of critical thinking, and level and use of research will vary across the registrant categories in relation to their scope of practice and responsibilities”. Nevertheless, for all registration categories, the registrant is expected to be able to:

  • Describe the principles of good research, how to access research, and interpret it for use as part of an evidence-based approach to practice

  • Apply an evidence-based approach to learning, practice, clinical judgement and decision-making and use critical thinking and problem-solving skills'.

Currently, the training and development of all care providers, from undergraduate through to the Intercollegiate Specialty Fellowship Examination (ISFE) for all dental specialties, has included critical appraisal as part of the assessment. This is changing.

To date, the content and processes used to assess whether an individual has reached the required standard in the domain have relied, in the vast majority, on using papers published in refereed journals. Such an approach fails the profession on several counts: it does not address the increased range of sources of information to which patients and clinicians have access, and perhaps most fundamentally, it requires a candidate to have knowledge over and above that of the referees used in the initial assessment before publication. Additionally, such an approach assumes that the journal's peer reviewers will have identified any flaws in the research report submitted.

This paper challenges both the current process of teaching critical appraisal and the lack of emphasis given on the subject matter in continuing professional development. It argues that the approach needs to take into account the increased range of sources and the widely varying nature of information used to influence both patients and clinicians. The paper outlines underlying principles on which critical appraisal training should be based and suggests an approach that those responsible for curricula could consider. To do this, the paper is divided into three further sections. The first section reinforces the importance of critical appraisal, the changing nature of sources of information and why current thinking needs to change. The second section suggests the objectives of any educational programme, and the third, a possible curriculum. Throughout this proposal, commonality has been adopted where possible as, while recognising that each dental specialism and, indeed, general practice has its own content, the underlying principles of the importance of critical appraisal remains.

Underlying principles for training in critical appraisal

Any proposed development of the critical appraisal component of the training programmes is underpinned by two aspects: first, the extent to which elements of the curriculum may have been covered in previous educational programmes and, second, how any individual would use the competence being assessed as part of their working practices.

With respect to the first element, the principle applied is to build on existing experiences and training, which would include those obtained at Bachelor of Dental Surgery (BDS) (or its equivalent) and possible Master of Science programmes. All dental graduates have been exposed to various elements associated with evidence-informed healthcare, as necessitated by the General Dental Council. Three examples listed state that an individual registrant should be able to:

  • Explain, evaluate and apply the principles of an evidence-based approach to learning, clinical and professional practice and decision-making

  • Critically appraise approaches to dental research and integrate with patient care

  • Recognise the scientific principles underpinning the use of materials and biomaterials and evaluate their limitations and selection, with emphasis on those used in dentistry.

It is helpful to view critical appraisal as a range of competencies.

In a competence-based system, each element has two components: the field and the level (Fig. 1). As an individual expands their knowledge and skills through their career, development can occur in two ways: gaining additional competence in an existing field and growing a new field.8 Given the evolving nature of sources of information, in order to maintain competence, an individual may require additional skills. As part of the ongoing requirement for continuing professional development, using a competence-based framework, the fields covering critical appraisal could be defined, and as the area evolves, the developments that have occurred build on the previous competencies. This may be through an additional ‘field' or simply a higher level of competence of an existing one. The proposed curriculum development approach adopted in this paper is that the curriculum content of training, for example, that assessed at ISFE, builds on the competencies identified within BDS when appropriate but also identifies those new ‘fields' which are necessary to ensure care of the highest qualities.

Fig. 1
figure 1

A competence-based curriculum. Reproduced with permission from P. Batchelor et al., ‘Issues concerning the development of a competency-based assessment system for dentistry', British Dental Journal, vol 185, pp 141-144, 1998, Springer Nature8

The second aspect concerns encouragement for the learner to recognise the value of critical appraisal to their careers. To help support this aspect, Meyers and Nulty9 identified what they termed a set of ‘curriculum design principles', namely:

‘To maximise the quality of student learning outcomes we, as academics, need to develop courses in ways that provide students with teaching and learning materials, tasks and experiences which:

  • Are authentic, real-world and relevant

  • Are constructive, sequential and interlinked

  • Require students to use and engage with progressively higher order cognitive processes

  • Are all aligned with each other and the desired learning outcomes, and

  • Provide challenge, interest and motivation to learn'.6

They went on to suggest that ‘the effect of applying these principles is to manipulate the learning system in ways that require students to adopt a deep learning approach in order to meet the course's assessment requirements - which, in turn, meets the desired course learning outcomes'.

This ambition is in line with the goal of all dental student examinations and we recommend that any critical appraisal course should adhere to these principles.

Objectives of a training programme

The next step in designing the assessment process is to provide clarity on the objectives of the training programmes. The research literature has highlighted a number of crucial elements that impact on the teaching of critical appraisal skills in healthcare settings.

In a systematic review of the effectiveness of critical appraisal skills training for clinicians, Taylor et al.10 noted that while their review provided some evidence of the benefits of teaching critical appraisal in terms of methodological knowledge and statistical issues, the quality of the studies was poor. Subsequently, in a systematic review of the topic, Horsely et al.11 highlighted the paucity of data underpinning the subject at that time. Indeed, their main finding highlighted that none of the reported studies evaluated the process of care or patient outcomes.

Recently, Wyer12 suggested that ‘clinical educators need to be reassured that the goals of training clinical learners in literature skills are very different from those of equipping epidemiologists and scientific reviewers to do publishable appraisals of the literature'.

This is important as the purpose is to assess the extent to which care providers have the skills to help patients make informed choices about their care. Guyatt et al.13noted that the vast majority of clinicians have little interest in the nuances of methodological appraisal of research literature. What they need, however, is a working familiarity with critical appraisal principles and the ways that flaws in research design may invalidate study findings.

If the above argument is accepted, the principles underlying the purpose of critical appraisal in the learning process are twofold: to assess the extent to which existing knowledge and skills have developed from those before entry to undergraduate education and, second, to equip the learner to apply evidence in clinical practice to help ensure the qualities of care that the clinician provides. It follows that any curriculum would have three main objectives:

  1. 1.

    Recognise claims about the effects of treatments which have an unreliable basis

  2. 2.

    Understand whether comparisons of treatments are fair and reliable

  3. 3.

    Make informed choices about treatments.

These objectives were proposed by Chalmers et al.14 In their work undertaken through the Informed Health Choices project, the authors went on to use these concepts to provide a framework to help people make informed health choices. In a more detailed paper, the authors outlined the key areas for each area. For example, under the first - recognising an unreliable basis for a claim - they suggested a set of concepts (Box 1). These could form the basis for identifying a more detailed set of exercises to underpin learning and, if such an approach is acceptable, the development of the assessment tool(s) to validate a candidate's competence.

Assessing critical appraisal competence

A number of authors have commented on the assessment arrangements for critical appraisal. In a summary of the work in the field, MacRae et al.15 described the literature as ‘pessimistic', noting a major deficiency arising from the use of poorly studied or inappropriate outcome measures in prior studies. The authors went on describe an assessment arrangement that required the reading of an article, completion of a rating scale assessing methodological quality and some free-text responses in a number of areas.

Diviani et al.,16 in a systematic review addressing how to critically appraise health information, argued that the field is covered by six core competences: 1) basic competence; 2) predisposition; 3) identification competence; 4) critical evaluation competence; 5) selection competence; and 6) application competence.

If the proposals concerning the objectives are acceptable, then the concepts that Chalmers et al. have identified can be converted into an assessment framework based upon the work of Diviani et al. Such an approach would meet the goals of the education arrangements initially proposed and not least, help ensure the quality of care.

Recent developments in published literature have seen the case being made for a ‘publications facts label', offering guidance as to why the research is trustworthy. Willensky and Pimental,17 highlighting the growth in open access, argued that with such developments come responsibilities for journal publishing platforms to address the arrival of an ‘age of misinformation'. However, as mentioned previously, journals are simply one among many other sources of information that the public and profession now use to access information.

Summary

This paper set out to prompt a discussion on how to develop the critical appraisal component for care providers. Currently, the evidence underpinning the present approach is very limited.18 The current emphasis in teaching critical appraisal has highlighted a limited range of higher order thinking and a focus, to date, on definitions of the processes involved in critical appraisal as opposed to its practical application for patient care.

Two broad principles to underpin the development of the arrangements are identified, namely building on existing competencies gained through prior learning experiences and ensuring that the skills have real world value to an individual's future working practices. Using these principles, the objectives of the critical appraisal component are threefold, namely: to be able to recognise claims about the effects of treatments which have an unreliable basis; to understand whether comparisons of treatments are fair and reliable; and subsequently, to help the patient make informed choices about treatments.

If this is accepted, a framework for designing the assessment tools and associated processes can follow. Indeed, given the nature of the work, there would be potential to develop a common examination standard across all dental disciplines to help ensure all dental care professionals can best support patients.