Table 2 Summary information for all of the included papers

From: Large language model integrations in cancer decision-making: a systematic review and meta-analysis

First author

Year of publication

Country

Targeted condition(s)

Evaluator type

Data type

Evaluation type(s)

Aspect of decision-making

Prompt engineering?

IRB approval?

Rahsepar et al.17

2023

USA

Lung cancer

Radiological scientist

Mixed

3

Patient-centered information

No

Not specified

Yeo et al.53

2023

Korea

Cirrhosis and hepatocellular carcinoma (HCC)

Board-certified/eligible transplant hepatologist reviewers

Mixed

1, 5, 7

Patient-centered information

No

Not specified

Fink et al.54

2023

Germany

Lung cancer

Unspecified (existing dataset)

N/A

N/A

Clinician-centered information

No

Not specified

Rao et al.55

2023

USA

Breast cancer

Unspecified expert reviewers

Quantitative

2

Patient-centered information

Yes

Not specified

Coskun et al.56

2023

Turkey

Prostate cancer

Urologists

Mixed

1

Patient-centered information

No

Not specified

Hermann et al.57

2023

USA

Cervical cancer

Attending gynecologic oncologists

Quantitative

3

Clinician-centered information

No

Not specified

Horiuchi et al.58

2024

Japan

Tumors in neuroradiology

Unspecified (existing dataset)

N/A

N/A

Clinician-centered information

No

Exempt

Nguyen et al.59

2023

USA

Four cancers: breast, ovarian, colorectal, and lung

Unspecified medical students

N/A

N/A

Clinician-centered information

Yes

Not specified

Lukac et al.18

2023

Germany

Breast cancer

Unspecified (existing dataset)

N/A

N/A

Clinician-centered information

No

Yes

Gebrael et al.60

2023

USA

Metastatic prostate cancer

Unspecified (existing dataset)

N/A

N/A

Clinician-centered information

No

Yes

Chung et al.21

2023

USA

Prostate cancer

Radiation oncologists

Quantitative

1, 3, 7, 8, 9

Clinician-centered information

No

Exempt

Kuşcu et al.34

2023

Turkey

Head and neck cancer

Experienced head and neck surgeons

Qualitative

1

Patient-centered information

No

Exempt

Janopaul-Naylor et al.61

2024

USA

Unspecified cancer

Oncologists with 5 to 10 years of clinical experience

Quantitative

1

Patient-centered information

No

Exempt

Haemmerli et al.36

2023

Switzer-land

Brain glioma

Neuro-oncologists, radio-oncologists, radiologists, neurosurgeons, neuropathologists and neurologists

N/A

N/A

Clinician-centered information

No

Yes

Choi et al.16

2024

Korea

Kidney cancer

Urological oncologists specializing in kidney cancer

Mixed

1, 4

Patient-centered information

No

Yes

Wei et al.62

2024

USA

Head and neck cancer

Unspecified reviewers

Mixed

1, 6, 7

Patient-centered information

No

Yes

Choo et al.63

2024

Korea

Colorectal cancer

Unspecified (existing dataset)

Quantitative

4

Clinician-centered information

No

Yes

Campbell et al.39

2024

USA

Thyroid cancer

Unspecified graders

Quantitative

3

Patient-centered information

Yes

Exempt

Griewing et al.64

2023

Germany

Breast cancer

Unspecified (existing dataset)

Quantitative

1

Clinician-centered information

No

Not specified

Alanezi et al.65

2024

Saudi Arabia

Unspecified cancer

Unspecified (existing dataset)

N/A

N/A

Patient-centered information

No

Yes

Lyu et al.22

2023

USA

Lung cancer

Experienced radiologists

N/A

N/A

Clinician-centered information

Yes

Not specified

Benary et al.66

2023

Germany

Cancers with genetic alterations (lung cancer and others)

Unspecified (existing dataset)

N/A

N/A

Clinician-centered information

No

Exempt

Braithwaite et al.67

2024

USA

Breast cancer

Experts in the fields of general internal medicine, family medicine, geriatric medicine, population health, cancer control, and radiology

Mixed

2, 6

Patient-centered information

No

Not specified

Peng et al.19

2024

China

Colorectal cancer

Unspecified evaluators and practitioners

Quantitative

1, 7

Patient-centered information

No

Yes

Köroğlu et al.68

2023

Turkey

Thyroid cancer

Endocrinologists

Mixed

1, 13, 14

Clinician-centered information

No

Exempt

Ozgor et al.69

2024

Turkey

Urological cancers: prostate, bladder, kidney, and testicular cancers

Experienced physicians in uro-oncology

Qualitative

14

Patient-centered information

No

Not specified

Davis et al.40

2024

USA

Oropharyngeal cancer

Unspecified physician-graders

Mixed

1, 7

Patient-centered information

No

Exempt

Son et al.70

2023

Korea

Breast, prostate, and lung cancer

Unspecified (existing dataset)

N/A

N/A

Clinician-centered information

No

Yes

Lee et al.71

2023

USA

Head and neck cancer

Experienced head and neck surgeons

N/A

N/A

Patient-centered information

No

Exempt

Arasteh et al.72

2024

Germany

Esophageal cancer

Unspecified (existing dataset)

N/A

N/A

Clinician-centered information

Yes

Yes

Pinto et al.73

2024

Brazil

Prostate cancer

Urologists

Quantitative

1

Clinician-centered information

Yes

Exempt

Valentini et al.37

2024

Austria

Sarcoma, a rare malignant tumor

Sarcoma experts (orthopedic oncologists and medical oncologists)

Mixed

1, 2, 7

Patient-centered information

No

Exempt

Thia et al.41

2024

Australia

Urological malignancies

Urology specialists

N/A

N/A

Patient-centered information

No

Exempt

Atarere et al.74

2024

USA

Colorectal cancer

Board-certified Internal Medicine physicians (internists) and an oncologic Gastroenterologist with significant colorectal cancer experience

Qualitative

2, 4

Patient-centered information

No

Not specified

Lim et al.75

2024

Singapore

Colorectal cancer

Unspecified (existing dataset)

Quantitative

1, 2

Clinician-centered information

Yes

Not specified

Choi et al.76

2023

Korea

Breast cancer

Breast cancer patients

N/A

N/A

Clinician-centered information

Yes

Yes

Wu et al.77

2024

China

Thyroid cancer

Unspecified reader

N/A

N/A

Clinician-centered information

No

Yes

Pereyra et al.78

2024

Argentina

Colorectal cancer

Board-certified gastroenterologists with more than 10 years of clinical experience

Quantitative

1

Clinician-centered information

No

Yes

Iannantuono et al.12

2024

USA

Unspecified cancer

Unspecified expert reviewers

Quantitative

1, 4, 5, 6, 15

Clinician-centered information

No

Not specified

Chow et al.79

2023

Canada

Unspecified cancers in radiotherapy

Unspecified participants

N/A

N/A

Patient-centered information

Yes

Exempt

Yalamanchili et al.20

2024

USA

Unspecified cancer

Radiation oncologists and physicists

N/A

N/A

Patient-centered information

Yes

Exempt

Kozel et al.80

2024

USA

Brain tumors

Neurosurgeons

Quantitative

1

Clinician-centered information

No

Not required

Wang et al.35

2024

China

Thyroid cancer

Doctors

N/A

N/A

Clinician-centered information

No

Yes

Yang et al.81

2024

China

Bone tumors

Experienced physicians (intermediate and senior doctors)

N/A

N/A

Clinician-centered information

No

Exempt

Hasani et al.82

2023

USA

Unspecified cancers in radiology

Urology fellows

N/A

N/A

Clinician-centered information

Yes

Yes

Zhou et al.38

2023

China

Gastric cancer

Unspecified graders

Quantitative

2, 11

Clinician-centered information

Yes

Not specified

Cocci et al.83

2023

Denmark

Urological related cancers

Urologist

N/A

N/A

Clinician-centered information

No

Not specified

Yucel et al.84

2024

Turkey

Unspecified cancer

Medical oncologists

Mixed

1, 6, 11

Patient-centered information

Yes

Not specified

Huo et al.85

2024

Canada

Colorectal cancer

Patients

Quantitative

1

Patient-centered information

Yes

Not specified

Sarangi et al.86

2023

India

Unspecified radiological cancers

Radiologists

Mixed

1, 3, 7, 10

Clinician-centered information

Yes

Not specified

Lent et al.42

2024

Denmark

Dermato-oncology

Dermatologists

Quantitative

1, 7, 12, and 14

Clinician-centered information

Yes

Not specified

Floyd et al.87

2024

USA

Unspecified cancers

Radiation oncology resident physicians

Quantitative

1, 7

Patient-centered information

No

Not specified

Gabriel et al.88

2023

UK

Cancer treatments

Urologists

Quantitative

1, 2, 15

Clinician-centered information

No

Not specified

Sievert et al.89

2024

Germany

Oropharyngeal squamous cell carcinoma

Surgeons and pathologists graders

N/A

N/A

Clinician-centered information

Yes

Yes

Hao et al.43

2024

USA

Unspecified cancer

Older adult cancer patient participants and cancer-related clinician participants

N/A

N/A

Patient-centered information

No

Yes

Holmes et al.13

2023

USA

Unspecified cancer

Medical physics residents and research fellows, as well as non-physicians with expertise in radiation physics

Quantitative

1

Clinician-centered information

Yes

Not specified

  1. Included papers and associated information, including publication year, first author’s country, target condition(s) of the paper, participant/grader roles, number of clinical professionals and non-expert participants, the decision-making aspect of the research, and whether the study involves prompt engineering. Fifteen types of evaluation metrics were identified in the reviewed literature; each was assigned a category number as follows: (1) general accuracy, (2) appropriateness, (3) correctness, (4) reliability, (5) reproducibility, (6) readability, (7) comprehensiveness, (8) ease of understanding, (9) harm, (10) simplicity, (11) stability, (12) currency of information, (13) safety, (14) usability, and (15) relevancy. Summary statistics and visualizations for the evaluation metrics are presented in Fig. 4. Detailed definitions of each metric and the corresponding scales referenced in the papers included in this systematic review are presented in Supplementary Table 3.
  2. Represents the papers included in the meta-analysis.