Table 7 The extraction effects of all methods.

CRF³⁹

0.5245

0.3494

0.4194

0.6039

0.5476

0.3746

0.4449

0.9214

NaïveBayes+CRF³⁹

0.4837

0.3649

0.4160

0.6243

0.5143

0.4715

0.4920

0.9263

BiLSTM+CRF⁴⁵

0.5681

0.4375

0.4943

0.6379

0.7407

0.7598

0.7501

0.9858

IDCNN+CRF⁴⁴

0.5827

0.4653

0.5174

0.6611

0.7442

0.7740

0.7588

0.9866

LSTM+BiLSTM+CRF⁴⁴

0.6146

0.5277

0.5678

0.6754

0.7620

0.7607

0.7614

0.9869

Base(Word2vec+...+POS)

0.4937

0.3781

0.4282

0.6145

0.8118

0.8177

0.8147

0.9490

Base+BiLSTM+ANN+CRF¹

0.8500

0.7078

0.7724

0.8970

0.8246

0.8193

0.8219

0.9511

Base+BERT⁵

0.7873

0.6715

0.7248

0.8371

0.8511

0.8348

0.8429

0.9433

Base+BERT+HAS⁵

0.8176

0.6523

0.7251

0.8851

0.8616

0.8454

0.8534

0.9543

Base+BERT+SSA⁵

0.8194

0.6647

0.7340

0.8860

0.8632

0.8463

0.8547

0.9548

ERBTF⁴⁶

0.8215

0.6831

0.7459

0.8637

0.8355

0.8214

0.8284

0.8947

KnowCTI⁴⁷

0.8346

0.6946

0.7582

0.8618

0.8492

0.8265

0.8377

0.8969

ITIRel⁴⁸

0.8254

0.6786

0.7448

0.8475

0.8347

0.8143

0.8244

0.8604

Base+FI+BiLSTM+ANN+CRF(ours)

0.8146

0.6810

0.7418

0.8765

0.8273

0.8158

0.8215

0.9683

Base+BERT+FI+BiLSTM+ANN+CRF (ours)

0.8547

0.7034

0.7717

0.8992

0.8687

0.8433

0.8558

0.9875

Prebuilt BERT+FI+BiLSTM+ANN+CRF (ours)

0.8370

0.6877

0.7550

0.8846

0.8567

0.8471

0.8519

0.9896

CRF³⁹

0.5170

0.2701

0.3520

0.7848

0.3216

0.2975

0.2661

0.4671

NaïveBayes+CRF³⁹

0.459

0.363

0.403

0.7435

0.3478

0.343

0.3454

0.4956

BiLSTM+CRF⁴⁵

0.4018

0.4664

0.4317

0.8374

0.3311

0.3171

0.3239

0.4814

IDCNN+CRF⁴⁴

0.5000

0.4646

0.4817

0.854

0.4567

0.4255

0.4405

0.6043

LSTM+BiLSTM+CRF⁴⁴

0.5674

0.5479

0.5575

0.8753

0.4663

0.4558

0.4611

0.6764

Base (Word2vec+...+POS)

0.4651

0.5118

0.4843

0.8455

0.4179

0.403

0.4103

0.6422

Base+BiLSTM+ANN+CRF¹

0.5433

0.5279

0.5355

0.823

0.5297

0.5135

0.5165

0.7268

Base+BERT⁵

0.5952

0.5965

0.5947

0.8417

0.543

0.5346

0.5388

0.739

Base+BERT+HAS⁵

0.5892

0.6201

0.6043

0.8799

0.5633

0.5472

0.5551

0.7538

Base+BERT+SSA⁵

0.5804

0.6306

0.6044

0.8807

0.5675

0.5491

0.5581

0.7604

ERBTF⁴⁶

0.5834

0.6513

0.6155

0.8532

0.6381

0.6486

0.6433

0.8172

KnowCTI⁴⁷

0.5796

0.6686

0.6209

0.8644

0.6523

0.6657

0.6589

0.8015

ITIRel⁴⁸

0.5793

0.6671

0.6201

0.8675

0.6586

0.6561

0.6573

0.8126

Base+FI+BiLSTM+ANN+CRF(ours)

0.5647

0.5385

0.5513

0.8317

0.6314

0.6247

0.6280

0.8003

Base+BERT+FI+BiLSTM+ANN+CRF (ours)

0.5973

0.6424

0.6190

0.8863

0.6681

0.6765

0.6723

0.8122

Pre_built BERT+FI+BiLSTM+ANN+CRF (ours)

0.596

0.6412

0.6178

0.8834

0.6643

0.6748

0.6695

0.8113