Table 1 The characteristics comparison table of three methods for evaluating the reliability of expert ratings.

From: Research on rating behaviors of bidding evaluation experts in construction projects based on novel integrated methods about reliability and decision entropy

Number

Method Name

Summary of Characteristics

1

Discreteness-based method

The degree of dispersion refers to the measurement index of the difference in understanding between an expert and other experts when evaluating the project. This method used the dispersion is to use the existing data to measure the difference or consistency between an expert’s score and other experts.

2

Method based on reliability coefficient

Reliability is an estimate of the degree of consistency of measurement, while expert reliability is the consistency of multiple experts’ scoring data for a project. The greater the reliability coefficient, the consistency between the evaluation experts is the higher. This method used reliability coefficient to use the existing data to measure the difference or consistency between an expert’s score and other experts.

3

Method based on decision entropy

The decision entropy method first uses the power method in numerical algebra to obtain the score closest to the real score of the project, and then calculates the decision entropy of the individual expert. It reflects the objectivity and accuracy of decision entropy.

Back to article page