Table 2 General recommendations for users and developers of kinetic inference tools^a

(i)

As a general consistency test, the inferred kinetic model (connectivity and rate constants) can be simulated and the output of the simulation compared to the original input data. For example, the simulator used herein is publicly available as a simple and powerful (MATLAB) tool to test whether the proposed model can generate data analogous to the original input, e.g. regarding FRET histogram, smFRET traces, etc.

(ii)

Potential biases in the analysis (e.g. regarding model size, state occupation, etc.) can be revealed by subjecting the re-simulated data (with known ground truth) to the same analysis approach as the experimental data.

(iii)

Where possible, kinetic models with a specific number and connectivity of states are preferred over mean residence times, since the latter leave the individual transition rate constants undetermined for more than 2 states.

(iv)

Uncertainty measures are necessary indicators of significance, and a unified standard would greatly improve their comparability. The 95% confidence interval was the most frequently used uncertainty measure in this study, and we encourage its use as a common standard for the future.

(v)

Benchmarking new analysis tools using datasets of varied complexity – including models with more than 2 states – can reveal systematic errors, e.g. regarding the weighting of multiple rate constants that depopulate a given state, an issue encountered in this study.

(vi)

Benchmarking new software with established test data helps the potential users to judge the added benefits of newly introduced analysis tools. The diverse datasets used herein are publicly available and can serve to assess a tool’s performance under varied experimental conditions.

(vii)

Supporting broadly accepted file formats for newly developed analysis tools facilitates fast dissemination in the field. We offer the simple format described in Supplementary Note 1, which proved to be very useful for this study.