Extended Data Fig. 1: Organization of the Benchpress data set.

The entry point is the benchpress directory, and SDK specific flags and options are set in the default.conf file. Optionally, tests taking longer than a specified timeout can be automatically included in the skipfile.txt. Abstract test definitions are included in the workouts directory, and test inputs in the form of OpenQASM files or Hamiltonians are included in the qasm and hamiltonian folders, respectively. Tests specific to each SDK are located in the corresponding “*_gym” directories. Inside each “gym”, tests are organized in groups based on the target functionality to be tested.