Table 1 Model effectiveness and model efficiency without and with ExCS.
Model | SR@1 | SR@5 | SR@10 | MRR | Search time |
|---|---|---|---|---|---|
BM25+CodeEx | 0.4056 | 0.5386 | 0.6220 | 0.4523 | 50 ms/query |
DeepCS | 0.4125 | 0.5214 | 0.6173 | 0.4072 | 1100 ms/query |
ExCS with DeepCS | 0.4712 (\(\uparrow\)14.2%) | 0.6061 (\(\uparrow\)15.2%) | 0.7037 (\(\uparrow\)14.1%) | 0.4685 (\(\uparrow\)15.0%) | 110 ms/query (\(\downarrow\)90.0%) |
CARLCS-CNN | 0.4331 | 0.5241 | 0.6335 | 0.4271 | 400 ms/query |
ExCS with CARLCS-CNN | 0.4830 (\(\uparrow\)11.5%) | 0.5922 (\(\uparrow\)13.3%) | 0.7095 (\(\uparrow\)12.1%) | 0.4713 (\(\uparrow\)10.3%) | 80 ms/query (\(\downarrow\)80.0%) |
TabCS | 0.6134 | 0.7145 | 0.8056 | 0.5946 | 400 ms/query |
ExCS with TabCS | 0.6168 (\(\uparrow\)0.5%) | 0.7179 (\(\uparrow\)0.5%) | 0.8103 (\(\uparrow\)0.6%) | 0.5986 (\(\uparrow\)0.7%) | 60 ms/query (\(\downarrow\)85.0%) |
SANCS | 0.6628 | 0.7509 | 0.8369 | 0.6435 | 500 ms/query |
ExCS with SANCS | 0.6537 (\(\downarrow\)1.1%) | 0.7341 (\(\downarrow\)2.2%) | 0.8280 (\(\downarrow\)1.2%) | 0.6367 (\(\downarrow\)1.1%) | 70 ms/query (\(\downarrow\)86.0%) |
