Table 7 Comparison of model predictions on ambiguous passages from the WikiQA Dataset.
Question | Passage snippet | Ambiguity type | Ground truth answer | RoBERTa prediction | XLNet prediction | DistilBERT-MRC prediction |
|---|---|---|---|---|---|---|
When did Lincoln die? | Lincoln was shot on April 14 and died the next morning. | Temporal | April 15 | April 14 | April 14 | April 15 |
Who founded Amazon? | Amazon was created by Jeff Bezos. He left his job at D. E. Shaw to pursue the idea. | Referential | Jeff Bezos | D. E. Shaw | D. E. Shaw | Jeff Bezos |
What is the capital of Mercury? | Mercury is a planet, not a country. | Lexical | No answer | Mercury | Mercury | No answer |
Who is the president of Apple? | Apple’s CEO Tim Cook introduced the iPhone. The president of Apple Europe was also present. | Referential | Tim Cook | Apple Europe | Apple Europe | Tim Cook |
When did WW2 start? | WW2 started in 1939, although some tensions had been rising since 1938. | Structural | 1939 | 1938 | 1938 | 1939 |
