Fig. 7: Comparison of the base tokenizer and the quantum-native tokenizer on QASM input. | npj Quantum Information

Fig. 7: Comparison of the base tokenizer and the quantum-native tokenizer on QASM input.

From: Symbolic analysis of Grover search algorithm via Chain-of-Thought reasoning and quantum-native tokenization

Fig. 7: Comparison of the base tokenizer and the quantum-native tokenizer on QASM input.The alternative text for this image may have been generated using AI.

Each grey and non-grey segment represents a distinct token. The base tokenizer fragments the syntax into subword units, while the quantum-native tokenizer preserves gate operations and qubit references as cohesive tokens, resulting in more compact and efficient representations.

Back to article page