Xiao et al. introduce ‘capability density’, defined as capability per parameter, as a metric for evaluating large language models. They report an empirical trend, the ‘densing law’, which states that capability density doubles approximately every 3.5 months, indicating that equivalent model performance can be achieved with exponentially fewer parameters over time.
- Chaojun Xiao
- Jie Cai
- Maosong Sun