Table 1 Overview of questions that can be answered by SemanticLens

Search

‘Has my model learned to encode a specific concept?’ through convenient ‘search-engine’-like text or image descriptions

Fig. 2a and Supplementary Figs. C.1 to C.3

‘Which components have encoded a concept, how is it used and which data are related?’

Fig. 2d

Describe

‘What concepts has my model learned?’ in a structured, condensed and understandable manner through textual descriptions

Fig. 2b and Supplementary Figs. D.1 to D.5

‘What concepts are contributing to a decision and how?’ by visualizing concept interactions throughout the model

Fig. 2c and Supplementary Fig. D.7

‘What do I not yet understand about my model?’, by offering an understanding of unexpected concepts and their role for the model and origin in data

Fig. 2d and Supplementary Figs. F.4 to F.11

Compare

‘What concepts are shared between two models, and which are unique to each one?’ by comparing learned concepts qualitatively and quantitatively

Supplementary Figs. E.1 and E.2

‘How do my concepts in my model change when the architecture or training changes?’ by comparing and tracking semantics of components

Supplementary Figs. E.1 and D.4

Audit

‘Does my model rely on valid information only?’ by separating learned concepts into valid, spurious and unexpected knowledge

Figs. 3 and 4 and Supplementary Fig. F.1

Evaluate

‘How interpretable is my model?’ with easy-to-compute measures

Fig. 5b

‘How can I improve the interpretability of my model?’ by evaluating interpretability measures when the model architecture or training procedure change

Fig. 5c and Supplementary Tables G.1 to G.5