Table 7 External validation of modeled estimates using the Penn. Policy Database Project: 2007-2016.

No.	Major Topic	Machine Learning Estimates			Legacy Keyword Model
No.	Major Topic	Precision	Recall	F1	Precision	Recall	F1
2	Civil Rights	0.42	0.27	0.33	0.58	0.10	0.17
3	Health	0.67	0.77	0.72	0.73	0.45	0.55
4	Agriculture	0.44	0.52	0.47	0.59	0.41	0.48
5	Labor	0.24	0.50	0.32	0.18	0.23	0.20
6	Education	0.60	0.70	0.65	0.53	0.34	0.42
7	Environment	0.61	0.38	0.47	0.53	0.15	0.24
8	Energy	0.47	0.41	0.43	0.58	0.39	0.47
10	Transportation	0.43	0.37	0.40	0.41	0.20	0.27
12	Legal	0.56	0.62	0.59	0.22	0.02	0.04
13	Welfare	0.21	0.33	0.26	0.70	0.10	0.18
14	Construction	0.15	0.26	0.19	0.29	0.18	0.22
16	Military	0.42	0.88	0.57	0.75	0.71	0.73
17	Communications	0.10	0.71	0.17	0.35	0.11	0.17
21	Public Lands	0.64	0.42	0.51	0.41	0.07	0.13
24	Local Government	0.32	0.68	0.43	0.24	0.40	0.30
	Macro-average	0.42	0.54	0.44	0.47	0.26	0.30
	Micro-average	0.46	0.57	0.50	0.43	0.25	0.29

Notes: Macro-average is computed with each policy area equally weighted, and micro-average is computed with policy areas weighted by their respective number of bills coded.

Quick links

Search