Table 1 Log-likelihood comparisons across data sets.

	S. Bar	Inv. S. Bar	Bars and stripes	MNIST
CD-1	−20.42	−20.73	−61.08	−152.42
PCD-1	−21.71	−21.64	−57.01	−140.43
PT	−20.57	−20.57	−51.99	−142.00
MT	−19.85	−19.86	−50.79 (−41.82)	−136.42
Exact	−19.77	−19.77	−41.59	–

We report the highest achieved log-likelihoods over 50,000 gradient updates on a 9 × 4 restricted Boltzmann machine (RBM) across various RBM types (standard, enhanced-RBM, centered-RBM) and training techniques (contrastive divergence (CD), persistent-CD (PCD), parallel tempering (PT)) as reported in previous work¹⁶ compared with mode-assisted training (MT) on a standard RBM. In the table, rows correspond to different training techniques and columns are different data sets. For each technique, the best achieved log-likelihood score across 25 runs is reported. In parenthesis are results for a 9 × 9 RBM. For these small datasets we can also compare with the exact result. For the MNIST dataset, the trained networks trained had 16 hidden nodes and PCD-1 was used as the gradient update, and average log-likelihood is reported.
The highest log-likelihood achieved on a given data set is shown in bold.

Quick links

Search