Target Propagation for Recurrent Neural Networks (TPRNN)
This is a self-contained software accompanying the paper titled: Training Language Models using Target-Propagation. The code allows you to reproduce our results on two language modeling datasets, Penntree Bank (character and word) and wikitext, using various training methods.
The code implements the following training algorithms for RNNs:
- Standard BPTT training
- Penalty Method (PM)
- Alternating Direction Method of Multipliers (ADMM)
- Augmented Lagrangian Method (ALM)
It also allows you to play around with various hyper-parameters, including the recurrent model architecture, learning rates and others.