From abb9d24c9f74de68c50cb3de0803ac3799fc835c Mon Sep 17 00:00:00 2001 From: Nicholas Tomlin Date: Mon, 12 Aug 2019 09:42:09 -0700 Subject: [PATCH 1/3] Update lm_ptb_train.sh --- runs/examples/lm/lm_ptb_train.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/runs/examples/lm/lm_ptb_train.sh b/runs/examples/lm/lm_ptb_train.sh index 1f74de4..3e4b526 100644 --- a/runs/examples/lm/lm_ptb_train.sh +++ b/runs/examples/lm/lm_ptb_train.sh @@ -1,5 +1,5 @@ python $SEQ2SEQROOT/main.py \ - --run runs/example/ \ + --run runs/examples/ \ --model LSTMLanguageModel \ --train data/lm/train \ --dev data/lm/valid \ From 64975b6212d703466bede10634b488c0f5766f90 Mon Sep 17 00:00:00 2001 From: Nicholas Tomlin Date: Mon, 12 Aug 2019 09:43:04 -0700 Subject: [PATCH 2/3] Update README.md --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 5ea0212..5c6e445 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ The framework is still largely a work in progress. I'd like to add more examples To train an LSTM language model on the Mikolov data with the specified hyperparameters in `lm.py`, first specify the framework directory and run the example script with: export SEQ2SEQROOT=/directory/of/repository - bash runs/example/lm_ptb_train.py + bash runs/examples/lm_ptb_train.py The hyperparmeters specify a very large language model (512 input dim, 1024 hidden dim, 3 layer LSTM)! I trained using a Titan X (12GB), and tuned the memory accordingly. With it, training was possible at 383s per epoch, and after epoch 31 the validation shows: @@ -21,9 +21,9 @@ The hyperparmeters specify a very large language model (512 input dim, 1024 hidd Done. Validation loss: 338222.015259. Average Token loss: 4.585439. Perplexity: 98.046266. Monitored quantity improved. -The validation results and checkpointed models should be saved in `runs/example/` (specified in the script). Test the checkpointed model with: +The validation results and checkpointed models should be saved in `runs/examples/` (specified in the script). Test the checkpointed model with: - bash runs/example/lm_ptb_test.sh + bash runs/examples/lm_ptb_test.sh I was able to achieve these results on the test set: @@ -42,7 +42,7 @@ In this example, we will use the silver parse trees in `data/parse/` generated f To train a seq2seq parser with the specified hyperparameters in `seq2seq.py`, run: - bash runs/example/seq2seq_ptb_train.sh + bash runs/examples/seq2seq_ptb_train.sh The hyperparmeters specify a very large seq2seq model (512 input dim, 512 hidden dim, 2 layer encoder/decoder)! I trained using a Titan X (12GB), and tuned the memory accordingly. With it, training was possible at 349s per epoch, and after epoch 45 the validation shows: From 24766c318ba1540fbcec89e93a371ae9a2323a3f Mon Sep 17 00:00:00 2001 From: nickatomlin Date: Mon, 12 Aug 2019 09:48:38 -0700 Subject: [PATCH 3/3] fixed more filepaths --- runs/examples/lm/lm_ptb_test.sh | 2 +- runs/examples/seq2seq/seq2seq_ptb_test.sh | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/runs/examples/lm/lm_ptb_test.sh b/runs/examples/lm/lm_ptb_test.sh index 90b3cd5..bc2b278 100644 --- a/runs/examples/lm/lm_ptb_test.sh +++ b/runs/examples/lm/lm_ptb_test.sh @@ -1,5 +1,5 @@ python $SEQ2SEQROOT/test.py \ - --run runs/example/ \ + --run runs/examples/ \ --model LSTMLanguageModel \ --populate lm.model \ --test data/lm/test \ diff --git a/runs/examples/seq2seq/seq2seq_ptb_test.sh b/runs/examples/seq2seq/seq2seq_ptb_test.sh index 90b3cd5..bc2b278 100644 --- a/runs/examples/seq2seq/seq2seq_ptb_test.sh +++ b/runs/examples/seq2seq/seq2seq_ptb_test.sh @@ -1,5 +1,5 @@ python $SEQ2SEQROOT/test.py \ - --run runs/example/ \ + --run runs/examples/ \ --model LSTMLanguageModel \ --populate lm.model \ --test data/lm/test \