Test div

Clickable Dropdown

Click on the buttons to select which experiments will be included in the table (each experiment is one row); you can filter by task, encoder type, decoder type, attention type, learning rate, and hidden size.

Any task Question formation Tense reinflection
Any encoder SRN GRU LSTM Tree GRU ON-LSTM
Any decoder SRN GRU LSTM Tree GRU ON-LSTM
Any attention None Location-based Content-based
Any 0.00005 0.0001 0.0005 0.001 0.005 0.01 0.05
Any 32 64 128 256 512 1024 2048

Select which columns to include in the table

First, aspects of the experiment:

Name Dataset Encoder Decoder Attention Learning rate Hidden size

Second, full-sentence metrics of accuracy: Test full and Gen full are full-sentence accuracy on the test and generalization set, while POS indicates accuracy on getting the correct sequence of parts-of-speech

Test full Test POS Gen full Gen POS

Second, first-word metrics of accuracy: Gen main is the percent of generalization set outputs that started with the main auxiliary; Gen first is the percent that started with the first auxiliary; Gen other aux is the percent that started with some other auxiliary; and Gen other is the percent that started with something that wasn't an auxiliary.

Gen main Gen first Gen other aux Gen other

Generalization set first-word accuracy where the relative clause on the subject was of a particular type

Object relative clause Transitive subject relative clause Intransitive subject relative clause

Percent of outputs that belonged to a particular form along the lines of Crain and Nakayama

Delete first, prepose first Delete first, prepose second Delete first, prepose other Delete second, prepose first Delete second, prepose second Delete second, prepose other Delete none, prepose first Delete none, prepose second Delete none, prepose other Other

Example outputs given by the trained model, whether on the test set or generalization set; and the command that can be used to re-run this experiment.

Test examples Gen examples Command