3.1 The Network

For reasons outlined above (see section 2.1), neural network models which are used to process structured natural language input need a mechanism to store information about the context in which each input element appears. Unsurprisingly, then, the network architecture used for CLASPnet is that of an Elman-type recurrent neural network. (Note 1) Figure 3.1 shows the network. The input layer consists of two parts, one for the orthographic representations, and one for the semantic representations; both parts feed into a smaller hidden layer with a recurrent layer; the information from these two hidden layers is then combined in a single large hidden layer with a recurrent layer; the large hidden layer in turn feeds the output layer. All layers with links are also fully interconnected -- e.g. all the units on the semantic input layer have a connection to all the units on the small semantic hidden layer. As a consequence, there are 25x15 = 375 connections between these two layers. The total number of connections in the network was 19790.

The three layers of units -- input, hidden, and output -- will now be discussed separately.

Note 1
There are also Jordan-type recurrent neural networks. The main difference between the two is that Jordan nets also have a recurrent layer for the output units, in addition to the recurrent layer(s) for the hidden layer(s). Elman nets only have the latter type of recurrent layers.