3.1.3 The Output Layer
The output layer of CLASPnet has 17 units, which, using a localist representation, represent properties about the sentence and the clause which the network is processing at any given moment. Every time a new word is presented to the input units, the activation values of the output units are updated to reflect this change. In this manner, one can 'see' the analysis which the net gives to a sentence or clause evolve word per word. Sequential processing of this kind is naturally interesting from a psycholinguistic point of view, as one can compare results of experiments with humans with those with neural networks. (Whether a close fit between the two is of any scientific relevance, however, depends on a lot of factors -- e.g. whether any time has been spent on getting the network results to correspond with the human data.)
Before I discuss the individual output units, I briefly want to motivate my choice to focus on clausal properties. Hudson's (1984) work on Word Grammar proves that it is not entirely obvious to do so: in Word Grammar, sentences, clauses, and even nominal or verbal phrases are abandoned in favor of a 'companion' relationship between words, and the dependency relation between heads and their modifiers. Still, even Hudson (1984: 81) mentions in passing that there is a language, Kalkatungu, in which words can occur just about anywhere in a clause, but they never cross clause boundaries. Similar data have been reported by Taylor (1995: 180) for the Zulu clitic 'ke': it is used for topicalization of various linguistic structures, but only one can appear in each clause. Developmental evidence for accepting clauses as useful linguistic entities is given in Slobin (1985: 222f.): he reports that, cross-linguistically, children prefer separate negation morphemes which are put just outside of the normal clause, and without the words in this clause being reordered. And Langacker (1990b: 212) has written: "The finite clause is a pivotal unit of grammatical organization". Taken together, all this seems to make a strong case for the existence of the clause as a mental entity of some sorts: the words in a clause can be thought of constructing a single conceptual scene, in which -- at least typically -- a single action or state is taking place. At the center of each clause then is a verb phrase which -- at least partly -- describes this action or state. Or, turning the tables, each verb phrase indicates the presence of a separate clause. It is this last rule which I have followed for the definition of clauses in CLASPnet. (The definition of a sentence is even less controversial: all the words before a full stop, question mark, or exclamation mark.)
Figure 3.2 summarizes the interpretation of the output units of CLASPnet:
While the combination of the 17 output units certainly does not express everything that could to be said about any English clause -- aspect, tense, and pragmatic functions come to mind -- it does provide a lot of information about such a clause. Enough so, I feel, that a recurrent neural network like CLASPnet could be expected to have a hard time coming up with the desired values on the output units. Before I discuss the actual results of the simulations, however, it is useful to take a closer look at how the training and testing corpora were created.
There are of course many more moods than the three used here. Bybee (1985: 23) gives the following list: "imperative, indicative, negative, probable, interrogative, optative, conjunctive, conditional, and dubitative" (Pawnee has eight!), and Hudson (1984) also mentions the exclamation as a separate mood. Implementing more moods in CLASPnet would be easy because of the type of grammar used to generate sentences and the type of parser used to analyze them (see 3.2).
The 'be' in the latter sentence is not a mistake -- see 3.2.2 below for more details.