3.3.4.3 Double or Quits

It is well known that human parsing mechanisms are robust, whereas those of even the largest symbolic translation programs can break down quite easily. Hence, it is interesting to see how a connectionist model fares in this regard; as it simply does not 'know' the concept of a list of valid parse trees, it seems reasonable to expect that it will be more tolerant of 'invalid' constructions than its symbolic counterparts.

As a first experiment, I presented the network with sentences which contain double nouns or noun phrases, like 'the man man see the women near the yacht.', 'we we will not sleep.' and 'the man see the women experts near the yacht.'(Note 23) To put it metaphorically, the network didn't even blink. If one looks at the word by word analysis for the second sentence, then nothing out of the ordinary can be detected:

Word	Ind	Imp	Int	ID1	ID2	ID3	Sta	Dec	Ord	WH	YN	Rel	Con	Com	Inf	Voi	Pol
we	1	0	0	1	0	0	1	1	0	0	0	0	0	0	0	1	0
	1	0	0.03	1	0	0.01	1	1	0	0	0.02	0	0	0	0	0.78	0.46
we	1	0	0	1	0	0	1	1	0	0	0	0	0	0	0	1	0
	1	0	0	1	0	0	1	1	0	0	0	0	0	0	0	1	0.90
will	1	0	0	1	0	0	1	1	0	0	0	0	0	0	0	1	0
	1	0	0	1	0	0	1	1	0	0	0	0	0	0	0	1	0.74
not	1	0	0	1	0	0	1	1	0	0	0	0	0	0	0	1	0
	1	0	0	0.99	0.01	0	1	1	0.02	0	0	0	0	0	0.01	0.95	0
sleep	1	0	0	1	0	0	1	1	0	0	0	0	0	0	0	1	0
	1	0	0	1	0	0	1	1	0.02	0	0	0	0	0	0	1	0
,	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0

Table 3.6: Word by word analysis of the sentence 'we we will not sleep.'

In 'can will the girls like the fishes?' there was some confusion about the clause type: it went from Yes/No-Question to declarative on 'will', but the network then slowly raised the activation value of the Yes/No unit again at the expense of the Declarative unit. And the rather confusing sentence 'it be said announced that we you be not the best president postman for the united kingdom states.' was even parsed as if it was absolutely normal, as was a sentence with a double locative prepositional phrase: 'the man love the falcons near the roof on the boat.'

Going the other way, I also looked at how the network behaves when words are missing from the input. In many cases, the net again fails to notice that there is something special about the sentence: for example, 'the see the women near the yacht.', 'the friendly girl love.', 'the girls be going to the fishes?' and 'the man love the falcons near the on boat.' were are all correctly interpreted for all their words. In other cases, however, weird behavior ensued. For example, in 'the very happy sleep.' the network drops the activation of the Polarity unit when it sees 'sleep'. In 'will not sleep.' the clausal type switches from Yes/No to Order when the 'not' is processed -- apparently, the network notices that a 'not' occurs as the second word in a clause, and associates this input with the 'do not ...' imperatives -- though the Interrogative mood unit remains highly active for the entire sentence. Similar interference seems to be at work in 'the man the moody boys.' when 'the moody boys' is analyzed as the beginning of a relative clause -- sentences like 'the man we like ...' do indeed occur in the training corpus. The final interesting sentence is 'it be that we be not the best for the united.' in which 'said' or 'announced' has been omitted from the matrix clause. While complement clauses introduced by 'that' are normally always recognized immediately, in this case the network misses the cue from the preceding 'said' and fails to give the complement clause any analysis at all. So, what we have here is another clear example of the context-sensitivity of the network.

Useful though the behavior of the network may be with respect to using similar methods for improving the performance of various kinds of language recognition systems -- especially sloppy speech, of course -- a valid psycholinguistic concern is that the network might perform too well. Connectionist models want to simulate human behavior, not improve on it. And while real people also sometimes overlook repeated and missing words, this is certainly not always the case. Consequently, it is still a challenge for CLASPnet to show that it can detect anomalies in its input. But it is unclear how one should go about training a network to react to certain patterns in a specific way, if, per definition, these patterns cannot be part of the training corpus. Still, there is already some evidence that connectionist models can fail in psychologically plausible ways: the random sentence 'under . highly ! in man moody which sad worshiping very palace boat' gives CLASPnet fits -- i.e. it causes strange patterns on the activation values of the output units.

Note 23
The last sentence is of course perfectly grammatical in normal English, but it is useful in this context because it is ungrammatical as far as the grammar used for generating the corpora is concerned.