3.3.3.2 ID and Status Units

Figure 3.12 shows the overall performance of the network on these four units:

Here, too, we find very good results, with all the percentages ranging between 96% and 100%. But unlike what we have for the sentential mood units, it is not as easy to find an explanation for these results: the ID numbers are not associated in a straightforward manner with the different clauses in a sentence. The same relative clause might occur as the second clause in one sentence but as the third or fourth in another sentence. Simply increasing a counter by one every time a new clause begins is actually a typically symbolic task, and not one which one would expect a neural network to learn without difficulty. A possible explanation might be that the network simply counts the commas, which separate most of the clauses -- but this hypothesis seems unlikely given the good results of nets which were trained on corpora without punctuation marks (see 3.4.6.2). A factor which does seem to have played a role is that there are not that many sentences in the corpora which have multiple clauses: only about 600 words out of approximately 43,000 are part of the third or fourth clause in a sentence. (But their small number notwithstanding, the network still manages to classify 76% of them correctly for the training corpus and 73% for the test corpus.) With most of the words belonging either to the first clause (approx. 28,000) or the second (approx. 14,000), the network may have found it relatively easy to keep the two apart.

Surprisingly, overall performance on the single Status unit is not as good as that on the ID units. Although the two tasks are related, one would expect there to be more cues available to the network to determine the status of a clause: for example, all the declarative clauses are also matrix clauses, but none of the relative clauses, connector clauses or complement clauses are (except for 'the more ..., the less ...'-like constructions in which both clauses are considered matrix clauses). From reading the word by word analyses of how CLASPnet parses sentences, it can be learnt that the net makes two types of errors with regard to the Status unit: first, complements of verbs of asking introduced by 'if' (e.g. 'do the men inquire if you be not lovely?' or 'shall the small children, dying, ask the large men in a barn if this large boy should not be old?') are often classified incorrectly as belonging to the matrix clause -- but similar complements introduced by 'whether' do not suffer from the same problem; second, in some sentence-final subordinate clauses, the activation value of the Status unit increases for the last word or words -- apparently in anticipation of a new matrix clause at the beginning of the next sentence.

If we look at the missed and spurious errors for these units (see Figure 3.13), then there is one clear datum in need of explanation. The mismatch between the 0% for missed ID3 in the training corpus and 100% for missed ID3 in the test corpus is, however, simple to explain: in the training corpus there are no words belonging to the fourth clause in a sentence (recall that the ID units use a binary encoding scheme), so the network has learnt to leave the ID3 unit off at all times. But in the test corpus, there are 5 words belonging to the fourth clause in a sentence. As might be expected, all 5 words are classified incorrectly, resulting in the 100% missed bar seen in Figure 3.13.