3.4.3.1 Deprivation of Orthography or Semantics

In order to see how CLASPnet would cope with partial input, I rewrote the first test corpus so as to replace all the values of either the semantic representations or the orthographic representations with zeros. In this manner, no activation could pass from the relevant part of the input layer to the corresponding hidden layer, and this layer, in turn, could no longer influence the large hidden layer.

The leftmost bar of each group in Figure 3.25 always shows the performance of the 'normal' test corpus, with the orthography-only corpus second, and the semantics-only corpus third. It is not surprising to see that the semantics-only corpus nearly always leads to the worst result -- the tasks are, after all, fairly syntactic in nature, and the semantic representation itself with its perceptual, Gestalt, and instinctive units seems poorly equipped to determine whether, for example, a clause is passive or active. However, Figure 3.25 also shows that the orthography-only corpus leads to results which are noticeably worse than the normal corpus. On average, the orthography-only corpus misses about 40% of the positive classifications which it should have made. For all these cases, the difference between the orthography-only corpus and the normal corpus can only be explained by the absence of the semantic input representations.

So, what this experiment demonstrates is that when a connectionist network like CLASPnet is given the option of using even very limited local semantic information for classification tasks involving high-level structural properties, it will make use of this information and come to depend on it. Take it away and the network is crippled.