Learning & Teaching Foreign Languages

Chargement du moteur de recherche...

Neural Networks

Read and Reflect

Connectionist models of learning generally rely on a model of learning based on neural networks.  Read this short description of Alan Turing's early model.

Connectionist researchers Rumelhart and McClelland used this type of network to model language acquisition.  They trained a virtual network in the past tense of English.  They used the past tense forms of both regular and irregular verbs, and ones which occur frequently or more rarely.

After adapting weighting through comparing the input and output of many verbs over many repetitions, the network produced correct past tense forms for the training verbs, and was also able to generate correct forms for unfamiliar verbs.  It had "learned" the English past tense.

Rumelhart and McClelland conclude that this virtual network, like Newell and Simon's logic machine, has reproduced human learning and helped understand it:

Our simple learning model shows, to a remarkable degree, the characteristics of young children learning the morphology of the past tense in English.  We have shown how our model generates the so-called U-shaped learning curve for irregular verbs and that it exhibits a tendence to overgeneralize that is quite similar to the pattern exhibited by young children. [..]
(Rumelhart & McClelland, 1986, p. 267)

However, unlike Newell and Simon's information processing system, which required conscious manipulation of a system of symbols, the connectionist model is sub-symbolic:

We have, we believe, provided a distinct alternative to the view that children learn the rules of English past-tense formation in any explicit sense.  [..] The child need not figure out what the rules are, nor even that there are rules.  The child need not decide whether a verb is regular or irregular.  [..] There isn't even a question (as far as generating the past-tense form is concerned) as to whether a verb form is one encountered many times or one that is being generated for the first time.  A uniform procedure is applied for producing the past-tense form in every case.  The base form is supplied as input to the past-tense network and the resulting pattern of activation is interpreted as a phonological representation as the past form of that verb.  This is the procedure whether the verb is regular or irregular, familiar or novel.

(pp. 267-8)

Check your understanding of the material on this page by answering the following questions:

1. Why did Rumelhart and McClelland select regular and irregular verbs, as well as frequent and rare ones?

2. In what ways did their network mimic child language acquisition?

3. Why do they believe connectionist models of learning are superior to symbolic systems?

 

ANSWERS

1. Why did Rumelhart and McClelland select regular and irregular verbs, as well as frequent and rare ones?

To make the input to the network similar to the input received by a human learner.  The child learner of English hears common regular (talk/talked) and irregular (go/went) verbs, as well as rarer verbs (whisper/whispered; wreak/wrought).

2. In what ways did their network mimic child language acquisition?

It displayed overgeneralisation, or U-shaped learning, for irregular verbs: correct irregular form, incorrect regularised form, then correct irregular form

3. Why do they believe connectionist models of learning are superior to symbolic systems?

There is no need to claim that young learners are learning rules either implicitly or explicitly, and no need for separate mechanisms for regular and irregular or common and rare verbs.  The model of learning is therefore simpler and thus, some would claim, superior.

Read about a similar experiment in SLA: L2 past

Valid XHTML 1.0 Transitional