In PSYCHOLINGUISTICS, computational models are becoming increasingly important both for helping us understand and develop our theories and for deriving empirical predictions from those theories. The behavior of a theory of language processing usually depends not just on the mechanics of the model itself, but also on the properties of the linguistic input the theory operates on. Even when the theory is conceptually simple, the interaction between theory and language is often too complex to be explored without the benefit of computer simulations. It is no suprise then that computational models have been at the centre of some of the most significant recent developments in psycholinguistics.
The main area of contact with empirical data has been made by models operating roughly at the level of the word. Although there is an active and productive enterprise directed at developing models of higher level processes such as syntactic parsing (Kempen and Vosse 1989; McRoy and Hirst 1990; Marcus 1980) and discourse (Kintch and van Dijk 1978; Kintch 1988; Sharkey 1990), the complexity of these processes makes it harder to derive detailed experimental predictions.
The Neighborhood Activation Model (Luce, Pisoni and Goldinger 1990) gives a computational account of isolated word recognition, but only TRACE (McClelland and Elman 1986) and Shortlist (Norris 1994a) have been applied to the more difficult problem of how words can be recognised in continuous speech, where the input may contain no reliable cues to indicate where one word ends and another begins. Both of these models are descendants of McClelland and Rumelhart's (1981) connectionist Interactive Activation Model (IAM) of VISUAL WORD RECOGNITION. The central principle of both models is that the input can activate multiple word candidates, represented by nodes in a network, and that these candidates then compete with each other by means of inhibitory links between overlapping candidates. So, the spoken input 'get in' might activate 'tin' as well as 'get' and 'in,' but 'tin' would be inhibited by the other two overlapping words. TRACE and Shortlist represent opposite positions in the debate over whether SPOKEN WORD RECOGNITION is an interactive process or not. In TRACE there is continuous interaction between the lexical and phonemic levels of representation, whereas Shortlist has a completely bottom-up, modular architecture. They also differ in their solution to the problem of how to recognize words beginning at different points in time. TRACE uses a permanent set of complete lexical networks beginning at each point where a word might begin. In Shortlist the network performing the lexical competition is created dynamically and contains only those candidates identified by a bottom-up analysis of the input. On a purely practical level, at least, this has the advantage of enabling Shortlist to perform simulations with realistically sized lexicons of twenty or thirty thousand words. Shortlist has also been extended (Norris, McQueen and Cutler 1996) to incorporate the Metrical Segmentation Strategy of Cutler and Norris (1988), which enables the model to make use of metrical cues to word boundaries.
The most significant non-connectionist model of spoken word recognition has been Oden and Massaro's (1978) Fuzzy Logical Model of Perception (FLMP). FLMP differs from TRACE and Shortlist in that it can be seen as a generic account of how decisions are made on the basis of information from different sources. FLMP itself has nothing to say, for example, about the competition process which is vital for the recognition of words in continuous speech in both TRACE and Shortlist. Comparisons between FLMP and TRACE in terms of their treatment of the relationship between lexical and phonemic information led to a major revision of the IAM framework and the development of the Stochastic Interactive Activation Model (Massaro 1989; McClelland 1991).
IAMs and spreading activation models have also been predominant in the area of LANGUAGE PRODUCTION (Dell 1986, 1998; Harley 1993; Roelofs 1992; see also Houghton 1990). Dell's model is designed to account for the nature and distribution of speech errors. It takes as its input an ordered set of word units (lemmas), representing the speaker's intended production, and produces as its output a string of phonemes which may be corrupted or misordered. In Dell (1988) the model consists of a lexical network in which word nodes are connected to their constituent phonemes by reciprocal links, and a word shape network which reads out successive phonemes in the appropriate syllable structure. The main effect of the reciprocal links from phonemes to words is to give the model a tendency for its errors to form real words rather than nonsense words. Whether the production system really does contain these feedback links has been the topic of extensive debate between Dell and Levelt. Levelt, Roelofs and Meyer (in press) describe the latest computational implementation of the WEAVER++ model, which is a non-interactive spreading-activation model designed to account for an extensive body of RT data on production as well as speech error data.
The most controversial connectionist model of language has been Rumelhart and McClelland's (1986) account of the acquisition of the past tense of verbs. Conventional linguistic accounts of quasi-regular systems such as the past tense assume that the proper explanation is in terms of rules and a list of exceptions. Rumelhart and McClelland modelled the acquisition of the past tense using a simple pattern-associator which mapped the phonology of verb roots (e.g. kill, run) onto their past tense forms (killed, ran). They claimed that their model not only explained important facts about the acquisition of verbs, but that it did so without using linguistic rules. The model was fiercely criticized by Pinker and Prince (1988). Later models by MacWhinney and Leinbach (1991) and Plunket and Marchman (1991; 1993), using backpropagation with hidden units, rectified some of the technical deficiencies of the original model, and claimed to give a more accurate account of the developmental data, but the debate between the connectionist and symbolic camps continues. Recently both neuropsychological and neuroimaging data have been reported which hint at a neuroanatomical distinction between mechanisms underlying the rule-based and non-rule based processes (e.g. Marslen-Wilson and Tyler in press)
A parallel set of arguments has surrounded models of reading aloud. The relationship between spelling and sound is another example of a quasi-regular system where back-propagation networks have been used to give a unitary account of the READING process rather than incorporating spelling-to-sound rules and a list of exception words (e.g. yacht, choir) which are not pronounced according to the rules (Seidenberg and McClelland 1989; Plaut et al. 1996). The more traditional two-process view is represented by Coltheart et al. (1993) while an interactive activation model by Norris (1994b) takes an intermediate stance.
-- Dennis Norris
REFERENCES AND FURTHER READINGS
Coltheart, M., B. Curtis, P. Atkins, and M. Haller. (1993). Models of reading aloud: dual-route and parallel-distributed-processing approaches. Psychological Review 100: 589-608.
Cutler, A. and D. Norris. (1988). The role of strong syllables in segmentation for lexical access. Journal of Experimental Psychology: Human Perception and Performance 14: 113-121.
Dell, G.S. (1986). A spreading-activation theory of retrieval in sentence production. Psychological Review 82: 407-428.
Dell, G.S. (1988). The retrieval of phonological forms in production: tests of predictions from a connectionist model. Journal of Memory and Language 27: 124-142.
Dijkstra, T. and K. de Smedt (Eds.). (1996). Computational Psycholinguistics. London: Taylor and Francis.
Harley, T.A. (1993). Phonological activation of semantic competitors during lexical access in speech production. Language and Cognitive Processes 8: 291-309.
Houghton, G. (1990). The problem of serial order: a neural network model of sequence learning and recall. In R. Dale, C. Mellish, and M. Zock (Eds.), Current Research in Natural Language Generation. London: Academic Press, pp. 289-319.
Kempen, G. and T. Vosse. (1989). Incremental syntactic tree formation in human sentence processing: a cognitive architecture based on activation decay and simulated annealing. Connection Science 1: 273-290.
Kintch, W. (1988). The role of knowledge in discourse comprehension: a construction-integration model. Psychological Review 95: 163-182.
Kintch, W. and T.A. van Dijk. (1978). Towards a model of text comprehension and production. Psychological Review 85: 363-394.
Levelt, W.J.M, A. Roelofs, and A.S. Meyer. (In press). A theory of lexical access in speech production. Brain and Behavioural Sciences
Luce, P.A., D.B. Pisoni, and S.D. Goldinger. (1990). Similarity neighborhoods of spoken words. In G.T.M. Altmann (Ed.), Cognitive Models of Speech Processing: Psycholinguistic and Computational Perspectives. Cambridge, MA: MIT Press, pp. 122-147.
MacWhinney, B. and J. Leinbach. (1991). Implementations are not conceptualizations: revising the verb learning model. Cognition 40: 121-157.
Marcus, M.P. (1980). A Theory of Syntactic Recognition for Natural Language. Cambridge, MA: MIT Press.
Massaro, D.W. (1989). Testing between the TRACE model and the Fuzzy Logical Model of Perception. Cognitive Psychology 21: 398-421.
McClelland, J.L. (1991). Stochastic interactive processes and the effects of context on perception. Cognitive Psychology 23: 1-44.
McClelland, J.L. and J.L. Elman. (1986). The TRACE model of speech perception. Cognitive Psychology 18: 1-86.
McClelland, J.L. and D.E. Rumelhart. (1981). An interactive activation model of context effects in letter perception: 1. An account of the basic findings. Psychological Review 88: 375-407.
McRoy, S.W. and G. Hirst. (1990). Race-based parsing and syntactic disambiguation. Cognitive Science 14: 313-353.
Marslen-Wilson, W.M and L.K. Tyler. (In press). Dissociating types of mental computation. Nature
Norris, D. (1994a). Shortlist: a connectionist model of continuous speech recognition. Cognition 52: 189-234.
Norris, D. (1994b). A quantitative multiple-levels model of reading aloud. Journal of Experimental Psychology: Human Perception and Performance 20(6): 1212-1232.
Norris, D., J.M. McQueen, and A. Cutler. (1995). Competition and segmentation in spoken word recognition. Journal of Experimental Psychology: Learning, Memory, and Cognition 21: 1209-1228.
Oden, G.C. and D.W. Massaro. (1978). Integration of featural information in speech perception. Psychological Review 1985: 172-191.
Pinker, S. and A. Prince. (1988). On language and connectionism: analysis of a parallel distributed processing model of language acquisition. Cognition 28: 73-193.
Plaut, D.C., J.L. McClelland, M. Seidenberg, and K.E. Patterson. (1996). Understanding normal and impaired word reading: computational principles in quasi-regular domains: Psychological Review 103: 56-115.
Plunkett, K., and V. Marchman. (1991). U-shaped learning and frequency effects in a multi-layered perceptron; implications for child language acquisition. Cognition 38: 43-102.
Plunkett, K., and V. Marchman. (1993). From rote learning to system building: acquiring verb morphology in children and connectionist nets. Cognition 48: 21-69.
Roeloffs, A. (1996). Serial order in planning the production of successive morphemes of a word. Journal of Memory and Language 35: 854-876.
Roelofs, A. (1992). A spreading-activation theory of lemma retrieval in speaking. Cognition 42: 107-142.
Rumelhart, D.E. and J.L. McClelland. (1986). On learning the past tense of English verbs. In J.L. McClelland and D.E. Rumelhart (Eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Cambridge, MA: MIT Press.
Seidenberg, M. and J.L. McClelland. (1989). A distributed developmental model of word recognition and naming. Psychological Review 96: 523-568.
Sharkey, N.E. (1990). A connectionist model of text comprehension. In D.A. Balota and G.B. Flores d'Arcais (Eds.), Comprehension Processes in Reading. Hillsdale, NJ: Lawrence Erlbaum, 487-514.