From: IN%"CYBSYS-L@BINGVMB" "Cybernetics and Systems" 29-MAR-1990 11:00:32.31 To: "Peter H. Roosen-Runge" , RICHARD CLARK CC: Subj: Dynamic neural nets Received: from JNET-DAEMON by YUSol; Thu, 29 Mar 90 11:00 EDT Received: From BINGVMB(MAILER) by YUSOL with Jnet id 8489 for CS100006@YUSOL; Thu, 29 Mar 90 11:00 EDT Received: by BINGVMB (Mailer X1.25) id 7487; Thu, 29 Mar 90 10:25:41 ECT Date: Thu, 29 Mar 90 10:19:27 EDT From: CYBSYS-L Moderator <@NEXUS:cybsys@bingvaxu.cc.binghamton.edu> Subject: Dynamic neural nets Sender: Cybernetics and Systems To: "Peter H. Roosen-Runge" , RICHARD CLARK Reply-to: Cybernetics and Systems Really-From: Peter Cariani Date: Thu, 29 Mar 90 00:37:00 est Dave Mills has perceptively commented that the dynamic, multistable aspect of neural nets has been generally overlooked. I think this is because neural nets have been cast as reliable, deterministic devices for computation rather than as parts of robots coordinating percepts and actions in real time. The nature of perception and action in the physical world is one of ongoing activity and sequences of behaviors, while the nature of computation and syntactic symbol manipulation is essentially a-temporal and a one-time operation (compute something once, why compute it again?). I've been reading McCulloch lately (highly recommended! Embodiments of Mind, MIT Press, 1965, 1988) and one sees that the initial ideas of McCulloch and Pitts also involved recurrent networks, those having cycles of self-production/modification relations. Here the sequences of outputs form stable behaviors rather than one output or decision. My impression is that these circular self-production networks (which were first proposed by Nicolas Rashevsky in the 1930's and which form the basis for Rosen's Metabolism, Repair systems and Maturana & Varela's Autopoietic systems and Bill Powers' Control Theory) were abandoned by the neural nets people in the late 50's in favor of feed-forward ones because of the latter's greater stability and predictability. Recurrency and cyclicity are of course ultimately feed-back relations. Recurrent networks are also directly related to autocatalytic cycles, as Eric Minch has explicitly pointed out. Neural networks are a subclass of reaction networks. Of specific interest here is a Road Not Taken: the neural network ideas of Peter H. Greene, who discussed the behavior of networks of oscillators coupled in various ways. There are three very intriguing papers from 1962 which suggest that such a network will have resonant modes and periodic behaviors within each mode. The network behavior could be switched from one resonant mode to another, but all of the possible modes are always latent in the underlying dynamics of the network. I think we could get open-ended creation of new modes of behavior out of such a system precisely because it is analog and doesn't bottom-out in complexity (as in a digital system). Greene apparently went on to work on the coordination of motion in animals, where it is very useful to have rhythmic modes of activity, as in walking: the higher centers steer the lower ones into generating the rhythmic outputs needed to drive the muscles and off we go! It seems from the Citation Index that Peter Kugler and Scott Kelso, who do work on dynamical theory, neural nets, and Gibsonian theories of behavior both cite Greene's work, but I haven't tracked down these papers yet...For ongoing dynamics of neural nets, I recommend looking at their work. Does anyone out there have any ideas about what happened historically to the idea of recurrent networks? Is it due to their less tractable, their non-straight-forward nature? Is there any work being done on recurrent neural networks? (There seemed to be no evidence of this kind of perspective at the IJCNN conference in January....) I think we should build devices along the lines which Greene suggested. Does anyone know of any such attempts? References -------------- Greene, Peter H (1962) On looking for neural networks and "cell assemblies" that underlie behavior. Bulletin of Mathematical Biophysics 24:247-275 & 395-411 (2 papers). There is also a shorter intro, On the Representation of Information by Neural Net Models, in Self-Organizing Systems, 1962, Yovits et al eds, Pergamon Press. Kelso, Scott, Sholtz, & Schoner (1988) Dynamics governs switching among patterns of coordination in biological movement. Physics Letters A 134(1): 8-12 (12 Dec 1988). Kugler, Peter, Scott Kelso, & MT Turvey (1980) On the concept of coordinative structures as dissipative structures. In: Tutorials in Motor Behavior. Stelmach & Requib, eds, North-Holland. Minch, Eric (1989) The Representation of Hierarchical Structure in Evolving Networks. PhD dissertation, Systems Science Dept, SUNY- Binghamton, University Microfilms, Ann Arbor. Powers, William (1973) Behavior: Control of Perception. Rashevsky, Nicolas. Mathematical Biophysics: Physico-Mathematical Foudations of Biology. Dover, 1938, 1960. Vols I & II. Rosen, Robert. Anticipatory Systems. Pergamon Press, 1985. 'Nuff said, over & out. --Peter Cariani, peterc@chaos.cs.brandeis.edu 37 Paul Gore St, Jamaica Plain, MA 02130, tel 617-524-0781