From:	IN%"CYBSYS-L@BINGVMB"  "Cybernetics and Systems" 29-MAR-1990 11:00:32.31
To:	"Peter H. Roosen-Runge" <CS100006@YUSOL.BITNET>, RICHARD CLARK <GUEST11@YUSOL.BITNET>
CC:	
Subj:	Dynamic neural nets

Received: from JNET-DAEMON by YUSol; Thu, 29 Mar 90 11:00 EDT
Received: From BINGVMB(MAILER) by YUSOL with Jnet id 8489 for CS100006@YUSOL;
 Thu, 29 Mar 90 11:00 EDT
Received: by BINGVMB (Mailer X1.25) id 7487; Thu, 29 Mar 90 10:25:41 ECT
Date: Thu, 29 Mar 90 10:19:27 EDT
From: CYBSYS-L Moderator <@NEXUS:cybsys@bingvaxu.cc.binghamton.edu>
Subject: Dynamic neural nets
Sender: Cybernetics and Systems <CYBSYS-L@BINGVMB>
To: "Peter H. Roosen-Runge" <CS100006@YUSOL.BITNET>, RICHARD CLARK
 <GUEST11@YUSOL.BITNET>
Reply-to: Cybernetics and Systems <CYBSYS-L@BINGVMB>

Really-From: Peter Cariani
 <peterc@chaos.cs.brandeis.edu.cs.brandeis.edu@RELAY.CS.NET>
Date: Thu, 29 Mar 90 00:37:00 est
 
Dave Mills has perceptively commented that the dynamic, multistable
aspect of neural nets has been generally overlooked. I think this is
because neural nets have been cast as reliable, deterministic devices
for computation rather than as parts of robots coordinating percepts
and actions in real time. The nature of perception and action in the
physical world is one of ongoing activity and sequences of behaviors,
while the nature of computation and syntactic symbol manipulation is
essentially a-temporal and a one-time operation (compute something
once, why compute it again?).
   I've been reading McCulloch lately (highly recommended! Embodiments
of Mind, MIT Press, 1965, 1988) and one sees that the initial ideas of
McCulloch and Pitts also involved recurrent networks, those having cycles
of self-production/modification relations. Here the sequences of outputs
form stable behaviors rather than one output or decision. My impression
is that these circular self-production networks (which were first proposed
by Nicolas Rashevsky in the 1930's and which form the basis for Rosen's
Metabolism, Repair systems and Maturana & Varela's Autopoietic systems
and Bill Powers' Control Theory)
were abandoned by the neural nets people in the late 50's in favor of
feed-forward ones because of the latter's greater stability and
predictability.    Recurrency and cyclicity are of course ultimately
feed-back relations.  Recurrent networks are also directly related to
autocatalytic cycles, as Eric Minch has explicitly pointed out. Neural
networks are a subclass of reaction networks.
   Of specific interest here is a Road Not Taken: the neural network
ideas of Peter H. Greene, who discussed the behavior of networks of
oscillators coupled in various ways. There are three very intriguing
papers from 1962 which suggest that such a network will have resonant
modes and periodic behaviors within each mode. The network behavior
could be switched from one resonant mode to another, but all of the
possible modes are always latent in the underlying dynamics of the
network. I think we could get open-ended creation of new modes of
behavior out of such a system precisely because it is analog and
doesn't bottom-out in complexity (as in a digital system). Greene
apparently went on to work on the coordination of motion in animals,
where it is very useful to have rhythmic modes of activity, as in
walking: the higher centers steer the lower ones into generating
the rhythmic outputs needed to drive the muscles and off we go! It
seems from the Citation Index that Peter Kugler and Scott Kelso,
who do work on dynamical theory, neural nets, and Gibsonian theories
of behavior both cite Greene's work, but I haven't tracked down these
papers yet...For ongoing dynamics of neural nets, I recommend looking
at their work.
   Does anyone out there have any ideas about what happened historically
to the idea of recurrent networks? Is it due to their less tractable,
their non-straight-forward nature? Is there any work being done on
recurrent neural networks? (There seemed to be no evidence of this
kind of perspective at the IJCNN conference in January....)
   I think we should build devices along the lines which Greene suggested.
Does anyone know of any such attempts?
 
References
--------------
 
Greene, Peter H (1962) On looking for neural networks and "cell
   assemblies" that underlie behavior. Bulletin of Mathematical
   Biophysics 24:247-275 & 395-411 (2 papers). There is also a
   shorter intro, On the Representation of Information by Neural
   Net Models, in Self-Organizing Systems, 1962, Yovits et al eds,
   Pergamon Press.
Kelso, Scott, Sholtz, & Schoner (1988) Dynamics governs switching
   among patterns of coordination in biological movement. Physics
   Letters A 134(1): 8-12 (12 Dec 1988).
Kugler, Peter, Scott Kelso, & MT Turvey (1980) On the concept of
   coordinative structures as dissipative structures. In: Tutorials
   in Motor Behavior. Stelmach & Requib, eds, North-Holland.
Minch, Eric (1989) The Representation of Hierarchical Structure in
   Evolving Networks. PhD dissertation, Systems Science Dept, SUNY-
   Binghamton, University Microfilms, Ann Arbor.
Powers, William (1973) Behavior: Control of Perception.
Rashevsky, Nicolas. Mathematical Biophysics: Physico-Mathematical
   Foudations of Biology. Dover, 1938, 1960. Vols I & II.
Rosen, Robert. Anticipatory Systems. Pergamon Press, 1985.
 
'Nuff said, over & out.
 
--Peter Cariani, peterc@chaos.cs.brandeis.edu
37 Paul Gore St, Jamaica Plain, MA 02130, tel 617-524-0781