Previous essays used a simple default ballistic forward motion action: the animal moved forward until it hit an obstacle or encountered food or an odor plume. Furthermore, all actions in the essay were continuous: at every time-step direction or speed could change. The animal was slug-like. However, since vertebrate is oscillatory — swimming or walking — powered by central pattern generators (CPGs), vertebrate motor commands are not continuous but modifiable only on correct timing of the oscillatory cycle.
This essay 21 adds some more realism to the simulated animal by creating distinct action syllables, each of which runs to completion. In addition, the default movement is a Lévy walk to more efficiently search for food.
Zebrafish bouts and mouse modules
Zebrafish larva move in discrete bouts [Johnson et al. 2020], punctuated by pauses. Each bout is on the order of 200ms to 1000ms and consists of stereotyped movement like a forward swimming stroke, and several turn types possibly followed by forward motion. The neural source of the bout timing is not know, although basal ganglia (striatal) defects can also produce jerky motion instead of smoothly linked motion.
Mouse movement is also comprised of small modules [Wiltschko et al. 2016] on the order of 60 modules in open exploration, each module lasting 200ms to 500ms. Unlike the zebrafish larva, sequences of mouse modules are linked smoothly instead of in jerky bouts.
Simulation action syllables
To match the vertebrate action syllables, the essay simulation now has action syllables at the lowest layers. The action syllables are fixed programs that last for several simulation ticks. For example a forward left turn might take 10 simulation ticks, depending on the tick resolution. In other words, these syllables are more like real-time actions with limited time resolution, not turn-based actions like a board game.
Unlike the current action completes, the system ignores new neural commands. In the future, specialized commands like freeze or panic escape might override the current action, like fast escape of zebrafish that bypass normal motor logic. In theory the system could also incorporate mid-action modulation, such as adding power to a swimming stroke without altering the basic action.
The new syllables should add realism and also introduce complications that vertebrates need to solve, such as timing for sequences of syllables. The syllables also introduce issues because prior-action memory as modeled by the striatum and nucleus isthmi need to persist for a syllable, not just a simulation tick.
Lévy walks
Along with the syllable changes, essay 21 adds better default movement. Previously if the animal wasn’t approaching food or avoiding an obstacle, it would move forward ballistically. That simplification only worked because the simulation is a simple bounded box, but in nature animals have better default search strategies.
Brownian motion is a simple default. The animal turns a random direction, then moves forward a random (gaussian) distance. Brownian motion does well at searching a neighborhood and is better than the essay’s ballistic strategy, but it tends to get stuck in a small area. When food is scarce and patchy, the brownian strategy won’t move the long distances needed to effectively find a new patch. The Lévy walk improves on this strategy by moving long distances when the current neighborhood is already searched [Abe 2020].
A Lévy walk is fractal (“scale-free”): larger walks have the same search structure as neighborhood search. If described in probabilistic terms, the distance traveled looks like a power law:
P(len) = 1/ len ^ 2
Where the exponent 2 (alpha) can be generalized between 1 and 3. The bounds are because the exponent 1 or less is ballistic and exponent 3 or more is essentially brownian. If alpha is closer to 1, the search is wider and moving further, while an alpha is closer to 3, the search is closer and more local.
Central pattern generators, criticality, and chaos
Animals do seem to use Lévy walks [Kölzsch et al. 2015] and the source appears to be internally generated [Berni et al. 2012] as opposed to a response to a fractal environment. The internal source of the randomness is not well know, although central pattern generators appear to be a possibility [Sims et al. 2018], [Reynolds 2019].
Near a critical point, fractal patterns appear and can be efficient computationally [Abe 2020], and a relatively simple chaotic system with only two random variables and produce Lévy walks. More broadly, other brain areas such as the cortex may also use criticality or near criticality to improve computation and generate longer-lasting signals to solve the timing problem [Hidalgo et al. 2014]. The timing problem is that fast neurons are 10ms but behavior needs to be responsive on the order of seconds and minutes.
Vertebrate source of Lévy walks
Although there is some knowledge of the search circuitry in fruit flies [Berni et al. 2012], the vertebrate circuitry seems entirely unknown.
As a thought experiment, consider the midbrain theta circuitry as part of the exploration circuit [Cisek 2022], and therefore related to the Lévy walk. If the vertebrate search is generated by a combination of central pattern generators then the source should be in the hindbrain, near those CPGs.
Theta clock cycles (4-12hz) in the brain are strongly correlated with exploratory movement, thought, and learning especially with the hippocampus (E.hc).
In the above, the hindbrain contains the chaotic search circuitry and generates theta, such as B.pno and B.vtn in conjunction with the reticulospinal motor command (B.rs). B.vtn is the forward movement analogue of the head direction nucleus B.dtn.
The medial septum (P.ms) is essentially the main theta clock for the hippocampus. Part of the theta cycle is generated internally to P.ms with spontaneous interaction of acetylcholine (ACh) and GABA inhibitory neurons, but exploration-related theta comes from the hypothalamic retromammilary (H.rm, aka supramammilary), which receives its theta from the hindbrain theta centers.
The movement restriction for theta depends on the median raphe (V.mr), which is one of the two main serotonin (5HT) centers. If V.mr is disabled, theta through H.rm always exists, not restricted to exploration. V.mr is in turn strongly influenced with the habenula (Hb.m) and interpeduncular (M.ip) complex.
Simulation
For now, I’m avoiding using criticality or chaotic variables in the simulation, although criticality is an interesting design to explore and very possibly how the vertebrate brain solves these problems.
The disadvantage is that the simulation would quickly become unclear and overcomplicated. While a few chaotic variables in the brainstem might be manageable, extending that idea to the striatum and cortex seems like it would become impenetrable. Since the purpose of the simulation is something like an executable thought experiment or executable diagram, and impenetrable simulation is defeating the purpose. Some alternatives like [Bartumeus and Levin 2008] fractal reorientation clocks might serve the purpose while remaining more clear.
References
Abe MS. Functional advantages of Lévy walks emerging near a critical point. Proc Natl Acad Sci U S A. 2020 Sep 29
Bartumeus F, Levin SA. Fractal reorientation clocks: Linking animal behavior to statistical patterns of search. PNAS. 2008
Berni J., Pulver S.R., Griffith L.C., Bate M. Autonomous circuitry for substrate exploration in freely moving Drosophila larvae. Curr. Biol. 2012
Berni J., Genetic dissection of a regionally differentiated network for exploratory behavior in Drosophila larvae. Curr. Biol. 25, 1319–1326 (2015).
Cisek P. Evolution of behavioural control from chordates to primates. Philos Trans R Soc Lond B Biol Sci. 2022 Feb 14
Hidalgo J, Grilli J, Suweis S, Muñoz MA, Banavar JR, Maritan A. Information-based fitness and the emergence of criticality in living systems. Proc Natl Acad Sci U S A. 2014 Jul 15
Kölzsch A., et al., Experimental evidence for inherent Lévy search behaviour in foraging animals. Proc. R. Soc. B Biol. Sci. 282, 20150424 (2015)
Maass W, Natschläger T, Markram H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 2002 Nov
Nurzaman SG, Matsumoto Y, Nakamura Y, Shirai K, Koizumi S, Ishiguro H. From Lévy to Brownian: a computational model based on biological fluctuation. PLoS One. 2011 Feb 3
Reynolds A. M., Current status and future directions of Lévy walk research. Biol. Open 7, bio030106 (2018)
Sims DW, Reynolds AM, Humphries NE, Southall EJ, Wearmouth VJ, Metcalfe B, Twitchett RJ. Hierarchical random walks in trace fossils and the origin of optimal search behavior. Proc Natl Acad Sci U S A. 2014 Jul 29
Sims D. W., Humphries N. E., Hu N., Medan V., Berni J., Optimal searching behaviour generated intrinsically by the central pattern generator for locomotion. eLife 8, e50316 (2019)
Wiltschko AB, Johnson MJ, Iurilli G, Peterson RE, Katon JM, Pashkovski SL, Abraira VE, Adams RP, Datta SR. Mapping Sub-Second Structure in Mouse Behavior. Neuron. 2015 Dec 16
Wolf S, Nicholls E, Reynolds AM, Wells P, Lim KS, Paxton RJ, Osborne JL. Optimal search patterns in honeybee orientation flights are robust against emerging infectious diseases. Sci Rep. 2016 Sep 12