While implementing the basic model, some issues came up, including issues already solved in earlier essays.
What controls “give-up”?
The foraging task needs to give-up on a non-promising odor, ignore it, leave from the current place, and explore for a new odor. In an earlier essay, odor habituation implemented give-up. If the seek didn’t find the food within the habituation time, the sense would disappear, disabling the seek action.
Animal circling food with no ability to break free.
The perseveration problem can be solved in many ways, including the goal give-up circuit in essay 17 and the odor habituation in an earlier essay. One approach cuts the sensor; the other disables the action. But two solutions raises the question of more possible solutions, any or all of which might affect the animal.
Sense habituation (cutting sensor)
Habenula give-up (inhibit action)
Motivational state – hypothalamus hunger/satiety
Circadian rhythm – foraging at twilight
Global periodic reset – rest / sleep
Give-up or leave?
The distinction between giving-up and leaving is between abandoning the current action and switching to a new, overriding action. Although the effect is similar, the implementing circuit differs. In a leave circuit, after the give-up time, the animal would actively leave the current area (place avoidance). Assuming the leave action has a higher priority than seeking, then lateral inhibition would disable the seek action. In foraging vocabulary, does failure inhibit exploitation or does it encourage exploration?
Distinct circuits for give-up and leave to curtail a failed odor approach.
As the diagram above shows, this distinction isn’t a semantic quibble, but represents different circuits. In the give-up circuit, the quit decision either inhibits the olfactory seek input and/or inhibits the seek action. With seek disable, the default action moves the animal away from the failed odor. In the leave circuit, the quit decision activates a leave action, which moves the animal away from the failed place, inhibiting the seek action laterally.
Leave or avoid?
Leaving an area is a primitive action and is a requirement for foraging. However, neuroscience papers don’t generally study foraging, they study place avoidance from aversive stimuli, which raises a question. Since the physical action of leaving and aversive place avoidance is identical, do the two actions share circuits or are they distinct?
Distinct leave and avoid actions compared to shared locomotion.
In the avoid circuit, danger avoidance is distinct from food-seeking, only sharing at the lowest motor layers. In the leave circuit, exploration leaving and place avoidance share the same mid-locomotor action.
Slow and fast twitch swimming
[Lacalli 2012] explores the evolution of chordate swimming, inspired by a discovery of mid-Cambrian fossils, which suggest that fast-twitch muscles are a later addition to a more basal chordate swimming, possibly to escape from new Cambrian predators. The paper explores the non-vertebrate Amphioxus motor circuitry in like of the fossil, suggesting two distinct motor circuits: normal swimming and escape.
Slow and fast paths for normal swimming and fast predator escape.
In this model, higher layers are independent paths that only resolve at the lowest motor command neuron level (such as B.rs). For the foraging tasks, this model that leaving an explored area would use a different system from leaving a noxious area (place aversion), despite being the same underlying motion.
Serotonin as muscle gain-control
In the zebrafish, [Wei et al. 2014] studied serotonin in V.dr (dorsal raphe) as gain-control for muscle output, amplifying the effect of glutamate signals. When they inhibited 5HT (serotonin), the muscle only produced 40% of its maximal strength. Serotonin acted as a gain-control, a multiplicative signal that amplified glutamate signals, allowing for a broader dynamic range.
[Kawashima et al. 2016] investigated 5HT in the context of task-learning for muscle effort, where 5HT caches the real-time adjustment by the cerebellum and pretectal areas. When 5HT is disabled, the real-time system still adjusts the muscle effort, but it doesn’t remember the adjustment for future bouts. That study considers the 5HT neurons as leaky integrators of motor-gated visual feedback, where zebrafish gauge the success of swimming effort by visual motion. Notably, the neurons only store visual information when the fish is actively swimming, as an action-outcome integrator.
The two studies focused on opposite muscle effects, both increasing effort and decreasing effort. 5HT can either inhibit or excite depending on the receptor type, suggesting that 5HT shouldn’t be interpreted as representing a specific value, either positive or negative, but instead possibly carrying either value.
Taking these studies as analogies, it seem reasonable to consider V.dr as an action-outcome accumulator for future effort in the 10-30 seconds range, not specific to either positive or negative amplification. Of course, because serotonin has diverse effects in multiple circuits, reality is likely more complicated.
Serotonin zooplankton dispersal and learning
Many aquatic animals have a larval zooplankton stage, where the larva disperses from its spawn point for several days or weeks, then descends to the sea floor for its adult life. A small number of serotonin neurons signal the switch to descend. Essentially, this is a single explore/exploit pair.
Larva exploring in a dispersal stage, switching to descend to the sea floor for adult life.
Habenula function circuit
Essay 17 is running with the model of the habenula as central to the give-up/move-on circuit. The following is a straw man model of the habenula based on the above discussion of quitting, leaving and avoiding circuits. Because essay 17 has no learning or higher areas like the striatum, the diagram ignores any learning functionality. This diagram is for a hypothetical pre-stratal habenular function.
Odor-based locomotion using the habenula.
Note, this locomotion only includes odor-based navigation. The audio-visual-touch locomotion uses a different system based on the optic tectum. This dual-locomotive system may be the result of a bilaterian chimaera brain [Tosches and Arendt 2013].
The habenula connectivity and avoidance path is loosely based on [Stephenson-Jones et al. 2012] on the lamprey habenula connectivity. The seek path is loosely based on [Derjean et al. 2010] for the zebrafish.
In this model, Hb.m (medial habenula) is primarily a danger-avoidance circuit, and M.ipn (interpeduncular nucleus) is a place avoidance locomotive region. Hb.l (lateral habenula) is a give-up circuit that both inhibits the seek function (giving up) and excites the shared leave locomotor region, implementing the foraging exploit to explore decision. Here, place avoidance and exploratory leaving are treated as equivalent. As mentioned above, this diagram is mean to be a straw man or a thought experiment, because it’s easier to work with a concrete model.
The locomotive model in essays 14 to 16 were non-vertebrate. Essay 17 takes the same problems, avoiding obstacles and seeking food, and with a model based on the vertebrate brain. Since these models are still Precambrian or early Cambrian, they don’t include the full vertebrate architecture, but try to find core components that might have been a basis for later vertebrate developments.
The animal is a slug-like creature with mucociliary forward movement, where propulsion is cilia or cilia-like and steering is muscular. This combination of slug-like motion and vertebrate brain is probably not evolutionary accurate, but it allows touch-based obstacle avoidance without the complications of vision of lateral-line senses.
The animal seeks food by following odor plumes, and avoids obstacles by turning away when touching them. The locomotion model includes the following components:
[Braitenberg 1984] navigation (simple crossed vs uncrossed signals for approach and avoid).
Obstacle avoidance with a direct touch-to-muscle circuit.
Odor-seeking with distinct “what” and “where” paths.
Perseveration fix with an explicit give-up circuit.
Motivation-state (satiety) control of odor-seeking (“why” path [Verschure et al. 2014])
Proto-vertebrate model
A diagram of the proto-vertebrate model, including analogous brain regions follows:
Proto-vertebrate locomotive model. Key: B.sp spinal motor, B.rs reticulospinal motor command (medial and lateral), B.ss spinal somatosensory, H.l lateral hypothalamus, Hb.l lateral habenula, M.lr midbrain locomotive region (M.ppt), Ob olfactory bulb, Snc substantia nigra pars compacta. DA dopamine, ACh acetylcholine.
For the sake of readability, the model simplifies the actual vertebrate midline crossing patterns, leaving only a single cross between B.rs (reticulospinal) and B.sp (spinal), which represents Braitenberg navigation.
In this model, obstacle avoidance is reflexive between B.ss (somatosensory touch) and B.rs. Odor navigation (“where”) flows through Snc (substantia nigra pars compacta) to M.lr (midbrain locomotive region). In the zebrafish, the Snc area is the posterior tuberculum, and the M.lr like represents M.ppn (pedunculopontine tegmental nucleus). The motivation-state (hunger or satiety) and “what” (food odor vs non-food) flow through H.l (lateral hypothalamus). The give-up circuit flows through Hb.l (lateral habenula).
Olfactory navigation path
[Derjean et al. 2010] traced a path in zebrafish from Ob (olfactory bulb) to the posterior tuberculum (mammal Snc) to the midbrain locomotive region (likely M.ppn), to the reticulospinal motor command neurons.
Zebrafish olfactory to motor path in [Derjean 2010].
I’ve labeled this path as a “where” path, based on simulation requirements, but as far as I know, that label has no scientific basis.
The Snc / posterior tubuculum area includes descending glutamate and dopamine (DA) neurons, although the Snc is better known for its ascending dopamine path. Since [Ryczko et al. 2016] reports a mammalian descending glutamate and DA path from Snc to M.ppn, portions of this descending path appears to be evolutionarily conserved. The DA appears to be an effort boost, increasing downstream activity, but most of the activity is glutamate.
Braitenberg navigation
[Braitenberg 1986] vehicles are a thought experiment for simple circuits to implement approach and avoid navigation. In the original, the vehicles have two light-detection sensors connected to drive wheels. Depending on the connection topology, sign and thresholds, the simple circuits can implement multiple behaviors.
Braitenberg circuits for approach and escape.
A circuit that combines the output of approach and avoid circuits with some lateral inhibition can implement both approach and avoidance with avoidance taking priority. In the essay simulation, if the animal touches a wall, it will turn away from the obstacle, temporarily ignoring any odor it might be following.
Circuit for combined odor approach and touch obstacle avoidance.
Mammalian locomotion appears to use a similar circuit between the superior colliculus (OT – optic tectum) and the motor driving B.rs neurons [Isa et al. 2021]. This circuit pattern implies that approach and avoidance are separate behaviors, only reconciled at the end. For example, a punishing reinforces that increases avoidance is not simply the mirror image of a non-reward that decreases approach. The two reinforcers modify different circuits.
“What” path vs “where” path
The mammalian visual system has separate “what” and “where” paths. One path detects what object is in focus, and one path keeps track of where the object location is. This division between object decision and navigation has been useful in the simulation, because navigation details are quickly lost in the circuit when deciding what to do with an odor.
“What” and “where” paths as configuring a switchboard.
When an animal senses an odor, say a food odor, the animal needs to identify it as a food odor, decide if the animal is hungry or sated, and decide if there’s a higher-priority task. All that processing and decision can lost the fine timing phase and amplitude details needed for precise navigation. Gradient following, for example, needs fine differences in timing or amplitude to decide whether to turn left or right. By splitting the long, complicated “what” decision from the short, simple “where” location, the circuit can benefit from both.
[Cohn 2015] describes the fruit fly mushroom body as a switchboard, where dopamine neurons configure the path for olfactory senses to travel. In the context of “what” and “where”, the “what” path configures the switchboard and the “where” path follows the connected circuit.
Some odor-based navigation has a more extreme division between “what” and “where.” Following odor in water isn’t always gradient-based navigation, because odors form clumps instead of gradient plumes. Instead of following a gradient, the animal moves against the current toward the odor source. In that latter situation, the “where” path uses entirely different senses for navigation, using water flow mechanosensors, not olfactory sensors [Steele et al. 2023].
Navigation against current toward an odor plume.
The diagram above illustrates a food-searching strategy for some animals in a current, both water and air. In water, the current is more reliable for navigation than an odor gradient. When there’s no scent, the animal swims back and forth across the current. When it detects a food odor, it swims against the current. If it loses the odor, it will return to back and forth swimming. In this navigation type, entirely different senses drive the “what” and “where” paths.
Foraging and give-up time
Giving up is an essential part of goal-directed behavior. If an animal cannot ever give up, it will be stuck on the goal without escaping. In the context of foraging, the give-up time is optimized with the marginal value theorem [Charnov 1976], suggesting that an animal should move to another patch when its current reward-gaining rate drops below the average rate for the environment. Animal behavior researchers like [Kacelnik and Brunner 2002] have observed animals roughly following this theorem, although using simpler heuristics.
In more complex animals, the failure to give up can be pathological, such as psychological perseveration.
Foraging state diagram illustrating the give-up timer
The give-up circuit needs some kind of internal timer or cost integrator, and a way to cancel the task. In this essay’s model, the lateral habenula (Hb.l) computes the give-up time or integrates the cost, and it cancels the task by suppressing the locomotive signal through Snc.
Habenula as a give-up circuit
Hb.l is positioned to act as a give-up circuit. It receives cost signals as non-rewarded bouts or as aversive events. [Stephenson-Jones et al. 2016] interprets the Hb.l input, P.hb (habenular-projecting pallidum), as evaluating action outcome. Hb.l can suppress both the midbrain dopamine and midbrain serotonin areas. In learned helplessness situations or depression, Hb.l is hyperactive [Webster et al. 2020], causing reduced activity.
Habenula circuit as a give-up mode in a locomotive circuit.
[Hikosaka 2012] suggests the habenula’s role as suppressing motor activity under aversive conditions, a role evolved from its close relationship to the pineal gland’s circadian scheduling.
In a review article, [Hu 2020] discusses the suppressive effects of the habenula, also remarking on its role as a reward-prediction error. In particular, noting that H.l (lateral hypothalamus) to Hb.l is aversive. The Hu article also notes that Hb.l knock-out abolishes the error signal from reward omission, not an error signal from aversive (shock or obstacles).
Once the threshold is crossed, the Hb.l to Snc signal produces behavioral avoidance, reduced effort and depressive-like behavior from learned helplessness. The Hb.l is the only brain area consistently hyperactive in animal models of depression.
Note, since this essay’s simulation is a non-learning behavioral model, the only “prediction” possible is an evolutionary intrinsically-attractive odor, and the only role for an error is giving up the current behavior. Here, I’m interpreting the H.l to Hb.l signal as a cost signal, integrated by Hb.l, that gives up when it crosses a threshold.
Vertebrate reference
For reference, here’s a functional model of the vertebrate brain.
Functional model of vertebrate brain.
The areas in this model cluster around the hindbrain isthmus divider. B.rs are hindbrain neurons near the isthmus. M.lr (M.ppn) are midbrain neurons that migrate from the hindbrain (r1) to the midbrain. Snc is the midbrain tegmental area (the V – value area), near the isthmus, and contiguous with M.ppn. Similarly the H.l area that projects to Snc is contiguous with it. The habenula is the most distant area, located above the thalamus near the pineal gland (not in the diagram as a simplification, but associated with the pallidum areas.) So, the areas discussed here are a small part of the entire brain, but interestingly clustered around the isthmus divider near the cerebellum.
Minimal viable straw man
I think it’s important to remember that the essay simulations are an engineering project not a scientific one. One difference is that the simulations necessary require decisions beyond science. Another difference is that the project needs a simple core that may not correspond to any evolutionary animal. For example, even simple animals have some rudimentary vision, if only two or three pigment spots. For another, learning centers like the mushroom body. And dealing with internal biological issues like breathing and blood pressure with motion.
This model in particular is more of a straw man or minimal viable product than an actual proposal for an ancestral proto-vertebrate mind. The model is intended to be a straw man, a target that might give a base framework to criticize or build on.
Alternative olfactory paths
Another potential “what” path for innate behavior goes through the medial habenula, which is responsive to odors and produces place avoidance [Amo et al. 2014], but [Chen et al. 2019] suggests it also supports attraction for food odors.
Olfactory innate path through habenula. Key: A.co cortical amygdala, H.l lateral hypothalamus, Hb habenula (medial and lateral), IPN interpeduncular nucleus, M.pag periaqueductal gray, Ob olfactory bulb.
In mammals, the olfactory path to H.l goes through the cortical amygdala (A.co) [Cádiz-Moretti et al. 2017]. While this essay is deliberately omitting the cortex, in the lamprey the olfactory path goes through the lateral pallium (LPa, corresponding to mammalian O.pir piriform cortex) to the posterior tubercular (Snc in mammals.)
For this essay, I’ve picked the Ob to Snc path instead of the alternatives for simplicity. The habenula path is very tempting, but would require exploring the IPN and serotonin (5HT) paths to the MLR, which is more complicated than a “what” path through H.l
Subthalamic nucleus as give-up circuit
The sub thalamic nucleus (H.stn) is associated with a “stop” action, stopping downstream motor actions, either because of a new, surprising stimulus, or from higher-level commands. Since a give-up signal stops the seek goal, the stop action from H.stn might play a part in the control
H.stn stop is in parallel to habenular give-up. Key: H.l lateral hypothalamus, H.stn subthalamic nucleus, Hb.l lateral habenula, M.lr midbrain locomotor region, Snc substantia nigra pars compacta, Snr substantia nigra part reticulata.
H.stn is believed to have a role in patience in decision making [Frank 2006] and in encoding reward and cost [Zénon et al. 2016], which is very similar to the role of the habenula, and H.stn projects to Hb.l via P.hb habenula-projecting pallidum.
However, the H.stn’s patience is more related to holding off (stopping) action before making a decision, related to impulsiveness, while the give-up circuit is more related to persistence, continuing an action. So, while the two capabilities are related, they’re different functions. Since current essay simulation does not have patience-related behavior arrest but does need a give-up time, the habenula seems a better fit.
Serotonin inhibition path
In zebrafish, the habenula inhibits the dorsal raphe (V.dr, serotonin neurons) but not Snc or dopamine [Okamoto et al. 2021]. The inhibition works through V.dr to the Snc/posterior tubuculum to the locomotive regions.
As with the alternative olfactory paths, this serotonin inhibition path may be more evolutionary primitive, but would add complexity to the essay’s model, so will be held off for later exploration.
Conclusions
As mentioned above, the purpose of this model is a basis for the current essay’s simulation, and as a straw man to focus alternatives to see if there might be a better minimal model.
