Attempting a toy model of vertebrate understanding

Tag: olfactory

Essay 26: Ignoring distracting odors

I’ve been ignoring distracting cues in the previous essays for simplification. Since the simulated animal only encountered a single odor at a time, it never needed to select one and ignore the other. In essay 26, I’ll implement a very simple first approximation to ignoring distractors, using the P.bf (basal forebrain) control of the Ob (olfactory bulb) as a switchboard to let the selected odor through and inhibit the ignored distractor.

Simulated animal (triangle) encountering two odor plumes (circles).

In the diagram above, the animal (triangle) is seeking food using the purple odor cue as a gradient direction. When it encounters the distractor odor in blue, it should ignore the distractor, otherwise the two odors will mingle into an incorrect summed gradient and the animal will seek in the wrong direction [Cisek 2022].

Temporal chemotaxis

For essay 26, I’m switching chemotaxis (odor seeking) to use the apical temporal gradient search, using Hb.m (medial habenula) and B.ip (interpeduncular nucleus) like the phototaxis in essay 24. The apical system follows the chimera brain model of [Tosches and Arendt 2013], which suggests that odor senses and actions are distinct systems from bilateral tactile senses. For the essays, the shift is from a bilateral, Braitenberg-like [Braitenberg 1984] system to a modulated random walk like the bacterial tumble-and-run.

Olfactory tumble-and-run system using Hb.m and B.ip for temporal gradient direction, and B.rs for the modulated random walk. B.ip interpeduncular nucleus, B.rs hindbrain reticulospinal motor area, Hb.m medial habenula, Ob olfactory bulb.

The above diagram shows the problem with distractor odors. Because the tumble-and-run system uses a single temporal gradient, it necessarily adds both odors together for its input. The summed input goes to the Hb.m (medial habenula) and B.ip (interpeduncular nucleus) system to modulate the random walk direction.

When the animal crosses into the overlapping distractor odor, it will follow the combined signal, distracted from the original seek target. To avoid distraction, the system can either amplify the current odor A, or inhibit the distractors like odor B.

Analogy with nucleus isthmi

An earlier essay 19 also had an attention / distractor problem, with a different issue of action consistency, and used a zebrafish circuit in P.ni (nucleus isthmi) as a solution. In larval zebrafish P.ni works together with OT (optic tectum) to sustain attention on prey during a hunt [Henriques et al 2019]. P.ni is an ACh (acetylcholine neurotransmitter) and GABA (inhibiting neurotransmitter) system that both amplifies the predicted prey location and inhibits surrounding areas.

Nucleus isthmi circuit as adapted by essay 19. ACh acetylcholine, OT optic tectum, Pni nucleus isthmi.

In the above diagram for the essay 19 circuit, a simultaneous left and right touch would select one action at random and sustain that choice for subsequent movement with the P.ni positive feedback circuit. The outputs are crossed because it’s an avoidance circuit: an obstacle on the left triggers a right turn.

Importantly, the positive feedback is modulatory; it doesn’t trigger an action by itself. At a synapse level, ACh triggers mAChR (ACh metabotropic receptor, Gs stimulatory type) on the sensor axon, amplifying the sensor’s neurotransmitter release. The ACh and mAChR act as the decay timer, because they have a slow time constant on the order of a few seconds. If the sensor doesn’t stimulate the circuit, as when successfully avoiding the obstacle, the attention will decay over a few seconds, resetting the system to its original state.

A similar function applies to Ob and P.bf (basal forebrain), where P.bf acts like P.ni to sustain attention to the selected odor. “Basal forebrain” is a general name for a collection of functionally-related subcortical areas in the ventral (“basal”) forebrain, all pallidal-like (P). The specific P areas for the Ob are P.hdb (horizontal diagonal band) and Po.me (magnocellular preoptic area), but I’ll use P.bf for simplicity.

Olfactory bulb as a switchboard

In this model, Ob acts like a switchboard controlled by P.bf. P.bf selects attended odor paths in Ob, where Ob either passes the odor signal to its destination or inhibits the signal if it’s a distractor. P.bf opens and closes gated circuits in Ob.

Although the architecture of the Ob and P.bf circuit resembles the P.ni circuit, Ob appears to rely more heavily on inhibitory GABA for the gating operation, although ACh is also important [Böhm et al 2020], [de Saint Jan et al 2020], [Nunez-Parra et al 2000]. Since this essay is a first cut, simplified model, I’m using a single signal that represents a gating attention / inhibition signal, and glossing over the ACh vs GABA distinction.

Olfactory bulb switchboard using basal forebrain to gate selected odors. B.ip interpeduncular nucleus, B.rs reticulospinal motor, Hb.m medial habenula, Omt mitral/tufted output cells, Osn olfactory sensor neurons, P.bf basal forebrain.

In the above diagram where the switchboard selects odor A and inhibits odor B, the apical seek circuit receives only odor A’s signal. P.bf gates odors from Osn (olfactory sensory neurons) to Omt (mitral/tufted output cells), which then add to form a single signal for the temporal gradient tumble-and-run seek. For simplicity, I’ve shown the P.bf ACh and GABA signal as a simple gating control.

Once the system detects odor A, P.bf configures the switchboard to pass through A and inhibit other odors, locking out the distractor. Because the selecting signals are modulators, they don’t drive a signal until an odor signal arrives. Like the P.ni circuit, attention will timeout as ACh and its slow mACh receptor decay. When the animal leaves the odor plume, the system resets because the absence of odor A collapses the feedback loop.

Although the essay’s switchboard is an improvement over the naive summation of odor signals, it’s still quite limited. There’s no active selection of a best odor, and the system can’t switch to a better odor cue. Also, since the global give-up circuit isn’t integrated with P.bf, giving up on odor A can’t select odor B. Instead the animal must leave the plume and reset the system.

Slightly more complete Ob switchboard

The Ob is a surprisingly complex system; it’s not just a simple odor system. In addition to the P.bf, Opir (olfactory piriform cortex) also modulates the Ob system, and Ob itself has lateral inhibition between Omt (mitral cell output), which is plastic, learning to discriminate odors itself, as well as modulatory input from the serotonin and noradrenaline system.

In the real Ob, many Osn for the same odor feed into a single Ogl (olfactory glomeruli), which provides input to several Omt, all representing the same odor. Each odor feature has its own Ogl system, several hundred in mammals (two in the essay simulation). Ogl is where the neuropil of the Osn axons meet the Omt dendrites, in a fan-in to fan-out system. Also, each Ogl has many inhibitory Opg (periglomerular inhibitors) with multiple variations, and each Omt has several inhibitory Ogc (olfactory granule cells). The basic fan-in and fan-out structure looks like the following diagram.

Olfactory bulb glomerule fan-in and fan-out system. Bip interpeduncular nucleus, B.rs reticulospinal motor, Hb.m medial habenula, Ogc olfactory granule cell inhibitor, Ogl olfactory glomerule, Omt olfactory mitral/tufted output, Opg olfactory periglomerular inhibitor, Osn olfactory sensor neuron.

The switchboard diagram below focuses on the ACh and GABA control from P.bf. It combines multiple Osn, Opg, Omt and Ogc into single items.

Partial olfactory bulb switchboard circuit. B.ip interpeduncular nucleus, B.rs reticulospinal motor, Hb.m medial habenula, Ogc olfactory granule cell, Ogl olfactory glomeruli, Omt olfactory mitral/tufted output, Opg olfactory periglomerular inhibitor, Opir olfactory piriform cortex, Osn olfactory sensory neuron.

To break down the diagram, the core of the switchboard circuit is the Osn to Ogl to Omt to output path; everything else is gating to select or inhibit the signal.

Odor gating happens in two locations: modulating Omt’s input dendrite tree in Ogl by Opg and modulating Omt’s output by Ogc (olfactory granular cell). Because each Omt’s input Ogl is shared for several Omt, the Opg inhibition likely affects many or all Omt for a single Ogl. In contrast, the Ogc inhibition is individual, and the Omt and Ogc circuit creates and manages gamma oscillations, which amplifies and reduces noise from the signal.

Although I’m not planning on touching cortical areas for many essays, the Opir (olfactory piriform cortex) modules the Ob switchboard in a similar circuit as B.pf with some difference. Since the Opir input to the many Ogc and many Ogl is not odor selective [Boyd et al 2015], Ogc must learn the meaning of the Opir input through plasticity.

Global give-up circuit

The essay’s task engagement and give-up circuit currently uses H.l (lateral hypothalamus) and Hb.l (lateral habenula) with V.dr (dorsal raphe serotonin) [Hikosaka 2010], [Chowdhury and Yamanaka 2016]. When a seek fails Hb.l suppresses H.l, H.l ends seek, and the animal moves on [Post et al 2022].

Global give-up circuit. H.l lateral hypothalamus, Hb.l lateral habenula, V.dr dorsal raphe, 5HT serotonin.

Because the global give-up circuit is entirely disconnected from the olfactory selective attention from the essay, giving up means giving up on all odors, not just the current attended odor.

Simulation

For this essay, I refactored much of the simulation code to clean up ideas from previous essays. A new hindbrain module manages the main locomotion like the zebrafish hindbrain motor area [Dunn et al 2016], which is possibly different from the tetrapod / amniote locomotion in the midbrain. Because the essay animal is currently more primitive than amniotes, this simplification seemed appropriate and makes the code organization more clear.

Olfactory locomotion is now random-walk based following apical tumble-and-run, as opposed to the earlier bilateral path through Vta (ventral tegmental area / posterior tuberculum) and OT (tectum). In zebrafish both paths exist, which I might explore later, but this essay is restricted to the apical temporal gradient search.

The seek mode now slows the animal and adjusts the Levy walk parameters to simulate ARS (area restricted search). As I’ll cover in the problems section, switching to seek mode is still hardcoded.

I split the habenula seek from habenula give-up (Hb.m from Hb.l) and pulled the gradient seek and head direction from B.ip into the habenula seek. Conceptually, the habenula seek code now represents Hb.m and B.ip as a single complex.

Simulated odor seeking with target attention and distractor inhibition.

In the screenshot above, the animal is making a u-turn to return to the food when the odor gradient (blue semicircle) is opposite the head direction (black semicircle). In the upper right, the green box outlined in red represents the attended green odor signal, while the white box outline in blue represents the suppressed blue odor. Despite the Osn naively sensing both blue and green odors because the animal is in the overlap area, only the green odor passes through Omt to the seek system.

The square borders around the odor color represent P.bf modulation. Red is attended (100% pass through), blue is inhibited (10% pass through), and grey is unmodulated (50% pass through).

In the diamond-shaped homunculus, the bright blue triangle represents the u-turn nudge.

As the goal vector shows, the guessed goal direction isn’t very accurate, particularly when the animal is making a turn. Currently, the animal continues to update its guess even in the middle of a turn when the odor data and averages are not appropriate for the current direction.

References

Böhm E, Brunert D, Rothermel M. Input dependent modulation of olfactory bulb activity by HDB GABAergic projections. Sci Rep. 2020 Jul 1;10(1):10696. 

Boyd AM, Kato HK, Komiyama T, Isaacson JS. Broadcasting of cortical activity to the olfactory bulb. Cell Rep. 2015 Feb 24;10(7):1032-9.

Braitenberg, V. (1984). Vehicles: Experiments in synthetic psychology. Cambridge, MA: MIT Press. “Vehicles – the MIT Press”

Chowdhury S, Yamanaka A. Optogenetic activation of serotonergic terminals facilitates GABAergic inhibitory input to orexin/hypocretin neurons. Sci Rep. 2016;6:36039

Cisek P. Evolution of behavioural control from chordates to primates. Philos Trans R Soc Lond B Biol Sci. 2022 Feb 14

De Saint Jan D. Target-specific control of olfactory bulb periglomerular cells by GABAergic and cholinergic basal forebrain inputs. Elife. 2022 Feb 28;11:e71965.

Dunn, Timothy, Yu Mu, Sujatha Narayan, Owen Randlett, Eva A Naumann, Chao-Tsung Yang, Alexander F Schier, Jeremy Freeman, Florian Engert, Misha B Ahrens (2016) Brain-wide mapping of neural activity controlling zebrafish exploratory locomotion eLife 5:e12741

Henriques PM, Rahman N, Jackson SE, Bianco IH. Nucleus Isthmi Is Required to Sustain Target Pursuit during Visually Guided Prey-Catching. Curr Biol. 2019 Jun 3;29(11):1771-1786.e5. 

Hikosaka O. The habenula: from stress evasion to value-based decision-making. Nat Rev Neurosci. 2010 Jul;11(7):503-13. 

Nunez-Parra A, Cea-Del Rio CA, Huntsman MM, Restrepo D. The Basal Forebrain Modulates Neuronal Response in an Active Olfactory Discrimination Task. Front Cell Neurosci. 2020 Jun 5;14:141. 

Post RJ, Bulkin DA, Ebitz RB, Lee V, Han K, Warden MR. Tonic activity in lateral habenula neurons acts as a neutral valence brake on reward-seeking behavior. Curr Biol. 2022 Oct 24;32(20):4325-4336.e5.

Tosches, Maria Antonietta, and Detlev Arendt. The bilaterian forebrain: an evolutionary chimaera. Current opinion in neurobiology 23.6 (2013): 1080-1089.

Essay 20: Olfactory avoidance

Although the essays have implemented obstacle avoidance, they haven’t yet explored olfactory avoidance. Olfactory avoidance is distinct from obstacles, not just because obstacles have higher priority, but because the olfactory system is from an entirely different nervous system than the sensorimotor system. In the chimaeral brain theory [Tosches and Arendt 2013], bilaterian brains are composed of an apical nervous system (ANS) focused on chemo senses (olfactory external and hypothalamic internal), and a blastoporal nervous system (BNS) focused on sensorimotor control like obstacle avoidance.

Olfactory path

The paths for olfactory motion compared with obstacle motion shows the value of the chimaeral theory in making sense of the brain. Working backward from the midbrain locomotive region (MLR), the acetylcholine (ACh) MLR nuclei specialize: the pedunculopontine nucleus (M.ppt) supports the sensorimotor BNS, and the laterodorsal tegmental nucleus (M.ldt) supports the chemosensory ANS.

Sensor-locomotion paths: olfactory on top and somatosensory on bottom. B.ll lateral line, B.rs reticulospinal motor command, B.ss somatosensory, Hb.m medial habenula, M.ldt laterodorsal tegmental nucleus, M.ppt pedunculopontine nucleus, Ob.m medial olfactory bulb, OT tectum, R.vis visual input, ,Vta ventral tegmental area.

In the above diagram, food odors and warning odors use distinct paths to the MLR. Food odors from the olfactory bulb (Ob) pass through the ventral tegmental area (Vta – posterior tuberculum in zebrafish) to the MLR [Derjean et al. 2010]. Aversive odors like cadaverine pass through the medial habenula (Hb.m) to the M.ldt portion of the MLR [Stephenson-Jones et al. 2012]. The food and avoidance paths are distinct because hunger and satiety from the hypothalamus modulate the food path, while the avoidance path can pass through unmodulated. These olfactory locomotion paths correspond to the ANS.

Lamprey medial habenula path

All vertebrates share this basic architecture, including the lamprey, one of the most evolutionary-distant vertebrates. [Stephenson-Jones et al. 2012] traced the Hb.m circuit, showing that Hb.m inputs are from the olfactory path, the parapineal (light attraction), and an electron-sensory alarm to the interpeduncular nucleus (M.ip).

Lamprey olfactory warning path through the habenula to the MLR. M.ip interpeduncular nucleus.

The above diagram fills out the olfactory warning path. The interpeduncular nucleus is a key node in the avoidance circuit, and also key to locomotor-induced theta, and one of the two serotonin nodes. Mip has a major output to the serotonin areas: dorsal raphe (V.dr) and medial raphe (V.mr) and to the central grey (M.pag) [Quina et al. 2017] and M.ldt as well as structures associated with hippocampal (E.hc) theta [Lima et al. 2017].

Medial habenula behavior

In larval zebrafish, Hb.m supports olfactory avoidance [Choi et al. 2017], [Jeong et al. 2021], and light seeking [Zhang et al. 2017]. At least one study indicates that it may also affect food seeking [Chen et al. 2019]. The non-Ob input to Hb.m — the posterior septum (P.ps) — produce locomotion when stimulated [Ostu et al. 2018], suggesting that later evolved functionality maintains the original basal function.

In zebrafish, M.ip only projects to serotonin areas (V.dr and V.mr), not to dopamine or MLR areas. The lamprey connectivity suggests that the M.ip to M.ldt connection was lost in fish.

The Hb.m to M.ip connection is affected by nicotine. An interesting property is that low stimulation and high stimulation have opposite effects. Low stimulation uses glutamate connections and is attractive while high stimulation adds ACh and is aversive [Krishnan et al. 2014].

Developmental genetic notes

As an interesting aside, both Hb.m and avoidant layers of OT shared a genetic marker Brn3a (aka pou4f1) [Quina et al. 2009], [Fedtsova et al. 2008]. That marker also appears in the cerebellum’s inferior olive, trigeminal sensory areas, and the amphioxus motor LPN3 neuron [Bozzo et al. 2023].

M.ldt and M.ppt are sibling areas, deriving from the r1 rhombic lip [Machold et al. 2011].

Glutamate and GABA neurons in M.ip, Vta, and M.ldt all derive from r1 basal neurons [Lahti et al. 2016].

Locomotion switchboard

The addition of olfactory avoidance further complicates the switchboard combining the various locomotor streams, especially if the olfactory path uses serotonin as a modulator as opposed to a straight glutamate connection. Although I’ll probably use a fixed priority for essay 20, and as [Cisek 2022] notes, avoidance can be combined additively, at some point the switchboard will need more control, especially when essays add vision and consummatory actions.

References

Bozzo M, Bellitto D, Amaroli A, Ferrando S, Schubert M, Candiani S. Retinoic Acid and POU Genes in Developing Amphioxus: A Focus on Neural Development. Cells. 2023 Feb 14

Chen W-Y, Peng X-L, Deng Q-S, Chen M-J, Du J-L, Zhang B-B. Role of Olfactorily Responsive Neurons in the Right Dorsal Habenula-Ventral Interpeduncular Nucleus Pathway in Food-Seeking Behaviors of Larval Zebrafish. Neuroscience. 2019

Choi JH, Duboue ER, Macurak M, Chanchu JM, Halpern ME. Specialized neurons in the right habenula mediate response to aversive olfactory cues. Elife. 2021 Dec 8

Cisek P. Evolution of behavioural control from chordates to primates. Philos Trans R Soc Lond B Biol Sci. 2022 Feb 14

Derjean D, Moussaddy A, Atallah E, St-Pierre M, Auclair F, Chang S, Ren X, Zielinski B, Dubuc R. A novel neural substrate for the transformation of olfactory inputs into motor output. PLoS Biol. 2010 Dec 21

Fedtsova N, Quina LA, Wang S, Turner EE. Regulation of the development of tectal neurons and their projections by transcription factors Brn3a and Pax7. Dev Biol. 2008 Apr 1

Jeong YM, Choi TI, Hwang KS, Lee JS, Gerlai R, Kim CH. Optogenetic Manipulation of Olfactory Responses in Transgenic Zebrafish: A Neurobiological and Behavioral Study. Int J Mol Sci. 2021 Jul 3

Krishnan S, Mathuru AS, Kibat C, Rahman M, Lupton CE, Stewart J, Claridge-Chang A, Yen SC, Jesuthasan S. The right dorsal habenula limits attraction to an odor in zebrafish. Current Biology. 2014

Lahti L, Haugas M, Tikker L, Airavaara M, Voutilainen MH, Anttila J, Kumar S, Inkinen C, Salminen M, Partanen J. Differentiation and molecular heterogeneity of inhibitory and excitatory neurons associated with midbrain dopaminergic nuclei. Development. 2016 Feb 1

Lima LB, Bueno D, Leite F, Souza S, Gonçalves L, Furigo IC, Donato J Jr, Metzger M. Afferent and efferent connections of the interpeduncular nucleus with special reference to circuits involving the habenula and raphe nuclei. J Comp Neurol. 2017 Jul 1

Machold R, Klein C, Fishell G. Genes expressed in Atoh1 neuronal lineages arising from the r1/isthmus rhombic lip. Gene Expr Patterns. 2011 Jun-Jul

Otsu Y, Lecca S, Pietrajtis K, Rousseau CV, Marcaggi P, Dugué GP, Mailhes-Hamon C, Mameli M, Diana MA. Functional Principles of Posterior Septal Inputs to the Medial Habenula. Cell Rep. 2018 Jan 16

Quina LA, Wang S, Ng L, Turner EE. Brn3a and Nurr1 mediate a gene regulatory pathway for habenula development. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience. 2009

Stephenson-Jones M, Floros O, Robertson B, Grillner S. Evolutionary conservation of the habenular nuclei and their circuitry controlling the dopamine and 5-hydroxytryptophan (5-HT) systems. Proc Natl Acad Sci U S A. 2012 Jan 17

Tosches, Maria Antonietta, and Detlev Arendt. “The bilaterian forebrain: an evolutionary chimaera.” Current opinion in neurobiology 23.6 (2013): 1080-1089.

Zhang BB, Yao YY, Zhang HF, Kawakami K, Du JL. Left Habenula Mediates Light-Preference Behavior in Zebrafish via an Asymmetrical Visual Pathway. Neuron. 2017 Feb 22

Essay 17: Proto-Vertebrate Locomotion

The locomotive model in essays 14 to 16 were non-vertebrate. Essay 17 takes the same problems, avoiding obstacles and seeking food, and with a model based on the vertebrate brain. Since these models are still Precambrian or early Cambrian, they don’t include the full vertebrate architecture, but try to find core components that might have been a basis for later vertebrate developments.

The animal is a slug-like creature with mucociliary forward movement, where propulsion is cilia or cilia-like and steering is muscular. This combination of slug-like motion and vertebrate brain is probably not evolutionary accurate, but it allows touch-based obstacle avoidance without the complications of vision of lateral-line senses.

The animal seeks food by following odor plumes, and avoids obstacles by turning away when touching them. The locomotion model includes the following components:

  • [Braitenberg 1984] navigation (simple crossed vs uncrossed signals for approach and avoid).
  • Obstacle avoidance with a direct touch-to-muscle circuit.
  • Odor-seeking with distinct “what” and “where” paths.
  • Perseveration fix with an explicit give-up circuit.
  • Motivation-state (satiety) control of odor-seeking (“why” path [Verschure et al. 2014])

Proto-vertebrate model

A diagram of the proto-vertebrate model, including analogous brain regions follows:

Proto-vertebrate locomotive model. Key: B.sp spinal motor, B.rs reticulospinal motor command (medial and lateral), B.ss spinal somatosensory, H.l lateral hypothalamus, Hb.l lateral habenula, M.lr midbrain locomotive region (M.ppt), Ob olfactory bulb, Snc substantia nigra pars compacta. DA dopamine, ACh acetylcholine.

For the sake of readability, the model simplifies the actual vertebrate midline crossing patterns, leaving only a single cross between B.rs (reticulospinal) and B.sp (spinal), which represents Braitenberg navigation.

In this model, obstacle avoidance is reflexive between B.ss (somatosensory touch) and B.rs. Odor navigation (“where”) flows through Snc (substantia nigra pars compacta) to M.lr (midbrain locomotive region). In the zebrafish, the Snc area is the posterior tuberculum, and the M.lr like represents M.ppn (pedunculopontine tegmental nucleus). The motivation-state (hunger or satiety) and “what” (food odor vs non-food) flow through H.l (lateral hypothalamus). The give-up circuit flows through Hb.l (lateral habenula).

Olfactory navigation path

[Derjean et al. 2010] traced a path in zebrafish from Ob (olfactory bulb) to the posterior tuberculum (mammal Snc) to the midbrain locomotive region (likely M.ppn), to the reticulospinal motor command neurons.

Zebrafish olfactory to motor path in [Derjean 2010].

A similar olfactory to motor path has been traced in lamprey by [Suryanarayana et al. 2021] and [Beausejour et al. 2021].

I’ve labeled this path as a “where” path, based on simulation requirements, but as far as I know, that label has no scientific basis.

The Snc / posterior tubuculum area includes descending glutamate and dopamine (DA) neurons, although the Snc is better known for its ascending dopamine path. Since [Ryczko et al. 2016] reports a mammalian descending glutamate and DA path from Snc to M.ppn, portions of this descending path appears to be evolutionarily conserved. The DA appears to be an effort boost, increasing downstream activity, but most of the activity is glutamate.

Braitenberg navigation

[Braitenberg 1986] vehicles are a thought experiment for simple circuits to implement approach and avoid navigation. In the original, the vehicles have two light-detection sensors connected to drive wheels. Depending on the connection topology, sign and thresholds, the simple circuits can implement multiple behaviors.

Braitenberg vehicles for approach and escape
Braitenberg circuits for approach and escape.

A circuit that combines the output of approach and avoid circuits with some lateral inhibition can implement both approach and avoidance with avoidance taking priority. In the essay simulation, if the animal touches a wall, it will turn away from the obstacle, temporarily ignoring any odor it might be following.

Circuit for obstacle avoidance and food approach for simulated slug.
Circuit for combined odor approach and touch obstacle avoidance.

Mammalian locomotion appears to use a similar circuit between the superior colliculus (OT – optic tectum) and the motor driving B.rs neurons [Isa et al. 2021]. This circuit pattern implies that approach and avoidance are separate behaviors, only reconciled at the end. For example, a punishing reinforces that increases avoidance is not simply the mirror image of a non-reward that decreases approach. The two reinforcers modify different circuits.

“What” path vs “where” path

The mammalian visual system has separate “what” and “where” paths. One path detects what object is in focus, and one path keeps track of where the object location is. This division between object decision and navigation has been useful in the simulation, because navigation details are quickly lost in the circuit when deciding what to do with an odor.

“What” and “where” paths as configuring a switchboard.

When an animal senses an odor, say a food odor, the animal needs to identify it as a food odor, decide if the animal is hungry or sated, and decide if there’s a higher-priority task. All that processing and decision can lost the fine timing phase and amplitude details needed for precise navigation. Gradient following, for example, needs fine differences in timing or amplitude to decide whether to turn left or right. By splitting the long, complicated “what” decision from the short, simple “where” location, the circuit can benefit from both.

[Cohn 2015] describes the fruit fly mushroom body as a switchboard, where dopamine neurons configure the path for olfactory senses to travel. In the context of “what” and “where”, the “what” path configures the switchboard and the “where” path follows the connected circuit.

Some odor-based navigation has a more extreme division between “what” and “where.” Following odor in water isn’t always gradient-based navigation, because odors form clumps instead of gradient plumes. Instead of following a gradient, the animal moves against the current toward the odor source. In that latter situation, the “where” path uses entirely different senses for navigation, using water flow mechanosensors, not olfactory sensors [Steele et al. 2023].

Navigation against current toward an odor plume.

The diagram above illustrates a food-searching strategy for some animals in a current, both water and air. In water, the current is more reliable for navigation than an odor gradient. When there’s no scent, the animal swims back and forth across the current. When it detects a food odor, it swims against the current. If it loses the odor, it will return to back and forth swimming. In this navigation type, entirely different senses drive the “what” and “where” paths.

Foraging and give-up time

Giving up is an essential part of goal-directed behavior. If an animal cannot ever give up, it will be stuck on the goal without escaping. In the context of foraging, the give-up time is optimized with the marginal value theorem [Charnov 1976], suggesting that an animal should move to another patch when its current reward-gaining rate drops below the average rate for the environment. Animal behavior researchers like [Kacelnik and Brunner 2002] have observed animals roughly following this theorem, although using simpler heuristics.

In more complex animals, the failure to give up can be pathological, such as psychological perseveration.

Odor-following state diagram including give-up timer.
Foraging state diagram illustrating the give-up timer

The give-up circuit needs some kind of internal timer or cost integrator, and a way to cancel the task. In this essay’s model, the lateral habenula (Hb.l) computes the give-up time or integrates the cost, and it cancels the task by suppressing the locomotive signal through Snc.

Habenula as a give-up circuit

Hb.l is positioned to act as a give-up circuit. It receives cost signals as non-rewarded bouts or as aversive events. [Stephenson-Jones et al. 2016] interprets the Hb.l input, P.hb (habenular-projecting pallidum), as evaluating action outcome. Hb.l can suppress both the midbrain dopamine and midbrain serotonin areas. In learned helplessness situations or depression, Hb.l is hyperactive [Webster et al. 2020], causing reduced activity.

Habenula circuit as a give-up mode in a locomotive circuit.

[Hikosaka 2012] suggests the habenula’s role as suppressing motor activity under aversive conditions, a role evolved from its close relationship to the pineal gland’s circadian scheduling.

In a review article, [Hu 2020] discusses the suppressive effects of the habenula, also remarking on its role as a reward-prediction error. In particular, noting that H.l (lateral hypothalamus) to Hb.l is aversive. The Hu article also notes that Hb.l knock-out abolishes the error signal from reward omission, not an error signal from aversive (shock or obstacles).

Once the threshold is crossed, the Hb.l to Snc signal produces behavioral avoidance, reduced effort and depressive-like behavior from learned helplessness. The Hb.l is the only brain area consistently hyperactive in animal models of depression.

Note, since this essay’s simulation is a non-learning behavioral model, the only “prediction” possible is an evolutionary intrinsically-attractive odor, and the only role for an error is giving up the current behavior. Here, I’m interpreting the H.l to Hb.l signal as a cost signal, integrated by Hb.l, that gives up when it crosses a threshold.

Vertebrate reference

For reference, here’s a functional model of the vertebrate brain.

Functional model of vertebrate brain.

The areas in this model cluster around the hindbrain isthmus divider. B.rs are hindbrain neurons near the isthmus. M.lr (M.ppn) are midbrain neurons that migrate from the hindbrain (r1) to the midbrain. Snc is the midbrain tegmental area (the V – value area), near the isthmus, and contiguous with M.ppn. Similarly the H.l area that projects to Snc is contiguous with it. The habenula is the most distant area, located above the thalamus near the pineal gland (not in the diagram as a simplification, but associated with the pallidum areas.) So, the areas discussed here are a small part of the entire brain, but interestingly clustered around the isthmus divider near the cerebellum.

Minimal viable straw man

I think it’s important to remember that the essay simulations are an engineering project not a scientific one. One difference is that the simulations necessary require decisions beyond science. Another difference is that the project needs a simple core that may not correspond to any evolutionary animal. For example, even simple animals have some rudimentary vision, if only two or three pigment spots. For another, learning centers like the mushroom body. And dealing with internal biological issues like breathing and blood pressure with motion.

This model in particular is more of a straw man or minimal viable product than an actual proposal for an ancestral proto-vertebrate mind. The model is intended to be a straw man, a target that might give a base framework to criticize or build on.

Alternative olfactory paths

Another potential “what” path for innate behavior goes through the medial habenula, which is responsive to odors and produces place avoidance [Amo et al. 2014], but [Chen et al. 2019] suggests it also supports attraction for food odors.

Olfactory innate path through habenula. Key: A.co cortical amygdala, H.l lateral hypothalamus, Hb habenula (medial and lateral), IPN interpeduncular nucleus, M.pag periaqueductal gray, Ob olfactory bulb.

In mammals, the olfactory path to H.l goes through the cortical amygdala (A.co) [Cádiz-Moretti et al. 2017]. While this essay is deliberately omitting the cortex, in the lamprey the olfactory path goes through the lateral pallium (LPa, corresponding to mammalian O.pir piriform cortex) to the posterior tubercular (Snc in mammals.)

For this essay, I’ve picked the Ob to Snc path instead of the alternatives for simplicity. The habenula path is very tempting, but would require exploring the IPN and serotonin (5HT) paths to the MLR, which is more complicated than a “what” path through H.l

Subthalamic nucleus as give-up circuit

The sub thalamic nucleus (H.stn) is associated with a “stop” action, stopping downstream motor actions, either because of a new, surprising stimulus, or from higher-level commands. Since a give-up signal stops the seek goal, the stop action from H.stn might play a part in the control

H.stn stop is in parallel to habenular give-up. Key: H.l lateral hypothalamus, H.stn subthalamic nucleus, Hb.l lateral habenula, M.lr midbrain locomotor region, Snc substantia nigra pars compacta, Snr substantia nigra part reticulata.

H.stn is believed to have a role in patience in decision making [Frank 2006] and in encoding reward and cost [Zénon et al. 2016], which is very similar to the role of the habenula, and H.stn projects to Hb.l via P.hb habenula-projecting pallidum.

However, the H.stn’s patience is more related to holding off (stopping) action before making a decision, related to impulsiveness, while the give-up circuit is more related to persistence, continuing an action. So, while the two capabilities are related, they’re different functions. Since current essay simulation does not have patience-related behavior arrest but does need a give-up time, the habenula seems a better fit.

Serotonin inhibition path

In zebrafish, the habenula inhibits the dorsal raphe (V.dr, serotonin neurons) but not Snc or dopamine [Okamoto et al. 2021]. The inhibition works through V.dr to the Snc/posterior tubuculum to the locomotive regions.

As with the alternative olfactory paths, this serotonin inhibition path may be more evolutionary primitive, but would add complexity to the essay’s model, so will be held off for later exploration.

Conclusions

As mentioned above, the purpose of this model is a basis for the current essay’s simulation, and as a straw man to focus alternatives to see if there might be a better minimal model.

References

Amo, Ryunosuke, et al. “The habenulo-raphe serotonergic circuit encodes an aversive expectation value essential for adaptive active avoidance of danger.” Neuron 84.5 (2014): 1034-1048.

Beauséjour PA, Zielinski B, Dubuc R. Olfactory-induced locomotion in lampreys. Cell Tissue Res. 2022 Jan

Braitenberg, V. (1984). Vehicles: Experiments in synthetic psychology. Cambridge, MA: MIT Press. “Vehicles – the MIT Press”

Cádiz-Moretti B, Abellán-Álvaro M, Pardo-Bellver C, Martínez-García F, Lanuza E. Afferent and efferent projections of the anterior cortical amygdaloid nucleus in the mouse. J Comp Neurol. 2017 

Charnov, Eric L. “Optimal foraging, the marginal value theorem.” Theoretical population biology 9.2 (1976): 129-136.

Chen, Wei-yu, et al. “Role of olfactorily responsive neurons in the right dorsal habenula–ventral interpeduncular nucleus pathway in food-seeking behaviors of larval zebrafish.” Neuroscience 404 (2019): 259-267.

Cohn R, Morantte I, Ruta V. Coordinated and Compartmentalized Neuromodulation Shapes Sensory Processing in Drosophila. Cell. 2015 Dec 17

Derjean D, Moussaddy A, Atallah E, St-Pierre M, Auclair F, Chang S, Ren X, Zielinski B, Dubuc R. A novel neural substrate for the transformation of olfactory inputs into motor output. PLoS Biol. 2010 Dec 21

Frank, Michael J. “Hold your horses: a dynamic computational role for the subthalamic nucleus in decision making.” Neural networks 19.8 (2006): 1120-1136.

Hikosaka, Okihide. The habenula: from stress evasion to value-based decision-making. Nature reviews neuroscience 11.7 (2010): 503-513.

Hu, Hailan, Yihui Cui, and Yan Yang. “Circuits and functions of the lateral habenula in health and in disease.” Nature Reviews Neuroscience 21.5 (2020): 277-295.

Isa, Tadashi, et al. “The tectum/superior colliculus as the vertebrate solution for spatial sensory integration and action.” Current Biology 31.11 (2021)

Kacelnik, Alex, and Dani Brunner. “Timing and foraging: Gibbon’s scalar expectancy theory and optimal patch exploitation.” Learning and Motivation 33.1 (2002): 177-195.

Okamoto H, Cherng BW, Nakajo H, Chou MY, Kinoshita M. Habenula as the experience-dependent controlling switchboard of behavior and attention in social conflict and learning. Curr Opin Neurobiol. 2021 Jun;68:36-43. doi: 10.1016/j.conb.2020.12.005. Epub 2021 Jan 6. PMID: 33421772.

Ryczko D, Cone JJ, Alpert MH, Goetz L, Auclair F, Dubé C, Parent M, Roitman MF, Alford S, Dubuc R. A descending dopamine pathway conserved from basal vertebrates to mammals. Proc Natl Acad Sci U S A. 2016 Apr 26

Steele TJ, Lanz AJ, Nagel KI. Olfactory navigation in arthropods. J Comp Physiol A Neuroethol Sens Neural Behav Physiol. 2023

Stephenson-Jones M, Floros O, Robertson B, Grillner S. Evolutionary conservation of the habenular nuclei and their circuitry controlling the dopamine and 5-hydroxytryptophan (5-HT) systems. Proc Natl Acad Sci U S A. 2012 Jan 17;109(3)

Stephenson-Jones M, Yu K, Ahrens S, Tucciarone JM, van Huijstee AN, Mejia LA, Penzo MA, Tai LH, Wilbrecht L, Li B. A basal ganglia circuit for evaluating action outcomes. Nature. 2016 Nov 10

Suryanarayana SM, Pérez-Fernández J, Robertson B, Grillner S. Olfaction in Lamprey Pallium Revisited-Dual Projections of Mitral and Tufted Cells. Cell Rep. 2021 Jan 5

Verschure PF, Pennartz CM, Pezzulo G. The why, what, where, when and how of goal-directed choice: neuronal and computational principles. Philos Trans R Soc Lond B Biol Sci. 2014 Nov 5

Webster JF, Vroman R, Balueva K, Wulff P, Sakata S, Wozny C. Disentangling neuronal inhibition and inhibitory pathways in the lateral habenula. Sci Rep. 2020 May 22

Zénon A, Duclos Y, Carron R, Witjas T, Baunez C, Régis J, Azulay JP, Brown P, Eusebio A. The human subthalamic nucleus encodes the subjective value of reward and the cost of effort during decision-making. Brain. 2016 Jun;139(Pt 6):1830-43.

Essay 16: Learning to Ignore

Essay 16 extends the odor seeking (chemotaxis) of essay 15 by adding a single memory item. The memory caches a failed odor search, avoiding the cost of searching for false odors. The neuroscience source is the fruit fly Drosophila. The simulation is still based on a Braitenberg slug with distinct circuits for chemotaxis and for obstacle avoidance.

Mushroom body model

The fruit fly mushroom body (MB) is the learning center. MB is a modulating system: if it’s knocked out, the fruit fly behaves normally, although with only intrinsic, unlearned behavior. Essay 16 focuses on a single MBON short term memory (STM) output neuron, which may specifically be the γ2 neuron.

Architecture of fruit fly mushroom body.
Mushroom body architecture, adapted from [Aso et al. 2014]

For simplicity and focus, essay 16 isn’t implementing the KC yet. Instead, the γ2 MBON receives its input directly from a small number of odor projection neurons (PN) that was implemented in essay 15. Essentially, the input is a small set of primitive odors, where the full KC is a massive combinatorial odor spectrum.

Candidate odors from evolution

To motivate why evolution might develop learning, consider the food-seeking slug from essay 15. Since the food odor in essay 15 perfectly predicted food, there was no reason to learn anything about food. The simulation’s “evolution” has perfectly solved the artificially perfect world, selecting exactly those odors needed to find food.

Choosing the right set of candidate odors is a dilemma for evolution. Too many candidates means wasted search time. Too few candidates avoids wasted time, but misses out on opportunities, which may be a smaller problem than too many candidates because the animal can fall back to random, brownian-motion search. This beast against including semi-predictive odors might mean that early Precambrian evolution might only favor the highest predictors and skip semi-productive odors.

Candidate odor surrounded by distractors

The preceding image represents odors potentially available to the slug from an evolutionary design perspective. If the beige color is the only candidate for food, the slug will ignore the blue-ish colors because it never senses the odor. There’s no need for a circuit or behavior to distinguish the two. For the animal, those odors don’t exist.

Food odors don’t perfectly predict food, either because of lingering odors or simply candidate odors that aren’t always from nutritious food. For example, the fruit fly can taste sweet and it can also sense nutrition from a rest in blood sugar. That distinction between sweet and nutritious is reflected in the mushroom body with specific neurons for each [Owald et al. 2015].

Classical association

For this essay, let’s explore what could be the most trivial memory, in the context of the fruit fly MB. The MB output has only 24 neurons in 15 distinct compartments per hemisphere. Each compartment appears to have specialized roles, such as short term memory (STM) vs long term memory (LTM) [Bouzaiane et al. 2015], and water seeking compared to sugar seeking [Owald et al. 2015].

Although learning studies typically use classical association (Pavlovian) terminology, where a conditioned stimulus (CS) like the food odor becomes associated with an unconditioned stimulus (US) like consuming food, I don’t think that framing is useful for the odor-seeking behavior of the simulated slug.

Naive animal missing food before classical training

In the diagram above, which follows the classical model, the animal (arrow) missing the food (brown square) despite being in the candidate odor’s plume because it hasn’t learned the associate the odor (CS) with the food (US). It only learns the association if it finds the food through brownian random search. Even then, if it randomly hits another food source with a different odor, it will forget the first, limiting the gain from this learning.

Even the non-learning algorithm of essay 14 performs better, because naive searching of all candidate odors is relatively successful, even if slightly time inefficient. Behaviorally, the difference is between default-approach or default-ignore. Default-approach needs to learn to ignore and default-ignore needs to learn to approach.

Learning to ignore

Learning to ignore is an alternative to the classical associative way of looking at the problem. It’s not an argument against classical conditioning in total, but it is a different perspective that highlights different features of the problem.

Successful approach to food and failure

The diagram above shows a successful approach to food and an unsuccessful approach. Both candidate odors potentially signal food because evolution ignores useless odors, but in this neighborhood the reddish odor is a non-food signal. As in essay 14, habituation will rescue the animal from perseveration, spending infinite time exploring a useless odor, but once an odor is found useless, ignoring it from the start would improve search efficiency.

Since odors in a neighborhood are likely similar, the animal is likely to encounter the useless odor soon. So, remembering a single item, like a single item cache, will improve the search by avoiding cost, until the animal reaches an area that does have nutritious food. The single-item cache lets the animal ignore patches of non-predictive odors.

Single item cache (short term memory)

A single mushroom body output neuron (MBON) and its associated dopamine neuron (DAN) can implement a single item cache by changing the weights of the KC to MBON synapses with long-term depression (LTD). Following the previous discussion, since it’s more efficient to remember the last failure than the last success, the learning is LTD at the synapse between the odor and the MBON. In fruit flies, short term memory (STM) is on the order of 2h. (For a fuller discussion between “short” and “long” term see [Sossin 2008].)

Reduced MB circuit for negative learning

In the above diagram, the O2 synapse with a ball represents the LTD cache item. If the animal senses odors for either O1 or O3, it approaches the odor. If it senses O2, it ignores the odor because of the LTD at the synapse. (The colors follow the mnemonic model. Purple represents primary sensor/odor, and blue represents apical/limbic/odor and motivation areas.)

The DAN needs to implement a failure signal to implement LTD, which is actually relatively complicated. Unlike success, which has an obvious direct stimulus when finding food, failure is ambiguous. How long the animal should persist before giving up is a difficult problem, and at very least requires a timer even for the simplest strategy. Because habituation already implements a timeout, an easy solution is to copy the circuit or possibly use its output. So, if the animal exists the odor plume because of habituation, the DAN might signal failure.

Another possibly strategy is for the DAN to continuously degrade the active signal as in habituation, and only rescue the synapse when discovering food. Results from [Berry et al. 2018] show the needed degrading over time (ramping LTD) in their study of MBON-γ2, although that study didn’t explore the rescuing of approach by finding food that we seed.

So, the second strategy might require a second, opposing neuron, which I’ll probably explore later. For this essay, the DAN will produce a failure signal on timeout and a success signal on finding food, something like a reward prediction error signal from reinforcement learning [Sutton and Barto 2018], but without using a reinforcement learning architecture.

Mammalian correlates

In mammals, the KC/MBON synapse with DAN modulation circuit functionally resembles the hippocampus CA3 to CA1 synapse (E.hc, E.ca1, E.ca3) with locus coerulus (V.lc). In mammals, V.lc is known as the primary source of noradrenaline, associated with surprise and orientation, but it also contains dopamine neurons, and strongly innervates the hippocampus.

[Aston-Jones and Cohen 2005] discuss the locus coeruleus involvement in decision-making, specifically in explore vs exploit decision. If time passes and exploitation continues to fail by not finding food, V.lc signaling encourages moving on and exploring different options, a behavior similar to ours.

The E.ca3 to E.ca1 connection (and E.ec, entorhinal cortex) is believed to detect novelty, and V.lc is active during exploration, Like the fruit fly MBON, the hippocampus uses LTD to learn a new place, using V.lc signal [Lemon et al. 2012] like the DAN.

In contrast with my simulated slug, since the E.hc novelty output doesn’t directly drive food approach, because the mammalian brain is far more complex and abstract, the comparison isn’t exact, but it is an interesting similarity.

Simulation: shared habituation

In essay 15, the simulated slug approached an intrinsically attractive odor to find food, but needed a habituation circuit to avoid perseveration. The fruit fly LN1 neurons between the ~50 main olfactory sensory neurons (ORN) and the ~150 olfactory projection neurons (PN) implement primary olfactory habituation. In this essay, I’m essentially adding a second odor to the system. Although the fruit fly has separate habituation circuits for each of the 50 primary odors, it’s interesting to see what a shared habituation circuit might look like in the simulation.

Shared habituation pre-learning circuit

The simulation heat map shows the animal spends much of its time between the odor plumes because the habituation timeout keeps refreshing, despite encountering different odors. While habituation is active, the animal doesn’t approach either odor plume, but mostly moves in the default semi-random pattern. Only when habituation times out will it approach a new odor.

Shared habituation. Warm colors are food-predicting odors, blue are distractors.

Split habituation

The fruit fly has a split habituation unlike the previous simulation. Each primary odor has an independent habituation circuit, which is synapse specific.

Split habituation in pre-learning circuit

In the simulation of split habituation, the animal spends more time investigating the odors because each new odor has its own habituation timeout. It can move from a failed odor and immediately explore a new odor.

Simulation heat map for split habituation

Although the animal spends much of its time exploring the distractor candidate odors, it’s still a big improvement over random search, because it’s more likely to find food instead of a near miss.

Single distractor

Since a single distractor exactly fits the single item cache, it’s unsurprising that adding the cache immediately solves the distractor problem. In the following heat map, the animal only explores the successful candidate odor and ignores the distractor.

Simulation heat map for single-item negative cache

Multiple odors and distractors

Multiple distractor odors is more interesting for a single item cache because it introduces miss-rate as a prominent issue and allows comparison between negative caching and positive caching (classical association). The table below is a summary of feeding time as a success metric for each strategy.

AlgorithmFeeding time
No odor approach0.8%
No learning7.4%
LTD (cache)8.0%
LTP (classical)5.7%
Success comparison for multiple algorithms, measured by feeding time

No odor approach

As a baseline, the first simulation disables all odor approach. The animal only reaches food when it runs into it randomly. While it’s above the food, the animal will slow, improving its efficiency somewhat. This strategy was explored in essay 14, and resembles the feeding of Trichoplax in [Smith et al. 2015].

Simulation heat map for odor-ignoring animal.

As the heat map shows, this strategy is pretty terrible. Because the animal only finds food by randomly crossing it, its success rate is purely a matter of the area covered by food. Although this strategy may have been effective with Precambrian bacteria mats, where finding food isn’t an issue, it’s a problem when finding food is a necessary task.

No learning

Intrinsic chemotaxis is important as a baseline for the learning strategies. In the fruit fly intrinsic odor approach behavior is in the lateral horn. When the MB is disabled, the lateral horn continues to approach odors.

Simulation heat map for non-learning odor-seeking animal.

As the above heat map shows, intrinsic odor approach is a vast improvement over non-chemotaxis, improving food time from 0.8% to 7.4% in this environment.

Negative caching (LTD)

The single item caching that’s the focus of this post improves the food time from 7.4% to 8% by avoiding some of the time spent on non-food odors. The difference isn’t as dramatic as adding odor approach itself, but it’s an improvement.

Simulation heat map for single-item negative cache

In this strategy, the animal remembers the last failure odor, and ignores the odor plume the next time it reaches it. The animal explores all other odors, including failure odors. On a cache miss (failure), the animal remembers the new failure and forgets the old one.

Classical conditioning (LTP)

The next strategy tries to simulate what classical conditioning might look like if it was used for behavior. In this simulation, the animal only follows the odor after it’s associated with food, which means the animal needs to randomly discover the food first.

Simulation heat map for single-item classical association learning

This strategy is actually worse than the non-learning case, because it only finds one food source at a time. Although the heat map shows both being visited, the areas are actually alternating. One source is the only found food for a long time until the other is randomly discovered, when the roles switch and the first is now ignored.

Simulation limitations

I think it’s important to point out some simulation limitations, particularly since I’ve added performance numbers for comparison. The simulation environment and timings can affect the numbers dramatically. For example, the odor plume size dramatically affects the classical conditioning algorithm. If finding food without following the odor is difficult, the classical conditioning animal will have great difficulty finding a new odor.

Specifically, if the gain from following the odor is large, then classical conditioning will always have a penalty, because it loses out on that gain until it makes its association. In contrast an explore-first strategy will always gain the odor-exploring advantage. If the gain of explore-first outweighs its cost, then a non-learning explore-first will win against associative learning.

cost_cache = p_miss * cost_miss + (1 - p_miss) * cost_hit
cost = cost_cache + cost_noncache

Consider the rough cache cost model above to see some of the issues with the negative cache. If the non-cacheable cost greatly outweighs the cache miss cost, then it doesn’t matter if the animal learns to avoid irrelevant odors. Contrariwise, if the miss cost is very large, then the miss rate is critical.

In addition, the miss rate is highly dependent on spatial and temporal locality. If similar odors are tightly grouped, even a small cache will have a low miss rate. But if there are many different distractor types spread randomly, the cache will miss most of the time.

Links

References

Aso Y, Hattori D, Yu Y, Johnston RM, Iyer NA, Ngo TT, Dionne H, Abbott LF, Axel R, Tanimoto H, Rubin GM. “The neuronal architecture of the mushroom body provides a logic for associative learning.” Elife. 2014

Aston-Jones, Gary, and Jonathan D. Cohen. “An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance.” Annu. Rev. Neurosci. 28 (2005): 403-450.

Berry JA, Phan A, Davis RL. “Dopamine Neurons Mediate Learning and Forgetting through Bidirectional Modulation of a Memory Trace.” Cell Rep. 2018

Bouzaiane E, Trannoy S, Scheunemann L, Plaçais PY, Preat T. “Two independent mushroom body output circuits retrieve the six discrete components of Drosophila aversive memory.” Cell Rep. 2015 May 26;11(8):1280-92.

Lemon N, Denise Manahan-Vaughan, “Dopamine D1/D5 Receptors Contribute to De Novo Hippocampal LTD Mediated by Novel Spatial Exploration or Locus Coeruleus Activity, Cerebral Cortex,” Volume 22, Issue 9, September 2012, Pages 2131–2138.

Owald D, Felsenberg J, Talbot CB, Das G, Perisse E, Huetteroth W, Waddell S. “Activity of defined mushroom body output neurons underlies learned olfactory behavior in Drosophila“. Neuron. 2015 Apr 22

Smith CL, Pivovarova N, Reese TS. “Coordinated Feeding Behavior in Trichoplax, an Animal without Synapses.” PLoS One. 2015 Sep 2

Sossin, Wayne S. “Defining memories by their distinct molecular traces.Trends in neurosciences 31.4 (2008): 170-175.

Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction. MIT press.

Essay 15: Odor Navigation

Since essay 15 is exploring the fruit fly mushroom body, which is olfactory-focused, the simulated animal needs odor-based behavior, following attractive odors toward food. The simulation assumes an odor gradient navigation system without implementing the details. Basically, it will cheat and assume a direction vector toward the odor, because the details don’t seem to matter for the mushroom body.

In reality, the animal should use a timed gradient calculation with a single sensor (klinotaxis) or a directional navigation with multiple lateral sensors (tropotaxis). Simple animals use either navigation technique, and even bacteria can use klinotaxis with a run-and-tumble strategy.

Odor plumes in water

The gradient itself is a big oversimplification for a marine slug as used in essay 15, because odors in water don’t have a simple gradient but clump instead. [Steele et al. 2023]. Since the odor plumes, clumps and filaments drive on the waters current, following the current upstream is a more effective than computing gradients.

To following a clumped odor plume in a water current, animals move upstream, against the flow toward the source. The navigation is based on the current flow mechanosensors, not an odor gradient. The odor sensing merely enables current following, which is an interesting circuit between chemosensory and mechanosensory circuits. Odor detection provides timing and go/no-go while the mechanosensory circuit navigates, somewhat like the “what” vs “where” split in the visual cortex.

In the diagram above, the odor control of the contra-flow navigation is inhibitory, a common pattern in vertebrate brain. For example, the striatum complex (basal ganglia) tonically inhibits its output, including midbrain locomotion or optic tectum. When an action is selected, the striatum disinhibits the midbrain command neurons. Despite the complication of disinhibition – double inhibition – the system improves signal noise.

When an inhibitory neuron disables a command, the added noise doesn’t matter because the behavior is disabled, and the extra control signal noise doesn’t harm the command. When the inhibitory control is taken away, the system has clean, undisturbed sensory data. As a contrast, in an excitatory system where the odor sensor positively excited the command, the odor control signal would add noise to the mechanical sensors, reducing precision. So, despite the extra complication of a double-negative inhibitory system, it’s behaviorally superior.

Essay 15 relevance

Although this odor navigation probably won’t be part of the essay 15 simulation, I think it’s important to describe what’s left out when simplifying a model. If the simulation becomes too simplified, it can lose the essence of the behavior. The simplification is necessary to keep the model uncluttered and focused, but the dividing line is a judgement call.

References

Steele TJ, Lanz AJ, Nagel KI. “Olfactory navigation in arthropods.” J Comp Physiol A Neuroethol Sens Neural Behav Physiol. 2023 Jul;209(4):467-488. doi: 10.1007/s00359-022-01611-9. Epub 2023 Jan 20. PMID: 36658447; PMCID: PMC10354148.

Powered by WordPress & Theme by Anders Norén