Attempting a toy model of vertebrate understanding

Month: January 2024

Essay 27: Feeding State Machine

Essay 27 returns to feeding, which essay 23 had an earlier sketch of. While the animal in earlier essays could eat while moving, like snails and worms, this essay will add the requirement of stopping before eating, which requires extra control mechanisms to manage the state transition.

A filter feeder like amphioxus, a non-vertebrate chordate that may hint at pre-vertebrate feeding, might move to find a better feeding zone, but then settles down as a static filter feeder. Tunicates, which are more closely related to vertebrates settle down permanently as adults and dissolve their brain as no longer needed. Because I want to keep the essay simple, I’m imaging something more like licking, which is more studied in rodents, as opposed to a more alien filter feeding. The main problem for the essay to introduce locomotion and eating as distinct actions.

As a contrast to further explore the idea of states and state transitions, the essays also explores the transition between roaming and dwelling: global wide-ranging search vs area restricted search. Roaming and dwelling are more amorphous motivational states as opposed to the strict motor division between moving and eating.

Feeding states

Below is a more detailed diagram of the foraging and feeding states, revolving around the core foraging task. The animal passively roams until is finds an odor cue for a food target, which starts a seek to the target. If it finds food, the animal sops and eats.

In this model, the roam state and dwell state can be separate from seeking a target, depending on the animal’s environmental niche. A seek can start in a roam state or a dwell state, and seek cues may or may not initiate dwell state. For example, dwell state might only start when the animal eats nutritious food, indicating that food is nearby.

Feeding state diagram for the essay. ach (acetylcholine neurotransmitter) agrp (hunger peptide), ARS (area restricted search), cgrp (alarm/bitter taste peptide), da (dopamine), glp-1 (satiety/sickness peptide), ox (orexin wakefulness/action peptide), set (somatostatin peptide), V.dr (dorsal raphe), 5HT (serotonin)

The diagram includes important failure states. If seeking fails, the animal gives up and leaves the area, and must ignore the last cue to avoid perseveration. If the taste is bitter or toxic, the animal rejects the food. For now, I’m postponing longer failure states like the food lacking nutritional value or causing food poisoning.

To avoid perseveration, seeking the failed cue forever, the avoid state moves the animal away from the failed cue and ignores seek cues. A more sophisticated brain could remember the failed cue for a short time, but the current essays lack short term memory.

Eating here means specifically licking or filter feeding. I’m being precise here because the simulation requires it, and more vague neuroscience terms like “reward” are often unclear about exactly what it’s relation to actual eating are.

The connection between the dwell state and serotonin is from [Flavell et al 2013], [Ji et al 2021] which founds serotonin marking the dwell state in the flatworm C. elegans, and [Marques et al 2020] finding serotonin for a zebrafish dwell (“exploit”) state.

Roaming and dwelling

Food search phases have multiple strategies, broadly divided into roaming and dwelling. Roaming is a broader, more general search without a specific area or target. Dwelling or ARS (area restricted search) is slower, with tighter turning, where the current area is believed to be more likely to have food. [Horstick et al 2017] describes dwell as four properties: reduction in travel distance, increased change in orientation, increased path complexity, and a directional bias.

For this essay, dwelling is a motivational drive not a motor command, meaning it can overlap with other motivations and doesn’t provide a strict action state requirement. For example, dwell isn’t required to seek a target, which can occur in the roaming state, for example in C. elegans [Ji et al 2021].

In the C. elegans the dwell state is associated with serotonin and the roam state with PDF (pigment dispensing factor) [Flavell et al 2013]. In zebrafish the dwell state is associated with V.dr (dorsal raphe) serotonin [Marques et al 2020], the roam state is associated with SST (somatostatin peptide) [Horstick et al 2017]. While arousal isn’t quite the same as well, [Lovett-Barron et al 2017] found SST as a low-arousal marker, while CART, ACh (acetylcholine), NE (norepinephrine), serotonin, dopamine and NPY (neuropeptide Y) as signs of high arousal.

Triggers for the dwell state depend on the animal’s species [Dorfman et al 2020]. In C. elegans, which feeds on bacteria, nutritional feedback extends the dwell state [Ben Arous et al 2009]. In some animals a food cue triggers dwell, while in others only eating nutritious food triggers dwell. In zebrafish lack of a food cue causes H.c (caudal hypothalamus) activation decay [Wee et al 2019].

Reflexive eating

This essay models reflexive eating as a hindbrain system controlled by B.pb (parabrachial nucleus) with downstream motor and sensory in B.nts (nuclei tractus solitarius), M.mdd (reticular medulla), and B.3g (trigeminal – orofacial sensorimotor). The simulation isn’t as detailed, treating the hindbrain eating as a single low-level module.

Hindbrain modules involved in reflexive eating. B.3g (trigeminal), B.mdd (reticular medulla), B.nts (nucleus tractus solitarius), B.pb (parabrachial nucleus).

This innate circuit can with without input from higher areas [Watts et al 2022]. For example if rodents lack any dopamine, they won’t move or eat and will starve even if food is near them. However, if food or water is placed at their lips, which activates the innate circuit, the rodents will eat [Rossi et al 2016].

The B.pb area also processes sweet, bitter or salt, and can reject food without requiring higher areas. The higher areas modulate B.pb behavior, such as suppressing B.pb’s innate rejection of sour when drinking lemonade.

Because the B.pb innate eating and the MLR (midbrain locomotor region) are independent, some system much coordinate switching between moving and eating.

The illusion of state machine atomicity

The feeding state diagram suggests a simple atomic transition from seeking food to eating the food, but this transition needs management from some neural circuits. For example, when braking during driving, drivers need to pay attention to the stopping distance. Braking stops a car, but the state transition isn’t a simple atomic transition. For this essay’s eating task, some neural circuit must keep track of the animal’s stopping after seeking and only allow eating when locomotion has stopped.

State transition from seeking to eating, emphasizing the stopping state. H.pstn (parasubthalamic nucleus), H.stn (subthalamic nucleus).

H.stn (subthalamic nucleus) is involved with stopping, waiting, and switching tasks [Isoda and Hikosaka 2008]. Since H.stn also receives motor efference copies via T.pf (thalamus parafascicular nucleus) and Ppt (peduncular pontine nucleus), H.stn is in a good position to manage the stopping transition and can prevent eating until the locomotion has ended. The diagrams shows H.pstn (parasubthalamic nucleus) as a parallel area for gaiting eating, following [Barbier et al 2021].

H.stn and H.pstn state transition circuit

H.stn and H.pstn are well-placed to fulfill the transitions between seeking and eating. To flesh this idea out, here’s a simplified model of the seal to eat state transition circuit.

The main action paths are horizontal: moving is from H.stn to MLR to B.rs (reticulospinal motor neurons) and eating is from H.pstn to B.nts to orofacial licking motor neurons. The rest of the circuit manages the transition between the two states.

State transition circuit for move state to eat state. B.nts (nucleus tractus solidarus), fb (feedback), H.pstn (parasubthalamic nucleus), H.stn (subthalamic nucleus), MLR (midbrain locomotor region), Snr (substantia nigra pars reticulata), T.cl (centrolateral thalamus), T.pf (parafascicular thalamus).

Control over the transition comes from S.nr (substantia nigra pars reticulata), which inhibits eating when the animal is moving, and inhibits moving while the animal is eating. To know when the animal has stopped moving, H.stn receives motor efferent copies from T.cl and T.pf (centrolateral and parafascicular thalamus, aka intralaminar). As a note, T.cl contains cerebellum output, so H.stn may receive fine-grained motor timing feedback. H.pstn receives parallel eating efferent copies from B.pb and B.nts to know when the animal has stopped eating.

This circuit has the same structure as a lateral inhibition decision circuit, but the function is about handling timing and transition, not deciding between competing options.

Note: [Shah et al 2022] suggest H.pstn is more specific to suppressing feeding for aversive situations like food poisoning or a predator threat, but not the motor control as described here.

A note on this model: the actual neural circuit isn’t as clean, parallel and logical, because evolution isn’t an intelligent designer. Furthermore, this brain region is part of the neuropeptide core, where neuropeptide broadcast-like signaling can be more important than point-to-point circuit diagrams. Specifically, the disinhibition of B.pb eating is more likely peptides from the hypothalamus, not S.nr tonic inhibition.

H.l food zone

Studies on H.l (lateral hypothalamus) show two interesting results relevant here [Jennings et al 2015]:

  • Two distinct GABA neuron populations gate eating and seeking.
  • Two distinct neuron populations are active in a food zone or outside a food zone.

The food zone neurons partially explain how H.l decides between seeking and eating. How does this animal knows when it’s reached the food? In C. elegans there are dopamine chemosensory neurons that sense when the animal passes over food bacteria, and signals the animal to slow [Sawin et al 2000]. Dopamine chemosensory neurons also signal for the animal to turn more when leaving food (dwell-like state) [Hills et al 2004]. For this essay, using B.pb and B.nts to sense nearby food seems like a reasonable simplification because the simulation animal is aquatic and aquatic taste is a chemosensory system, similar to a close-range olfaction.

Food zone modulation of seeking and eating. fz (food zone), H.l (lateral hypothalamus).

The essay uses a signal when the animal is in a food zone or not in a food zone. The food zone signal inhibits eating or seeking actions when the animal is in a non-appropriate place. The essay uses a signal from B.pb as mentioned above.

In mammals H.l receives input from more sophisticated location systems than a bare chemosensory signal, such as E.sub.d (dorsal subiculum of hippocampus), S.ls (lateral septum, which processes hippocampal output), A.bl (basolateral amygdala, highly connected to hippocampus), S.msh (medial shell striatum receiving large hippocampus input) as well as the bare B.pb as for the simulation. All these areas incorporate more complicated environmental context. When the essays start investigating environmental context, I’ll need to revisit the H.l food zone with more sophisticated input.

H.sum as driving seek

Fleshing out the drivers of the seek circuit, consider H.sum (supramammillary nucleus, aka retromammillary) and its role in exploring (roaming and seeking). [Ferrell et al 2021] study a subset of H.sum neurons that express tac1 peptide (tachykinin, aka substance-P or neurokinin). These H.sum neurons correlate highly with movement velocity, a second before the action. Since they precede action, they’re upstream in the locomotive path.

H.sum is also involved in wakefulness [Liang et al 2023], [Plaisier et al 2020], motivation [Kesner et al 2021], and specifically food motivation [Le May et al 2019], and is modulated by hunger peptides like GLP-1 [Vogel et al 2016], [López-Ferreras et al 2018].

H.sum also participates in threat avoidance [Escobedo et al 2023], but that circuit is through Poa (preoptic area) and is outside this essay, although it would be interesting if any of the downstream circuitry is shared. H.sum is also well know for its role in hippocampal theta oscillations, novelty [Chen et al 2020], temporal and spatial memory [Cui et al 2013], and social memory, although those are outside the scope of this essay.

The diagram below shows a possible explore-related path of mammalian H.sum via the tac1 neurons.

Exploration locomotion driven through H.sum. H.l (lateral hypothalamus), H.sum (supramammillary nuleus), Hb.l (lateral habenula), MLR (midbrain locomotor region), M.pag (periaqueductal gray), P.ms (medial septum), V.dr (dorsal raphe – serotonin), Vta (ventral tegmental area – dopamine)

It may be important that H.sum and Vta (ventral tegmental area) are both neighbors and H.sum includes dopamine neurons and those dopamine neurons are sometimes considered an extension of the Vta [Yetnikoff et al 2014].

The following diagram gives an extremely rough idea of the adjacency of these areas. In a smaller primitive pre-vertebrate, these might not only be neighbors but mingled earlier functionality. The diagram includes H.zi (zona incerta) because it’s a neighbor, and also because H.zi is a food-seeking area [Ye et al 2023], but I’m postponing consideration of H.zi to a future essay.

Neighbors of the lateral habenula and supramammillary nucleus. H.l (lateral hypothalamus), H.pstn (parasubthalamic nucleus), H.stn (subthalamic nucleus), H.sum (supramammillary nucleus), H.zi (zona incerta), MLR (midbrain locomotive region), Ppt (Pedunculopontine pontine nucleus), Snc (substantia nigra pars compacta – dopamine), Snr (substantia nigra pars reticulata), Vta (ventral tegmental nucleus – dopamine), ZLI (zona limitans intrathalamica).

In addition, the rostral part of Vta nearest H.sum is part of p3 in the prosomeric embryonic model, which is a source of hypothalamic cells [Kim et al 2022]. For pre-vertebrates in this essay, then, there might not be a distinct between H.sum and Vta / posterior tuberculum, particularly since the essays are currently focusing on downstream connections, not upstream dopamine to a future striatum. Zebrafish downstream dopamine circuits directly modulate locomotor movement [Ryczko et al 2020], [Reinig et al 2017]. I think it’s reasonable to simplify this circuit for now and consider H.sum as directly projecting to MLR.

State transition circuit for seek to eat

Putting these ideas together yields something like the diagram below. Like the earlier simplified diagram, horizontal paths drive core seeking and eating behavior, and other circuits manage the state transition. Seeking uses the top path from H.l to H.sum to MLR to B.rs, which produces the final locomotion. Eating uses the bottom path from H.l to H.pstn to B.nts, which controls reflexive eating.

State management circuit for seek to eat transition. B.nts (nucleus tracts solitarius), fb (feedback), fz (food zone), H.l (lateral hypothalamus), H.pstn (parasubthalamic nucleus), H.stn (subthalamic nucleus), H.sum (supramammillary nucleus), MLR (midbrain locomotor region), T.cl (centrolateral thalamus), T.pf (parafascicular thalamus).

The left contains motivational drivers. The food zone and non food zone systems restrict seeking and eating, only allowing seeking and eating in appropriate locations.

In the center H.stn and its parallel H.stn enforce a smooth transition between seeking and eating, using motor efferent copies to pause transition until active motor stops. The smooth transition creates the illusion of an atomic state transition.

As a diagram note, I’ve used red for the H.l inhibitory neurons that gate seek and eat because they’re playing the same role as Snr neurons. Technically they should be blue, if following normal essay conventions.

Modulation of eating

The eating and feeding modulation systems are complicated and overlapping, which is too detailed for this essay, but two part are interesting. First, B.pb tonically inhibits eating with the CGRP peptide to B.nts. To enable eating, H.arc (hypothalamus arcuate) disinhibits B.nts eating by sending AgRP (a hunger peptide) to B.pb [Campos et al 2016].

Modulation of reflexive eating. AgRP (a hunger peptide), B.nts (nucleus of the solitary tract), B.pb (parabrachial nucleus), CGRP (an anti-eating peptide), H.arc (hypothalamus arcuate).

Although the essays have used the disinhibition pattern before, the pattern has generally ben GABA disinhibition, while this feeding disinhibition uses peptide signaling. As mentioned above, there are many feeding-related peptides that inhibit, excite, and modulate the feeding system without using connection based synapses.

As a parallel, a drinking modulation path goes through the basal ganglia Snr and OT (optic tectum) [Rossi et al 2016]. This path though the basal ganglia and OT coordinates anticipatory licking, while the earlier B.nts path is reflexive eating.

Control of anticipatory licking. B.mdd (medulla licking motor), OT.dl (deep, lateral optic tectum), Snr.l (lateral substantia nigra pars reticulata)

Another drinking path involves S.a (central/striatal amygdala), midbrain, and hindbrain circuits [Zheng et al 2022]. M.dp (deep mesencephalic nucleus) extends licking but doesn’t initiate it. So M.dp might extend eating after tasting. Similarly B.plc extends eating [Gong et al 2020]. S.a sst (somatostatin peptide) neurons promote eating and drinking [Kim et al 2017].

Sustained eating with an amygdala circuit. B.mdd (medulla motor eating), B.pb (parabrachial nucleus), M.dp (deep mesencephalic nucleus), S.a.sst (set-expressing neurons of the central amygdala).

Another path for tasting and eating runs through S.v (ventral striatum). [Sandoval-Rodríguez et al 2023] founds S.v directly controlling feeding using hindbrain taste input to extend eating, and using hindbrain GLP-1 (anti-eating peptide) to inhibit eating. Unlike most striatum circuits, these striatum neurons project directly to the hindbrain motor areas.

Ventral striatum taste exciting and food inhibition circuit with the hindbrain. B.ap (area postrema – nutrient sensing), B.mdd (medulla motor), B.nts (nucleus of the solitary tract), B.pb (parabrachial nucleus), Sv (ventral striatum / nucleus accumbens).

Because this essay is already complicated enough, this simulation isn’t covering all of these details. For simplicity, the simulation will use a simple continuation circuit inspired by the central amygdala and postpone other control circuits for later exploration.

Simplified eating continuation circuit with the central amygdala. B.mdd (medulla motor), B.pb (parabrachial nucleus), Sa.sst (central amygdala, sst projecting neurons)

The important point for now is that eating modulation uses multiple paths, some controlled through synaptic circuits and others through broadcast motivational peptides. The system is not one or the other, but a messy combination. To model this messiness, the simulation needs to handle both systems.

References

Barbier M, Risold PY. Understanding the Significance of the Hypothalamic Nature of the Subthalamic Nucleus. eNeuro. 2021 Oct 4.

Ben Arous J, Laffont S, Chatenay D. Molecular and sensory basis of a food related two-state behavior in C. elegans. PLoS One. 2009 Oct 23;4(10):e7584. 

Campos CA, Bowen AJ, Schwartz MW, Palmiter RD. Parabrachial CGRP Neurons Control Meal Termination. Cell Metab. 2016 May 10;23(5):811-20.

Chen S, He L, Huang AJY, Boehringer R, Robert V, Wintzer ME, Polygalov D, Weitemier AZ, Tao Y, Gu M, Middleton SJ, Namiki K, Hama H, Therreau L, Chevaleyre V, Hioki H, Miyawaki A, Piskorowski RA, McHugh TJ. A hypothalamic novelty signal modulates hippocampal memory. Nature. 2020

Cui Z, Gerfen CR, Young WS 3rd. Hypothalamic and other connections with dorsal CA2 area of the mouse hippocampus. J Comp Neurol. 2013 Jun 1;521(8):1844-66. 

Dorfman A, Hills TT, Scharf I. A guide to area-restricted search: a foundational foraging behaviour. Biol Rev Camb Philos Soc. 2022 Dec;97(6):2076-2089. 

Escobedo Abraham, Holloway Salli-Ann, Votoupal Megan, Cone Aaron L, Skelton Hannah E, Legaria Alex A., Ndiokho Imeh, Floyd Tasheia, Kravitz Alexxai V., Bruchas Michael R., Norris Aaron J. (2023) Glutamatergic Supramammillary Nucleus Neurons Respond to Threatening Stressors and Promote Active Coping eLife 12:RP90972

Farrell JS, Lovett-Barron M, Klein PM, Sparks FT, Gschwind T, Ortiz AL, Ahanonu B, Bradbury S, Terada S, Oijala M, Hwaun E, Dudok B, Szabo G, Schnitzer MJ, Deisseroth K, Losonczy A, Soltesz I. Supramammillary regulation of locomotion and hippocampal activity. Science. 2021 Dec 17;374(6574):1492-1496. 

Flavell SW, Pokala N, Macosko EZ, Albrecht DR, Larsch J, and Bargmann CI (2013). Serotonin and the neuropeptide PDF initiate and extend opposing behavioral states in C. elegans. Cell 154, 1023–1035.

Gong R, Xu S, Hermundstad A, Yu Y, Sternson SM. Hindbrain Double-Negative Feedback Mediates Palatability-Guided Food and Water Consumption. Cell. 2020 Sep 17;182(6):1589-1605.e22. 

Hills T, Brockie PJ, Maricq AV (2004) Dopamine and glutamate control area-restricted search behavior in Caenorhabditis elegans. J Neurosci 24: 1217–1225

Horstick EJ, Bayleyen Y, Sinclair JL, Burgess HA. Search strategy is regulated by somatostatin signaling and deep brain photoreceptors in zebrafish. BMC Biol. 2017 Jan 26;15(1):4. 

Isoda M, Hikosaka O. Role for subthalamic nucleus neurons in switching from automatic to controlled eye movement. J Neurosci. 2008 Jul 9;28(28):7209-18.

Jennings JH, Ung RL, Resendez SL, Stamatakis AM, Taylor JG, Huang J, Veleta K, Kantak PA, Aita M, Shilling-Scrivo K, Ramakrishnan C, Deisseroth K, Otte S, Stuber GD. Visualizing hypothalamic network dynamics for appetitive and consummatory behaviors. Cell. 2015 Jan 29;160(3):516-27.

Ji N, Madan GK, Fabre GI, Dayan A, Baker CM, Kramer TS, Nwabudike I, and Flavell SW (2021). A neural circuit for flexible control of persistent behavioral states. eLife 10. 10.7554/eLife.62889.

Kesner AJ, Shin R, Calva CB, Don RF, Junn S, Potter CT, Ramsey LA, Abou-Elnaga AF, Cover CG, Wang DV, Lu H, Yang Y, Ikemoto S. Supramammillary neurons projecting to the septum regulate dopamine and motivation for environmental interaction in mice. Nat Commun. 2021 May 14;12(1):2811.

Kim J, Zhang X, Muralidhar S, LeBlanc SA, Tonegawa S. Basolateral to Central Amygdala Neural Circuits for Appetitive Behaviors. Neuron. 2017 Mar 22;93(6):1464-1479.e5.

Kim DW, Place E, Chinnaiya K, Manning E, Sun C, Dai W, Groves I, Ohyama K, Burbridge S, Placzek M, Blackshaw S. Single-cell analysis of early chick hypothalamic development reveals that hypothalamic cells are induced from prethalamic-like progenitors. Cell Rep. 2022 Jan 18;38(3):110251.

Le May MV, Hume C, Sabatier N, Schéle E, Bake T, Bergström U, Menzies J, Dickson SL. Activation of the rat hypothalamic supramammillary nucleus by food anticipation, food restriction or ghrelin administration. J Neuroendocrinol. 2019 Jul;31(7):e12676.

Liang M, Jian T, Tao J, Wang X, Wang R, Jin W, Chen Q, Yao J, Zhao Z, Yang X, Xiao J, Yang Z, Liao X, Chen X, Wang L, Qin H. Hypothalamic supramammillary neurons that project to the medial septum modulate wakefulness in mice. Commun Biol. 2023 Dec 12;6(1):1255. 

López-Ferreras L, Eerola K, Mishra D, Shevchouk OT, Richard JE, Nilsson FH, Hayes MR, Skibicka KP. GLP-1 modulates the supramammillary nucleus-lateral hypothalamic neurocircuit to control ingestive and motivated behavior in a sex divergent manner. Mol Metab. 2019 Feb;20:178-193. 

Lovett-Barron M, Andalman AS, Allen WE, Vesuna S, Kauvar I, Burns VM, Deisseroth K. Ancestral Circuits for the Coordinated Modulation of Brain State. Cell. 2017 Nov 30;171(6):1411-1423.e17.

Marques JC, Li M, Schaak D, Robson DN, Li JM. Internal state dynamics shape brainwide activity and foraging behaviour. Nature. 2020 Jan;577(7789):239-243.

Plaisier F, Hume C, Menzies J. Neural connectivity between the hypothalamic supramammillary nucleus and appetite- and motivation-related regions of the rat brain. J Neuroendocrinol. 2020 Feb;32(2):e12829.

 Reinig S, Driever W, Arrenberg AB. The Descending Diencephalic Dopamine System Is Tuned to Sensory Stimuli. Curr Biol. 2017 Feb 6;27(3):318-333. 

Rossi MA, Basiri ML, Liu Y, Hashikawa Y, Hashikawa K, Fenno LE, Kim YS, Ramakrishnan C, Deisseroth K, Stuber GD. Transcriptional and functional divergence in lateral hypothalamic glutamate neurons projecting to the lateral habenula and ventral tegmental area. Neuron. 2021 Dec 1;109(23):3823-3837.e6. 

Ryczko D, Grätsch S, Alpert MH, Cone JJ, Kasemir J, Ruthe A, Beauséjour PA, Auclair F, Roitman MF, Alford S, Dubuc R. Descending Dopaminergic Inputs to Reticulospinal Neurons Promote Locomotor Movements. J Neurosci. 2020 Oct 28;40(44):8478-8490.

Sandoval-Rodríguez R, Parra-Reyes JA, Han W, Rueda-Orozco PE, Perez IO, de Araujo IE, Tellez LA. D1 and D2 neurons in the nucleus accumbens enable positive and negative control over sugar intake in mice. Cell Rep. 2023 Mar 28;42(3):112190. 

Sawin ER, Ranganathan R, Horvitz HR (2000) C. elegans locomotory rate is modulated by the environment through a dopaminergic pathway and by experience through a serotonergic pathway. Neuron 26: 619–631 

Shah T, Dunning JL, Contet C. At the heart of the interoception network: Influence of the parasubthalamic nucleus on autonomic functions and motivated behaviors. Neuropharmacology. 2022 Feb 15;204:108906.

Vogel H, Wolf S, Rabasa C, Rodriguez-Pacheco F, Babaei CS, Stöber F, Goldschmidt J, DiMarchi RD, Finan B, Tschöp MH, Dickson SL, Schürmann A, Skibicka KP. GLP-1 and estrogen conjugate acts in the supramammillary nucleus to reduce food-reward and body weight. Neuropharmacology. 2016 Nov;110(Pt A):396-406.

Watts AG, Kanoski SE, Sanchez-Watts G, Langhans W. The physiological control of eating: signals, neurons, and networks. Physiol Rev. 2022 Apr 1;102(2):689-813. 

Wee CL, Song EY, Johnson RE, Ailani D, Randlett O, Kim JY, Nikitchenko M, Bahl A, Yang CT, Ahrens MB, Kawakami K, Engert F, Kunes S. A bidirectional network for appetite control in larval zebrafish. Elife. 2019 Oct 18;8:e43775. 

Ye Q, Nunez J, Zhang X. Zona incerta dopamine neurons encode motivational vigor in food seeking. Sci Adv. 2023 Nov 15;9(46):eadi5326.

Yetnikoff L, Lavezzi HN, Reichard RA, Zahm DS. An update on the connections of the ventral mesencephalic dopaminergic complex. Neuroscience. 2014 Dec 12;282:23-48.

Zheng D, Fu JY, Tang MY, Yu XD, Zhu Y, Shen CJ, Li CY, Xie SZ, Lin S, Luo M, Li XM. A Deep Mesencephalic Nucleus Circuit Regulates Licking Behavior. Neurosci Bull. 2022 Jun;38(6):565-575. 

Essay 26 issues: olfactory attention

Unsurprisingly since essay 26 was a first cut at selective attention, it exposed a number of problems with both the neuroscience and the simulation model itself.

Specific give up

The current give up circuit is a global circuit, which doesn’t depend on the current stimulus. For this essay, the animal has two potential and because the give up is global, when the animal gives up, it gives up on both odors.

Global give-up circuit for olfactory seek.
Global give-up circuit for olfactory seek. H.l lateral hypothalamus, Hb.l lateral habenula, Vdr dorsal raphe, 5HT serotonin.

An improvement would be a cue-specific give up capability. When the animal gives up on odor A, it should investigate odor B. Instead it gives up on both. I need to add some mechanism to create a cue-specific give up capability.

As a possible neural analog, the adenosine receptor can work as a local give-up circuit by integrating neural activity. Since adenosine is essentially a waste produce from neural activity, long activity will accumulate adenosine. The A1 adenosine receptor detects the adenosine and inhibits activity, since it’s a Gi receptor.

Olfactory complexity and attention

The essay’s odor model is extremely oversimplified, because odor receptors are feature detectors, not molecule receptors, and odors are combinations of molecules. Since a specific odor is a combination of features, P.bf (basal forebrain) can’t be a simple winner-take-all inhibitory circuit as implemented in this essay. Instead, attention needs to be a set of features that excludes the distractor odor’s features.

Olfactory gamma and beta

Although the essay treats the olfactory bulb data as direct signals, oscillations are a major feature of the olfactory bulb. Strong odors trigger gamma (40-100Hz) signals in Omt (mitral/tufted output cells), enhanced by ACh (acetylcholine) from P.bf. Feedback from O.pir (olfactory piriform cortex) triggers beta (15-30Hz) oscillations. In addition, interactions with breathing in mammals synchronized with theta (4-10Hz). Although, in the last case, since the simulation animal is aquatic, breathing isn’t an appropriate synchronizer.

Temporal gradient seek issues

Odor seeking in essay 26 uses temporal gradient descent modulated by head direction in Hb.m (medial habenula) and B.ip (interpeduncular nucleus). The animal combines its head direction with the temporal gradient to estimate the odor direction, and it saves the result as a goal vector. As the animal turns, it can improve the direct estimate. In the phototaxis example of essay 25, the saved goal vector direction helped with intermittent data, where it could remember the light location for a few seconds.

Problems with the current odor direction. A quick switch in location incorporates data from the old direction, leading to an incorrect estimate.

However, the system as implemented in the model is extremely limited. It can’t truly triangulate to locate the odor, but can only improve the single direction. In the diagram above, the animal can only select one of the two vectors as an estimate. It can’t combine the two into a better estimate of the center. Also, in the diagram, the earlier estimate is no longer useful because the animal has moved.

Now, the issue might be purely in the simulation. If B.ip and Vdr (dorsal raphe serotonin) are calculating this kind of estimate, it’s likely their computation is better than the current simulation.

The selection is a trade off where a stronger gradient is likely a better estimate, but if the animal moves too far from the earlier sample, the old direction is no longer relevant. Since the animal lacks the sophistication of an allocentric map to resolve the discrepancy, it discards the old value.

The current implementation decays the old estimate to allow newer estimates to overwrite it even if the later gradient is weaker. Essentially the memory is like a leaky integrator, as is appropriate for placing it in the serotonin neurons and/or associate glia with short term (5s) memory as in simple zebrafish motor memory [Dragomir et al 2020].

Bayesian updates

In a future essay, it might be interesting to explore this issue to see if a simple Bayesian system could be implemented in low-complexity circuits, where stronger data would update the current model more than the current model.

Self motion and gradient vectors

When the animal is turning, the running average no longer represents a straight line. For the gradient vector, the system assumes the recent average was measured along the current head direction, but turns violate this assumption. To avoid miscalculating gradient vectors, the animal should suppress measurement during turns.

Swimming and theta

The gradient seek issues above are compounded with swimming with a fixed head. Early vertebrates would have had a fixed head like sharks, meaning that each swimming stroke would move the head from side to side. That sideways movement would affect the odor gradient and head direction.

Inconsistent head vs body direction and odor measurement while swimming with a fixed head.

A simplistic fix would take an odor gradient sample only on each swim stroke, only reporting at the stroke end for consistency and to average from the beginning of the stroke to the end. That solution would give a consistent measurement in a reasonably consistent direction, as opposed to sampling randomly in a cycle.

Log encoding vs linear encoding

For simplicity, I’e used linear encoding for signals in the essays, because the basic functional architecture remains the same, and the simulation isn’t precise enough to need more complexity. But for odors, the dynamic range between a single molecule detection and an overpowering odor doesn’t scale well with a linear representation.

In particular, the odor weight from the simple distance gradient, together with above mentioned temporal gradient issues might be better modeled with a log signal. Basically, the issue I raised above with gradient vector sampling might be more tractable with a different encoding, and log encoding might make the actual neural circuit less finicky than the current linear model.

Seek mode switching

The essay’s simulation lacks a specific mode switching circuit. In vertebrates the peptide core (hypothalamus, PAG, B.pb area) switches action modes from roaming to seek to eating to rest and sleep. These modes are motivated and depend on internal needs and scheduling impulses programmed by evolution.

References

Dragomir EI, Štih V, Portugues R. Evidence accumulation during a sensorimotor decision task revealed by whole-brain imaging. Nat Neurosci. 2020 Jan;23(1):85-93.

Essay 26: Ignoring distracting odors

I’ve been ignoring distracting cues in the previous essays for simplification. Since the simulated animal only encountered a single odor at a time, it never needed to select one and ignore the other. In essay 26, I’ll implement a very simple first approximation to ignoring distractors, using the P.bf (basal forebrain) control of the Ob (olfactory bulb) as a switchboard to let the selected odor through and inhibit the ignored distractor.

Simulated animal (triangle) encountering two odor plumes (circles).

In the diagram above, the animal (triangle) is seeking food using the purple odor cue as a gradient direction. When it encounters the distractor odor in blue, it should ignore the distractor, otherwise the two odors will mingle into an incorrect summed gradient and the animal will seek in the wrong direction [Cisek 2022].

Temporal chemotaxis

For essay 26, I’m switching chemotaxis (odor seeking) to use the apical temporal gradient search, using Hb.m (medial habenula) and B.ip (interpeduncular nucleus) like the phototaxis in essay 24. The apical system follows the chimera brain model of [Tosches and Arendt 2013], which suggests that odor senses and actions are distinct systems from bilateral tactile senses. For the essays, the shift is from a bilateral, Braitenberg-like [Braitenberg 1984] system to a modulated random walk like the bacterial tumble-and-run.

Olfactory tumble-and-run system using Hb.m and B.ip for temporal gradient direction, and B.rs for the modulated random walk. B.ip interpeduncular nucleus, B.rs hindbrain reticulospinal motor area, Hb.m medial habenula, Ob olfactory bulb.

The above diagram shows the problem with distractor odors. Because the tumble-and-run system uses a single temporal gradient, it necessarily adds both odors together for its input. The summed input goes to the Hb.m (medial habenula) and B.ip (interpeduncular nucleus) system to modulate the random walk direction.

When the animal crosses into the overlapping distractor odor, it will follow the combined signal, distracted from the original seek target. To avoid distraction, the system can either amplify the current odor A, or inhibit the distractors like odor B.

Analogy with nucleus isthmi

An earlier essay 19 also had an attention / distractor problem, with a different issue of action consistency, and used a zebrafish circuit in P.ni (nucleus isthmi) as a solution. In larval zebrafish P.ni works together with OT (optic tectum) to sustain attention on prey during a hunt [Henriques et al 2019]. P.ni is an ACh (acetylcholine neurotransmitter) and GABA (inhibiting neurotransmitter) system that both amplifies the predicted prey location and inhibits surrounding areas.

Nucleus isthmi circuit as adapted by essay 19. ACh acetylcholine, OT optic tectum, Pni nucleus isthmi.

In the above diagram for the essay 19 circuit, a simultaneous left and right touch would select one action at random and sustain that choice for subsequent movement with the P.ni positive feedback circuit. The outputs are crossed because it’s an avoidance circuit: an obstacle on the left triggers a right turn.

Importantly, the positive feedback is modulatory; it doesn’t trigger an action by itself. At a synapse level, ACh triggers mAChR (ACh metabotropic receptor, Gs stimulatory type) on the sensor axon, amplifying the sensor’s neurotransmitter release. The ACh and mAChR act as the decay timer, because they have a slow time constant on the order of a few seconds. If the sensor doesn’t stimulate the circuit, as when successfully avoiding the obstacle, the attention will decay over a few seconds, resetting the system to its original state.

A similar function applies to Ob and P.bf (basal forebrain), where P.bf acts like P.ni to sustain attention to the selected odor. “Basal forebrain” is a general name for a collection of functionally-related subcortical areas in the ventral (“basal”) forebrain, all pallidal-like (P). The specific P areas for the Ob are P.hdb (horizontal diagonal band) and Po.me (magnocellular preoptic area), but I’ll use P.bf for simplicity.

Olfactory bulb as a switchboard

In this model, Ob acts like a switchboard controlled by P.bf. P.bf selects attended odor paths in Ob, where Ob either passes the odor signal to its destination or inhibits the signal if it’s a distractor. P.bf opens and closes gated circuits in Ob.

Although the architecture of the Ob and P.bf circuit resembles the P.ni circuit, Ob appears to rely more heavily on inhibitory GABA for the gating operation, although ACh is also important [Böhm et al 2020], [de Saint Jan et al 2020], [Nunez-Parra et al 2000]. Since this essay is a first cut, simplified model, I’m using a single signal that represents a gating attention / inhibition signal, and glossing over the ACh vs GABA distinction.

Olfactory bulb switchboard using basal forebrain to gate selected odors. B.ip interpeduncular nucleus, B.rs reticulospinal motor, Hb.m medial habenula, Omt mitral/tufted output cells, Osn olfactory sensor neurons, P.bf basal forebrain.

In the above diagram where the switchboard selects odor A and inhibits odor B, the apical seek circuit receives only odor A’s signal. P.bf gates odors from Osn (olfactory sensory neurons) to Omt (mitral/tufted output cells), which then add to form a single signal for the temporal gradient tumble-and-run seek. For simplicity, I’ve shown the P.bf ACh and GABA signal as a simple gating control.

Once the system detects odor A, P.bf configures the switchboard to pass through A and inhibit other odors, locking out the distractor. Because the selecting signals are modulators, they don’t drive a signal until an odor signal arrives. Like the P.ni circuit, attention will timeout as ACh and its slow mACh receptor decay. When the animal leaves the odor plume, the system resets because the absence of odor A collapses the feedback loop.

Although the essay’s switchboard is an improvement over the naive summation of odor signals, it’s still quite limited. There’s no active selection of a best odor, and the system can’t switch to a better odor cue. Also, since the global give-up circuit isn’t integrated with P.bf, giving up on odor A can’t select odor B. Instead the animal must leave the plume and reset the system.

Slightly more complete Ob switchboard

The Ob is a surprisingly complex system; it’s not just a simple odor system. In addition to the P.bf, Opir (olfactory piriform cortex) also modulates the Ob system, and Ob itself has lateral inhibition between Omt (mitral cell output), which is plastic, learning to discriminate odors itself, as well as modulatory input from the serotonin and noradrenaline system.

In the real Ob, many Osn for the same odor feed into a single Ogl (olfactory glomeruli), which provides input to several Omt, all representing the same odor. Each odor feature has its own Ogl system, several hundred in mammals (two in the essay simulation). Ogl is where the neuropil of the Osn axons meet the Omt dendrites, in a fan-in to fan-out system. Also, each Ogl has many inhibitory Opg (periglomerular inhibitors) with multiple variations, and each Omt has several inhibitory Ogc (olfactory granule cells). The basic fan-in and fan-out structure looks like the following diagram.

Olfactory bulb glomerule fan-in and fan-out system. Bip interpeduncular nucleus, B.rs reticulospinal motor, Hb.m medial habenula, Ogc olfactory granule cell inhibitor, Ogl olfactory glomerule, Omt olfactory mitral/tufted output, Opg olfactory periglomerular inhibitor, Osn olfactory sensor neuron.

The switchboard diagram below focuses on the ACh and GABA control from P.bf. It combines multiple Osn, Opg, Omt and Ogc into single items.

Partial olfactory bulb switchboard circuit. B.ip interpeduncular nucleus, B.rs reticulospinal motor, Hb.m medial habenula, Ogc olfactory granule cell, Ogl olfactory glomeruli, Omt olfactory mitral/tufted output, Opg olfactory periglomerular inhibitor, Opir olfactory piriform cortex, Osn olfactory sensory neuron.

To break down the diagram, the core of the switchboard circuit is the Osn to Ogl to Omt to output path; everything else is gating to select or inhibit the signal.

Odor gating happens in two locations: modulating Omt’s input dendrite tree in Ogl by Opg and modulating Omt’s output by Ogc (olfactory granular cell). Because each Omt’s input Ogl is shared for several Omt, the Opg inhibition likely affects many or all Omt for a single Ogl. In contrast, the Ogc inhibition is individual, and the Omt and Ogc circuit creates and manages gamma oscillations, which amplifies and reduces noise from the signal.

Although I’m not planning on touching cortical areas for many essays, the Opir (olfactory piriform cortex) modules the Ob switchboard in a similar circuit as B.pf with some difference. Since the Opir input to the many Ogc and many Ogl is not odor selective [Boyd et al 2015], Ogc must learn the meaning of the Opir input through plasticity.

Global give-up circuit

The essay’s task engagement and give-up circuit currently uses H.l (lateral hypothalamus) and Hb.l (lateral habenula) with V.dr (dorsal raphe serotonin) [Hikosaka 2010], [Chowdhury and Yamanaka 2016]. When a seek fails Hb.l suppresses H.l, H.l ends seek, and the animal moves on [Post et al 2022].

Global give-up circuit. H.l lateral hypothalamus, Hb.l lateral habenula, V.dr dorsal raphe, 5HT serotonin.

Because the global give-up circuit is entirely disconnected from the olfactory selective attention from the essay, giving up means giving up on all odors, not just the current attended odor.

Simulation

For this essay, I refactored much of the simulation code to clean up ideas from previous essays. A new hindbrain module manages the main locomotion like the zebrafish hindbrain motor area [Dunn et al 2016], which is possibly different from the tetrapod / amniote locomotion in the midbrain. Because the essay animal is currently more primitive than amniotes, this simplification seemed appropriate and makes the code organization more clear.

Olfactory locomotion is now random-walk based following apical tumble-and-run, as opposed to the earlier bilateral path through Vta (ventral tegmental area / posterior tuberculum) and OT (tectum). In zebrafish both paths exist, which I might explore later, but this essay is restricted to the apical temporal gradient search.

The seek mode now slows the animal and adjusts the Levy walk parameters to simulate ARS (area restricted search). As I’ll cover in the problems section, switching to seek mode is still hardcoded.

I split the habenula seek from habenula give-up (Hb.m from Hb.l) and pulled the gradient seek and head direction from B.ip into the habenula seek. Conceptually, the habenula seek code now represents Hb.m and B.ip as a single complex.

Simulated odor seeking with target attention and distractor inhibition.

In the screenshot above, the animal is making a u-turn to return to the food when the odor gradient (blue semicircle) is opposite the head direction (black semicircle). In the upper right, the green box outlined in red represents the attended green odor signal, while the white box outline in blue represents the suppressed blue odor. Despite the Osn naively sensing both blue and green odors because the animal is in the overlap area, only the green odor passes through Omt to the seek system.

The square borders around the odor color represent P.bf modulation. Red is attended (100% pass through), blue is inhibited (10% pass through), and grey is unmodulated (50% pass through).

In the diamond-shaped homunculus, the bright blue triangle represents the u-turn nudge.

As the goal vector shows, the guessed goal direction isn’t very accurate, particularly when the animal is making a turn. Currently, the animal continues to update its guess even in the middle of a turn when the odor data and averages are not appropriate for the current direction.

References

Böhm E, Brunert D, Rothermel M. Input dependent modulation of olfactory bulb activity by HDB GABAergic projections. Sci Rep. 2020 Jul 1;10(1):10696. 

Boyd AM, Kato HK, Komiyama T, Isaacson JS. Broadcasting of cortical activity to the olfactory bulb. Cell Rep. 2015 Feb 24;10(7):1032-9.

Braitenberg, V. (1984). Vehicles: Experiments in synthetic psychology. Cambridge, MA: MIT Press. “Vehicles – the MIT Press”

Chowdhury S, Yamanaka A. Optogenetic activation of serotonergic terminals facilitates GABAergic inhibitory input to orexin/hypocretin neurons. Sci Rep. 2016;6:36039

Cisek P. Evolution of behavioural control from chordates to primates. Philos Trans R Soc Lond B Biol Sci. 2022 Feb 14

De Saint Jan D. Target-specific control of olfactory bulb periglomerular cells by GABAergic and cholinergic basal forebrain inputs. Elife. 2022 Feb 28;11:e71965.

Dunn, Timothy, Yu Mu, Sujatha Narayan, Owen Randlett, Eva A Naumann, Chao-Tsung Yang, Alexander F Schier, Jeremy Freeman, Florian Engert, Misha B Ahrens (2016) Brain-wide mapping of neural activity controlling zebrafish exploratory locomotion eLife 5:e12741

Henriques PM, Rahman N, Jackson SE, Bianco IH. Nucleus Isthmi Is Required to Sustain Target Pursuit during Visually Guided Prey-Catching. Curr Biol. 2019 Jun 3;29(11):1771-1786.e5. 

Hikosaka O. The habenula: from stress evasion to value-based decision-making. Nat Rev Neurosci. 2010 Jul;11(7):503-13. 

Nunez-Parra A, Cea-Del Rio CA, Huntsman MM, Restrepo D. The Basal Forebrain Modulates Neuronal Response in an Active Olfactory Discrimination Task. Front Cell Neurosci. 2020 Jun 5;14:141. 

Post RJ, Bulkin DA, Ebitz RB, Lee V, Han K, Warden MR. Tonic activity in lateral habenula neurons acts as a neutral valence brake on reward-seeking behavior. Curr Biol. 2022 Oct 24;32(20):4325-4336.e5.

Tosches, Maria Antonietta, and Detlev Arendt. The bilaterian forebrain: an evolutionary chimaera. Current opinion in neurobiology 23.6 (2013): 1080-1089.

Powered by WordPress & Theme by Anders Norén