essays on mind

Attempting a toy model of vertebrate understanding

Tag: dopamine

Essay 29: Sleep and Basal Ganglia

By

On March 29, 2024

In Uncategorized

The original impetus for this sleep essay was the idea that the basal ganglia could best be understood as a sleep and wake circuit [Kazmierczak and Nicola 2022]. After reviewing the rest of the brainstem sleep circuitry, it’s time to tackle the original problem.

Snr as a sleep/wake gate

Snr (substantia nigra pars reticulata) is the output node of the basal ganglia. It’s a set of GABA neurons that tonically suppress the majority of all brainstem motor areas including MLR (midbrain locomotor region), OT (optic tectum), and R.rs (hindbrain reticulospinal motor command) with corollary discharge to the thalamus. Snr can inhibit initiation of eating and motion [Rossi et al 2016], but don’t disrupt ongoing actions [Liu et al 2018]. Disruption of Snr can cause hyperactivity and insomnia [Geraschenko et al 2006]. The caudal Snr derives from hindbrain r1 (rhombomere r1 near the midbrain-hindbrain boundary) [Achim et al 2012], [Lahti et al 2015], [Partanen and Achim 2022], suggesting it may be evolutionarily old, possibly older than other basal ganglia regions.

Sleep as gating motive from action or sleep from action. Wake as disinhibiting sleep. Snr (substantia nigra pars reticulata).

As described in part 1 this essay, sleep suppresses senses, motivation and action. To implement this suppression, sleep could disconnect senses and motivation neurons from action neurons. In the above diagram, the gate is conceptual. The circuit could also inhibit the sense or action nodes directly instead of requiring specific gating neurons. This gating architecture has the advantage of simplicity because the sleep circuit can be localized in the gate, while the senses and actions can be mostly free of sleep circuitry.

As a preview, sleep neurotransmitters and peptides in BG (basal ganglia) include AD (adenosine), enk (enkephalin), MOR (μ-opioid receptor), and wake neurotransmitters include DA (dopamine), tac1 (tachykinin 1 aka neurokinin 1 aka substance P), dyn (dynorphin), and DOR (δ-opioid receptor).

If the vertebrate brain follows this architecture, Snr is well-placed to control that gate. Snr.m (medial Snr) projections have many collaterals to distinct motor areas and suppressing the wake-promoting areas covered earlier in this essay, which suggests widespread suppression as opposed to fine-grained control.

Snr.m gad2 connectivity. 60% of Snr.m inputs are from motor, motivation and wake areas. H.l (lateral habenula), H.stn (subthalamic nucleus), H.zi (zona incerta), M.pag (periaqueductal gray), OT.m (medial optic tectum), P.ge (external global pallidus), Ppt (pedunculopontine nucleus), R.rs (reticulospinal motor command), S.d (dorsal striatum), Snr.m (medial substantia nigra pars reticulata).

As the above diagram illustrates, despite its description as basal ganglia output, 60% of the gad2 (genetic marker), Snr.m inputs are outside of the basal ganglia, particularly from the midbrain (30%) and hypothalamus (10%) [Liu et al 2020]. Snr.m has two independent neuron types marked by gad2 and pv (parvalbumin), which are topographically organized with gad2 in Snr.m and pv in Snr.l (lateral Snr). While Snr.l.pv seems to be strictly motor related, Snr.m.gad2 are sleep related [Liu et al 2020]. However, [Lai et al 2021] reports Snr.l as sleep related.

Functional sleep and action requirements. Any ongoing action should suppress sleep, and sleep should suppress all actions.

Snr’s widespread motor and motivation connectivity suggests a possible primitive role in sleep. Sleep needs to suppress all actions, but any ongoing action needs to suppress sleep, because an animal shouldn’t fall asleep while eating or moving. It seems plausible that a primitive proto-vertebrate could have used Snr for sleep regulation without needing the rest of the basal ganglia.

Because astrocytes can integrate inputs spatially and temporally and are associated with sleep, it’s plausible that Snr astrocyte would be involved in this circuit. Interestingly Snr astrocytes are sensitive to dopamine and become hyperactive in the absence of dopamine [Bosson et al 2015] and are sensitive to glutamate from H.stn [Barat et al 2015].

Dopamine D2.i sleep / wake circuit

Although the independent Snr circuit is a functional sleep / wake gating circuit, it tonically inhibits the sense to action circuit, adding noise. An improvement to the circuit enables the gate when a signal is available, using the striatum to selectively open the gate. This circuit uses dopamine to open and close the gate. High dopamine is a wake signal and low dopamine is a sleep signal.

In the above diagram, Snr and S.d2 (D2.i associated striatum projection neurons) are sleep-promoting regions and S.d1 (D1.s associated striatum projection neurons) is a wake-promoting region. D2.i (inhibitory Gi-protein dopamine receptor) disconnects inputs, as opposed to inhibiting a neuron directly. When DA is available, S.d2 is disconnected, and S.d1 inhibits Snr, opening the gate. When DA is low, S.d2 is active, which inhibits S.d1, disinhibiting Snr, closing the gate and producing sleep. The D2i between S.d2 and S.d1 is from [Dobbs et al 2016].

The idea of the circuit is that the sense signal disinhibits itself during wake, but sleep prevents sense from disinhibiting itself. The minimal system only requires D2i circuits [Oishi et al 2017]. Wake enables the gate, and sleep disables the gate. Although I’ll cover D1s later, D2i is more fundamental because disabling D1s can be reversed by sufficient arousal, but disabling D2i can’t [Kazmierczak and Nicola 2022].

Note the diagram is somewhat incorrect, because direct S.d2 to S.d1 connection is weak [Tepper 2008]. Instead, S.d2 GABA inhibits S.d1 input at distal dendrites as opposed to inhibiting the neuron soma itself.

P.v ventral pallidum and S.core

While S.d2 neurons in model above suppresses motor for sleep, S.d2 in S.core (ventral striatum core aka nucleus accumbens) can produce sleep pressure by inhibiting the wake supporting P.v (ventral pallidum) [Oishi et al 2017]. P.v is a tonically active, wake-promoting nucleus, primarily inhibiting sleep areas or disinhibiting wake areas.

Sleep/wake control adding P.v as a tonic wake producing node. DA (dopamine), D2i (inhibitory Gi-coupled dopamine receptor), H.l (lateral hypothalamus), Hb.l (lateral habenula), M.pag (periaqueductal gray), Ppt (pedunculopontine nucleus – ACh), P.v (ventral pallidum), S.d1 (D1-associated striatum projection neuron), S.d2 (D2-associated striatum projection neuron), Snr (substantia nigra pars reticulata), V.mr (median raphe – serotonin), Vta (ventral tegmental area – dopamine).

P.v fill a similar wake-promoting role as S.d1, but unlike S.d1 it’s tonically active and affects the motivation loop of H.l, Hb.l, and Vta instead of gating sense from action. Where P.v supports general wake, S.d1 supports specific wake for an action. Like the previous basal ganglia sub-circuit, this sub-circuit only requires D2i receptors.

P.v promotes wake by inhibiting Hb.l sleep-producing system [Li et al 2023]. It also promotes wake through Vta by disinhibiting GABA interneurons [Li et al 2021]. (It could also disinhibit H.l orexin but I don’t have a reference).

In the model above, stimulating S.d2 inhibits wake-producing P.v, which disinhibits sleep-producing areas like Hb.l and inhibits wake-producing areas like H.l and Vta through GABA interneurons. Conversely, stimulating the D2i receptor by high DA inhibits S.d2, which disinhibits Pv, allowing it so promote wake. Disabling the D2i receptor activates S.d2, promoting sleep even with high dopamine [Qu et al 2010].

Note that S.d1 also connects to P.v and can produce wake [Zhang et al 2023]. P.v has multiple sub-populations with opposing functions. For example, it has both a hedonic hot spot for liked food and a cold spot for disliked food [Castro et al 2015]. For the sake of simplicity the diagram only shows a sleep-promoting path through S.d2, but there may be a wake-promoting path through S.d2 to an opposing P.v subpopulation.

D1s – stimulator dopamine receptors

Although using only D2i as a mode switch to the sleep path is functional, it can be improved by also enhancing the wake path with D1s (stimulatory Gs-coupled dopamine receptor).

D1s as enhancing the basal ganglia wake path. DA (dopamine), D1s (stimulatory Gs-coupled dopamine receptor), D2i (inhibitory Gi-coupled dopamine receptor), S.d1 (D1-associated striatum projection neuron), S.d2 (D2-associated striatum projection neuron), Snc (substantia nigra pars compacta – dopamine), Snr (substantia nigra pars reticulata).

The improved circuit works exactly like the D2i-only circuit but enhances the wake path when DA is available. Dopamine boosts both the signals from the sense to S.d1 and the signal from S.d1 to Snr [Salvatore 2024], [Kliem 2007], [Rice and Patel 2015]. When dopamine is available, it boots the sense to S.d1 signal with D1s, which more strongly disinhibits the gate by inhibiting Snr, which is also boosted by D1s.

The D1s in Snr and dopamine may be more important for motor suppression than dopamine in the striatum [Salvatore 2024]. In Parkinson’s disease and also normal aging, bradykinesia (slow movement) correlates with dopamine in Snr more closely than dopamine in the striatum. Motor symptoms in Parkinson’s disease don’t generally occur until striatal dopamine is reduced by 80%, but the effect on Snr is more immediate with only a small drop of dopamine.

Note that the Snc (substantia nigra pars compacta) to Snr dopamine comes from somatodendritic broadcast, not from an axon synapse. Snc dendrites in Snr produce dopamine to enhance the S.d1 to Snr connection.

Although the previous diagrams show the basic logic of the circuit, the basal ganglia use adenosine as a sleep-producing neurotransmitter, competing with dopamine.

Adenosine in striatum sleep

Adenosine is a product of the energy molecule ATP and is produced by neural activity, and also as a astrocyte transmission molecule. Although adenosine can accumulate in a circadian manner, particularly in P.bf (basal forebrain), it’s typically a shorter term sleep pressure. Caffeine is wake promoting by suppressing adenosine receptors.

Dopamine and adenosine are paired, opposing neurotransmitters in the basal ganglia: dopamine produces wake and adenosine promotes sleep. As an opposing signal to dopamine, the adenosine circuit is a flip version of the dopamine circuit.

Parallel adenosine sleep circuit in the basal ganglia. AD (adenosine), A1i (inhibitory Gi-coupled adenosine receptor), A2a.s (stimulatory Gs-coupled adenosine receptor), S.d1 (D1-associated striatum projection neuron), S.d2 (D2-associated striatum projection neuron), Snr (substantia nigra pars reticulata).

When adenosine is active in the above circuit, it cuts off S.d1 input and output and enhances S.d2’s suppression of S.d1. With S.d2 fully suppressed, Snr is free to suppress the gate and therefore suppress sleeping action.

Since adenosine is low in the morning, sleep is suppressed, which is enhanced by high ultradian morning dopamine. If A2a.s (stimulating Gs-coupled adenosine receptor) are stimulated in the striatum, the animal is more likely to sleep even in the morning [Yuan et al 2017], specifically in S.core not S.sh (ventral striatum shell aka nucleus accumbens) [Oishi et al 2017].

The dual signal system allows for interesting combinations at the boundary between sleep and wake. If adenosine is high with sleep pressing, then a large amount of dopamine motivation is required to continue wake. In fact, sleep deprivation down regulates D2i receptors, moving from the neuron membrane to the interior [Volkow et al 2012], which tips the balance toward sleep by diminishing the D2i-mediated wake signal. Caffeine inhibits both the A1i (inhibitory Gi-coupled adenosine receptor) and A2a.s receptors, tipping the balance to dopamine wake.

Dorsal striatum indirect path

The full S.d (dorsal striatum) path includes an indirect path, but this path may be more related to pure motor control, not sleep. As mentioned above, Snr divides into two populations Snr.l with pv neurons and Snr.m with gad2 neurons, and the Snr.l neurons are motor related, not sleep related [Liu et al 2020]. Similarly, the indirect path including P.ge (external globus pallidus) and H.stn (sub thalamic nucleus) may not be sleep related. Nevertheless, I’ll include it here, in case it is sleep related.

S.d model with indirect path included. DA (dopamine), D1s (stimulatory Gs-coupled dopamine receptor), D2i (inhibitory Gi-coupled dopamine receptor), H.stn (subthalamic nucleus), P.ge (external globus pallidus), S.d1 (D1-associated striatum projection neuron), S.d2 (D2-associated striatum projection neuron), Snc (substantia nigra pars compacta), Snr (substantia nigra pars reticulata).

Note that both P.ge and H.stn are tonically active, and they oscillate together at beta frequencies (roughly 10hz), which suppresses action. An excessive beta oscillation in this P.ge and H.stn circuit is a Parkinson’s disease symptom that suppresses motion and can also interrupt sleep. D2i receptors in H.stn mean that dopamine suppresses H.stn output [Shen et al 2012].

One significant experiment showed that lesioning P.ge increased wake by 40%, particularly eliminating normal circadian night-time sleep, replacing it with day-time like napping [Qiu et al 2016], which would suggest that P.ge is a major sleep center like Po.vl (ventrolateral preoptic area) [Vetrivelan et al 2010]. Note that this analysis would suggest that my basal ganglia sleep diagram is entirely wrong, because P.ge as a sleep center is basically incompatible with its position in the circuit.

P.ge – external globus pallidus

Lesioning P.ge increases wake by 40%, almost entirely eliminating circadian sleep [Qiu et al 2016]. However, this produces hyperactive chewing, weight loss, abnormal motor behavior and death in 3-4 weeks [Vetrivelan et al 2010]. Other manipulations of P.ge produce hyperactivity, abnormal movement, and odd stereotypical behavior [Gittis et al 2014]. So, it’s unclear to me that P.ge is a sleep center, but removing P.ge produces excessive action which then suppresses sleep.

In addition, P.ge is a heterogenous area with at least three major cell types with distinct projections and roles. Arkypallidal neurons project strongly and exclusively to the striatum. Lhx6 neurons project strongly to Snc and to some areas of H.stn, excluding the center. Pv neurons project to all of H.stn and also to T.pf (parafascical thalamus) [Gittis et al 2014].

Distinct projection neuron types of P.ge. H.stn (subthalamic nucleus), P.ge (external globus pallidus), Snc (substantia nigra pars compacta), Snr (substantia nigra pars reticulata), Spn (striatal projection neuron), Spv (pv marked striatum interneuron), T.pf (parafascicular thalamus).

With three projection types, it’s possible that they have entirely separate functions. For example, the lhx6 projections are functionally compatible with a sleep promoting role, and lhx6 neurons in H.zi (zona incerta) are sleep promoting [Liu et al 2017].

References

Achim K, Peltopuro P, Lahti L, Li J, Salminen M, Partanen J. Distinct developmental origins and regulatory mechanisms for GABAergic neurons associated with dopaminergic nuclei in the ventral mesodiencephalic region. Development. 2012 Jul;139(13):2360-70. 

Barat E, Boisseau S, Bouyssières C, Appaix F, Savasta M, Albrieux M. Subthalamic nucleus electrical stimulation modulates calcium activity of nigral astrocytes. PLoS One. 2012;7(7):e41793. 

Bosson A, Boisseau S, Buisson A, Savasta M, Albrieux M. Disruption of dopaminergic transmission remodels tripartite synapse morphology and astrocytic calcium activity within substantia nigra pars reticulata. Glia. 2015 Apr;63(4):673-83. 

Castro DC, Cole SL, Berridge KC. Lateral hypothalamus, nucleus accumbens, and ventral pallidum roles in eating and hunger: interactions between homeostatic and reward circuitry. Front Syst Neurosci. 2015 Jun 15;9:90. 

Dobbs LK, Kaplan AR, Lemos JC, Matsui A, Rubinstein M, Alvarez VA. Dopamine Regulation of Lateral Inhibition between Striatal Neurons Gates the Stimulant Actions of Cocaine. Neuron. 2016 Jun 1;90(5):1100-13.

Gerashchenko D, Blanco-Centurion CA, Miller JD, Shiromani PJ. Insomnia following hypocretin2-saporin lesions of the substantia nigra. Neuroscience. 2006;137(1):29-36.

Gittis AH, Berke JD, Bevan MD, Chan CS, Mallet N, Morrow MM, Schmidt R. New roles for the external globus pallidus in basal ganglia circuits and behavior. J Neurosci. 2014 Nov 12;34(46):15178-83.

Kaźmierczak M, Nicola SM. The Arousal-motor Hypothesis of Dopamine Function: Evidence that Dopamine Facilitates Reward Seeking in Part by Maintaining Arousal. Neuroscience. 2022 Sep 1;499:64-103. 

Kliem MA, Maidment NT, Ackerson LC, Chen S, Smith Y, Wichmann T. Activation of nigral and pallidal dopamine D1-like receptors modulates basal ganglia outflow in monkeys. J Neurophysiol. 2007 Sep;98(3):1489-500.

Lahti L, Haugas M, Tikker L, Airavaara M, Voutilainen MH, Anttila J, Kumar S, Inkinen C, Salminen M, Partanen J. Differentiation and molecular heterogeneity of inhibitory and excitatory neurons associated with midbrain dopaminergic nuclei. Development. 2016 Feb 1;143(3):516-29. 

Lai YY, Kodama T, Hsieh KC, Nguyen D, Siegel JM. Substantia nigra pars reticulata-mediated sleep and motor activity regulation. Sleep. 2021 Jan 21;44(1):zsaa151. 

Li YD, Luo YJ, Xu W, Ge J, Cherasse Y, Wang YQ, Lazarus M, Qu WM, Huang ZL. Ventral pallidal GABAergic neurons control wakefulness associated with motivation through the ventral tegmental pathway. Mol Psychiatry. 2021 Jul;26(7):2912-2928. 

Li Y, Zhang X, Li Y, Li Y, Xu H. Activation of Ventral Pallidum CaMKIIa-Expressing Neurons Promotes Wakefulness. Neurochem Res. 2023 Aug;48(8):2502-2513.

Liu K, Kim J, Kim DW, Zhang YS, Bao H, Denaxa M, Lim SA, Kim E, Liu C, Wickersham IR, Pachnis V, Hattar S, Song J, Brown SP, Blackshaw S. Lhx6-positive GABA-releasing neurons of the zona incerta promote sleep. Nature. 2017 Aug 31;548(7669):582-587. doi: 10.1038/nature23663. Epub 2017 Aug 23.

Liu, D., Ma, C., Zheng, W., Yao, Y., & Dan, Y. (2018). Sleep and motor control by a basal ganglia circuit. BioRxiv, 405324.

Liu D, Li W, Ma C, Zheng W, Yao Y, Tso CF, Zhong P, Chen X, Song JH, Choi W, Paik SB, Han H, Dan Y. A common hub for sleep and motor control in the substantia nigra. Science. 2020 Jan 24;367(6476):440-445.

Oishi Y, Suzuki Y, Takahashi K, Yonezawa T, Kanda T, Takata Y, Cherasse Y, Lazarus M. Activation of ventral tegmental area dopamine neurons produces wakefulness through dopamine D2-like receptors in mice. Brain Struct Funct. 2017 Aug;222(6):2907-2915. 

Partanen J, Achim K. Neurons gating behavior-developmental, molecular and functional features of neurons in the Substantia Nigra pars reticulata. Front Neurosci. 2022 Sep 6;16:976209.

Qiu MH, Yao QL, Vetrivelan R, Chen MC, Lu J. Nigrostriatal Dopamine Acting on Globus Pallidus Regulates Sleep. Cereb Cortex. 2016 Apr;26(4):1430-9.

Qu WM, Xu XH, Yan MM, Wang YQ, Urade Y, Huang ZL. Essential role of dopamine D2 receptor in the maintenance of wakefulness, but not in homeostatic regulation of sleep, in mice. J Neurosci. 2010 Mar 24;30(12):4382-9.

Rice ME, Patel JC. Somatodendritic dopamine release: recent mechanistic insights. Philos Trans R Soc Lond B Biol Sci. 2015 Jul 5;370(1672):20140185. 

Rossi MA, Li HE, Lu D, Kim IH, Bartholomew RA, Gaidis E, Barter JW, Kim N, Cai MT, Soderling SH, Yin HH. A GABAergic nigrotectal pathway for coordination of drinking behavior. Nat Neurosci. 2016 May;19(5):742-748.

Salvatore MF. Dopamine Signaling in Substantia Nigra and Its Impact on Locomotor Function-Not a New Concept, but Neglected Reality. Int J Mol Sci. 2024 Jan 17;25(2):1131.

Shen KZ, Johnson SW. Regulation of polysynaptic subthalamonigral transmission by D2, D3 and D4 dopamine receptors in rat brain slices. J Physiol. 2012 May 15;590(10):2273-84.

Tepper JM, Wilson CJ, Koós T. Feedforward and feedback inhibition in neostriatal GABAergic spiny neurons. Brain Res Rev. 2008 Aug;58(2):272-81. 

Vetrivelan R, Qiu MH, Chang C, Lu J. Role of Basal Ganglia in sleep-wake regulation: neural circuitry and clinical significance. Front Neuroanat. 2010 Nov 23;4:145.

Volkow ND, Tomasi D, Wang GJ, Telang F, Fowler JS, Logan J, Benveniste H, Kim R, Thanos PK, Ferré S. Evidence that sleep deprivation downregulates dopamine D2R in ventral striatum in the human brain. J Neurosci. 2012 May 9;32(19):6711-7.

Yuan XS, Wang L, Dong H, Qu WM, Yang SR, Cherasse Y, Lazarus M, Schiffmann SN, d’Exaerde AK, Li RX, Huang ZL. Striatal adenosine A2Areceptor neurons control active-period sleep via parvalbumin neurons in external globus pallidus. Elife. 2017 Oct 12;6:e29055.

Zhang J, Peng Y, Liu C, Zhang Y, Liang X, Yuan C, Shi W, Zhang Y. Dopamine D1-receptor-expressing pathway from the nucleus accumbens to ventral pallidum-mediated sevoflurane anesthesia in mice. CNS Neurosci Ther. 2023 Nov;29(11):3364-3377. 

Essay 29: Sleep – circadian

By

On March 22, 2024

In Uncategorized

As mentioned in the previous post, sleep is often divided into circadian sleep and homeostatic sleep, although this model is an oversimplification in part because of the metabolic cycles [Borbély et al 2016]. Despite the caveats, I think starting from circadian circuits is a good start.

Also as mentioned previously, circadian cycles may have started as an oxidation-reduction cycle to project from oxygen’s toxicity after the Great Oxidation Event [Edgar et al 2012]. One of the early solutions is melatonin, a powerful natural antioxidant [Tosches et al 2014].

Melatonin

Melatonin exists in almost all animals except sponges. Along with its antioxidant properties, it signals for the zooplankton diel vertical migration, swimming toward the light at dusk and sinking at night [Tosches et al 2014].

System for melatonin-controlled zooplankton vertical migration. ACh (acetylcholine).

In the migrating zooplankton, melatonin triggers ACh (acetylcholine) neurons, which rhythmically spike and these spikes disrupt the cilia, disorganizing them and allowing the plankton to sink.

Reptile and mammal complications

As a complication to understanding the vertebrate circuits, both reptiles and mammals have sleep requirements at odds with aquatic vertebrates. Because land temperatures change more than water temperatures, and reptiles are cold-blooded, their sleep and wake is necessarily strongly tied to temperature as well as the common light/dark connection [Rial et al 2022]. So, sleep and temperature are highly correlated, which makes the Poa (preoptic area) combination of temperature and sleep functions more reasonable than a seemingly random combination.

Mammals have the additional complication of the evolutionary nocturnal bottleneck [Rial et al 2022], meaning the simple heuristic of nighttime melatonin for sleep isn’t sufficient. The pineal melatonin is still at night for nocturnal animals [cite], and the light signaling needs to flip. Although diurnal mammals are no longer nocturnal, their clock circuitry retains the heritage of a nocturnal flip.

As a specific example, all non-mammalian vertebrates use the pineal gland as a circadian oscillator, not H.scn (subthalamic nucleus) [Vatine et al 2011].

Pineal gland and habenula

The pineal gland in the midbrain is the vertebrate’s main source of melatonin. Evolutionarily, the pineal gland is derived from photosensitive cells that directly convert light and dark into melatonin. In non-mammals, the pineal gland is still photosensitive. In the zebrafish, the pineal photoreceptor is still effect and entrains circadian cycles [Vatine et al 2011], and an analogous region in the non-vertebrate chordate Amphioxus provides a similar function, showing the pineal gland’s conserved function in vertebrates [Lacalli 2022].

Hb.m and Hb.l (medial and lateral habenula) derive from the pineal complex, and may have originally been effectors of the pineal gland, serving a nervous function analogous to melatonin [Hikosaka 2010]. Hb.l in particular is well-suited to control neurotransmitters associated with wake, such as dopamine from Vta (ventral tegmental area), serotonin from V.dr and V.mr (dorsal and medial raphe), and norepinephrine from V.lc (locus coeruleus). Note that melatoninAs explored in essay 20, Hb.m is involved in primitive phototaxis and chemotaxis and is well-placed to inhibit those actions during sleep.

Habenula control of sleep by gating motive from action. Hb (habenula), pineal (pineal gland).

In the above diagram, a primitive habenula function is to suppress sensation, motivation and action for sleep by suppression wake-supporting neurotransmitters. Although the diagram illustrates the habenula as disconnecting motive from action, it could also disconnect sense from action, as in phototaxis or chemotaxis in Hb.m.

Hb.m includes an internal entrainable circadian clock, unlike Hb.l. The Hb.m clock is necessary for ultradian foraging. The foraging ultradian is around four hours, generally on waking. Both dopamine and NE (norepinephrine) are elevated [Wang et al 2023] and reciprocally the circadian clock is set by dopamine and NE [Salaberry and Mendoza 2022].

Some misc notes: Hb.l is required for some anesthesia (propofol) and stimulating Hb.l strongly induces NREM, and suppresses motor [Gelengen et al 2018]. Hb.l stimulus produces NREM [Goldstein 1983]. Hb.l is more active mid and late day and early night [Aizawa et al 2013] (possibly producing morning ultradian activity). Hb.l manipulation produces wake fragmentation in the wake period and sleep fragmentation in the sleep period via orexin in H.l [Gelengen et al 2018].

Cell clocks

As mentioned in the introduction, the oxidation-reduction protection may have led to the development of cellular clocks. Essentially all cells have circadian cycle in protein expression, including metabolic and detoxification cells in the liver, heart, kidneys and digestion [Dibner et al 2010], even including gut bioflora. The clocks are synchronized by multiple signals, including feeding patterns, but most studied by light.

For example, dopamine is under clock control and is modulated by melatonin [Ashton and Jagannath 2020]. In S.v (ventrial striatum aka nucleus accumbens) dopamine is at a daily low at night. DAT (dopamine transporter), affected by cocaine, is regulated by clock genes [Alsonso et al 2021], possibly under control of astrocytes. Dopamine is particularly tonically high in early morning before eating with an ultradian cycle of about four hours. Two four-ish hour dopamine cycles are known: the FEO (food entrainable oscillator), which produces pre-feeding activity [Dibner et al 2010], and MASCO (methamphetamine-sensitive circadian oscillatory) [Tataroglu et al 2006], which may be the same system.

The retina itself is under circadian control, modulated by dopamine and D2i (inhibitory Gi-coupled dopamine receptor) [Yujinovsky et al 2006], including in frogs [Cahill and Besharse 1991].

And astrocytes in S.v are under circadian cell clock control [Becker-Krail et al 2022]. Astrocytes are well-placed to manage sleep because they have widespread connections to many synapses and are connected to other astrocytes with gap junctions, allowing for integration over time and space and widespread broadcast signaling.

H.scn circadian entrainment

The circadian system has three distinct components that can either work on their own or work together:

  • Cell clocks
  • Light / dark photoreceptors or feeding signals and behavior
  • Entraining the cell clock to the signal (zeitgeber)

If the eye area of the mollusk sea hare is lesioned, circadian entrainment is eliminated, but because of other photoreceptors, the animal still follows light and dark cycles as long as the light changes. The deficit is only exposed when the lesioned mollusks are placed under continual dark [Vorster et al 2014], [Newcomb et al 2014]. Similarly, in zebrafish many cells are photoreceptive without entraining the cellular clocks.

Mammals use H.scn (suprachiasmatic nucleus) to coordinate circadian cellular clocks. The H.scn name is important, but it’s located above the optic crossing (suprachiasmatic) and developmentally the retina develops from the hypothalamus adjacent to H.scn.

Abstract representation of the mammalian brain highlighting the proximity of the retina and H.scn. arc (H.arc – arcuate nucleus), C (cortex), CB (cerebellum), H.l (lateral hypothalamus), ip (R.ip interpeduncular nucleus), mb (H.mb mammillary bodies), MHB (midbrain-hindbrain boundary), P (pallidum), S (striatum), scn (suprachiasmatic nucleus), sum (supramammillary nucleus), Vta (ventral tegmental area), r1 (rhombomere 1), ZLI (zona-limitans intrathalamica)

The diagram above shows the rough location of the retina development area and H.scn, which both develop from the hypothalamus. A primitive eye with only a few photoreceptors would have been part of the hypothalamus, and like the mollusk the photoreceptor would be near the clock entrainment circuit that became H.scn.

The H.scn clock signal is somewhat indirect, with an interim projection to H.scz to H.dm (dorsomedial hypothalamus) and finally to H.l (lateral hypothalamus) for wake and Po.vl (ventrolateral preoptic area) for sleep. H.scn uses dopamine from Vta as part of its synchronization [Grippo et al 2017].

Ultradian DA – morning foraging

The sleep / wake cycle has an additional boost during normal foraging times such as immediately after waking. In the subjective morning (dark for rodents), wake is encouraged, homeostatic sleep is suppressed, and dopamine levels are higher. After the foraging boost ends, but still in the wake period, tonic dopamine levels drop and the animals take more frequent naps. This hut radian boost of about four hours affects learning and behavior as well as modulating drug abuse [Ruby et al 2013].

Sleep / wake cycle showing morning boost. ZT (zeitgeber time).

Because this ultradian foraging boots wake and suppresses sleep significantly, studies that stimulate or inhibit sleep and wake can specifically affect the ultradian boost without affecting other sleep / wake periods. So it’s very important to look at the hourly effects because the experimental modulation might reduce the foraging boost specifically, but a summary might show a general sleep increase.

H.scn circadian entrainment uses dopamine. DA from either Vta [Grippo et al 2017], [Tang et al 2022] and/or H.sum (supramammillary nucleus) [Luo et al 2018] can entrain food circadian cycles. Note that since the dopamine “A10” area extends beyond the Vta to include H.sum and M.pag.v on opposite ends of the Vta, these studies may be reporting the same area.

As mentioned above, there are also the food entrained oscillator [Liu et al 2012], [Gallardo et al 2014], [Pendergast and Yamazaki 2019], [Ashton and Jagannath 2020], and the meth-sensitive oscillator [Tataroglu et al 2006], which are also dopamine related and may be part of the same system.

Neurotransmitters and peptides

The inhibitory neurotransmitter GABA is associated with sleep, and many sleep drugs are GABA stimulants. GABA neurons in Snr (substantia nigra pars reticulata), H.zi (zona incerta), Vta, and Po.vl are all associated with sleep. As mentioned above GABA from mitochondria and in Hydra are used as a sleep promoting neurotransmitter.

While GABA is associated with sleep, other major neurotransmitters like NE, DA, 5HT (serotonin), ACh (acetylcholine) and histamine are associated with wake maintenance of the execution of wakefulness. As discussed in the previous post, ongoing actions need to suppress sleep. NE, DA, and 5HT are all maintain wake while the animal is active and drop when the animal is winding down activity to sleep. Cortical wake requires activity in ACh-rich area in Ppt (pedunculopontine nucleus), P.ldt (laterodorsal tegmental nucleus), and P.bf (basal forebrain).

Produced by H.l, orexin (aka hypocretin) appears to be a wake-maintenance peptide since removal of orexin produces narcolepsy. H.l orexin projects to essentially all of the other wake-maintaining neurotransmitters, including NE, DA, 5HT and ACh. Orexin is slow, waking after tens of seconds, while stimulating V.lc NE is around two seconds [Yamaguchi et al 2018]. On counterargument is that orexin can ramp later in the day [Grady et al 2006], [Mogavero et al 2023], which would suggest that it’s not part of the ultradian foraging system, although it’s also highly tied to foraging. (Suggesting I need to read more articles to see if the contradiction has been resolved.)

Although orexin is the most dramatic of H.l wake, H.l also includes wake and sleep producing GABA and glutamate neurons that may be even more important for wake, independent of the orexin function. Unfortunately, H.l is complex enough that the different functions haven’t been fully pulled apart.

Adenosine is a sleep-promoting molecule derived from the energy molecule ATP, and has extensive receptor throughout the brain, notable in the striatum. Because it’s a product of ATP, it measures local neural activity and possibly sleep need. Its measurement of global brain activity for homeostatic sleep seems more questionable, but adenosine does accumulate throughout the wake period in P.bf [Porkka-Heiskanan et al 2000].

Inflammation peptides like IL-1β are also sleep-promoting [Imeri and Opp 2009]. In addition to their inflammation-related sleep, they seem to be part of normal homeostatic sleep signaling. In Drosophila sleep-need astrocytes produce IL-1β as a signaling peptide [Blum et al 2021]. In zebrafish, sleep deprivation correlates with immune signaling [Williams et al 2007].

Next: ignition and maintenance circuits

After this general discussion on sleep wake, the next post will cover some of the specific sleep and wake circuits, particularly those associated with wake ignition, wake maintenance and sleep maintenance.

References

Aizawa H, Cui W, Tanaka K, Okamoto H. Hyperactivation of the habenula as a link between depression and sleep disturbance. Front Hum Neurosci. 2013 Dec 10;7:826. 

Ashton A, Jagannath A. Disrupted Sleep and Circadian Rhythms in Schizophrenia and Their Interaction With Dopamine Signaling. Front Neurosci. 2020 Jun 23;14:636. 

Becker-Krail DD, Walker WH 2nd, Nelson RJ. The Ventral Tegmental Area and Nucleus Accumbens as Circadian Oscillators: Implications for Drug Abuse and Substance Use Disorders. Front Physiol. 2022 Apr 27;13:886704.

Blum ID, Keleş MF, Baz ES, Han E, Park K, Luu S, Issa H, Brown M, Ho MCW, Tabuchi M, Liu S, Wu MN. Astroglial Calcium Signaling Encodes Sleep Need in Drosophila. Curr Biol. 2021 Jan 11;31(1):150-162.e7. 

Borbély AA, Daan S, Wirz-Justice A, Deboer T. The two-process model of sleep regulation: a reappraisal. J Sleep Res. 2016 Apr;25(2):131-43.

Cahill GM, Besharse JC. Resetting the circadian clock in cultured Xenopus eyecups: regulation of retinal melatonin rhythms by light and D2 dopamine receptors. J Neurosci. 1991 Oct;11(10):2959-71. 

Dibner, C., Schibler, U., & Albrecht, U. (2010). The mammalian circadian timing system: organization and coordination of central and peripheral clocks. Annual review of physiology, 72, 517-549.

Edgar RS, Green EW, Zhao Y, van Ooijen G, Olmedo M, Qin X, Xu Y, Pan M, Valekunja UK, Feeney KA, Maywood ES, Hastings MH, Baliga NS, Merrow M, Millar AJ, Johnson CH, Kyriacou CP, O’Neill JS, Reddy AB. Peroxiredoxins are conserved markers of circadian rhythms. Nature. 2012 May 16;485(7399):459-64.

Gallardo CM, Darvas M, Oviatt M, Chang CH, Michalik M, Huddy TF, Meyer EE, Shuster SA, Aguayo A, Hill EM, Kiani K, Ikpeazu J, Martinez JS, Purpura M, Smit AN, Patton DF, Mistlberger RE, Palmiter RD, Steele AD. Dopamine receptor 1 neurons in the dorsal striatum regulate food anticipatory circadian activity rhythms in mice. Elife. 2014 Sep 12;3:e03781.

Gelegen C, Miracca G, Ran MZ, Harding EC, Ye Z, Yu X, Tossell K, Houston CM, Yustos R, Hawkins ED, Vyssotski AL, Dong HL, Wisden W, Franks NP. Excitatory Pathways from the Lateral Habenula Enable Propofol-Induced Sedation. Curr Biol. 2018 Feb 19;28(4):580-587.e5.

Goldstein, R. (1983). A GABAergic habenulo-raphe pathway mediation of the hypnogenic effects of vasotocin in cat. Neuroscience 10, 941–945.

Grady, S. P., Nishino, S., Czeisler, C. A., Hepner, D., & Scammell, T. E. (2006). Diurnal variation in CSF orexin-A in healthy male subjects. Sleep, 29(3), 295-297.

Grippo RM, Purohit AM, Zhang Q, Zweifel LS, Güler AD. Direct Midbrain Dopamine Input to the Suprachiasmatic Nucleus Accelerates Circadian Entrainment. Curr Biol. 2017 Aug 21;27(16):2465-2475.e3. 

Hikosaka O. The habenula: from stress evasion to value-based decision-making. Nat Rev Neurosci. 2010 Jul;11(7):503-13.

Lacalli T. An evolutionary perspective on chordate brain organization and function: insights from amphioxus, and the problem of sentience. Philos Trans R Soc Lond B Biol Sci. 2022 Feb 14;377(1844):20200520.

Liu YY, Liu TY, Qu WM, Hong ZY, Urade Y, and Huang ZL (2012) Dopamine is involved in food-anticipatory activity in mice. J Biol Rhythms 27:398–409.

Luo YJ, Ge J, Chen ZK, Liu ZL, Lazarus M, Qu WM, Huang ZL, Li YD. Ventral pallidal glutamatergic neurons regulate wakefulness and emotion through separated projections. iScience. 2023 Aug 5;26(8):107385.

Mogavero MP, Godos J, Grosso G, Caraci F, Ferri R. Rethinking the Role of Orexin in the Regulation of REM Sleep and Appetite. Nutrients. 2023 Aug 22;15(17):3679.

Newcomb JM, Kirouac LE, Naimie AA, Bixby KA, Lee C, Malanga S, Raubach M, Watson WH 3rd. Circadian rhythms of crawling and swimming in the nudibranch mollusc Melibe leonina. Biol Bull. 2014 Dec;227(3):263-73. 

Pendergast JS, Yamazaki S. The Mysterious Food-Entrainable Oscillator: Insights from Mutant and Engineered Mouse Models. J Biol Rhythms. 2018 Oct;33(5):458-474.

Porkka-Heiskanen T, Strecker RE, McCarley RW. Brain site-specificity of extracellular adenosine concentration changes during sleep deprivation and spontaneous sleep: an in vivo microdialysis study. Neuroscience. 2000;99(3):507-17. 

Rial RV, Canellas F, Akaârir M, Rubiño JA, Barceló P, Martín A, Gamundí A, Nicolau MC. The Birth of the Mammalian Sleep. Biology (Basel). 2022 May 11;11(5):734. doi: 10.3390/biology11050734. 

Ruby NF, Hwang CE, Wessells C, Fernandez F, Zhang P, Sapolsky R, Heller HC. Hippocampal-dependent learning requires a functional circadian system. Proc Natl Acad Sci U S A. 2008 Oct 7;105(40):15593-8.

Salaberry NL, Mendoza J. The circadian clock in the mouse habenula is set by catecholamines. Cell Tissue Res. 2022 Feb;387(2):261-274.

Tang Q, Assali DR, Güler AD, Steele AD. Dopamine systems and biological rhythms: Let’s get a move on. Front Integr Neurosci. 2022 Jul 27;16:957193. 

Tataroglu O, Davidson AJ, Benvenuto LJ, Menaker M. The methamphetamine-sensitive circadian oscillator (MASCO) in mice. J Biol Rhythms. 2006 Jun;21(3):185-94. 

Tosches MA, Bucher D, Vopalensky P, Arendt D. Melatonin signaling controls circadian swimming behavior in marine zooplankton. Cell. 2014 Sep 25;159(1):46-57.

Vatine G, Vallone D, Gothilf Y, Foulkes NS. It’s time to swim! Zebrafish and the circadian clock. FEBS Lett. 2011 May 20;585(10):1485-94. 

Vorster AP, Krishnan HC, Cirelli C, Lyons LC. Characterization of sleep in Aplysia californica. Sleep. 2014 Sep 1;37(9):1453-63. 

Wang F, Wang W, Gu S, Qi D, Smith NA, Peng W, Dong W, Yuan J, Zhao B, Mao Y, Cao P, Lu QR, Shapiro LA, Yi SS, Wu E, Huang JH. Distinct astrocytic modulatory roles in sensory transmission during sleep, wakefulness, and arousal states in freely moving mice. Nat Commun. 2023 Apr 17;14(1):2186. 

Yamaguchi H, Hopf FW, Li SB, de Lecea L. In vivo cell type-specific CRISPR knockdown of dopamine beta hydroxylase reduces locus coeruleus evoked wakefulness. Nat Commun. 2018 Dec 6;9(1):5211. 

Yujnovsky I, Hirayama J, Doi M, Borrelli E, Sassone-Corsi P. Signaling mediated by the dopamine D2 receptor potentiates circadian regulation by CLOCK:BMAL1. Proc Natl Acad Sci U S A. 2006 Apr 18;103(16):6386-91.

 Williams JA, Sathyanarayanan S, Hendricks JC, Sehgal A. Interaction between sleep and the immune response in Drosophila: a role for the NFkappaB relish. Sleep. 2007 Apr;30(4):389-400. 

Essay 22 issues: subthalamic nucleus simulation

By

On November 14, 2023

In Uncategorized

The essay 22 simulation explored a striatum model where the two decision paths competed: odor seeking vs random exploration, using dopamine to bias between exploration and seeking. This model resembled striatum theories like [Bariselli et al. 2020] that consider the stratum’s direct and indirect paths as competing between approach and avoidant actions.

Issues in essay 22 include both neuroscience divergence and simulation problems. Although the simulation is a loose functional model, that laxity isn’t infinite and it may have gone too far from the neuroscience.

Adenosine and perseveration

Seeking and foraging have a perseveration problem: the animal must eventually give up on a failed cue, or it will remain stuck forever. The give-up circuit in essay 22 uses the lateral habenula (Hb.l) to integrate search time until it reaches a threshold to give up. An alternative circuit in the stratum itself involves the indirect path (S.d2), the D2 dopamine receptor and adenosine, with a behaviorally relevant time scale.

When fast neurotransmitters are on the order of 10 milliseconds, creating a timeout on the order of a few minutes is a challenge. Two possible solutions in that timescale are long term potentiation (LTP) where “long” means about 20 minutes, and astrocyte calcium accumulation, which is also about 10 to 20 minutes.

Adenosine receptors (A2r) in the striatum indirect path (S.d2) measure broad neural activity from ATP byproducts that accumulate in the intercellular space. Over 10 minutes those A2r can produce internal calcium ion (Ca) in the astrocytes or via LTP to enhance the indirect path. Enhancing the indirect path (exploration), eventually causes a switch from the direct path (seeking) to exploration, essentially giving-up on the seeking.

Ventral striatum

Although the essay models the dorsal striatum (S.d), the ventral striatum (S.v aka nucleus accumbens) is more associated with exploration and food seeking. In particularly, the olfactory path for food seeking goes through S.v, while midbrain motor actions use S.d. In salamanders, the striatum only processes midbrain (“collo-“) thalamic inputs, while olfactory and direct senses (“lemno-“) go to the cortex [Butler 2008]. Assuming the salamander path is more primitive, the essay’s use of S.d in the model is a likely mistake.

But S.v raises a new issue because S.v doesn’t use the subthalamus (H.stn) [Humphries and Prescott 2009]. Although, that model only applies to the S.v shell (S.sh) not the S.v core (S.core).

Ventral striatum pathway. MLR midbrain locomotive region, P.v ventral pallidum, S.sh ventral striatum shell, Vta ventral tegmental area.

In the above diagram of a striatum shell circuit, an odor-seek path is possible through the ventral tegmental area (Vta) but there is no space for an alternate explore path.

Low dopamine and perseveration

[Rutledge et al. 2009] investigates dopamine in the context of Parkinson’s disease (PD), which exhibits perseveration as a symptom. In contrast to the essay, PD is a low dopamine condition, and adding dopamine resolves the perseveration. But that resolve is the opposite of essay 22’s dopamine model, where low dopamine resolved perseveration.

Now, it’s possible that give-up perseveration and Parkinson’s perseveration are two different symptoms, or it’s possible that the complete absence of dopamine differs from low tonic dopamine, but in either case, the essay 22 model is too simple to explain the striatum’s dopamine use.

Dopamine burst vs tonic

Dopamine in the striatum has two modes: burst and tonic. Essay 22 uses a tonic dopamine, not phasic. The striatum uses phasic dopamine to switch attention to orient to a new salient stimulus. The phasic dopamine circuit is more complicated than the tonic system because it requires coordination with acetylcholine (ACh) from the midbrain laterodorsal tegmentum (V.ldt) and pedunculopontine (V.ppt) nuclei.

A question for the essays is whether that phasic burst is primitive to the striatum, or a later addition, possibly adding an interrupt for orientation to an earlier non-interruptible striatum.

Explore semantics

The word “explore” is used differently by behavioral ecology and in reinforcement learning, despite both using foraging-like tasks. These essays have been using explore in the behavioral ecology meaning, which may cause confusion on the reinforcement learning sense. The different centers on a fixed strategy (policy) compared with changing strategies.

In behavioral ecology, foraging is literal foraging, animals browsing or hunting in a place and moving on (giving up) if the place doesn’t have food [Owen-Smith et al. 2010]. “Exploring” is moving on from an unproductive place, but the policy (strategy) remains constant because moving on is part of the strategy. The policy for when to stay and when to go [Headon et al. 1982] often follows the marginal value theorem [Charnov 1976], which specifies when the animal should move on.

In contract, reinforcement learning (RL) uses “explore” to mean changing the policy (strategy). For example, in a two-armed bandit situation (two slot machines), the RL policy is either using machine A or using machine B, or a fixed probabilistic ratio, not a timeout and give-up policy. In that context, exploring means changing the policy not merely switching machines.

[Kacelnick et al. 2011] points out that the two-choice economic model doesn’t match vertebrate animal behavior, because vertebrates use an accept-reject decision [Cisek and Hayden 2022]. So, while the two-armed bandit may be useful in economics, it’s not a natural decision model for vertebrates.

Avoidance (nicotinic receptors in M.ip)

The simulation uncovered a foraging problem, where the animal remained around an odor patch it had given up on, because the give-up strategy reverts to random search. Instead, the animal should leave the current place and only resume search when its far away.

Path of simulated animal after giving up on a food odor.

In the diagram above, the animal remains near the abandoned food odor. The tight circles are the earlier seek before giving up, and the random path afterwards is the continued search. A better strategy would leave the green odor plume and explore other areas of the space.

As a possible circuit, the habenula (Hb.m) projects to the interpeduncular nucleus (M.ip) uses both glutamate and ACh as neurotransmitters, where ACh amplifies neural output. For low signals without ACh, the animal approaches the object, but high signals with ACh switch approach to avoidance. This avoidance switching is managed by the nicotine receptor (each) which is studied for nicotine addiction [Lee et al. 2019].

An interesting future essay might explore using nicotinic aversion to improve foraging by leaving an abandoned odor plume.

References

Bariselli S, Fobbs WC, Creed MC, Kravitz AV. A competitive model for striatal action selection. Brain Res. 2019 Jun 15;1713:70-79.

Butler, Ann. (2008). Evolution of the thalamus: A morphological and functional review. Thalamus & Related Systems. 4. 35 – 58.

Charnov, Eric L. Optimal foraging, the marginal value theorem. Theoretical population biology 9.2 (1976): 129-136.

Cisek P, Hayden BY. Neuroscience needs evolution. Philos Trans R Soc Lond B Biol Sci. 2022 Feb 14;377(1844):20200518.

Headon T, Jones M, Simonon P, Strummer J (1982) Should I Stay or Should I Go. On Combat Rock. CBS Epic.

Humphries MD, Prescott TJ. The ventral basal ganglia, a selection mechanism at the crossroads of space, strategy, and reward. Prog Neurobiol. 2010 Apr;90(4):385-417.

Kacelnik A, Vasconcelos M, Monteiro T, Aw J. 2011. Darwin’s ‘tug-of-war’ vs. starlings’ ‘horse-racing’: how adaptations for sequential encounters drive simultaneous choice. Behav. Ecol. Sociobiol. 65, 547-558.

Lee HW, Yang SH, Kim JY, Kim H. The Role of the Medial Habenula Cholinergic System in Addiction and Emotion-Associated Behaviors. Front Psychiatry. 2019 Feb 28

Owen-Smith N, Fryxell JM, Merrill EH. Foraging theory upscaled: the behavioural ecology of herbivore movement. Philos Trans R Soc Lond B Biol Sci. 2010 Jul 27;365(1550):2267-78. 

Rutledge RB, Lazzaro SC, Lau B, Myers CE, Gluck MA, Glimcher PW. Dopaminergic drugs modulate learning rates and perseveration in Parkinson’s patients in a dynamic foraging task. J Neurosci. 2009 Dec 2

Essay 22: Subthalamic Nucleus

By

On October 29, 2023

In Uncategorized

After essay 21 changed the animal’s default movement to a Lévy exploration, it’s immediate to ask whether that random search is a full action, just like a seek turn or an avoid turn. An if exploration is a controlled action, then the model needs to treat exploration as a full action, like approach or avoid.

Exploration as a full locomotive system at the level of approach and avoid.

[Cisek 2020] identifies a vertebrate system for exploration, including the hippocampus (E.hc) and its associated nuclei such as the retromammilary hypothalamus (H.rm aka supramammilary). Essay 22 considers the idea of treating the subthalamic nucleus (H.stn) as part of the exploration circuit.

Subthalamic nucleus

H.stn is a hypothalamic nucleus from the same area as H.rm, which is part of the hippocampal theta circuit, which synchronizes exploration and spatial memory and learning. However, H.stn is part of the basal ganglia and not directly connected with the exploration system.

[Watson et al. 2021] finds a locomotive function of H.stn, where specific stimulation by the parafascicular thalamus (T.pf) to H.stn starts locomotion. If the stimulation is one-sided, the animal moves forward with a wide turn to the contralateral side. T.pf includes efference copies of motor actions from the MLR as well as from other midbrain actions.

Locomotion induced in the H.stn by T.pf stimulation. H.stn sub thalamic nucleus, T.pf parafascicular nucleus, MLR midbrain locomotor region.

For essay 22, let’s consider the H.stn locomotion as exploration. Since H.stn is part of the basal ganglia, the bulk of essay 22 is considering how exploration might fit into the proto-striatum model of essay 18.

Striatal attention and persistence

Since the current essay simulation animal is an early Cambrian proto-vertebrate, it doesn’t have a full basal ganglia. Evolutionarily, the full basal ganglia architecture could not have sprung into being fully formed; it must have developed in smaller step. Following a hypothetical evolutionary path, the essays are only implementing a simplified striatal model, adding features step-by-step. Unfortunately, because there’s no living species with a partial basal ganglia — all vertebrates have the full system — the essay’s steps are pure invention.

The initial striatum of essay 18 was a partial solution to a simulation problem: persistence. When the animal hit a wall head on, activating both touch sensors, it would choose randomly left or right, but because the simulation is real-time not turn-based, at the next tick both sensors remained active and the animal would choose randomly again, jittering at the wall until enough turns of the same direction escaped the barrier.

proto-striatum circuit for persistence by attention.
Proto-striatum for persistence by attention. Action feedback biases the choice to the last option: win-stay. B.rs reticulospinal motor command, Ob olfactory bulb, MLR midbrain locomotor region, Snc substantia nigra pars compacta (posterior tuberculum).

The main sense-to-action path is from the olfactory bulb (O.b) through the substantia nigra (Snc aka posterior tuberculum in zebrafish) to the midbrain locomotor region (MLR) and to the reticulospinal motor command neurons (B.rs), following the tracing and locomotive study of [Derjean et al. 2010] in zebrafish and Vta/Snc control of locomotion in [Ryczko et al. 2017]. The proto-striatum circuit is built around that olfactory-seeking circuit, acting persistent attention.

The proto-striatal model uses an efference copy of the last action from the MLR to bias the choice of the next action via a MLR to T.pf to striatum path. The model biases the choice through removing inhibition of the odor to action path. If the last action as left, the left odor is disinhibited, making it more likely to win.

The striatal system uses disinhibition for noise reasons. [Cohen et al. 2009] studied attention in the visual system and found that attention removed coherent noise by removing inhibition. By removing inhibition, the attended circuit is less affected by the controlling circuit’s noise.

Note: essay 19 considered an alternative solution to the attention issue by following the nucleus isthmi system in zebrafish as studied in [Grubert et al. 2006], where the attention to the win-stay odor used acetylcholine (ACh) amplification to bias the choice.

Striatal columns: approach and avoid

An immediate difficulty with the simple proto-striatal model is the lack of priority. Although left vs right have equal priority, avoiding a predator is more important than seeking a potential food source. Unfortunately, the proto-striatum treats all options equally. As a solution, essay 18 split the striatum into columns, where each column resolves an internal conflict without priority (“within-system”) and the columns are compared separately (“between-systems”), where “within-system” and “between-system” are from [Cisek 2019].

Proto-striatum columns for maintaining attention.
Dual striatum column for approach and avoid, where MLR resolves the final conflict. B.rs reticulospinal command neuron, B.ss somatosensory (touch), MLR midbrain locomotive region, M.pag periaqueductal gray, Ob olfactory bulb, S.ot olfactory tubercle, S.d dorsal striatum.

Subthalamic nucleus and exploration

If we now treat exploration as a distinct action system, then it needs its own control system and column in the proto-striatum. The within-system choice for exploration is the left and right turns for a random walk, and the between-system choices are between the exploration system and the odor-seeking system.

As a possible neural correlate of exploration, consider the sub thalamic nucleus (H.stn). The sub thalamic nucleus is derived from the hypothalamus, specifically from the same area as the retromammilary area (H.rm aka supramammilary), which is highly correlated with hippocamptal theta, locomotion and exploration.

[Watson et al. 2021] finds a locomotive function of H.stn, where specific stimulation by the parafascicular thalamus (T.pf) produces locomotion via the midbrain locomotive region (MLR). T.pf includes efference copies of motor actions from the MLR as well as other midbrain action efference copies. In the proto-striatum model, the feedback from MLR to striatum uses T.pf.

Exploration locomotive path through H.stn. H.stn sub thalamic nucleus, MLR midbrain locomotive region, T.pf parafascicular thalamus.

Seek and explore with dual striatal columns

Suppose the striatum manages both odor seeking (chemotaxis) and default exploration (Lévy walk). The two actions are conflicting with a complex priority system. When a food odor first appears, the animal should seek toward it (priority to seek), but if no food exists the animal should resume exploration (priority to explore). To resolve the between-system conflict, the two strategies need to columns with lateral inhibition to ensure that only one is selected.

Dual striatum columns for seek and explore strategies. B.rs reticulospinal motor command, H.stn sub thalamic nucleus, Ob olfactory bulb, P.ge globus pallidus external, S.d1 direct striatum projection, S.d2 indirect striatum projection, Snc substantia nigra pars compacta, Snr substantia nigra pars reticulata.

Selecting the seek column enables the odor sense to MLR path, seeking the potential food odor. Selecting the explore column enables the H.stn to MLR path, randomly searching for food.

Note: the double inversion in both paths is to reduce neuron noise [Cohen et al. 2009]. Removing inhibition reduces noise, where adding excitation would add noise. In the essay stimulation, this double negation isn’t necessary.

Striatum with dopamine/habenula control

The previous dual column circuit isn’t sufficient for the problem, because it lacks a control signal to switch between exploit (seek) and explore. The striatum dopamine circuit might help this problem by bringing in the foraging implementation from essay 17.

A major problem in essay 17 was the tradeoff between persistence and perseverance in seeking an odor. Persistence ensures that seeking an odor will continue even when the intermittent. Perseverance is a failure mode where the animal never gives up, like a moth to a flame. As a model, consider using dopamine in the striatum as persistence or effort [Salamone et al. 2007], and control of dopamine by the habenula as solving perseverance with a give-up circuit.

Explore and exploit (seek) columns controlled by dopamine. H.l lateral hypothalamus, Hb.l lateral habenula, H.stn sub thalamic nucleus, MLR midbrain locomotive region, Ob olfactory bulb, P.em pre thalamic eminence, P.ge globus pallidus external, S.d1 striatum direct projection, S.d2 striatum indirect projection, Snc substantia nigra pars compacta, Snr substantia nigra pars reticulata.

The striatum uses two opposing dopamine receptors named D1 and D2. D1 is a stimulating modulator though a G.s protein path, and D2 is an inhibiting modulator through a G.i protein path. In the above diagram, high dopamine will activate the seek column via D1 and inhibiting the explore column via D2. Low dopamine inhibits the seek column and enables the explore column. So dopamine becomes an exploit vs explore controller.

In many primitive animals, dopamine is a food signal. In c.elegans the dopamine neuron is a food-detecting sensory neuron. In vertebrates, the hunger and food-seeking areas like the lateral hypothalamus (H.l) strongly influence midbrain dopamine neurons both directly and indirectly. Indirectly, H.l to lateral habenula (Hb.l) causes non-reward aversion [Lazaridis et al. 2019].

For the essay, I’m taking H.l as multiple roles (H.l is a composite area with at least nine sub-areas [Diaz et al. 2023]), both calculating potential reward (odor) via the H.l to Vta/Snc connection, and cost (exhaustion of seek task without success) via the H.l to Hb.l to Vta/Snc connection.

References

Cisek P. Resynthesizing behavior through phylogenetic refinement. Atten Percept Psychophys. 2019 Oct

Cisek P. Evolution of behavioural control from chordates to primates. Philos Trans R Soc Lond B Biol Sci. 2022 Feb 14

Cohen MR, Maunsell JH. Attention improves performance primarily by reducing interneuronal correlations. Nat Neurosci. 2009 Dec;12(12):1594-600.

Derjean D, Moussaddy A, Atallah E, St-Pierre M, Auclair F, Chang S, Ren X, Zielinski B, Dubuc R. A novel neural substrate for the transformation of olfactory inputs into motor output. PLoS Biol. 2010 Dec 21

Diaz, C., de la Torre, M.M., Rubenstein, J.L.R. et al. Dorsoventral Arrangement of Lateral Hypothalamus Populations in the Mouse Hypothalamus: a Prosomeric Genoarchitectonic Analysis. Mol Neurobiol 60, 687–731 (2023).

Gruberg E., Dudkin E., Wang Y., Marín G., Salas C., Sentis E., Letelier J., Mpodozis J., Malpeli J., Cui H. Influencing and interpreting visual input: the role of a visual feedback system. J. Neurosci. 2006;26:10368–10371

Lazaridis I, Tzortzi O, Weglage M, Märtin A, Xuan Y, Parent M, Johansson Y, Fuzik J, Fürth D, Fenno LE, Ramakrishnan C, Silberberg G, Deisseroth K, Carlén M, Meletis K. A hypothalamus-habenula circuit controls aversion. Mol Psychiatry. 2019 Sep

Ryczko D, Grätsch S, Schläger L, Keuyalian A, Boukhatem Z, Garcia C, Auclair F, Büschges A, Dubuc R. Nigral Glutamatergic Neurons Control the Speed of Locomotion. J Neurosci. 2017 Oct 4

Salamone JD, Correa M, Nunes EJ, Randall PA, Pardo M. The behavioral pharmacology of effort-related choice behavior: dopamine, adenosine and beyond. J Exp Anal Behav. 2012 Jan

Watson GDR, Hughes RN, Petter EA, Fallon IP, Kim N, Severino FPU, Yin HH. Thalamic projections to the subthalamic nucleus contribute to movement initiation and rescue of parkinsonian symptoms. Sci Adv. 2021 Feb 5

Essay 18: Proto-striatum

By

On September 29, 2023

In Uncategorized

A problem with essay 17 was the lack of action stickiness, which became a problem for avoiding obstacles. When the animal hits an obstacle head-on, both touch sensors fire and the animal chooses a direction randomly. Because the decision repeats every tick (30ms) and chooses randomly to break ties, the animal flutters between both choices and remains stuck until enough random choices are in the same direction to escape the obstacle. What’s needed is a stick choice system to keep a direction once it’s selected. In some decision studies, this is a “win-stay” capability.

A previous essay solved this issue with muscle-based timing or a dopamine-based system, but some of the theories of the striatum function suggest it might solve the problem. The core idea uses the dopamine as a feedback enhancer to sway choice to “stay.”

Simplified proton-striatum circuit for “win-stay.” B.ss somatosensory (touch), B.rs reticulospinal motor control, M.lr midbrain locomotive region, S.pv parvalbumin GABA inhibitory interneuron, Snc substantia nigra pars compacta, S.spn striatum spiny projection neuron (aka medium spiny neuron), ACh acetylcholine, DA dopamine.

The circuit is intended not as the full vertebrate basal ganglia, but a possible core function for a pre-vertebrate animal in the early Cambrian. The circuit here represents only the direct path and specifically only the striostome (patch) circuit, and only represents the downstream connections, and ignores the efferent copy and upstream enhancements. Despite being simplified, I think it’s still to complicated as a single evolutionary step.

Simplified proto-circuit

If that simplified striatal circuit is too complicated for an evolutionary step, but lateral inhibition is a reasonable circuit.

Simplified photo-circuit with lateral inhibition.

The above simplified circuit is a simple lateral inhibition circuit with an added reset function from the motor region.

The main path is through the somatosensory touch (B.ss), through the substantia nigra pars compacta (Snc – posterior tubuculum in zebrafish) to the midbrain locomotive region (M.lr). [Derjean et al. 2010] traced a similar path for olfactory information. I’m just replacing odor with touch.

The reset function might be a simple efferent copy from the central pattern generator for timing. In a swimming animal like an eel, the spinal cord controls the oscillation of body undulation, moving the animal forward. Because the cycle is periodic, when the motor system fires at a specific phase such as an initial-segment muscle twitch, it can send a copy of the motor signal upstream as an efferent copy. That signal is periodic, clock-like, something like the theta oscillation in vertebrates, and upper layers can use that clock.

Zebrafish larva swim in discrete bouts, each on the order of 500ms to 2sec. Since the specific mechanism that organizes bouts isn’t known, any model is just a guess, but might motivate some of the striatal circuitry. Specifically, the acetylcholine (ACh) path in the striatum. The motor swimming clock could break movement into bouts with a reset signal.

Since the sense to Snc to M.lr is a known circuit [Derjean et al. 2010], lateral inhibition is a common circuit, and motor efferent copy of central pattern oscillation is also common, this simplified circuit seems like a plausible evolutionary step.

Improved circuit

Some problems in the simplified circuit lead to improvements in the full circuit. The simplified circuit is susceptible to noise, leading to twitchy behavior, because sensors and nerves are noisy. Secondly, when two options compete, a weaker signal might win the competition if it arrives first. An accumulator system that averages the signals will give better comparisons.

To improve the decisions, the new circuit adds a single pair of inhibition neurons, specializes the existing neurons, and changes the connections.

Circuit improving noise and decision.

To improve decision making, the S.spn neurons are now accumulators, averaging inputs over 100ms or so, just long enough to reduce noise without harming response time too much. As an implementation detail, the S.spn neurons might either accumulate calcium (Ca) itself, or a partner astrocyte might accumulate Ca.

To improve noise behavior, the added Snc inhibition neurons tonically inhibit the Snc neurons, so a stray signal from B.ss to Snc won’t inadvertently trigger the action before the decision. The dual inhibition is a slightly complicated circuit which reduces noise because an active path (disinhibited) has only sense inputs; the modulatory signals are taken away.

The dopamine feedback has the benefit of being a modulator instead of a pure feedback signal. Because it’s a multiplicative modulator, dopamine doesn’t trigger the cycle itself. When the signal ends, the dopamine feedback doesn’t continue a ghost reverberation signal.

Choice decisions: drift diffusion

Psychologists, economists, and neuroscientists have several useful models for decision making, primarily deriving from the drift diffusion model [Ratcliff and McKoon 2008], which extends a random walk model to decision-making. While most of the research appears to be centered on visual choice in the cortical (C) visual system, such as the lateral intraparietal area (C.lip), the concepts are general and the circuits simple, which could apply to many neural circuits, even outside of the mammalian cortex.

Drift-diffusion is a variation of a random walk. Each new datum adds a vector to an accumulator, walking a step, until the result crosses a threshold.

Circuits for leaky competing accumulator (LCA) and feed-forward models of two-choice decision.

One simple model is the leaky competing accumulator (LCA) of [Usher and McClelland 2001], where each choice has an accumulator, and the accumulators inhibit each other laterally. Another model use feedforward inhibition instead of lateral inhibition, where each sense inhibits its competitors. For this essay, these models seem a good, simple options for the simulation.

In the context of the striatum, [Bogacz and Gurney 2007] analyze the basal ganglia and cortex as a choice-based decision system. They interpret the direct path (S.d1) as the primary accumulator, and the indirect path (S.d2 / P.ge / H.stn) as feed-forward inhibition. They suggest that the basal ganglia could produce near-optimal decision in the two-choice task.

References

Bogacz R, Gurney K. The basal ganglia and cortex implement optimal decision making between alternative actions. Neural Comput. 2007;19:442–477

Derjean D, Moussaddy A, Atallah E, St-Pierre M, Auclair F, Chang S, Ren X, Zielinski B, Dubuc R. A novel neural substrate for the transformation of olfactory inputs into motor output. PLoS Biol. 2010 Dec 21

Ratcliff, R., & Childers, R. (2015). Individual differences and fitting methods for the two-choice diffusion model of decision making. Decision, 2(4), 237.

Usher, M., & McClelland, J. L. (2001). On the time course of perceptual choice: The leaky competing accumulator model. Psychological Review, 108, 550–592.

Wang, X.-J. (2002). Probabilistic decision making by slow reverberation in cortical circuits. Neuron, 36, 1–20.

Essay 15: Exploring Fruit Fly Mushroom Bodies

By

On August 7, 2023

In Uncategorized

In the fruit fly Drosophila, the mushroom body is an olfactory associative learning hub for insects, similar to the amygdala (or hippocampus or piriform, O.pir) of vertebrates, learning which odors to approach and which to avoid. Although the mushroom body (MB) is primarily focused on olfactory senses, it also receives temperature, visual and some other senses. The genetic markers for MB embryonic development in insects match the markers for cortical pyramidal neurons and other learning neurons like the cerebellum granule cells.

While most studies seem to focus on the mushroom body as an associative learning structure, others like [Farris 2011] compare it to the cerebellum for action timing and adaptive filtering.

Essay 15 will likely expand the essay 14 slug that avoided obstacles, adding odor tracking. The mushroom body should show how to improve the slug behavior without adding too much complexity.

Mushroom body architecture

The mushroom body is a highly structured system, comprised of Kenyon cells (KC), mushroom body output neurons (MBONs), and dopamine (DAN) and optamine (OAN) neurons (collectively MBIN – mushroom body input neurons). The outputs are organized into exactly output compartments, each with only one or two output neurons (MBONs) [Aso et al. 2014].

Mushroom body architecture. Diagram adapted from [Aso et al. 2014].

The 50 primary olfactory neurons (ORN) and 50 projection neurons (PN) project to 2,000 KC neurons. Each KC neuron connects to 6-8 PNs with claw-like dendrites. The 2,000 KC neurons then converge to 34 MBONs. In vertebrates, divergence-convergence pattern occurs in the cerebellum with granule cells to Purkinje cells, in the hippocampus with dentate gyrus to CA3, in the striatum with MSN (medium spiny neurons) to S.nr/S.nc (substantia nigra), and in cortical areas with large granular areas.

Habituation (ORN, PN, LN1)

Olfactory habituation is handled by the LN1 circuit from the olfactory receptor neuron (ORN) to projection neuron (PN) connection. The fly will ignore an odor after 30 minutes, and the odor remains ignored for 30 to 60 minutes. Since the LN1 GABA inhibition neuron implements the habituation, suppressing LN1 can reverse habituation nearly instantly.

Habituation is odor-specific because it’s synapsed based, but a single neuron inhibits multiple odors. (There are multiple LN1 inhibitors, but they’re not odor-specific and inhibit multiple odors.) Habituation is odor-specific because it inhibits on a per-synapse system. LN1 doesn’t inhibit the PN, but inhibits the synapse from an ORN to the target PN. So, only figuring PNs trigger habituation, despite relying on a shared inhibitory neuron. [Shen 2020].

Sparse KC firing (KC, APL)

The 50 projection neurons (PNs) diverge into 2,000 Kenyon cells(KC). Each KC has 6-8 claw-like dendrites to 6-8 PNs. Each fly has a random connection between the PNs and its KCs. The KCs that fire for an odor form a “tag,” which is essentially a hash of the odor inputs.

About 5% of the KCs fire for any odor, which is strictly enforced by the APL circuit. APL is a single inhibitory GABA neuron per side that reciprocally connects with all 2,000 KCs. The APL inhibition threshold ensures only the strongest 5% of KCs will fire.

A computer algorithm by [Dasgupta et al. 2017] uses this KC tag for a “fly hash” algorithm, which computes a hash using a random, sparse connection for locality-sensitive hashing (LSH). LSH retains information from the origin data to retain distance between hashes as reflective as distance between odors. Interestingly, they use the fly hash with visual input, treating pixels as equivalent to odors. Despite the significant differences between unique odors and arbitrary pixels, the results for the fly hash are better than other similar LSH systems.

Dopamine and compartments (MBON, DAN)

The output compartments for the mushroom body are broadly organized into attractive and repellant groups with seven repelling compartments and nine attracting compartments. Three of the compartments (g1-g3) feed forward into other compartments.

Each compartment is fed by one or two dopamine neuron types (DAN), for a total of 20 DAN types for 16 MBON compartments. The repellant section (PPL1) is even more restricted with exactly one dopamine neuron per compartment. The attractive section (PAM) has a bound 20 DAN cells per compartment.

In a study of larval Drosophila [Eichler et al. 2020] studied the entire connectome of the mushroom body and found highly specific reciprocal MBON connections, inhibitory, directly excitatory and axon excitatory.

An [Eschbach et al. 2020] study looked at dopamine (DAN) connective. Each DAN type has distinct inputs from raw value stimuli (unconditioned stimuli – US), and also internal state, and feed back from the mushroom body. So, while the DANs do convey unconditioned stimuli for learning, they’re not a simplistic reward and punishment signal.

Essay 15 simulation direction

Essay 15 will likely continue the slug-like animal from essay 14, and add odor approaching. Because I think the slug will need habituation immediately, I’ll probably explore habituation first.

The essay will probably next explore a single KC to a single MBON, simulating a single odor.

I’m tempted to explore multiple MBON compartments before exploring multiple KCs. Partially for general contrariness: pushing against the simple reward/punishment model to see if it goes anywhere, or if multiple dopamine sources needlessly complicated learning.

And, actually, since I want to push against learning as a solution for everything, to see how much of the mushroom body function can work without any learning at all, treating habituation as short term memory, not learning.

References

Aso Y, Hattori D, Yu Y, Johnston RM, Iyer NA, Ngo TT, Dionne H, Abbott LF, Axel R, Tanimoto H, Rubin GM. “The neuronal architecture of the mushroom body provides a logic for associative learning.” Elife. 2014 Dec 23;3:e04577. doi: 10.7554/eLife.04577. PMID: 25535793; PMCID: PMC4273437.

Dasgupta S, Stevens CF, Navlakha S. “A neural algorithm for a fundamental computing problem.” Science. 2017 Nov 10;358(6364):793-796. doi: 10.1126/science.aam9868. PMID: 29123069.

Eichler, Katharina, et al. “The complete connectome of a learning and memory centre in an insect brain.” Nature 548.7666 (2017): 175-182.

Eschbach, Claire, et al. “Recurrent architecture for adaptive regulation of learning in the insect brain.” Nature Neuroscience 23.4 (2020): 544-555.

Farris, Sarah M. “Are mushroom bodies cerebellum-like structures?.” Arthropod structure & development 40.4 (2011): 368-379.

Shen Y, Dasgupta S, Navlakha S. “Habituation as a neural algorithm for online odor discrimination.” Proc Natl Acad Sci U S A. 2020 Jun 2;117(22):12402-12410. doi: 10.1073/pnas.1915252117. Epub 2020 May 19. PMID: 32430320; PMCID: PMC7275754.

Powered by WordPress & Theme by Anders Norén