The neural correlates of exploration

Show simple item record

dc.contributor.author Hassall, Cameron Dale
dc.date.accessioned 2019-08-28T19:33:42Z
dc.date.available 2019-08-28T19:33:42Z
dc.date.copyright 2019 en_US
dc.date.issued 2019-08-28
dc.identifier.uri http://hdl.handle.net/1828/11074
dc.description.abstract Like other animals, humans explore to learn about the world, and exploit what we have learned in order to maximize reward. The trade-off between exploration and exploitation is a widely-studied topic that cuts across multiple domains, including animal ecology, economics, and computer science. This work approaches the explore-exploit dilemma from the perspective of cognitive neuroscience. In particular, how are our decisions to explore or exploit represented computationally? And how is that representation implemented in the brain? Experiment 1 examined neural signals following outcomes in a risk-taking task. Explorations – defined as slower responses – were preceded by an enhancement of the P300, a component of the human event-related brain potential thought to reflect a phasic release of norepinephrine from locus coeruleus. Experiment 2 revealed that the same neural signal precedes feedback in a learning task called a two-armed bandit. There, a reinforcement learning model was used to classify responses as either exploitations or explorations; exploitations were driven by previous rewards, and explorations were not. Experiments 3 and 4 extended these results in three important ways. First, evidence is presented that the neural signal observed in Experiments 1 and 2 was driven not only by the upcoming decision, but also by the preceding decision (perhaps even more so). Second, Experiments 3 and 4 involved increasingly larger action spaces. Experiment 3 involved choosing from among either 4, 9, or 16 options. Experiment 4 involved searching for rewards in continuous two-dimensional map. In both experiments, the feedback-locked P300 was enhanced following exploration. Third, exploitation was the more common strategy in Experiments 1 and 2. Thus, it was unclear whether the exploration-related P300 enhancement observed there was due to exploration per se, to exploration rate, or to the fact that exploration was rare compared to exploitation. Experiment 3 partially address this by eliciting different rates of exploration; the exploration-related P300 effect correlated with rate of exploration. In Experiment 4, exploration was more common than exploitation (in contrast to Experiments 1–3); even so, exploration was followed by a P300 enhancement. Together, Experiments 1–4 suggest the presence of a general neural system related to exploration that operates across multiple task types (discrete to continuous), regardless of whether exploration or exploitation is the more common task strategy. The proposed purpose of this neural signal is to interrupt one mode of decision-making (exploration) in favour of another (exploitation). en_US
dc.language English eng
dc.language.iso en en_US
dc.rights Available to the World Wide Web en_US
dc.subject P300 en_US
dc.subject decision making en_US
dc.subject reinforcement learning en_US
dc.subject event-related potential en_US
dc.subject learning en_US
dc.subject reward positivity en_US
dc.subject N200 en_US
dc.subject win-stay, lose-shift en_US
dc.subject computational modelling en_US
dc.subject P200 en_US
dc.subject explore-exploit dilemma en_US
dc.title The neural correlates of exploration en_US
dc.type Thesis en_US
dc.contributor.supervisor Krigolson, Olave E.
dc.contributor.supervisor Holroyd, Clay Brian
dc.degree.department Interdisciplinary Graduate Program en_US
dc.degree.level Doctor of Philosophy Ph.D. en_US
dc.identifier.bibliographicCitation Hassall, C. D., Holland, K., & Krigolson, O. E. (2013). What do I do now? An electroencephalographic investigation of the explore/exploit dilemma. Neuroscience, 228, 361–370. https://doi.org/10.1016/j.neuroscience.2012.10.040 en_US
dc.identifier.bibliographicCitation Hassall, C. D., McDonald, C. G., & Krigolson, O. E. (2019). Ready, set, explore! Event-related potentials reveal the time-course of exploratory decisions. Brain Research, 1719, 183–193. https://doi.org/10.1016/j.brainres.2019.05.039 en_US
dc.description.scholarlevel Graduate en_US

Files in this item

This item appears in the following Collection(s)

Show simple item record

Search UVicSpace


My Account