Decisions under Risk and Uncertainty

In this world nothing can be said to be certain, except death and taxes. ~ Benjamin Franklin

In unravelling the causal neural mechanisms that underpin decision making, neuroeconomics formulates the very function and evolution of the nervous systems in terms of decision making [1-2]. Neuroeconomic theory posits that evolutionary processes shape the nervous system to generate choice-behaviours that maximize an organism’s fitness [3]. As Glimcher [1] notes, “The goal of behaviour is to make the right choices; to choose the course of action that maximizes the survival of an organism’s genetic code.”

Of course, phyletic and architectural evolutionary constraints will limit the nervous system from being able to select and execute truly optimal behaviours. Put differently, the sensori-motor and perceptual capacities of an organism restrict the knowledge it can have about the world and thus, its ability to detect and respond optimally to stimuli [2]. Furthermore, environmental stochasticity produces unpredictable changes to the environmental conditions that an organism confronts. This being said, we should expect to observe mechanisms in the brain that evolved for governing choice-behaviours which reflect the uncertainty that is inherent to an organism’s perception and the environment in which it navigates [4]. How, then, would these mechanisms operate in order to select the right choices in the face of imminent risk and uncertainty? Economic models have already proven useful in addressing this question [5]. If we can formulate a plausible mechanism and specify a mathematical model for it, then we can form some clue as to what we should be looking for in the underlying neural activity of the brain. This is the essence of neuroeconomics.

# Distinguishing between risk and uncertainty

Economists generally make a formal distinction between decisions involving uncertainty and those involving risk. Technically speaking, however, risk is only considered to be a specific sub-type of uncertainty. In decisions under risk, the chooser is aware of the specific mathematical probabilities associated with each of the outcomes corresponding to the choices available to them [6]. Decisions involving uncertainty or ambiguity, on the other hand, describe when a chooser must select among multiple outcomes for which the probabilities are either partially or completely unknown [6].

# Neoclassical economic models

Neoclassical economics is a framework of decision-making founded on the notion that choosers behave rationally – that is, they consistently make optimal decisions which minimize losses and maximize gains [7]. Thus, economic models arising from the neoclassical or rational-choice traditions make an implicit assumption that organisms behave rationally when making decisions under risk and uncertainty.

## Expected-value theory

In describing how humans made decisions under conditions of uncertainty, early neoclassical theorists advanced the principle of expected value first posed by Peirre de Fermat and Blaise Pascal in 1654. Expected-value theory is a mathematical formulation that provides an estimate for the value of some event with gain x, given a known probability of its occurrence, p. It is equivalent to the value associated with an event if its outcome were certain, multiplied by the actual probability that the event will occur. Alternatively, it can be interpreted as the mean value, or average pay-off, of a discrete random variable. Stated formally, the expected value (EV) of a gamble X is

(1)
\begin{align} (EV)X=\sum_{x} p(x)\cdot x \end{align}

How can this formulation be employed to discern an optimal course of action? Suppose a vendor presents you with the following wager involving the roll of a fair, six-sided die. If you can correctly guess the number on which the die lands, you will win $300.00. However, if you guess incorrectly, you must pay$50.00 to the vendor. Is the wager worth taking? The probability of correctly guessing the outcome is 1/6, or approximately 17%, while the probability of incorrectly guessing the outcome is 5/6, or approximately 83%. Accordingly, the expected value of correctly guessing the number on which die lands is $50.00 (1/6 x$500.00). The expected value of incorrectly guessing the outcome, on the other hand, is $41.67 (5/6 ×$50.00). Thus, the expected value of winning the gamble is greater than losing it. Therefore, according to neoclassical economics, a rational agent will always accept the wager provided that he or she can afford to pay the vendor should they lose the gamble. Combining probability with value to derive a systematic measure of value under conditions of uncertainty was a critical insight that laid the foundation for modern probability theory [1].

Payoff Distribution of the St. Petersburg Paradox

Figure 1. Expected value remains at $1.00 for each coin flip, but cumulatively approaches infinity. Adapted from Weber, E. U. & Johnson, E. J. (2009). Decisions under uncertainty: Psychological, economic and neuroeconomic explanations of risk preference. In Glimcher et al., Neuroeconomics: Decision making and the brain (127-143). Academic Press: San Diego By the 18th century, expected-value theory had come to dominate economic thought and was reshaping commercial and industrial practices all throughout the West. However, as Nicholas Bernoulli would formally bring to light, the theory yielded a strange paradox, one which would ultimately lead economists to reconsider the universality of the theory in understanding and predicting human decision making [1][8]. Suppose that a vendor has a fair coin and, for a price bargained upon, allows you to enter the following gamble: If, on the first toss, the coin lands as heads, the vendor pays you$2.00 and the game concludes. If, however, the coin lands as tails, he pays you nothing; but, at no extra cost, he offers you an additional toss for which he agrees to pay you $4.00 (if the outcome is heads). Moreover, for every consecutive round that the coin lands as tails, the vendor agrees to another toss with a pay-out double that of the previous round (e.g. three tails and a head:$8.00, four tails and a head: $16.00, etc.) Notice that the expected value for each round remains at$1.00. It is, however, cumulative. In other words, the expected value increases by $1.00 for every consecutive toss, which, hypothetically, could be an infinite number. Accordingly, the gamble has an infinite expected value (see Fig. 1). Therefore, a rational agent should be willing to pay any price to enter the wager [1][8]. In reality, however, this is not what is observed. In fact, few are willing to pay much more than$4.00 to partake in the gamble as was initially realized by casino operators in St. Petersburg [1].

## Risk-aversion

In response to this paradox, Daniel Bernoulli, brother of Nicholas Bernoulli, pointed out that although expected-value theory predicted the, “mathematical expected value… [It] made an unreasonable claim about humans,” namely, that they were insensitive to risk [1]. To illustrate this point consider the following: given a choice between a 50% chance of winning $2000.00 or a sure chance of having$1000.00, which would you prefer? If you are like most people, you would have opted for the latter. Although the expected value is equal in both cases, humans have a tendency to prefer the more certain outcome – a phenomenon known as risk aversion [1].

## Expected-utility hypothesis

 Expected-utility of Wealth Figure 2. Concave utility function, u(x) = x0.5, converting wealth into its expected utility. An increase in wealth from 0 to 1000 units yields greater utility for an individual than an increase from 1000 to 2000 units of wealth [8].

Don't Panic.
In making this insight, Daniel Bernoulli proposed a new economic model that could better account for how humans made decisions under risk and uncertainty. Bernoulli postulated that rather than making decisions in order to maximize expected value, humans instead make choices in order to maximize expected utility [8]. Stated formally [9], if x is the monetary value of a lottery prize with a finite set of selected outcomes, where the function F(x) is the probability of receiving less than or equal to x dollars, and U is a representation of a choosers preferences in expected utility, then for all F:

(2)
\begin{align} U(F)= \int U(x) \cdot dF(x) \end{align}

Utility is a measure of the satisfaction or usefulness that a product or outcome brings to an individual. Utility is a relative value that can depend on various factors relevant to an individual’s situation (e.g. food scarcity or abundance, number of mating opportunities, income level, etc.). Bernoulli’s modification of expected-value theory established a formulation which can account for risk-aversion and inter-individual variability in economic preferences for risk [8]. Differences in risk preference can be influenced by any number of situational factors that are unique to a given individual. Consider again the prospects proposed in the previous section where you are given a choice between either a sure outcome of gaining $1’000.00 or a 50/50 outcome of gaining$2’000.00 or $0.00. An individual who currently has a net wealth of, let’s say, one million dollars is considerably more likely to gamble on winning the$2’000.00 than an individual who has a net wealth of zero dollars. This trend in risk preference highlights the relative utility of wealth (see Fig. 2) [8][10]. Again, wealth does not only need to be considered in terms of monetary gains. It can also be considered in terms of fitness. Suppose the mating season for the eastern gray squirrel is coming to an end. For a male squirrel that has yet to have successfully mated with a female, there is greater utility associated with a sure chance of mating with one female than taking a 50:50 gamble on being able to mate with two females or none at all. Although considerably more accurate than expected-value theory, expected-utility models do sometimes yield inconsistencies with how humans actually make decisions involving risk and uncertainty [10]. Nevertheless, expected-utility models have still proven very informative in understanding the neurobiology of decision making [5].

# Neuroeconomics of decisions under risk and uncertainty

## Neurophysiology of decision variables

 Expected Gain Encoded by LIP Figure 3. Activity of a neuron in LIP differentially correlates with the reward magnitudes associated with different target saccades [5].

In an experiment conducted by Platt and Glimcher, single cell recordings taken from the lateral intraparietal cortex (area LIP) of macaques during an occulo-motor task revealed patterns of neural activity that correlated with representations of probability and expected gain (i.e. subjective value of juice reward) [5]. The authors monitored intraparietal neurons while the monkeys perform a cued saccade task, in which a colour change of the central fixation stimulus indicated to the subjects which of two possible eye-movements would yield a juice-reward. The animals performed successive blocks of 100 trials. Importantly, across trial blocks, the authors systematically varied the probability in which a target saccade would produce a juice reward or the volume of juice that would be delivered. The researchers found that the firing rate of LIP neurons correlated with both the probability that a given saccade would yield a juice reward (outcome probability) and the volume of juice reward to be delivered (expected gain). The experimenters also observed that eye movements associated with LIP neurons seemed to encode both the prior and posterior probabilities relating to the colour change of the central fixation point (red or green; 80:20 and 20:80) and the saccadic eye movement that would be rewarded. Most interestingly, the electrophysiological recordings from LIP neurons during blocks in which the valuation of reward was being manipulated showed that the firing rates plateaued as additional (equivalent) increments of juice volume were added to the reward – as in Fig. 2 [1][5]. This suggests, as Bernoulli had first proposed on a behavioural level, that the brain does not encode the value or gain of a consequence, but instead encodes its utility [1]. Given the evidence showing that neurons in area LIP encode probability and value, or perhaps even utility, it seems plausible that LIP may integrate this information to compute something that resembles expected utility [5]. In more recent studies looking at human subjects, there is accumulating evidence which suggests that subjective value, or the valuation of available choices, is encoded by the mesocorticolimbic dopamine system, a critical brain circuit associated with motivation, reward processing and prediction error [20][21]. The major brain regions comprising this pathway are the ventral tegmental area (VTA), the ventral striatum and the vmPFC [21].

## Neural processing of risk

While traditional economic approaches towards decisions under risk have implicitly assumed some form of cognitive evaluation or appraisal, they have largely neglected the role that emotions might play in risk assessment [11]. Recent psychological and neuroscientific approaches, however, demonstrate that emotions are likely to play a significant role in how organisms process risk [11][14]. In fact, the anterior insula (AI), a neuroanatomical structure strongly implicated in processing aversive emotions such as anxiety, guilt, disappointment, and regret, to name a few, has been consistently demonstrated to be a critical brain region involved in risk assessment [11-15]. Extending previous findings, Kuhnen and Knutson [16], for example, show that activation of the AI correlated with risk-aversion during financial decision-making, in addition to risk-aversion in behavioural tasks, as was initially shown by Paulus et al. [17]. In an fMRI meta-analysis, Mohr and coworkers reveal quite conclusively that activation of the anterior insula (bilaterally) is correlated with decisions under risk, especially when choice outcomes involve potential losses [11]. Thus, it has been frequently suggested that the activity in the AI underlies risk or loss-aversion. However, other findings have suggested that the anterior insular cortex actually promotes risk-seeking behaviour [12]. Further research will be required to resolve these discrepancies. Also consistent with notion that affective states mediate the neural processing of risk is the observation that lesions to the orbitofrontal cortex, a region well-known to be involved in emotion regulation, is associated with an acquired insensitivity to risky gambles [12][18]. Equally important to the neural processing of risk are the cognitive mechanisms involved. Mohr and colleagues show that cognitive risk processing is correlated with bilateral activation of the dorsomedial prefrontal cortex (dmPFC) during both anticipation risk and decision risk. However, dmPFC activation was more likely under circumstances involving decision risk. Additionally, dmPFC activation is also observed for the processing of risky outcomes containing potential losses and potential gains [11]. Also implicated in the cognitive appraisal of risk is the right dorsolateral prefrontal cortex (DLPFC) and right parietal cortex. Unlike dmPFC, however, activation of DLPFC and parietal cortex during risk processing is exclusive to prospects that require an individual to make an explicit decision. Additionally, activity in the right DLPFC has been associated with reward valuation during reward-based decision making [19]. In considering the neural correlates associated with individual differences in risk preference (i.e. risk-seeking vs risk-averse), it has been observed that for risky decisions, individuals who tend to be risk-seeking show a significantly larger BOLD signal in their posterior parietal cortex compared to individuals who are risk-neutral or risk-averse [6]. Additionally, high risk-seeking behaviour is correlated with abnormalities – structural or functional – in the prefrontal cortex [18]. This is consistent with observations that substance-use-disorders, often characterized by dysregulated or pathological activity in the prefrontal cortex, causes addicts to perform significantly poorer on gambling tasks in comparison to healthy controls [13][18].

## Neural processing of uncertainty or ambiguity

In comparing the functional neuroanatomy between decisions under risk (i.e. known probabilities associated with outcomes) and decisions under ambiguity (unknown probabilities associated with outcomes), Huettel and colleagues found a significant correlation with increased activity in posterior part of the inferior frontal cortex (pIFG), the AI, and the posterior parietal cortex (PPC) [6]. However, among these regions of interest, the effect size was most prominent in the pIFG. Furthermore, pIFG activation was almost selective to trials that contained an ambiguous prospect, showing only minor activity during risky prospects. In comparing pIFG activation to participants’ behavioural data, the authors show that the magnitude of the BOLD response could reliably predict interindividual preferences for ambiguous decisions. To elaborate, during trials in which subjects were presented with a choice between either an ambiguous prospect of winning, for example, $35.00 or$0.00 and a certain prospect of winning $12.00 (or alternatively a risky prospect: e.g. 25% chance of$20.00 gain or 75% chance of \$12.00 gain), the frequency in which subjects preferred ambiguous prospects over certain or risky prospects was positively correlated with the level of activation in their pIFG. Conversely, behavioural preferences for risky prospects over those of certainty or ambiguity were proportional to the level of activation observed in subjects’ PPC [6]. In consideration of the above, the authors potentially identify patterns of selective and variable brain activation that highlight the dissociation between risk and uncertainty first proposed in 1921 by the economist Frank Knight [8].

Bibliography
1. Glimcher, P.W. Decisions, Uncertainty and the Brain: The Science of Neuroeconomics. (MIT Press, Cambridge, 2003).
2. Polister, P. Neuroeconomics: A Guide to the New Science of Making Choices. (Oxford Univ. Press, New York, 2008).
3. Zak, P. J. Neuroeconomics. Phil. Trans. R. Soc. Lond. B 359, 1737–1748 (2004).
4. Glimcher, P. W. Foundations of Neuroeconomic Analysis (Oxford Univ. Press, Oxford, 2011).
5. Platt, M. L. & Glimcher, P. W. Neural correlates of decision variables in parietal cortex. Nature 400, 233-238 (1999).
6. Scott, A., Huettel C., Stowe, J., Gordon, E. M., Warner, B.T., & Platt, M. L. Neural signatures of economic preferences for risk and ambiguity. Neuron 49, 765-775 (2006).
7. Glimcher, P.W., Dorris, M.C., & Bayer, H.M. Physiological utility theory and the neuroeconomics of choice. Game. Econ. Behav. 52, 213-256 (2005).
8. Weber, E.U. & Johnson, E.J. in Glimcher, P.W., Fehr, E., Camerer, C. & Poldrack, R. A (eds.). Neuroeconomics: Decision making and the brain. 127-143 (Academic Press, San Diego, 2009)
9. Levin, J. Choice under uncertainty. www.stanford.edu/~jdlevin/Econ%20202/Uncertainty.pdf (2006).
10. Kahneman, D., Tversky, A. Prospect theory: An analysis of decision under risk. Econometrica 47, 263-291 (1979).
11. Mohr, P.N.C., Biele, G. & Heekeren, H. R. Neural processing of risk. J. Neurosci. 30, 6613-6619 (2010).
12. Ishii, H., Ohara, S., Tobler, P.N., Tsutsui, K. & Iijima, T. Inactivating anterior insular cortex reduces risk taking. J. Neurosci. 32, 16031-16039 (2012).
13. Schultz, W. Potential vulnerabilities of neuronal reward, risk, and decision mechanisms to addictive drugs. Neuron 69, 603-617 (2011).
14. Harlé, K.M., Chang, L.J., van 't Wout, M. & Sanfey, A.G. The neural mechanisms of affect infusion in social economic decision-making: A mediating role of the anterior insula. NeuroImage 61, 32 40 (2012).
15. Michl, P., Meindl, T., Meister, F., Born, C., Engel, R.R., Reiser, M. & Hennig-Fast, K. Neurobiological underpinnings of shame and guilt: A pilot fMRI study. Soc. Cogn. Affect. Neurosc. 9, 150-157 (2014).
16. Kuhnen, C.M. & Knutson, B. The neural basis of financial risk taking. Neuron 47, 763-770 (2005).
17. Paulus, M.P., Rogalsky, C., & Simmons, A. Increased activation in the right insula during risk-taking decision making is related to harm avoidance and neuroticism. NeuroImage 19, 1439-1448 (2003).
18. Kishida, K.T., King-Casas, B. & Montague, P.R. Neuroeconomic approaches to mental disorders. Neuron 67, 543-554 (2010).
19. Mohr P. N., Biele G, Krugel L.K., Li S.C. & Heekeren H.R. Neural foundations of risk-return trade-off in investment decisions. NeuroImage 49, 2556 –2563 (2010).
20. Barkley-Levensona, E. & Galvána, A. Neural representation of expected value in the adolescent brain. Proc. Natl. Acad. Sci. U.S.A. 111, 1646–1651 (2014).
21. Goldstein, R.Z., & Volkow, N.D. Dysfunction of the prefrontal cortex in addiction: neuroimaging findings and clinical implications. Nat. Rev. Neurosci. 12, 652-669 (2011).