The Illusion of Free Will

Two boxes sit on a table in front of you. The first is transparent and holds 1,000 dollars. The second is opaque. You may take only the opaque box, or both boxes.

Yesterday, a machine studied you. It has run this game with thousands of people and has never been wrong. If it predicted that you would take only the opaque box, it put 1,000,000 dollars inside. If it predicted that you would take both, it left the opaque box empty. The machine has already finished; the money is either there or it is not. Nothing you do now can change what is in the box.

What do you take?

Most people find the answer obvious. The difficulty is that they split almost evenly on which answer is obvious, and each side tends to think the other is making a simple mistake. ^[1,2] The puzzle is called Newcomb's problem, and it is one of the sharpest ways of asking a very old question: if something could know in advance what you will choose, in what sense are you choosing?

Two Arguments That Cannot Both Win

The case for taking one box

The machine has never been wrong. People who take one box walk away with a million dollars. People who take both walk away with a thousand. If you want to be rich, be the kind of person who takes one box.

The case for taking both boxes

The money was placed yesterday. The box already contains 1,000,000 dollars or nothing. Whatever it contains, taking both boxes gives you exactly 1,000 dollars more than taking one. Leaving a visible 1,000 dollars on the table cannot make money appear in a box that was sealed before you walked in.

The four possible outcomes make the conflict visible:

Read across each column, and taking both boxes is always better by 1,000 dollars. Read down the rows, knowing the machine is almost always right, and taking one box is better by nearly a million.

The problem was invented by the physicist William Newcomb, and first analysed in print by the philosopher Robert Nozick in 1969. ^[1] Martin Gardner's 1973 column in Scientific American brought it to a wide audience and produced a flood of letters from readers, divided and often certain. ^[2] Nozick's own summary of his informal polling remains accurate: the problem looks easy to almost everyone, but not in the same way.

The Formal Description

Decision theory tries to say what a rational agent should do. Newcomb's problem separates two of its main versions.

Evidential decision theory asks: given that I make this choice, what should I expect to get? Suppose the machine is right with probability $p$. Then the expected payoffs are

Here $\mathbb{E}$ means the average payoff you would expect over many repetitions. Taking one box has the higher expectation whenever

In words: if the machine is even slightly better than a coin toss, the evidential answer is to take one box.

Causal decision theory asks a different question: what will my choice actually cause? Your choice cannot cause yesterday's prediction. Whatever the probability that the box is full, your action does not change it, so the 1,000 dollars is a free addition. The causal answer is to take both boxes. ^[3]

Both calculations are internally correct. They disagree because they answer different questions. That is why the problem is still unresolved: in a large survey of professional philosophers published in 2014, about 31% favoured two boxes, about 21% favoured one box, and the remaining respondents chose neither or were undecided. ^[4]

Why Newcomb's Machine Is About Free Will

The puzzle only works if the machine can be almost perfect. What would it need?

The physicist's version of the machine

In 1814 Pierre-Simon Laplace imagined an intellect that knew, at one instant, every force in nature and the position of every body. For such an intellect, he wrote, nothing would be uncertain: the future would lie open to it like the past. ^[5] Laplace's imagined predictor is the ancestor of Newcomb's machine.

The idea behind it is simple. If the world is governed by laws that turn one state into the next, then knowing the complete state of your brain and body, the surroundings, and the laws would, in principle, fix what you will do. You are made of particles. Particles obey equations. The conclusion seems to follow: your choice was settled before you felt yourself making it.

That thesis is determinism, and whether physics actually supports it is a separate and genuinely open question, covered in Determinism. Chaos makes such prediction practically impossible, and quantum mechanics may make it impossible in principle. Newcomb's machine sidesteps both objections by being very good rather than perfect. Even a predictor that reads only your personality, your history, and how you reason about puzzles might do far better than chance.

The feeling that remains

What is striking is that the feeling of choice survives the thought experiment. Standing in front of the boxes, you still deliberate. You still experience the decision as open. The illusion of free will is the claim that this experience misrepresents what is happening: that the sense of being the undetermined author of the choice is produced by the brain, not a window onto how choices are actually made.

What Psychology Adds

Philosophers argued about free will for more than two thousand years without much data. In the last half-century, psychology has studied the experience of choosing, which is a narrower question with experimental answers.

We explain choices we did not make

In 1977 Richard Nisbett and Timothy Wilson reviewed a series of studies in which people confidently explained their own choices with reasons that could not have been the real ones. ^[6] In one, shoppers asked to pick the best of four identical pairs of stockings showed a strong preference for the right-most item, then cited its quality. None mentioned position, and they rejected the suggestion that position mattered.

In 2005 Petter Johansson and colleagues pushed this further. ^[7] Participants chose which of two photographed faces they found more attractive. On some trials, using sleight of hand, the experimenter handed them the face they had not chosen and asked them to explain their choice. Most manipulations went unnoticed, and participants gave fluent reasons for a preference they had never expressed. The researchers called this choice blindness.

We can feel we caused what we did not

Daniel Wegner and Thalia Wheatley argued in 1999 that the feeling of willing an action is an inference: the mind notices a thought that comes just before an action, is consistent with it, and has no rival explanation, and concludes "I did that". ^[8] In one experiment, two people jointly moved a computer mouse over a screen of pictures. One was secretly an assistant who forced the cursor to stop on a particular picture. When the real participant heard the name of that picture through headphones a second or so before the stop, they rated the stop as more intentional, although they had not produced it.

Wegner developed the argument in his 2002 book The Illusion of Conscious Will. His conclusion was not that nothing causes our actions, but that the conscious feeling of willing is not a reliable report of what does.

The timing experiments

The best-known evidence, Benjamin Libet's 1983 finding that brain activity begins before people report deciding to move, has been heavily reinterpreted. Later work suggests the early signal may reflect background fluctuations rather than a hidden decision. That debate is covered in detail in Determinism.

Certainty note. These studies establish that introspection about choice is unreliable in specific, measurable ways. They do not establish that choices have no conscious causes, and they do not settle free will, which is partly a question about what "free" should mean.

Four Positions

The debate is usually divided into four positions. ^[9]

• Hard determinism holds that the world is determined and therefore free will does not exist. Popular books such as Sam Harris's Free Will (2012) and Robert Sapolsky's Determined (2023) argue a version of this.
• Illusionism holds that the experience of free will is systematically misleading, whether or not determinism is true. Wegner's work is the psychological version.
• Libertarianism (in the philosophical, not political, sense) holds that we do have free will, so our choices cannot be fully determined. Its difficulty is explaining how randomness would add control rather than luck.
• Compatibilism holds that free will does not require escaping causation. You act freely when your action comes from your own reasoning and values rather than from coercion, even if those have causes.

In the 2014 survey of philosophers, compatibilism was the most common view by a wide margin, at close to 60%. Libertarianism and the denial of free will were each held by around 12–14%. ^[4] That is a majority, not a consensus.

What the machine means for each

The positions give strikingly different readings of the box game.

For the hard determinist, there is no puzzle about freedom: you were always going to choose what the machine predicted. The only question is which reasoning style earns more money.

For the compatibilist, the machine is not a threat. It predicts you by modelling your reasoning, so your reasoning is exactly what produces the outcome. A predictable choice can still be your choice; a friend who always knows you will keep a promise does not make the promise unfree.

For the libertarian, a perfect predictor would be evidence against free will, which is one reason libertarians doubt such a predictor could exist.

Can the Machine Exist?

The announcement problem

Suppose the machine tells you its prediction before you choose. If it says "you will take both", you can take one out of spite, and the prediction fails. A predictor can only be reliable about an agent who hears the prediction if the agent cannot or does not act against it.

This is not only a trick. In 2008 the physicist David Wolpert proved general limits on what any physical inference device can predict about a universe that contains it, including limits on predicting other devices that can react to its predictions. ^[10] Complete self-inclusive prediction is ruled out by logic before physics even enters. Newcomb's machine avoids the problem by staying silent, but the result shows that "enough information predicts everything" has limits even in a deterministic world.

A backwards-looking loop

The one-box intuition becomes almost undeniable if the machine is not a predictor but a time traveller that simply watched what you did and came back to fill the box. Then your choice really does determine the contents, and taking one box is obviously right. Newcomb's problem sits between that case and an ordinary sealed box, which is why it resembles the consistency puzzles of time travel. Physics has a name for the rule that history must be consistent with its own loops: the Novikov self-consistency principle. A related case, in which an effect becomes its own cause, is the bootstrap paradox.

Does Believing It Matter?

If free will is an illusion, should anyone be told? In 2008 Kathleen Vohs and Jonathan Schooler reported that participants who read a passage arguing against free will cheated more on a later task. ^[11] The finding was widely quoted as a reason to protect the belief.

It did not hold up well. A larger set of studies published in 2020, with bigger samples, found that the reading reduced belief in free will but did not reliably increase cheating. ^[12]

The honest status is: a belief in determinism does not appear to make people behave worse, at least not in the ways that have been measured.

Limitations and Open Problems

• No perfect predictor of human behaviour exists. Real-world prediction of individual choices remains far from Newcomb's machine.
• Physics has not settled determinism. The strongest physical argument for the illusion depends on an assumption that quantum mechanics leaves open.
• The psychology is narrow. Choice blindness and misattributed intention are shown in simple, low-stakes tasks. How far they generalise to deliberate, high-stakes decisions is unknown.
• The question is partly conceptual. Whether free will exists depends on what one means by it. No experiment can decide between compatibilist and incompatibilist definitions.
• Consciousness itself is unexplained. Every position assumes there is someone who feels they are choosing. Why there is any such experience is an unsolved problem, discussed in Sentience.

The Choice That Remains

Newcomb's problem does not tell you what to take. It shows that "free will" names at least two different things: the feeling of an open future, and the fact that your own reasoning produces your actions. The first may be, in part, an illusion. Psychology has shown that the story we tell about our choices can be written after they are made. The second survives even a perfect predictor, because the predictor has to model you to work.

That distinction runs through other questions on this site. A person who refuses an unjust order, as in Civil Disobedience, is acting from reasons that belong to them, whether or not those reasons had causes. A person who refuses every commitment to keep all options open, as in the Puer Aeternus archetype, is guarding the first kind of freedom at the cost of the second. The same tension appears in the prisoner's dilemma, which the philosopher David Lewis argued in 1979 is a Newcomb problem in disguise: the other prisoner, reasoning as you do, is a mirror of your own choice. ^[13]

What we know is that the experience of choosing is unreliable in measurable ways. What physics permits is a world in which a sufficiently informed mind could foresee much of what we do. What remains unresolved is whether that would take anything away from us, or only from the story we tell about ourselves.

References

[1] Nozick, R. (1969). “Newcomb’s Problem and Two Principles of Choice.” In N. Rescher (Ed.), Essays in Honor of Carl G. Hempel, 114–146. Dordrecht: Reidel. https://doi.org/10.1007/978-94-017-1466-2_7

[2] Gardner, M. (1973). “Mathematical Games: Free will revisited, with a mind-bending prediction paradox by William Newcomb.” Scientific American, 229(1), 104–109. https://doi.org/10.1038/scientificamerican0773-104

[3] Weirich, P. “Causal Decision Theory.” Stanford Encyclopedia of Philosophy. https://plato.stanford.edu/entries/decision-causal/

[4] Bourget, D., & Chalmers, D. J. (2014). “What do philosophers believe?” Philosophical Studies, 170(3), 465–500. https://doi.org/10.1007/s11098-013-0259-7

[5] Laplace, P.-S. (1814; English translation 1951). A Philosophical Essay on Probabilities. New York: Dover Publications. https://archive.org/details/philosophicaless00lapl

[6] Nisbett, R. E., & Wilson, T. D. (1977). “Telling more than we can know: Verbal reports on mental processes.” Psychological Review, 84(3), 231–259. https://doi.org/10.1037/0033-295X.84.3.231

[7] Johansson, P., Hall, L., Sikström, S., & Olsson, A. (2005). “Failure to Detect Mismatches Between Intention and Outcome in a Simple Decision Task.” Science, 310(5745), 116–119. https://doi.org/10.1126/science.1111709

[8] Wegner, D. M., & Wheatley, T. (1999). “Apparent mental causation: Sources of the experience of will.” American Psychologist, 54(7), 480–492. https://doi.org/10.1037/0003-066X.54.7.480

[9] O’Connor, T., & Franklin, C. “Free Will.” Stanford Encyclopedia of Philosophy. https://plato.stanford.edu/entries/freewill/

[10] Wolpert, D. H. (2008). “Physical limits of inference.” Physica D: Nonlinear Phenomena, 237(9), 1257–1281. https://doi.org/10.1016/j.physd.2008.03.040

[11] Vohs, K. D., & Schooler, J. W. (2008). “The Value of Believing in Free Will: Encouraging a Belief in Determinism Increases Cheating.” Psychological Science, 19(1), 49–54. https://doi.org/10.1111/j.1467-9280.2008.02045.x

[12] Nadelhoffer, T., Shepard, J., Crone, D. L., Everett, J. A. C., Earp, B. D., & Levy, N. (2020). “Does encouraging a belief in determinism increase cheating? Reconsidering the value of believing in free will.” Cognition, 203, 104342. https://doi.org/10.1016/j.cognition.2020.104342

[13] Lewis, D. (1979). “Prisoners’ Dilemma Is a Newcomb Problem.” Philosophy & Public Affairs, 8(3), 235–240. https://www.jstor.org/stable/2265034