The basic problem with quantum mechanics is that it defines two different ‘dynamics’, meaning two different ways in which the physical reality changes over time. And they are both very weird.
The fundamental dynamics is the kind of thing we would expect. It is the ‘linear dynamics’, meaning logical and predictable. This is the sort of thing we are used to. The laws of Newtonian mechanics describe the linear, and completely predictable, dynamics of the ordinary world. This is how we are able to precisely predict the path of a soccer ball, and the actions to jump on a bus. This is called classical physics. Naturally, we are deeply familiar with this dynamics. Classical physics is simply how the everyday world works.
The Wave Function
The linear dynamics of the quantum is weird because although it is predictable, it defines reality only in terms of probabilities. Throw a ball, and physics can predict the trajectory with absolute precision. But throw a quantum and it goes in every possible direction at the same time.
The mathematics of the linear dynamics is the ‘wave function’. The image below is the wave function of a single quantum, such as an electron, in the corner of a square box. This is a wave of probability. The bigger the wave, the greater the probability of the quantum actually being at that position, if a measurement is made with a detector.
This is the first great weirdness. Although it is the basis of all physical reality, the quantum is not a real thing in the same way as the objects of everyday life. This was made clear by Werner Heisenberg who won the Nobel Prize in 1932 for the creation of quantum mechanics. The elementary particles, the quanta:
… form a world of potentialities or possibilities rather than one of things or facts. (1962, 128)
The sum total of zillions of quanta in each ordinary object adds up to produce the ordinary mechanics of Newtons laws, classical mechanics. This is how the quantum world gives rise to the ordinary word we are familiar with. This is why it is called a ‘quasi-classical world’, as if classical. It operates like an ordinary classical world, but is actually made of quanta which work quite differently.
Collapse
This is strange enough, but the totally incomprehensible thing is the ‘collapse dynamics’. When an observation is made, the quantum wave collapses to give rise to a specific tiny particle, such as a ‘real’ electron. But how can just looking at something cause the physical state to change? This is the great paradox of quantum theory.
We know this is what happens. One hundred years of precise experiments have given a unanimous answer. Measurement of the quantum state brings about change of the quantum state. But it makes no sense.
This is what we know. The wave function spreads out until an observation is made. This results in the collapse of the wave function to a specific subatomic particle. This is then the origin of a new wave function.
The Standard Formulation
This alternation of the two dynamics is formalised in the standard formulation of quantum mechanics. This is what the textbooks say. This is the ‘von Neumann-Dirac formulation of quantum mechanics’ (1932). Philosopher Jeffrey Barrett, who specialises in the measurement problem, provides a simplified form:
Linear: If no observation is made, then the quantum system evolves continuously according to the linear, deterministic dynamics.
Collapse: If an observation is made, then the quantum system instantaneously and randomly jumps to a state where it either determinately has or determinately does not have the property being observed. (2008; adapted)
Linear and collapse operate alternately in a cycle. This, essentially, is the quantum theory. And as he also states this is no longer a theory. It is simply how the quantum is known to work.
The Measurement Problem
As Barrett describes this is a superb theory:
The standard theory … is in one sense the most successful physical theory ever … it successfully predicts the behaviour of the basic constituents of all physical things. (1999, 1)
But as he goes on to explain, the theory seems inherently flawed:
… if one supposes that measuring devices are ordinary physical systems just like any other, constructed of fundamental particles interacting in their usual determinate way (and why wouldn’t they be?), then the standard theory is logically inconsistent since no system can obey both the deterministic and stochastic dynamics simultaneously. This is the measurement problem. (1999, 15)
The ‘usual determinate way’ to which he is referring is the linear dynamics, the dynamics of the wave function. The key point is that this gives rise to all possibilities. In the image above, the wave function defines every possible place the quantum could be, if it is observed.
Since, Barrett is saying, the device used to make an observation can only be made of ordinary stuff, that operates according to the linear dynamics, it must therefore wind up in all possible states. In other words, the device must produce every possible result of the observation. In a Yes / No experiment, it must show Yes and No. But only one single result is ever observed. This selection of one outcome or the other is the operation of the stochastic collapse dynamics.
Quantum Decoherence
By the way, you might see it suggested that ‘quantum decoherence’ solves the measurement problem. It does not, as stated by numerous experts.
The wave function of the universe defines all physical possibilities. Quantum decoherence is what transforms the wave function into specific versions of physical reality. This is an interference effect that alters the structure of the wave function, producing essentially classical states. This is what transforms the amorphous fundamental wave function into an ordinary world of the sort we are used to.
It has been proposed that this phenomenon explains the whole problem of the collapse of the wave function, and thus resolves the measurement problem. However, as stated by the physicist Guido Bacciagaluppi, a philosopher of physics mainly working on quantum mechanics:
Unfortunately, naïve claims of the kind that decoherence gives a complete answer to the measurement problem are still somewhat part of the ‘folklore’ of decoherence, and deservedly attract the wrath of physicists (e.g. Pearle 1997) and philosophers (e.g. Bub 1997, Chap. 8) alike. (2012)
This is from his article The Role of Decoherence in Quantum Mechanics in the Stanford Encyclopedia of Philosophy where there is a full discussion of the issue. The measurement problem continues to be acknowledged as an unresolved problem.
No Collapse
The measurement problem is the paradox that Hugh Everett essentially resolved back in 1957, but his explanation has never been understood. He shows that according to the wave function there is no collapse. As Barrett states, such a thing is not possible. Which of course means that the measurement instrument does indeed wind up in all possible states.
Nonetheless, Everett says, there is the appearance of collapse. In other words, only one specific outcome is experienced, subjectively, although both exist. Which of course makes no sense at all in physicalism.
If we suspend disbelief in physicalism, this works perfectly, but it two major implications. Firstly, this is the origin of the many-worlds concept. In a Yes / No experiment, one world has the Yes outcome and the other world has the No outcome. Half a century later this is increasingly accepted as the only feasible explanation. Secondly, it also means that something non-physical is happening to give rise to one experience or the other.
Two Dynamics – Two Worlds
As Barrett explains, the great puzzle is that no system can obey both the two dynamics simultaneously. They are incompatible:
The measurement problem arises from the fact that the standard theory’s two dynamical laws are incompatible: one is linear and the other nonlinear. … they constitute contradictory descriptions of the time-evolution of physical states (Barrett, 2011)
The resolution presented here is that the two dynamics are the operation of the two different types of world. This is the missing piece of the puzzle.
This automatically resolves the measurement problem. The linear dynamics is the time evolution of the physical world. All possible worlds come into being, each one a quasi-classical world of the current worldview. The collapse dynamics is the time evolution of the personal world of the individual, a relative world determinate only where observed.
The next section is Everett’s Solution.