Deep Learning with Quantum Decoherence

Getting back to metaphors in science, Wojciech Zurek’s so-called Quantum Darwinism is in the news due to a series of experimental tests. In Quantum Darwinism (QD), the collapse of the wave function (more properly the “extinction” of states) is a result of decoherence from environmental entanglement. There is a kind of replication in QD, where pointer states are multiplied, and then a kind of environmental selection as well. There is no variation per se, however, though some might argue that the pointer states imprinted by the environment are variants of the originals. Still, it makes the metaphor a bit thin at the edges, but it is close enough for the core idea to fit most of the floor-plan of Darwinism. Indeed, some champion it as part of a more general model for everything. Even selection among viable multiverse bubbles has a similar feel to it: some survive while others perish.

I’ve been simultaneously studying quantum computing and complexity theories that are getting impressively well developed. Richard Cleve’s An Introduction to Quantum Complexity Theory and John Watrous’s Quantum Computational Complexity are notable in their bridging from traditional computational complexity to this newer world of quantum computing using qubits, wave functions, and even decoherence gates.

Decoherence sucks for quantum computing in general, but there may be a way to make use of it. For instance, an artificial neural network (ANN) also has some interesting Darwinian-like properties to it. The initial weight distribution in an ANN is typically a random real value. This is designed to simulate the relative strength of neural connections. Real neural connections are much more complex than this, doing interesting cyclic behavior, saturating and suppressing based on neurotransmitter availability, and so forth, but assuming just a straightforward pattern of connectivity has allowed for significant progress. The connectivity strength then changes based on the stimulation patterns and, in backpropagation training models, based on the difference between the target pattern and the network’s performance. Micro-changes over learning epochs cause evolution of the network. Understanding the minutiae of these effects has always relegated ANNs to “black box” learning machines, subject to potential bias and unexpected outcomes relative to the data.

Still, there is an at least metaphorical similarity to the evolution of quantum systems through decoherence. Instead of environmental drag in the effect of measurements on the quantum system, the ANN state is modified by the error surface relative to the target state.  In other words, the environment is the training data that imprints on the network. Training large networks is so compute and time intensive that it is only recently that repurposing GPUs from video gaming as well as some specialized hardware has allowed very large networks with varying architectures to take on both old and new problems with much greater success. Being able to evolve the system asynchronously has significant performance implications.

But what if the ANNs consisted of qubits that we specifically wanted to decohere towards the target state? This would require somehow probing them proportionally to that error surface. This is clearly similar to quantum annealing in that the error landscape is being probed to find a global optimum, though the typical non-linear activation functions change the nature of the error landscape by transforming the space. It should be noted, too, that most interesting learning problems have been shown to have many local optima, hence interest in some kind of tunneling, parallel methods, or, recently, initialization of networks as auto-encoders.

It’s worth noting that machine learning and quantum ANNs have been developed theoretically since the mid-90s, though the directions of these efforts have not, as far as I can tell, explored this notion of using decoherence as an active component of learning. Generally decoherence associated with copying state from neuron to neuron is considered a nuisance or the topics of interest are more on whether quantum effects might play a part in real neurons or, generally, brain function. Other recent R&D efforts are here and here.

Leave a Reply

Your email address will not be published. Required fields are marked *