Theoretical Reorganization

April 13, 2019April 13, 2019 Mark DavisLeave a comment

Sean Carroll of Caltech takes on the philosophy of science in his paper, Beyond Falsifiability: Normal Science in a Multiverse, as part of a larger conversation on modern theoretical physics and experimental methods. Carroll breaks down the problems of Popper’s falsification criterion and arrives at a more pedestrian Bayesian formulation for how to view science. Theories arise, theories get their priors amplified or deflated, that prior support changes due to—often for Carroll—coherence reasons with other theories and considerations and, in the best case, the posterior support improves with better experimental data.

Continuing with the previous posts’ work on expanding Bayes via AIT considerations, the non-continuous changes to a group of scientific theories that arrive with new theories or data require some better model than just adjusting priors. How exactly does coherence play a part in theory formation? If we treat each theory as a binary string that encodes a Turing machine, then the best theory, inductively, is the shortest machine that accepts the data. But we know that there is no machine that can compute that shortest machine, so there needs to be an algorithm that searches through the state space to try to locate the minimal machine. Meanwhile, the data may be varying and the machine may need to incorporate other machines that help improve the coverage of the original machine or are driven by other factors, as Carroll points out:

We use our taste, lessons from experience, and what we know about the rest of physics to help guide us in hopefully productive directions.

The search algorithm is clearly not just brute force in examining every micro variation in the consequences of changing bits in the machine. Instead, large reusable blocks of subroutines get reparameterized or reused with variation.… Read the rest

Free Will and Algorithmic Information Theory (Part II)

March 28, 2019March 28, 2019 Mark DavisLeave a comment

So we get some mild form of source determinism out of Algorithmic Information Complexity (AIC), but we haven’t addressed the form of free will that deals with moral culpability at all. That free will requires that we, as moral agents, are capable of making choices that have moral consequences. Another way of saying it is that given the same circumstances we could have done otherwise. After all, all we have is a series of if/then statements that must be implemented in wetware and they still respond to known stimuli in deterministic ways. Just responding in model-predictable ways to new stimuli doesn’t amount directly to making choices.

Let’s expand the problem a bit, however. Instead of a lock-and-key recognition of integer “foodstuffs” we have uncertain patterns of foodstuffs and fallible recognition systems. Suddenly we have a probability problem with P(food|n) [or even P(food|q(n)) where q is some perception function] governed by Bayesian statistics. Clearly we expect evolution to optimize towards better models, though we know that all kinds of historical and physical contingencies may derail perfect optimization. Still, if we did have perfect optimization, we know what that would look like for certain types of statistical patterns.

What is an optimal induction machine? AIC and variants have been used to define that machine. First, we have Solomonoff induction from around 1960. But we also have Jorma Rissanen’s Minimum Description Length (MDL) theory from 1978 that casts the problem more in terms of continuous distributions. Variants are available, too, from Minimum Message Length, to Akaike’s Information Criterion (AIC, confusingly again), Bayesian Information Criterion (BIC), and on to Structural Risk Minimization via Vapnik-Chervonenkis learning theory.

All of these theories involve some kind of trade-off between model parameters, the relative complexity of model parameters, and the success of the model on the trained exemplars.… Read the rest

Free Will and Algorithmic Information Theory

March 5, 2019March 24, 2019 Mark Davis1 Comment

I was recently looking for examples of applications of algorithmic information theory, also commonly called algorithmic information complexity (AIC). After all, for a theory to be sound is one thing, but when it is sound and valuable it moves to another level. So, first, let’s review the broad outline of AIC. AIC begins with the problem of randomness, specifically random strings of 0s and 1s. We can readily see that given any sort of encoding in any base, strings of characters can be reduced to a binary sequence. Likewise integers.

Now, AIC states that there are often many Turing machines that could generate a given string and, since we can represent those machines also as a bit sequence, there is at least one machine that has the shortest bit sequence while still producing the target string. In fact, if the shortest machine is as long or a bit longer (given some machine encoding requirements), then the string is said to be AIC random. In other words, no compression of the string is possible.

Moreover, we can generalize this generator machine idea to claim that given some set of strings that represent the data of a given phenomena (let’s say natural occurrences), the smallest generator machine that covers all the data is a “theoretical model” of the data and the underlying phenomena. An interesting outcome of this theory is that it can be shown that there is, in fact, no algorithm (or meta-machine) that can find the smallest generator for any given sequence. This is related to Turing Incompleteness.

In terms of applications, Gregory Chaitin, who is one of the originators of the core ideas of AIC, has proposed that the theory sheds light on questions of meta-mathematics and specifically that it demonstrates that mathematics is a quasi-empirical pursuit capable of producing new methods rather than being idealistically derived from analytic first-principles.… Read the rest

The Elusive in Art and Artificial Intelligence

February 2, 2019February 2, 2019 Mark DavisLeave a comment

Per caption. — Deep Dream (deepdreamgenerator.com) of my elusive inner Van Gogh.

How exactly deep learning models do what they do is at least elusive. Take image recognition as a task. We know that there are decision-making criteria inferred by the hidden layers of the networks. In Convolutional Neural Networks (CNNs), we have further knowledge that locally-receptive fields (or their simulated equivalent) provide a collection of filters that emphasize image features in different ways, from edge detection to rotation-invariant reductions prior to being subjected to a learned categorizer. Yet, the dividing lines between a chair and a small loveseat, or between two faces, is hidden within some non-linear equation composed of these field representations with weights tuned by exemplar presentation.

This elusiveness was at least part of the reason that neural networks and, generally, machine learning-based approaches have had a complicated position in AI research; if you can’t explain how they work, or even fairly characterize their failure modes, maybe we should work harder to understand the support for those decision criteria rather than just build black boxes to execute them?

So when groups use deep learning to produce visual artworks like the recently auctioned work sold by Christie’s for USD 432K, we can be reassured that the murky issue of aesthetics in art appreciation is at least paired with elusiveness in the production machine.

Or is it?

Let’s take Wittgenstein’s ideas about aesthetics as a perhaps slightly murky point of comparison. In Wittgenstein, we are almost always looking at what are effectively games played between and among people. In language, the rules are shared in a culture, a community, and even between individuals. These are semantic limits, dialogue considerations, standardized usages, linguistic pragmatics, expectations, allusions, and much more.… Read the rest

Poetics and Humanism for the Solstice

December 19, 2018December 19, 2018 Mark Davis4 Comments

There is, necessarily, an empty center to secular existence. Empty in the sense that there is no absolute answer to the complexities of human life, alone or as part of the great societies that we have created. This opens us to wild, adventurous circuits through pain, meaning, suffering, growth, and love. Religious writers in recent years have had a tendentious tendency to denigrate this fantastic adventure, as Andrew Sullivan does in New York magazine. The worst possible argument is that everything is religion insofar as we believe passionately about its value. It’s wrong if for no other reason than the position of John Gray that Sullivan quotes:

Religion is an attempt to find meaning in events, not a theory that tries to explain the universe.

Many religious people absolutely disagree with that characterization and demand an entire metaphysical cosmos of spiritual entities and corresponding goals. Abstracting religion to a symbolic labeling system for prediction and explanation robs religion, as well as reason, art, emotion, conversation, and logic, of any independent meaning at all. So Sullivan and Gray are so catholic in their semantics that the words can be replanted to justify almost anything. Moreover, the subsequent claim about religion existing because of our awareness of our own mortality is not borne out by the range of concepts that are properly considered religious.

In social change Sullivan sees a grasping towards redemption, whether in the Marxist-idolatrous left or the covertly idolatrous right, but a more careful reading of history proves Sullivan wrong on the surface, at least, if not in the deeper prescription. For instance, it is not faith in progress that has been part of the liberal social experiment since the Enlightenment, but a grasping towards actual reasons and justifications for what is desired and how to achieve it.… Read the rest

Indifference and the Cosmos

October 28, 2018October 28, 2018 Mark DavisLeave a comment

I am a political independent, though that does not mean that I vote willy-nilly. I have, in fact, been reliably center left for most of my adult life, save one youthfully rebellious moment when I voted Libertarian, more as a statement than a commitment to the principles of libertarianism per se. I regret that vote now, given additional exposure to the party and the kinds of people it attracts. To me, the extremes of the American political system build around radical positions, and the increasingly noxious conspiracy theories and unhinged rhetoric is nothing like the cautious, problem-solving utopia that might make me politically happy, or at least wince less.

Some might claim I am indifferent. I would not argue with that. In the face of revolution, I would require a likely impossible proof of a better outcome before committing. How can we possibly see into such a permeable and contingent future, or weigh the goods and harms in the face of the unknown? This idea of indifference, as a tempering of our epistemic insights, serves as a basis for an essential idea in probabilistic reasoning where it even has the name, the principle of indifference, or, variously, and in contradistinction with Leibniz’s principle of sufficient reason, the principle of insufficient reason.

So how does indifference work in probabilistic reasoning? Consider a Bayesian formulation: we inductively guess based on a combination of a priori probabilities combined with a posteriori evidences. What is the likelihood of the next word in an English sentence being “is”? Indifference suggests that we treat each word as likely as any other, but we know straight away that “is” occurs much more often than “Manichaeistic” in English texts because we can count words.… Read the rest

Structure and Causality in Political Revolutions

September 28, 2018September 28, 2018 Mark DavisLeave a comment

Can political theories be tested like scientific ones? And if they can, does it matter? Alexis Papazoglou argues in the New Republic that, even if they can be tested, it is less important than other factors in the success of the political theory. In his signal case, the conflict between anti-globalist populists and the conventional international order is questioned as resulting in clear outcomes that somehow will determine the viability of one theory versus the other. It’s an ongoing experiment. Papazoglou breaks down the conflict as parallel to the notion that scientific processes ultimately win on falsifiability and rationality. In science, as per Kuhn’s landmark The Structure of Scientific Revolutions, the process is more paradigmatic agendas, powerful leaders, and less calculated rationality.

The scientific process may have been all of those things, of course, and may continue to be so in the future, but there are ongoing developments that make it less likely that sociological factors will dominate. And this is why the comparison with political theories is perhaps wrongheaded. There may be a community of political theorists but they are hardly the primary architects and spectators of politics, unlike science and scientists. We are all political actors, yet very few have the time or inclination to look carefully at the literature on the threat of successful authoritarian Chinese civilization versus Western liberal democracy, for instance. But we are not all scientific actors, despite being governed by the reality of the world around us. Politics yells and seethes while science quietly attends a conference. Even the consequences of science are often so gradualistic in their unfolding that we barely notice them; see the astonishing progress on cancer survival in the past decades and note the need for economic discounting for global climate change, where the slow creep of existential threats are somehow given dollar values.… Read the rest

Hypersensitive Conspiracy Disorder

August 27, 2018August 27, 2018 Mark DavisLeave a comment

I was once cornered in a bar in Suva, Fiji by an Indian man who wanted to unburden himself and complain a bit. He was convinced that the United States had orchestrated the coups of 1987 in which the ethnically Fijian-dominated military took control of the country. The theory went like this: ethnic Indians had too much power for the Americans to bear as we were losing Subic Bay as a deep water naval base in the South Pacific. Suva was the best, nearest alternative but the Indians, with their cultural and political ties to New Delhi, were too socialist for the Americans. Hence the easy solution was to replace the elected government with a more pro-American authoritarian regime. Yet another Cold War dirty tricks effort, like Mossaddegh or Allende, far enough away that the American people just shrugged our collective shoulders. My drinking friend’s core evidence was an alleged sighting of Oliver North by someone, sometime, chatting with government officials. Ollie was the 4D chess grandmaster of the late 80s.

It didn’t work out that way, of course, and the coups continued into the 2000s. More amazing still was that the Berlin Wall came down within weeks of that bar meetup and the entire engagement model for world orders slid into a brief decade of deconstruction and confusion. Even the economic dominance of Japan ebbed and dissipated around the same time.

But our collective penchant for conspiracy theories never waned. And with the growth of the internet and then social media, the speed and ease of disseminating fringe and conspiratorial ideas has only increased. In the past week there were a number of news articles about the role of conspiracy theories, from a so-called “QAnon” advocate meeting with Trump to manipulation of the government by Israel’s Black Cube group.… Read the rest

The Retiring Mind, Part IV: Phenology

August 8, 2018August 9, 2018 Mark DavisLeave a comment

An unexpectedly quick move to Northern Arizona thrust my wife and me into fire and monsoon seasons. The latter term is debatable: monsoons typically involved a radical shift in winds in Southeast Asia. Here the westerlies keep a steady rhythm though the year. The U.S. desert southwest has also adopted the Arabic term “haboob” in recent decades to refer to massive dust storms. If there is a pattern to loanword adoption, it might be a matter of economy. Where a single, unique term can take the place of an elongated description, the loanword wins, even if the nuances of the original get discarded. This continues our child language acquisition tendencies to view different words as being, well, different, even if a strong claim of “one word per meaning” is likely unjustified. We search for replacement terms that provide economy and even relish in the inside knowledge brought by the new lexical entry.

So, as afternoon breaks out into short, heavy downpours we dart in and out of hardware stores getting electrical fishing poles, screw anchors, and #10 8/32nd microbolts to rectify an installation difficulty with a ceiling fan. We meet with contractors and painters who rush through the intermittent squalls. And we break all this up with exploring new restaurants and hitting the local galleries, debating the subterfuge of this or that sculptor in undermining expectations about contemporary trends in southwestern art.

But there is a stability to the forest and canyon around our new house. Deer wander through, but less so as the rain has filled the red rock canyons with watering holes, allowing them to avoid long sojourns to Oak Creek for water. A bobcat nestled for half a morning on our lower deck overlooking the canyon, quietly scanning for prey.… Read the rest

Incompressibility and the Mathematics of Ethical Magnetism

June 22, 2018June 22, 2018 Mark DavisLeave a comment

One of the most intriguing aspects of the current U.S. border crisis is the way that human rights and American decency get articulated in the public sphere of discourse. An initial pull is raw emotion and empathy, then there are counterweights where the long-term consequences of existing policies are weighed against the exigent effects of the policy, and then there are crackpot theories of “crisis actors” and whatnot as bizarro-world distractions. But, if we accept the general thesis of our enlightenment values carrying us ever forward into increasing rights for all, reduced violence and war, and the closing of the curtain on the long human history of despair, poverty, and hunger, we must also ask more generally how this comes to be. Steven Pinker certainly has rounded up some social theories, but what kind of meta-ethics might be at work that seems to push human civilization towards these positive outcomes?