Month: March 2019

Free Will and Algorithmic Information Theory (Part II)

Mark DavisLeave a comment
Bad monkey

So we get some mild form of source determinism out of Algorithmic Information Complexity (AIC), but we haven’t addressed the form of free will that deals with moral culpability at all. That free will requires that we, as moral agents, are capable of making choices that have moral consequences. Another way of saying it is that given the same circumstances we could have done otherwise. After all, all we have is a series of if/then statements that must be implemented in wetware and they still respond to known stimuli in deterministic ways. Just responding in model-predictable ways to new stimuli doesn’t amount directly to making choices.

Let’s expand the problem a bit, however. Instead of a lock-and-key recognition of integer “foodstuffs” we have uncertain patterns of foodstuffs and fallible recognition systems. Suddenly we have a probability problem with P(food|n) [or even P(food|q(n)) where q is some perception function] governed by Bayesian statistics. Clearly we expect evolution to optimize towards better models, though we know that all kinds of historical and physical contingencies may derail perfect optimization. Still, if we did have perfect optimization, we know what that would look like for certain types of statistical patterns.

What is an optimal induction machine? AIC and variants have been used to define that machine. First, we have Solomonoff induction from around 1960. But we also have Jorma Rissanen’s Minimum Description Length (MDL) theory from 1978 that casts the problem more in terms of continuous distributions. Variants are available, too, from Minimum Message Length, to Akaike’s Information Criterion (AIC, confusingly again), Bayesian Information Criterion (BIC), and on to Structural Risk Minimization via Vapnik-Chervonenkis learning theory.

All of these theories involve some kind of trade-off between model parameters, the relative complexity of model parameters, and the success of the model on the trained exemplars.… Read the rest

A Southwestern History of Western Music

Mark DavisLeave a comment

Growing up in New Mexico I had an unusual collection of childhood experiences. My father was a professor of engineering at New Mexico State but had done post-docs in astrophysics at the Naval Observatory in Washington D.C. after a doctorate at University of Wisconsin at Madison. I rode along to cosmic ray observatories high in the mountains and joined him at Los Alamos National Laboratory during summer consulting gigs. He introduced an exotic young fellow professor and his wife to me after I became fascinated by insect vision and ideas for simulating how bugs see the world. That, in turn, led to me living with the couple (he with a doctorate in EE and graduate students in evolutionary biology; her with a doctorate in plant physiology doing cancer research) after my father died young. I slept under a workbench while they started a business and built early word processor systems, pivoted to commercial time sharing, then to consulting on weather monitoring systems for White Sands Missile Range.

I was always in special programs and doing science fairs, taking art classes at the university, and whatnot until it became uncool for me for some reason. In a perhaps not unexpected series of intersections by high school I had a collection of friends who were bright and precocious in their interests in a way that I thought perfectly normal at the time. They graduated early, investigated radical ideas and movements in the special collections of the college library, and we variously started university while still in high school. Later, in grad school, an equally creative collection of souls poured themselves into performance art projects and shows that were poorly attended but perhaps only because they were so radical and innovative that even the arts community felt at sea with all the new media forms we were inventing.… Read the rest

Free Will and Algorithmic Information Theory

Mark Davis1 Comment

I was recently looking for examples of applications of algorithmic information theory, also commonly called algorithmic information complexity (AIC). After all, for a theory to be sound is one thing, but when it is sound and valuable it moves to another level. So, first, let’s review the broad outline of AIC. AIC begins with the problem of randomness, specifically random strings of 0s and 1s. We can readily see that given any sort of encoding in any base, strings of characters can be reduced to a binary sequence. Likewise integers.

Now, AIC states that there are often many Turing machines that could generate a given string and, since we can represent those machines also as a bit sequence, there is at least one machine that has the shortest bit sequence while still producing the target string. In fact, if the shortest machine is as long or a bit longer (given some machine encoding requirements), then the string is said to be AIC random. In other words, no compression of the string is possible.

Moreover, we can generalize this generator machine idea to claim that given some set of strings that represent the data of a given phenomena (let’s say natural occurrences), the smallest generator machine that covers all the data is a “theoretical model” of the data and the underlying phenomena. An interesting outcome of this theory is that it can be shown that there is, in fact, no algorithm (or meta-machine) that can find the smallest generator for any given sequence. This is related to Turing Incompleteness.

In terms of applications, Gregory Chaitin, who is one of the originators of the core ideas of AIC, has proposed that the theory sheds light on questions of meta-mathematics and specifically that it demonstrates that mathematics is a quasi-empirical pursuit capable of producing new methods rather than being idealistically derived from analytic first-principles.… Read the rest