Author: Mark Davis

Entrepreneur, scientist, engineer, and author. Bent on worldwide understanding.

Trees of Lives

Mark DavisLeave a comment

Tree of LifeWith a brief respite between vacationing in the canyons of Colorado and leaving tomorrow for Australia, I’ve open-sourced an eight-year-old computer program for converting one’s DNA sequences into an artistic rendering. The input to the program are the allelic patterns from standard DNA analysis services that use the Short Tandem Repeat Polymorphisms from forensic analysis, as well as poetry reflecting one’s ethnic heritage. The output is generative art: a tree that overlays the sequences with the poetry and a background rendered from the sequences.

Generative art is perhaps one of the greatest aesthetic achievements of the late 20th Century. Generative art is, fundamentally, a recognition that the core of our humanity can be understood and converted into meaningful aesthetic products–it is the parallel of effective procedures in cognitive science, and developed in lock-step with the constructive efforts to reproduce and simulate human cognition.

To use Tree of Lives, install Java 1.8, unzip the package, and edit the supplied markconfig.txt to enter in your STRs and the allele variant numbers in sequence per line 15 of the configuration file. Lines 16+ are for lines of poetry that will be rendered on the limbs of the tree. Other configuration parameters can be discerned by examining com.treeoflives.CTreeConfig.java, and involve colors, paths, etc. Execute the program with:

java -cp treeoflives.jar:iText-4.2.0-com.itextpdf.jar com.treeoflives.CAlleleRenderer markconfig.txt
… Read the rest

Action on Hadoop

Mark DavisLeave a comment

hadoopinactionThe back rooms of everyone from Pandora to the NSA are filled with machines working in parallel to enrich and analyze data. And mostly at the core is Doug Cutting’s Hadoop that provides an open source implementation of the Google BigTable MapReduce framework combined with a distributed file system for replication and failover. With Hadoop Summit arriving this week (the 6th I’ve been to and the 7th ever), the importance and impact of these technologies continues to grow.

I hope to see you there and I’ll take this opportunity to announce that I am co-authoring Hadoop in Action, 2nd Edition with the original author, Chuck Lam. The new version will provide updates to this best-selling book and introduce all of the newest animals in the Hadoop zoo.… Read the rest

Vin versus the Vampires, Chapter 2: Dealing with the Creditors

Mark DavisLeave a comment

Vin dev Unpublished novel chapter about vampires taking over Hollywood, from the perspective of Vin Diesel. Vin arrives in London to work on a new film about vampires but is attacked by a strange creature while jogging in Hyde Park.

The doctor is a woman, brown, Indian or Pakistani, and, as usual when I first arrive in Britain, I am surprised that the accent can accompany any serious discussion at all. Yes, I had a tetanus shot three years ago. Actually, yes, I had the typhus series, too. No, it was definitely not a dog but admittedly, yes, I am not sure exactly what it was. I’m grinning at her as she projects standardized health system concern through the lilts and dips of pure Londoner. She keeps glancing at my grin, either not recognizing me or just concerned that I am drunk or high. It’s just the accent, I think about blurting out; I can’t take it seriously, sorry, an American oddity exaggerated by the pain in my knee and the early morning hours without much sleep. But I clam up and answer her questions only getting a bit peeved at the third round of, “Had you been drinking”

“No, I was jogging. I was jetlagged. I was jogging. Really.”

There were no stitches, just a bandage and a shot of broad-spectrum antibiotics. As I finished up and signed off, I thought about sneaking a peak at the chart to see if she had annotated “likely alcoholic” or something on the page, but it was almost 9 AM British Summer Time and I needed a nap before my meeting in the afternoon, so I scribbled where I needed to scribble and grabbed a cab back to the hotel, hobbling in past the front desk with my tattered sweats sweeping the marble of the lobby.… Read the rest

Inching Towards Shannon’s Oblivion

Mark Davis3 Comments

SkynetFollowing Bill Joy’s concerns over the future world of nanotechnology, biological engineering, and robotics in 2000’s Why the Future Doesn’t Need Us, it has become fashionable to worry over “existential threats” to humanity. Nuclear power and weapons used to be dreadful enough, and clearly remain in the top five, but these rapidly developing technologies, asteroids, and global climate change have joined Oppenheimer’s misquoted “destroyer of all things” in portending our doom. Here’s Max Tegmark, Stephen Hawking, and others in Huffington Post warning again about artificial intelligence:

One can imagine such technology outsmarting financial markets, out-inventing human researchers, out-manipulating human leaders, and developing weapons we cannot even understand. Whereas the short-term impact of AI depends on who controls it, the long-term impact depends on whether it can be controlled at all.

I almost always begin my public talks on Big Data and intelligent systems with a presentation on industrial revolutions that progresses through Robert Gordon’s phases and then highlights Paul Krugman’s argument that Big Data and the intelligent systems improvements we are seeing potentially represent a next industrial revolution. I am usually less enthusiastic about the timeline than nonspecialists, but after giving a talk at PASS Business Analytics Friday in San Jose, I stuck around to listen in on a highly technical talk concerning statistical regularization and deep learning and I found myself enthused about the topic once again. Deep learning is using artificial neural networks to classify information, but is distinct from traditional ANNs in that the systems are pre-trained using auto-encoders to have a general knowledge about the data domain. To be clear, though, most of the problems that have been tackled are “subsymbolic” for image recognition and speech problems.… Read the rest

Computing the Madness of People

Mark DavisLeave a comment

Bubble playing cardThe best paper I’ve read so far this year has to be Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-sample Performance by David Bailey, Jonathan Borwein, Marcos López de Prado, and Qiji Jim Zhu. The title should ring alarm bells with anyone who has ever puzzled over the disclaimers made by mutual funds or investment strategists that “past performance is not a guarantee of future performance.” No, but we have nothing but that past performance to judge the fund or firm on; we could just pick based on vague investment “philosophies” like the heroizing profiles in Kiplingers seem to promote or trust that all the arbitraging has squeezed the markets into perfect equilibria and therefore just use index funds.

The paper’s core tenets extend well beyond financial charlatanism, however. They point out that the same problem arises in drug discovery where main effects of novel compounds may be due to pure randomness in the sample population in a way that is masked by the sample selection procedure. The history of mental illness research has similar failures, with the head of NIMH remarking that clinical trials and the DSM for treating psychiatric symptoms is too often “shooting in the dark.”

The core suggestion of the paper is remarkably simple, however: use held-out data to validate models. Remarkably simple but apparently rarely done in quantitative financial analysis. The researchers show how simple random walks can look like a seasonal price pattern, and how by sending binary signals about market performance to clients (market will rise/market will fall) investment advisors can create a subpopulation that thinks they are geniuses as other clients walk away due to losses. These rise to the level of charlatanism but the problem of overfitting is just one of pseudo-mathematics where insufficient care is used in managing the data.… Read the rest

Saving Big Data from the Zeros

Mark DavisLeave a comment

ZerosBecause of the hype cycle, Big Data inevitably attracts dissenters who want to deflate a bit the lofty expectations that are built around new technologies that appear mystifying to those on the outside of the Silicon Valley machine. The first response is generally “so what?” and that there is nothing new here, just rehashing efforts like grid computing and Beowulf and whatnot. This skepticism is generally a healthy inoculation against aggrandizement and any kind of hangover from unmet expectations. Hence, the NY Times op-ed from April 6th, Eight (No, Nine!) Problems with Big Data should be embraced for enumerating eight or nine different ways that Big Data technologies, algorithms and thinking might be stretching the balloon of hope towards a loud, but ineffectual, pop.

The eighth of the list bears some scrutiny, though. The authors, who I am not familiar with, focus on the overuse of trigrams in building statistical language models. And they note that language is very productive and that even a short sentence from Rob Lowe, “dumbed-down escapist fare,” doesn’t appear in the indexed corpus of Google. Shades of “colorless green ideas…” from Chomsky, but an important lesson in how to manage the composition of meaning. Dumbed-down escapist fare doesn’t translate well back-and-forth through German via the Google translate capability. For the authors, that shows the failure of the statistical translation methodology linked to Big Data, and ties in to their other concerns about predicting rare occurrences or even, in the case of Lowe’s quote, zero occurrences.

In reality, though, these methods of statistical translation through parallel text learning date to the late 1980s and reflect a distinct journey through ways of thinking about natural language and computing.… Read the rest

Signals and Noise, Chapter 00010100 (Deprogramming)

Mark DavisLeave a comment

cover-design-epubA spiral is an ancient symbol—a snake, an eye, a womb—and a hypnotic focus for mesmerizing the compliant into a hypnagogic state. A spiral is a flow into a singularity. A spiral is a whirlwind. The spiral before Zach’s eyes was generated by a light projector, he knew, and by a filter that was spinning before the projector. He focused and heard only a faint dripping. The fuzziness was falling away from him like he was shedding a cocoon, though, and he soon felt bindings of his arms behind him, metallic and cold, mirroring the cold of the room around him. The spiral was spinning gently, like a pinwheel in a breeze, and Zach found it comforting. It was a flow into a black hole, the negation of everything material, yet the lines of flow never altered or diminished, but extended into forever. A cold universe, empty of the luminous, yet beautiful in its existence, is still cold, he reasoned as he felt the chill rise out of the chair, into his damp back, and his arms. The spiral kept spinning with clockwork regularity.

He finally heard steel slide against steel and a light bloomed to his left, incandescently warm and yellow. A human shadow marched in and stood quietly before him. He didn’t speak at first, waiting to try to see who it was, though suspecting a female form from the subtle hints of hip and slender arm as the shadow moved around him. She slid into the light of the spiral and he recognized Aphrodite from the beach, her hair tamed slightly by a band compressing the afro into three cottontail puffs, above, left and right. She finally spoke, low and even, declaring him cleansed and purified.… Read the rest

Humbly Evolving in a Non-Simulated Universe

Mark DavisLeave a comment

darwin-changeThe New York Times seems to be catching up to me, first with an interview of Alvin Plantinga by Gary Cutting in The Stone on February 9th, and then with notes on Bostrom’s Simulation Hypothesis in the Sunday Times.

I didn’t see anything new in the Plantinga interview, but reviewed my previous argument that adaptive fidelity combined with adaptive plasticity must raise the probability of rationality at a rate that is much greater than the contributions that would be “deceptive” or even mildly cognitively or perceptually biased. Worth reading is Branden Fitelsen and Eliot Sober’s very detailed analysis of Plantinga’s Evolutionary Argument Against Naturalism (EAAN), here. Most interesting are the beginning paragraphs of Section 3, which I reproduce here because it is a critical addition that should surprise no one but often does:

Although Plantinga’s arguments don’t work, he has raised a question that needs to be answered by people who believe evolutionary theory and who also believe that this theory says that our cognitive abilities are in various ways imperfect. Evolutionary theory does say that a device that is reliable in the environment in which it evolved may be highly unreliable when used in a novel environment. It is perfectly possible that our mental machinery should work well on simple perceptual tasks, but be much less reliable when applied to theoretical matters. We hasten to add that this is possible, not inevitable. It may be that the cognitive procedures that work well in one domain also work well in another; Modus Ponens may be useful for avoiding tigers and for doing quantum physics.

Anyhow, if evolutionary theory does say that our ability to theorize about the world is apt to be rather unreliable, how are evolutionists to apply this point to their own theoretical beliefs, including their belief in evolution?

… Read the rest

The Noble Gases of Social Theory

Mark DavisLeave a comment

elem_inertgas1“Intellectually inert” is an insult that I reserve only for vast elaborations that present little in the way of new knowledge. I use it sparingly and with hesitation. Ross Douthat usually doesn’t rise to that level, though he does tend to be obsessed with vague theories about the breakdown of traditional (read “conservative”) societal mores and the consequences to modern America.

But his recent blog post “Social Liberalism as Class Warfare” is so numbing in his rhetorical elaborations that it was the only phrase that came to mind after slogging my way through it. So what’s the gist of the post?

  1. Maybe rich, smart folks pushed through divorce and abortion because they thought it made them freer.
  2. But poor, not-so-smart folks lacked sufficient self-control to use these tools wisely.
  3. Therefore, the rich, smart folks inadvertently made poor, not-so-smart folks engage in adverse behaviors that tore-up traditional families.
  4. And we get increased income and social inequality as a result.

An alternative argument might be:

  1. Folks kept getting smarter and better educated (everyone).
  2. They wanted to be free of old stuffy traditions.
  3. There were no good, new traditions that took their place, and insufficient touchstones of the “elite” values in the cultural ecosystems of the underclass.
  4. And we get increased income and social inequality as a result.

And here we get to the crux of my suggestion of inertness: it doesn’t matter whether the unintended consequences of iconoclasty differentially impact socioeconomic strata. What matters is what can actually be done about it that is voluntary rather than imposed. After all, that is what the meritocracy of educated folks do in Douthat’s own calculus of assortative mating. And it won’t be that Old Time Religion because of (1) and (2), above.… Read the rest