Auto-Gladwell, or Can fractals be used to predict human history?

I just reviewed the book Bursts, by Albert-László Barabási, for Physics Today. But I had a lot more to say that couldn’t fit into the magazine’s 800-word limit. Here I’ll reproduce what I sent to Physics Today, followed by my additional thoughts.

The back cover of Bursts book promises “a revolutionary new theory showing how we can predict human behavior.” I wasn’t fully convinced on that score, but the book does offer a well-written and thought-provoking window into author Albert-László Barabási’s research in power laws and network theory.

Power laws–the mathematical pattern that little things are common and large things are rare–have been observed in many different domains, including incomes (as noted by economist Vilfredo Pareto in the nineteenth century), word frequencies (as noted by linguist George Zipf), city sizes, earthquakes, and virtually anything else that can be measured. In the mid-twentieth century, the mathematician Benoit Mandelbrot devoted an influential career to the study of self-similarity, deriving power laws for phenomena ranging from taxonomies (the distribution of the lengths of index entries) to geographical measurements. (I was surprised to encounter neither Zipf, Mandelbrot, nor Herbert Simon in the present book, but perhaps an excessive discussion of sources would have impeded the book’s narrative flow.)

Mandlebrot made a convincing case that nature is best described, not by triangles, squares, and circles, but by fractals–patterns that reveal increasing complexity when they are studies at finer levels. The shapes familiar to us from high-school geometry lose all interest when studied close up, whereas fractals–and real-life objects such mountains, trees, and even galaxies–are full of structure at many different levels. (Recall the movie Powers of Ten, which I assume nearly all readers of this magazine have seen at least once in their lives.)

A similar distinction between regularity and fractality holds in the social world, with designed structures such as bus schedules having a smooth order, and actual distributions of bus waiting times (say) having a complex pattern of randomness.

Trained as a physicist, Albert-László Barabási has worked for several years on mathematical models for the emergence of power laws in complex systems such as the Internet. In his latest book, Barabási describes many aspects of power laws, including a computer simulation of busy responses that went like this:

a) I [Barbasi] selected the highest-priority task and removed it from the list, mimicking the real habit I have when I execute a task.
b) I replaced the executed task with a new one, randomly assigning it a priority, mimicking the fact that I do not know the importance of the next task that lands on my list.

The resulting simulation reproduced the power-law distribution that he and others have observed to characterize the waiting time between responses to emails, web visits, and other data. But this is more than a cute model to explain a heretofore-mysterious stylized fact. As with Albert Einstein’s theory of Brownian motion, such latent-variable models suggest new directions of research, in this case moving from a static analysis of waiting time distributions to a dynamic study of the decisions that underlie the stochastic process.

For an application of this idea, Barabási discusses the “Harry Potter” phenomenon, in which hospital admissions in Britain were found to drop dramatically upon the release of each installment of the cult favorite children’s book. A similar pattern happened in relation to Boston’s professional baseball team: emergency-room visits in the city dropped when the Red Sox had winning days.

In addition to this sort of detail, Barabási makes some larger points, some of which are persuasive to me and some of which are not. He distinguishes between traditional models of randomness–Poisson and Gaussian distributions–which are based on statistically independent events, and bursty processes, which arise from feedback processes that at times suppress and at other times amplify variation. (A familiar example, not discussed in the book, is the system of financial instruments which shifted risk around for years before eventually blowing up.)

Barabási characterizes bursty processes as predictable; at one point he discusses the burstiness of people’s physical locations (we spend most of our time at home, school or work, or in between, but occasionally go on long trips). From here, he takes a leap–which I couldn’t follow at all–to conjecture a more general order within human behavior and historical events, in his opinion calling into question Karl Popper’s argument that human history is inherently unpredictable. The book also features a long excursion into Hungarian history, but the connection of this narrative to the scientific themes was unclear to me.

Despite my skepticism of the book’s larger claims, I found many of the stories in Bursts to be interesting and (paradoxically) unpredictable, and it offers an inside view on some fascinating research. I particularly liked how Barabási takes his models seriously enough that, when they fail, he learns even more from their refutation. I suspect there are quite a few more bursts to come in this particular research programme.

And now for my further thoughts, first on the structure of the book, then on some of its specific claims.

Structure

The book Bursts falls into what might be called the auto-Gladwell genre of expositions of a researcher’s own work, told through stories and personal anecdotes in a magazine-article style, but ultimately focused on the underlying big idea. Auto-Gladwell is huge nowadays; other examples in the exact same subfield as Barabási’s include Six Degrees (a book written by Duncan Watts, who was my Columbia colleague at the time, but which I actually first encountered it through a (serious) review in the Onion, of all places), Steven Strogatz’s Sync, last year’s Connected by Nicholas Christakis and James Fowler’s, as well as, of course, Barabási’s own Linked, published in 2002. These guys have collaborated with each other In different combinations, forming their own social network.

In keeping with the Gladwellian imperative, Bursts jumps around from story to story, often dropping the reader right into the middle of a narrative with little sense of how it connects to the big story. This makes for an interesting reading experience, but ultimately I’d be happier to see each story presented separately and to its conclusion. About half the book tells the story of a peasant rebellion in sixteenth-century Hungary (along with associated political maneuvering), but it’s broken up into a dozen chapters spread throughout the book, and I had to keep flipping back and forth to follow what was going on. (Also, as I noted above, I didn’t really see the connection between the story and Barabási’s scientific material.)

Similarly, Barabási begins with, concludes with, and occasionally mentions a friend of his, an artist with a Muslim-sounding name who keeps being hassled by U.S. customs officials. It’s an interesting story but does not benefit from being presented in bits and pieces. In other places, interesting ideas come up and are never resolved in the book. For example, chapter 15 features the story of a unified field theory proposed in 1919 (in Barabási’s words, “the two forces [gravity and electromagnetism] could be brought together if we assume that our world is not three- but five-dimensional”; apparently it anticipated string theory by several decades) and published on Albert Einstein’s recommendation in 1921. This is used as an example to introduce an analysis of Einstein’s correspondence, but it’s not clear to me exactly how the story relates to the larger themes of the book.

Specifics

As noted in the review above, I thought Barabási’s dynamic model of priority-setting was fascinating, and I would’ve liked to hear more details–ideally, something more specific than stories and statements that people are 90% predictable, but less terse than what’s in the papers that he and his collaborators have published in journals such as Science. On one hand, I can hardly blame the author for trying to make his book accessible to general audiences; still, I kept wanting more detail, to fill in the gaps and understand exactly how the mathematical models and statistical analyses fit into the stories and the larger claims.

My impression was that the book was making two scientific claims:

1. Bursty phenomena can often be explained by dynamic models of priority-setting.

2. In some fundamental way, bursty processes can be thought of as predictable and not so random. in particular, human behavior and even human history can perhaps be much more predictable than we thought?

How convincing in Barabási on these two claims? On the first claim, somewhat. His model is compelling to me, at least in the examples he focuses on. I think the presentation would’ve been stronger had he discussed the variety of different mathematical models that researchers have developed to explain power laws. Is Barabási’s model better or more plausible than the others? Or perhaps all these different models have important common features that could be emphasized? I’d like to know.

Of Barabási’s second claim, I’m not convinced at all. It seems like a big leap to go from being able to predict people’s locations (mostly they’re at work from 9-5 and home at other times) and forecasting human history as one might forecast the weather.

Also, I think Barabási is slightly misstating Karl Popper’s position from The Poverty of Historicism. Barabási quotes Popper as saying that social patterns don’t have the regularity of natural sciences–we can’t predict revolutions like we can predict eclipses because societies, unlike planets, don’t move in regular orbits. And, indeed, we still have difficulty predicting natural pheonomena such as earthquakes that do not occur on regular schedules.

But Popper was saying more than that. Popper had two other arguments against the predictability of social patterns. First, there is the feedback mechanism: if a prediction is made publicly, it can induce behavior that is intended to block or hasten the prediction. This sort of issue arises in economic policy nowadays. Second, social progress depends on the progress of science, and scientific progress is inherently unpredictable. (When the National Science Foundation gives our research group $200,000 to work on a project, there’s no guarantee of success. In fact, they don’t give money to projects with guaranteed success; that sort of thing isn’t really scientific research at all.)

I agree with Barabási that questions of the predictability of individual and social behavior are ultimately empirical. Persuasive as Popper’s arguments may be (and as relevant as they may have been when combatting mid-twentieth century Communist ideology), it might still be that modern scientists will be able to do it. But I think it’s only fair to present Popper’s full argument.

Finally, a few small points.

– On page 142 there is a discussion of Albert Einstein’s letters: “His sudden fame had drastic consequences for his correspondence. In 1919, he received 252 letters and wrote 239, his life still in its subcritical phase . . . By 1920 Einstein had moved into the supercritical regime, and he never recovered. The peak came in 1953, two years before his death, when he received 832 letters and responded to 476 of them.” This can’t be right. Einstein must have been receiving zillions of letters in 1953.

– On page 194, it says, “It is tempting to see life as a crusade against randomness, a yearning for a safe, ordered existence.” This echoes the famous idea from Schroedinger’s What is Life, of living things as entropy pumps, islands of low-entropy systems within a larger world governed by the second law of thermodynamics.

– On page 195, Barabási refers to Chaoming Song, “a bright postdoctoral research associate who joined my lab in the spring of 2008.” I hope that all his postdocs are bright!

– On page 199, he writes, “when it comes to the predicability of our actions, to our surprise power laws are replaced by Gaussians.” This confused me. The distribution of waiting times can’t be Gaussian, right? It would help to have some detail on exactly what is being measured here. I understand that, for accessibility reasons, the book has no graphs, but still it would be good to have a bit more information here.

5 thoughts on “Auto-Gladwell, or Can fractals be used to predict human history?

  1. How is this book compared to Taleb's Fooled by Randomness or Black Swan? Taleb is so anti-Guassian that he has very little respect for statisticians, and economists for that matter, which I think he's gone too far.

  2. Nor do venture capitalists give money to projects with guaranteed success.

    They are expecting about 1 in 7 to be finacially successful.

    Suspect expectations of _claimed_ success are much higher for funded research …

    and one of the reasons for the quality of much of published research?

    K?

  3. Much of human history actually is predictable. For example, I predict that people living in France will be better off on average, as measured by any reputable index of standard of living or human development, than people living in Mauritania 1 year, 10 years, and 100 years from now. Anybody care to bet against me?

    But, the parts of history that are predictable are also really boring, so nobody ever thinks about, say, France v. Mauritania, even though, objectively, it's an important question involving the welfare of millions.

    Admit it, you're probably trying to think up ways that Mauritania might possibly become better off than France in the future — say, a massive outbreak of zombieism in France! Or a new ice age covers France with a mile thick sheet of ice while turning the Sahara green. That's more fun to think about than what's most likely to happen regarding France and Mauritania: things keep on keeping on.

  4. Peter Turchin, a Russian-American professor of ecology, has been trying to develop elaborate quantitative models of history. He has a few basic patterns, of which these are ones I can remember

    – Ibn Khaldun's concept of asabiyya or social cohesiveness: a poor tribe with a one for all and all for one attitude charges out of the wilderness and overthrows a fractious civilization, then becomes fractious itself and is overthrown in turn by a higher asabiyya group.

    – Marchlords — Powerful conquerors are more likely to grow up on the edges of a culture than in the middle, where the stakes are lower (thus, Berlin and Vienna became German political capitals, not Frankfurt).

    – Elite overproduction in successful groups leading to fractiousness.

  5. Surely there is a problem with the prioritized sampling process as you have described it here. It looks to me like the only fixed point of this is where the distribution of remaining items converges to the lowest value in the distribution.

Comments are closed.