## History of Monte Carlo

Monte Carlo is the ubiquitous little beast of burden in Bayesian statistics. Val points to the article by Nick Metropolis “The Beginning of the Monte Carlo Method.” Los Alamos Science, No. 15, p. 125, 1987 about his years at Los Alamos (1943-1999) with Stan Ulam, Dick Feynman, Enrico Fermi and others. Some excerpts:

Among the attendees was Stan Ulam, who had rejoined the Laboratory after a brief time on the mathematics faculty at the University of Southern California. Ulam’s personality would stand out in any community, even where “characters” abounded. His was an informal nature; he would drop in casually, without the usual amenities. He preferred to chat, more or less at leisure, rather than to dissertate. Topics would range over mathematics, physics, world events, local news, games of chance, quotes from the classics—all treated somewhat episodically but always with a meaningful point. His was a mind ready to provide a critical link.

During his wartime stint at the Laboratory, Stan had become aware of the electromechanical computers used for implosion studies, so he was duly impressed, along with many other scientists, by the speed and versatility of the ENIAC. In addition, however, Stan’s extensive mathematical background made him aware that statistical sampling techniques had fallen into desuetude because of the length and tediousness of the calculations. But with this miraculous development of the ENIAC—along with the applications Stan must have been pondering—it occurred to him that statistical techniques should be resuscitated, and he discussed this idea with von Neumann. Thus was triggered the spark that led to the Monte Carlo method.

The spirit of this method was consistent with Stan’s interest in random processes—from the simple to the sublime. He relaxed playing solitaire; he was stimulated by playing poker; he would cite the times he drove into a filled parking lot at the same moment someone was accommodatingly leaving. More seriously, he created the concept of “lucky numbers,” whose distribution was much like that of prime numbers; he was intrigued by the theory of branching processes and contributed much to its development, including its application during the war to neutron multiplication in fission devices. For a long time his collection of research interests included pattern development in two-dimensional games played according to very simple rules. Such work has lately emerged as a cottage industry known as cellular automata.

John von Neumann saw the relevance of Ulam’s suggestion and, on March 11, 1947, sent a handwritten letter to Robert Richtmyer, the Theoretical Division leader (see “Stan Ulam, John von Neumann, and the Monte Carlo Method”). His letter included a detailed outline of a possible statistical approach to solving the problem of neutron diffusion in fissionable material.

Johnny’s interest in the method was contagious and inspiring. His seemingly relaxed attitude belied an intense interest and a well-disguised impatient drive. His talents were so obvious and his cooperative spirit so stimulating that he garnered the interest of many of us. It was at that time that I suggested an obvious name for the statistical method—a suggestion not unrelated to the fact that Stan had an uncle who would borrow money from relatives because he “just had to go to Monte Carlo.” The name seems to have endured.

[....]

Enrico Fermi helped create modern physics. Here, we focus on his interest in neutron diffusion during those exciting times in Rome in the early thirties. According to Emilio Segre, Fermi’s student and collaborator, “Fermi had invented, but of course not named, the present Monte Carlo method when he was studying the moderation of neutrons in Rome. He did not publish anything on the subject, but he used the method to solve many problems with whatever calculating facilities he had, chiefly a small mechanical adding machine.”

In a recent conversation with Segre, I learned that Fermi took great delight in astonishing his Roman colleagues with his remarkably accurate, “too-good-to-believe” predictions of experimental results. After indulging himself, he revealed that his “guesses” were really derived from the statistical sampling techniques that he used to calculate with whenever insomnia struck in the wee morning hours! And so it was that nearly fifteen years earlier, Fermi had independently developed the Monte Carlo method.

1. TCO says:

I'm just starting to check this site out.

a. What is the theme?

b. What is y'all's political leaning?

c. What do you think of MAKING IT COUNT (Leiberson)?

d. What do you think of the statistics argument wrt MBH98 hockey stick and such on the Climate Audit blog? http://www.climateaudit.org/

2. Martin Ringo says:

Andrew,
While I knew of Metropolis, I had never read anything directly written by him, and I found the Metropolis article interesting. When I started doing Monte Carlo experiments in the late 60s, my old man gave me, in an oral form, much the material in the article. As it turned out he — my dad — was a year ahead of Metropolis in the U of C physics program and may have been the only nuclear physicist in the program who did not work on the Manhattan project. The Navy would release him from the Naval Research Labs (Sonar) for a wild idea like nuclear fisson. Anyway after hearing much of the original and later uses in nuclear modeling, it was kind of nice to read about it from one of the early practioners.
Regarding the substance of the review, Metropolis views Monte Carlos like a physicist where it is often used as an exploratory tool. After nearly four decades of designing Monte Carlo experiments, I have a much simpler view of the technique, at least for the great majority of the work. Monte Carlos simply give the research the ability to integrate distributions that may be so intractable that even trying to write them down is difficult. This observation lead, in the 60s, to brief period in my field, econometrics, in which the tool was considered, shall we say, somewhat lowbrow. Fortunately, the field woke up and realized that integration by generation of (pseudo ) random numbers by arithmetical means, while sinful, is quite useful.

3. Anonymous says:

sounds sort of like doing numerical approximation for an integral that is difficult to evaluate analytically.

4. Gloria says:

I found a downloadable book on design and analysis of Monte Carlo experiments.
http://ideas.repec.org/p/dgr/kubcen/200417.html
http://www.exphs.org