I was inspired by this post by John Cook, “People want Swiss Army Knives,” to think more generally about the idea of statistical methods as pocket tools.
Cook argues that the Swiss Army Knife is a more useful tool than a scalpel because it can do so many more things, even if it does none of these things perfectly.
He might well be right but I think he’s confounding two useful aspects of the Swiss Army Knife. First is its multifunctionality, as mentioned above, but also relevant is that it’s foldable so you can just carry it around in your pocket. You can’t carry around a scalpel in your pocket or it will cut your pants open! Sure, you could take the scalpel apart and put it in a box but then it’s not so convenient to use as needed.
So there’s really an intermediate concept, something in between the scalpel (which can only do one thing and you can’t easily carry around in your pocket) and the Swiss Army Knife (which can do many things and you can carry around).
The intermediate item is a knife—it’s just a knife, it can only do one thing—but it folds up so you can carry it around in your pocket.
That is, a jackknife.
The statistical technique called the jackknife was so named by Tukey because it is an easy-to-use, portable tool that you can carry around with you . . . ummm, here’s a quote from a 1964 paper by Rupert Miller:
Tukey adopted the name of “jackknife” for this procedure, since a boy scout’s jackknife is symbolic of a rough-and-ready instrument capable of being utilized in all contingencies and emergencies.
This makes me think we need a statistical method called the Thin Mint, since this popular girl scout cookie is, ummmm, I don’t know exactly but I’m sure we could come up with something . . .
Anyway, back to the main point. It’s good to have methods that you can just carry around with you, formulas such as sqrt(p(1-p)/n) and (y+2)/(n+4) that are applicable in all sorts of situations.
However, as technology develops, we can carry around more in our pockets, things like phones and tape recorders and cameras are now no big deal.
Similarly in statistics. With Stan, we are on the cusp of having a convenient “jackknife” that can fit measurement error models, multilevel models, etc etc. It will take awhile but I think we’ll get there.
The point is to think of statistical inference not as a big deal, not as the building of a cathedral of a data analysis for each problem that comes in, but rather as a convenient tool, as part of a larger exploratory project.
To claim that the Swiss Army Knife is a more useful tool than a scalpel flies in the face of one of the fundamental principles of the Unix Philosophy.
The Unix approach is for every tool to do one thing and that one thing really really well. Dunno, but that choice has worked wonderfully well for me in practice. Nothing like an sweet, efficient ‘grep | cut | tr | sort’ pipe to chomp on a humongous data file. It ain’t fun to fiddle with the half-baked, slow, buggy, under-powered implementations of search / replace / sort in the menu of every GUI package installed on my PC.
Rahul:
It also depends on where you draw the line. Stan could be viewed as a scalpel that only does Bayesian inference and nothing more, or a Swiss army knife that fits any possible Bayesian model you can express.
The key reason I like the pocket knife analogy: it captures the simplicity of the scalpel with two other important features: (a) general use for many purposes, and, especially, (b) portability.
Stan could be viewed as … a Swiss army knife that fits any possible Bayesian model you can express.
You can’t calculate multivariate normal integrals in Stan, so any model that needs that can’t be fit in Stan. Or maybe it can be done by simulating draws from a multivariate normal and calculating proportions. I’ve thought about trying that, but I can get the job done with PyMC easily and quickly using a very efficient and accurate Fortran function from the SciPy statistics library.
More generally, the fact that there is a formal system for requesting features in Stan suggests that it can’t fit any Bayesian model, unless we construe “that you can express” to mean, tautologically, “that you can express in ways currently supported by Stan.”
Noah:
I did indeed mean “you can express” in that tautological way. I wondered if anyone would catch that!
More seriously, yes, there are some models that can’t be fit in Stan and we’re working on them, when we have the time.
To be more fair than I was in the earlier comment, I very much like Stan, and I’ve been using it for other analyses. Plus, I put in a feature request for the multivariate normal integral thing, so I still hold out hope that I can do all my Bayesian model fits in Stan eventually. Stan is great!
Rahul – this is disussed in John’s blog post.
UNIX itself is a Swiss Army Knife. You can even carry it on a thumbdrive in your pocket.
The unix philosophy relies on there being a simple shared interface (streams of text) at which the simple-tools-which-do-one-thing-well can be composed.
Sometimes this happens in statistics, but it seems to me that the presence of uncertainty usually makes it harder to compose simple tools in a routine manner. You might be able to compose, say, parts of a Bayesian generative model, but you need something more, something quite clever, to actually do inference for that composed model, you can’t usually just compose two different inference tools. Hence the need for things like Stan.
I agree.
I vote for the spreadsheet. Adding your own formulas and macros makes it wonderfully versatile.
I vote for the abacus.
Portable, analog computation for many purposes, and a toy for toddlers.
I don’t think toddlers can use Stan.
My Swiss Army Knife is basically the same as the “Explorer” model, that markets itself as intended for Boy Scouts. I’m missing some of the tools (no pliers, fish scaler, saw and a few hooks) and I’ve had it for over 30 years, at least since I first took it to China in 1982 (where it was very useful).
http://www.victorinox.com/us/product/Swiss-Army-Knives/Category/Everyday-Use/SwissChamp/53501
Recently had it repaired. $5 plus the cost of getting it to the repair facility. It has a lifetime guarantee.
I love my Buck “Vantage” knife, I carry it around everywhere using the clip that lets it clip to my pocket. I was in a Trader Joes and one of the workers there asked me what kind of knife I was carrying. He was clearly impressed with the good balance of large-enough size to be useful, and small enough to fit in a pocket, as well as good quality construction. Not very expensive either. I used to carry a swiss army knife, but found it wasn’t strong or safe enough for the things I tended to use it for. The locking feature and one-handed opening of my Buck knife work well for me. A few drops of olive oil every 6 months and it opens and closes like a charm without providing off flavors to the occasional apple or slice of cheese while hiking.
Are there places where they object to you carrying around your knife?
North Korea.
A few, basically anywhere they have a metal detector and a posted policy, which is basically nowhere that I go. I think I had to leave it in the car when I had jury duty, that’s about it.
Ok. I had a great, monogrammed Swiss Knife that was a gift & I used to have it in my backpack. One day I forgot it was in there & went to take a flight. Had to give it up there. I’m still sad about it. :(
I think that happened to me too after 9/11 (back in the 1990’s I actually used to walk onto planes with 3 inch buck knives, no problem. The security guys would hold it up to a template on the side of the x-ray machine to be sure it was short enough), which is another reason to get a larger knife that clips to your pocket. Harder to forget that you have it.
Gosh, literally last week I was wondering about the origins of the word “jackknife” in the statistical sense. Thanks!
Partial moments are the Swiss Army Knives of statistical tools…
https://www.scribd.com/collections/4308514/Nonlinear-Nonparametric-Statistics
The UNIX analogy is not quite right, and can lead to confusion.
I wrote some quick comments over at John Cook’s.
1) At any level of programming hierarchy, one has (A) a set of components (tools) and (the part unmentioned) (B) a set of compositional mechanisms that connect them. One makes progress by creating new tools, improving the existing ones, or adding new compositional mechanisms.
2) In UNIX (as of 1973), most tools were indeed pretty focused, and they had to be, since people were using PDP-11/45s, with 248KB total memory, and max 64K code + 64K data per process.
Within a command, one had a (A) small function library and (B) function calls.
At the next level up, one had (A) a set if commands and (B) a simple file format and linear pipes, with rudimentary extension via shell scripts. By 1975 we’d beefed up the shell enough to make it a much more practical compositional mechanism for scripting.
3) In the early years, there were continual discussions of the form: X almost does what we need, should we add an option to make X’, or is there a tool Y of general use such that X|Y does the job, but Y us useful elsewhere.
Good answers were not standalone, but depended in the existing art of tools and compositions. In particular, when shell programming got rolling 1975-1976, it got much easier to customize usage of command combinations rather than adding options or inventing new commands. Sometimes, frequent usage showed usefulness of adding options to simplify / speed data transformations: example: sort … | uniq led to sort -u, a real enhancement at trivial software cost.
4) Summary: these analogies need context:
A) a scalpel does one thing, but normally exists in a set.
B) a jackknife is more flexible, but is so without having a collection if features, ie it’s like UNIX cp(1), which copies files without worrying about their contents (unlike some OS’s if the time). That’s a really nice firm if generality.
C) A Swiss Army knife (or common bike tool) is a fixed collection of tools packaged as one physical object, but usable only one at a time, with no extension mechanism.
D) The UNIX programming environment is like a workbench with a collection of interoperable tools, to which one can add new ones, and use some simultaneously. Simple example: some socket wrenches of different sizes, a set if sockets, and some extenders of different lengths. Some shell scripts were more like robots that could select electric saws, bits for drill press, tools for lathe, and move wood pieces through the process.
I don’t know Stan well enough to comment, but maybe Andrew will comment on the nature of the components AND the compositional methods.
Excellent description.
Could you please explain sqrt(p(1-p)/n) and (y+2)/(n+4)? I’m not recognizing them, but would love to have them in my pocket!
Also, does anyone else have similar equations that they use all the time?
Suppose you’re estimating a proportion from the following data: n, the total number of observed cases; y, the number of observed cases that fall into a specific class you care about. If it’s reasonable to treat each observation as exchangeable with any other observation then (y+2)/(n+4) is a regularized estimate of the proportion (it arises in a Bayesian analysis as the posterior mean given a Beta(2,2) prior on the proportion and a binomial likelihood) and sqrt(p(1-p)/n) is a rough-and-ready standard error on any reasonable estimate of the proportion (the estimated value is p in the formula).
Hmmmm… what if we carry on with a “artificially intelligent statistician” in our pockets? Ok, it wouldn’t reach where a professional statistician can do but I’m pretty sure it would fit the needs of 8 out of 10 regular people…
As the person who introduced the German concept of the “Eierlegendewollmilchsau” (egg-laying, woolly-coated,
milk-producing pig) into the statistical literature (https://www.tandfonline.com/doi/pdf/10.1080/10543400701643855?needAccess=true) I have never been fond of these multi-purpoe tools. The German pig is an animal that “provides many staple items, but not very well. So too a clinical trial that attempts to answer too many questions at once, whilst it may be able to answer each it will do so neither efficiently nor optimally.”