From your first link, I agree this is a common (mis)perception of both students and faculty – “a math course like any other — formal skills to master with no thought as to their usefulness or meaning.”

“Students rarely believe us. Many complain half way through that this math course isn’t like any other they’ve taken.

Where are the formulas? As instructors we often find that hard to remember. Since we’re mathematicians, we’re

tempted to think (semi)formal mathematics is both more useful and more important than it really is.”

One of the many hurdles – “Unfortunately, many quantitative reasoning instructors are underpaid adjuncts stitching together multiple jobs to eke out a living. Perhaps it’s unfair to ask them to spend the extra time it can take to teach from our text. “

The context of University finances, realities and politics is probably a large part of the problem.

Hopefully efforts like your will help reduce the costs/risks for others.

]]>I suspect that there is a bit of a political problem that the school just wants someone from the Maths dpt to come teach “statistics”, not to mess with their heads by telling them that the way everyone sitting up-stairs is doing research is probably flawed. ]]>

Daniel: I see your point. And that’s what I was trying to make that the focus of my comment — that it’s better to train doctors how to evaluate evidence rather than being able to perform a few statistical tests. I completely agree that just because a study has a significant finding (p<.05) doesn't mean it provides strong evidence. But it also doesn't mean that the study is meaningless either. There's a lot that goes into evaluating what the available data tell us, and I would hope that doctors would be trained to properly weigh these considerations (so that they can make informed decisions and help their patients).

]]>No. It’s a disservice to give some semblance of pretending that this stuff is science.

Huge swaths of medical research is just holding coconut husks to your head and waving flags and waiting for the planes to land.

]]>First, I think that p-values should continue being taught to students. I know, I know. Many readers of this blog vehemently oppose the framework of hypothesis testing and would prefer a Bayesian approach to scientific discovery. Yet, aspiring doctors will almost certainly come across research couched within the frequentist framework of hypothesis testing, p-values, and the like. So you would be doing medical students a disservice to ignore these topics.

Yet, p-values can and should be discussed in the context of what they really tell us about medical evidence. And here the emphasis should be put on other concepts that are more important than just p<.05 = truth. Not only do p-values treat uncertainty as a binary decision (rather than a continuum), but they mask more important considerations like measurement error, effect sizes, sample representativeness, etc (it's actually quite a long list!).

Second, I think it's most important to teach students how to evaluate *good* research. That is, how doctors can become astute consumers (rather than producers) of empirical evidence (and statistics). Thus, students would be better served to learn about causal inference and uncertainty, among other things. I could go on…

BTW, next year, there is a Bayesian course at ESSLLI in Toulouse, taught by Mark Andrews (who is an outstanding teacher).

]]>However, there is a basic problem with the framing of the study. What is it that they are doing in those statistics classes that these students are passing? My impression is that they are basically teaching statistics as an arithmetic class. It’s not all about p values (I’m thinking that’s a couple of weeks at the end maybe) but about using formulas to calculate the mean and percents, defining the median as “the middle number” in a list of numbers that is no longer than half a line. Students may learn summation notation and permutations and combinations and they may or may not use a fancy calculator but definitely not a computer or Excel, but they are doing a tremendous amount of calculation by hand. They learn to use a normal table from the back of the book to look up probabilities. So … I would actually call this largely a variation on college algebra with a statistics wrapper. Not a bad thing, probably more useful than factoring polynomials. And what they are finding is that, sad but true, you really don’t need a lot of math to succeed in a lot of majors and the majors that need stats will offer their own courses.

This is math 5 (the EA) http://fsw01.bcc.cuny.edu/mathdepartment/Courses/Math/MTH05/math05.htm

This is the stats

http://www.bcc.cuny.edu/MathematicsComputerScience/math23.1.htm

I’m not saying that it is a bad thing to learn college algebra in the applied context of this kind of statistics course. I think it is a good thing, especially done with intentionality. If students have never mastered slope and intercept how can they know anything about regression? I just don’t buy it. Are they passing? Sure they can plug and chug the homework problems. Are they learning the same amount as in EA? Probably. But it really seems like a slight of hand to say that EA is remedial and the basic stats course is not.

]]>Earlier this year I took computer lab sessions teaching basic statistics to first year medicine students. Amusingly in the past the 2nd year students always told the first year students not to bother taking the course- “It’s only worth 5% of you overall mark, the assignment is kind of hard and who needs statistics anyway?”. After a few years it seems the message got filtered down that the 3rd and 4th year work actually included a fair bit of interpreting medical studies so maybe the statistics course was worth doing after-all.

Anyway, we were given 4 weeks with one hour of lecture and one hour in the lab each week.

The labs were

Week 1- Hey this is Excel, can you make a new variable? How about a scatter plot? Calculate the average height of all the males.

Week 2- Let’s create a pivot table. Please don’t put continuous variables in as a factor- now you have 100 rows. How about you put something simple like gender in instead?

Week 3- How to do a t-test in excel. Don’t worry if this doesn’t really make sense now- it probably never will.

Week 4- Let’s brainstorm about running a state wide study looking at a new medical treatment.

The lectures looked at similar stuff but also looked at making inferences, sampling from populations,random allocation to treatment groups,correlations, statistical power and observational vs controlled experiments. I don’t think they got up to ANOVA.

My question is, given that reason we are teaching young med students any statistics is in the hope that they will one day be able to interpret a medical study (the majority of which are going to have p-values everywhere) and have some ability to pick good studies from poor studies, does it still make sense to spend 1/4 of the lessons talking about t-tests and p-values? What would be more pressing subject matter with only 8 hours to play with? Do we have a duty to explain concepts like garden-of-forking-paths to all prospective doctors?

]]>So my position is that we teach badly because a) we don’t teach using the natural method by which people learn but instead rely on substituted abstractions as though they adequately convey the stories that numbers and symbols actually tell and b) we don’t even try to come up with examples that matter. (Your papers about examples are an obvious exception.) You’d think that after decades teachers could manage to share examples of how they see this or that or how they learned this or that … except they do, but only in lower grades. It’s as though we forget how much freaking learning is actually going on in young kids! We act as though they’re given simple stuff when it’s not the simplicity of the material but the amount they have to put together to make sense of the world, to read and write and even to walk and talk at all. So we abandon the method we use when we’re doing the most important and difficult learning, meaning that learning which occurs when the roadmap is in our heads, when we have to traverse the great divide between meaning and abstraction, between distance and effect, between objects here and gone and permanence versus impermanence.

]]>Re the issue of between-subjects versus within-subject design and analysis in psychology, see Molenaar, P. C. M. (2004). A manifesto on psychology as idiographic science: Bringing the person back into psychology, this time forever. MEASUREMENT: INTERDISCIPLINARY RESEARCH AND PERSPECTIVES, 2, 201–218, with comments by Tuerlinckx, von Eye, Thum, Rogosa, Nesselroade, Curran & Wirth, a discussion by Molenaar, and a separate reply to Rogosa.

]]>I’ve been seeing that what researchers want to do is simply stop mentioning p-values, significance, null hypothesis, etc but keep on with the same problems. For example, in this vaccine trial they briefly mention giving the vaccinees ibuprofen, then this is completely ignored as accounting for any decrease in diagnoses (fever is the primary sign of ebola). It is interesting to see there is extended discussion and concern about statistical issues like biased samples, but none about the scientific issues with the study: http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(16)32621-6/fulltext

I’m becoming concerned that by complaining about NHST, we are losing simple heuristics to gauge the competence of the researchers, without an improvement in reliability.

]]>I’ve (co)written Common Sense Mathematics, for college students who aren’t going on in science so should not redo high school algebra even when it’s called College Algebra.

The thrust is real problems (the text and the exercises all start with direct quotes from newspapers and the web) to think about estimation, percentage change, elementary descriptive statistics (teaching Excel), regression and regression nonsense, linear and exponential growth, credit card debt and mortgages, probability.

The last chapter on the prosecutor’s fallacy and dealing with false positives quotes this blog (with Andrew’s permission) in several places. (Use contingency tables and natural frequencies, not Bayes’ Theorem).

]]>This title reminds me of Meehl’s paper “Theory-testing in Psychology and Physics”:

“Because physical theories typically predict numerical values, an improvement in experimental precision reduces the tolerance range and hence increases corroborability. In most psychological research, improved power of a statistical design leads to a prior probability approaching 1/2 of finding a significant difference in the theoretically predicted direction. Hence the corroboration yielded by “success” is very weak, and becomes weaker with increased precision. “Statistical significance” plays a logical role in psychology precisely the reverse of its role in physics.”

The subject is not quite the same, yet the underlying notion of more study leading to worse results is there in each work.

]]>