Skip to content
 

Perhaps you could try a big scatterplot with one dot per dataset?

Joe Nadeau writes:

We are studying variation in both means and variances in metabolic conditions. We have access to nearly 200 datasets that involve a range of metabolic traits and vary in sample size, mean effects, and variance. Some traits differ in mean but not variance, others in variance but not mean, still others in both, and of course some in neither. These studies are based on animal models where genetics and environmental conditions are well-controlled and where pretty rigorous study designs are possible. We plan to report the full range of results, but would like to rank results according to confidence (power?) for each dataset. Some are obviously more robust than others. Confidence limits. which will be reported, don’t seem to quite right for ranking. We feel an obligation to share some sense of confidence, which should be based on sample size, variance in the contrast groups – between genotypes, between diets, between treatments, …… . We are of course aware of the trickiness in studies like these. Our question is based on getting the rest right, and wanting to share with readers and reviewers a sense of the ‘strengths, limitations’ across the datasets. Suggestions?

My reply: Perhaps you could try a big scatterplot with one dot per dataset? Also I would doubt that there are really traits that do not differ in mean or variance. Maybe it would help to look at the big picture rather than to be categorizing individual cases, which can be noisy. Similarly, rather than ranking the results, I think it would be better to just consider ways of displaying all of them.

3 Comments

  1. Is the issue an attempt to show which traits are consistent across multiple labs and multiple measurement techniques etc? Otherwise I don’t understand the goal. What are we trying to communicate?

  2. Anonymous says:

    “Some traits differ in mean but not variance, others in variance but not mean, still others in both, and of course some in neither.”

    I bet that all the means and variances vary, but your sample sizes are too small to detect it in some cases.

  3. Garnett says:

    Funnel plots?

    Funnel plots for comparing institutional performance. DJ Spiegelhalter – Statistics in medicine, 2005

Leave a Reply