Lies, damn lies, and statistics
"But the polls were wrong about Trump!" some of my otherwise astute friends exclaim. No, the aggregate polling, as compiled by Nate Silver and Sam Wang both were completely consistent with the outcome of the 2016 election. Clinton won, by nearly 3 million votes. The electoral college appointed Trump because the outcomes in three states, which were polling within the margin of error, happened to all go marginally for Trump. Nothing wrong with the aggregate polls (although individual polls can and do skew).
My point isn't to resurrect the 2016 election discussion. My point is that polling is based on statistics. We use statistics all the time as a guide to action. People pay lots of money to compile statistics to predict the future. Of course, statistics provide a probability, not metaphysical certitude.
But my point isn't to validate the use of statistical analysis. The history of statistical analysis speaks for itself.
My real point is "garbage in, garbage out." If you present badly worded questions, or if your study design doesn't distinguish between two equally plausible interpretations, then the statistics or polls are worthless. Not the fault of statistics.
In the linked commentary, the Pew Poll could be interpreted as meaning that young people skew liberal and geezers skew conservative because, you know, age. Or it could reflect the fact that each cohort's political preferences are established in youth, and rarely change with age. The post uses statistical analysis to re-frame the interpretation of statistical analysis. Critical thinking is hard, peeps!
"All of a sudden, that inexorable conservative drift with age disappears. Rather, people born during the LBJ and Nixon presidencies shifted ever bluer compared with those born before. Those who formed their ideologies during the unpopular Carter presidency, or during the Reagan years – basically, late Boomers and at the first half of Gen X – became the most conservative of all."
https://angrybearblog.com/2019/12/political-leanings-through-time-for-birth-cohorts.html#more-53517
My point isn't to resurrect the 2016 election discussion. My point is that polling is based on statistics. We use statistics all the time as a guide to action. People pay lots of money to compile statistics to predict the future. Of course, statistics provide a probability, not metaphysical certitude.
But my point isn't to validate the use of statistical analysis. The history of statistical analysis speaks for itself.
My real point is "garbage in, garbage out." If you present badly worded questions, or if your study design doesn't distinguish between two equally plausible interpretations, then the statistics or polls are worthless. Not the fault of statistics.
In the linked commentary, the Pew Poll could be interpreted as meaning that young people skew liberal and geezers skew conservative because, you know, age. Or it could reflect the fact that each cohort's political preferences are established in youth, and rarely change with age. The post uses statistical analysis to re-frame the interpretation of statistical analysis. Critical thinking is hard, peeps!
"All of a sudden, that inexorable conservative drift with age disappears. Rather, people born during the LBJ and Nixon presidencies shifted ever bluer compared with those born before. Those who formed their ideologies during the unpopular Carter presidency, or during the Reagan years – basically, late Boomers and at the first half of Gen X – became the most conservative of all."
https://angrybearblog.com/2019/12/political-leanings-through-time-for-birth-cohorts.html#more-53517
I teach this to my classes when they are learning about spreadsheets. I have them ask a question about favorite colors to the other classes. By grade I allow them to ask about favorite among x number of colors, x=their grade level. So second graders are limited to 2 colors and 5th get to offer a choice of 5 colors. They present their results and we talk about the differences and what made the results differ. It's very eye-opening to them and has made some parents think about polls.
ReplyDeleteI especially love doing this during election years!
Sounds like a great exercise! Training citizens of the future and a practical application of numbers!
Delete