Truth in the Ranch author “JW” is back with another survey, once again using statistics as proof points to support their arguments. And, again, they’ve significantly overstated things on the way to concluding that the results are “clear and resounding” and “highly indicative of the overall community sentiment”.
I’ve previously addressed the open rates (they’re inaccurate) and click-through rates (there is no industry standard) that are used to claim “a very impressive response” and infer broad community support for their arguments.
However, this time we have some information from the Association that we didn’t have for the last post, which is that there have been 3,450 ballots issued for the election. So that tells us our total voter universe and helps us determine exactly how many members need to be surveyed to deliver a representative result of community sentiment.
The answer: we would need to randomly survey the community and secure answers from 346 Covenant voters to deliver a statistically meaningful result that had a 5% margin of error at a 95% confidence level. JW had only 160 responses – less than half the number needed.
In addition, JW’s survey method is not random and likely skewed based on respondent self-selection bias. If you put up a banner ad that says, “Love Trump? Click Here!”, you’ll get a high percentage of Trump voters. Any survey results from such a self-selected group of supporters would break heavily for Trump, delivering results that are not representative of all voters. The same is surely happening here given that the survey contained leading questions.
But, wait, let’s assume that their survey was actually conducted properly via random phone interviews and that members of all viewpoints were included – despite the leading questions. A survey that yielded 160 responses against a voter pool of 3,450 has a margin of error of +/- 7.6%. We add that margin of error to the reported results, arriving at the conclusion that as many as 20.2% of Members would answer “Yes” to JW’s first question, “Should former members of the Association Board continue making serious allegation of wrongdoing by the current Board as it relates to the PPP loan?”
That means that as many as 697 members might want the investigation into the PPP loan to continue. That’s likely not the possible result that JW would want us to focus on from the survey.
Seriously, though, we shouldn’t put any stock in these “survey” results. Sending out surveys via email with leading questions from an anonymous account and attempting to assert that 160 responses are “highly indicative of the overall community sentiment” is lunacy, indeed.
.
