Tuesday, May 31, 2005

Polling: My (Further) Analysis

It seems that my earlier post on polling in federal politics has attracted the attention of a commentor who apparently is Glen P. Robbins of ROBBINS SCE RESEARCH. In the comment section for that post he made some comments that have made me think some more about polling and my analysis. I will be posting, sometime today, an update of the moving averages I posted earlier along with some nifty graphs.

Mr. Robbins makes two points that I would like to deal with in this post. First he says:
We think the snapshot in time idea has outlived its usefullness, and often serves to be as much an apology as anything else. "Nailing" the numbers is important because we are measuring the relative values of art and science in a poll. The science is of course the statistics which can be as complex as one wants it to be. The art is understanding, or getting a sense of what is happening over a particular time in history.
I have to agree that in some cases the "snapshot in time" characterisation is what he claims--apologism for inaccuracy, or worse. The problem is that when a company like his conducts a poll before an election has even been called they can't possibly expect that poll's results to hold for an indefinite period of time. Perhaps, the compromise that I see is that as polls are conducted closer to the election day they should move (quite naturally) from being useful as a "snapshot in time" to a predicted outcome. I'm not really sure what he means in the last sentence where he talks about the art of polling. I view polls as a tool that represent themselves as scientific. If he is saying that they should be an end onto themselves and that hard data should be re-interpreted by either changing the methodology of asking the questions or of presenting the results I would have to strongly disagree. I may be confused as to what his point here is though.

His other interesting argument is:

One thing that has to be considered, and I present this as a critic of the current polling culture, is the relationship between the establishment, including government, and the media.


Accordingly, it is our contention that there are more pollsters who have had historical relationships with Liberal governments and if polls are inclined to lean towards one party over another, it would be the Liberals (the establishment party), than the Conservatives, or the N.D.P.

Between these two main points of the argument he makes specific claims about the biased nature of the media and the polls they comission. If you like, see the comment section of the post linked to above for his argument in toto.

Frankly, this argument that Conservatives (and others outside of government) have made popular is growing very tiresome. There may be a media bias against the CPC. Some reporters and commentators definitely display a flagrant bias. Polling companies may be complicit. The problem is that it is a foolish argument to make because it is not at all testable. I firmly believe that some reporters or commentators or even entire media outlets have a definite bias. The argument depends on a convulated, and circular path that goes something like this: 1. When in government the Liberals reward their friends (put aside the obvious Adscam corruption style of reward for now) in media; 2. This media helps the Liberals stay in power by pumping them while dumping the opposition; 3. The voting public, for whatever reason, does not perceive or is not critical of these biases; 4. This uncritical voting public re-elects Liberals and the cycle perpetuates. The counter is: 1. People generally agree with Liberal policies; 2. Media is just trying to sell more product by giving people what they want; 3. Governments are only trying to use the best tools available to communicate with the population about important programmes. This boils done to an unresolvable chicken or the egg problem. Which came first: People agreeing with Liberal policy or Liberals pre-disposing people to like their policy?

Furthermore, no one is willing to recognise that biases go both ways. For every right-winger who accuses the Toronto Star of bias there is a left-winger who says the same about the National Post. They are both wasting their time and energy arguing unproveable arguments.

It just seems like people making these sort of conspiracy arguments are refusing to "play by the rules". Whenever they lose an argument by being presented with data like an opinion poll they claim that data is biased. And then they claim that these biased polls create biased election results. My point is that both sides will cheat and it seems like people are more or less equally divided in their opinion between the Liberals and Tories and therefore the resources for cheating will be equally distributed. And that's why I feel that the averaged result of these polls will provide a clearer picture of where public opinion really lies.


Anonymous ROBBINS Sce Research said...

I like the commentary it is clever, but it is sounds to me like something the media would write, its sort of a daisy chain circular reasoning that with the intention to get away from a problem and dismiss as he said she said.

The response does not properly address the depth of the potential problem and ignores obvious historical signposts and rationale evidence.

Pollster make money from government and from poltical parties. They also make money from advertisers that are linked to publications. Imagine for a moment you are President of this polling firm. Because of your relationship to parties, media and others you make $2,000,000 per year.

All of a sudden a situation comes up where this party you are linked to runs into a major problem. Let's say this is Quebec. What is the problem is so deep that you risk having any recognition in that province.

Are any of the pollsters that are clamoring for work, like lawyers who scramble like rats to cheese for Government insurance defense contracts, and you are in a position where a new party could come into power. What if that new party said no more contracts for any of you, we aren't using pollsters or we are using ROBBINS.
He is the only one with the jam to tell it like it is. What if you friends in the media and government tell you that this situation is a matter of national unity?

What happens to that word science? Science in polling can mean political science and or statistical science. Both can and are used in polls. There are a number of variances that can be used in the calculation of mean average, average average, etc, where any statistics in any polls can be manipulated to produce a result.

Frankly, I think the media can order up and poll results it wants.

Lastly, to understand polling properly one needs to look at the art of polling, Angus Reid, Gallup Roper and others all talk about this. Without an understanding of the art of polling the entire discussion is as you aptly describe it simply rhetorical.

ROBBINS is so accurate because we address the raw data (I get on the phone with my callers to hear respondents-we don't hire people for $8.00 per hour). Look at the data within the context of the questions asked, whether the questions are static or slightly maleable to 'test' within the poll.

Than we consider the data collection over what specific area, and if applicable we can adjust for population density, or even calculate for vote turn-out in specific constituency (high population density does not necessarily translate into high voter turn-out). Also, who was our pool of respondents, were they voters, were they citizens, were they homeowners, taxpayers, or general.

During the elections in the Ukraine there were many polls conducted with 10,000 sample sizes. This was obviously because no-one trusted the government numbers. I would say that here in Canada we need much higher sample sizes nationally 5,000 and 6,000 size samples, but the media who sponsors these does not want to pay for them, they want a poll a week to control and manage the political debate. Rhetorick, this is where we all need to go.

There is a reason that the polling if frequent and volatile. The establishment has a sense of loss of control because crediblity and validity of authority are flying out the window, and only an election will clear the slate, but the status quo, the Liberal party is not ready to lose.

I really think it is wilful blindness to wrap up fair commentary about a problem construct as conspiracy, that contra assertion is soooo played. It is likely more important for the bright minds like yours to more deeply explore all of this information rather than take your poignant and very brilliant analysis and apply it in primae facie fashion.

Don't ignore the voice in the 'wilderness', it might just be bringing you to where you ought to be.

12:49 AM  

Post a Comment

<< Home