An Open Letter to Consultores 21

Estimados Consultores 21,

I read your communiqué published yesterday with some confusion, and the distinct sense that you really haven’t grasped the full magnitude of the damage last Sunday did to your firm’s reputation. I’m afraid bland assurances that you acted professionally and broad assertions of your staff’s professionalism fall far short of an adequate response to the reputational debacle you face.

Pointing to your forecasts’ success in the previous six elections was an especially misguided touch: it reads like the White Star Line responding to the sinking of the Titanic with a press release noting that their previous 75 transatlantic crossings had gone off without a hitch.

The reputational hole you’re in is far too deep for that.

To begin to claw your way back to respectability, bold action will be needed.

In fact, I can think of only one course of action with a real chance of success: you need to reveal all the details of that final survey that led you to forecast 7-O wrong.

Yes, all the details: the full questionnaire, a detailed description of your sampling strategy, a full question-by-question breakdown of responses, including comprehensive cross-tabs and, most importantly of all, an in-depth explanation of the way you assigned non-responders to the pro-government or pro-opposition column in your projection. Everything.

Nothing short of this radical action will really address the legitimate concerns of the millions of Venezuelans who rested their hopes and dreams on your analysis. We’re simply not going to take your word for an internal investigation that keeps the underlying data out of reach, because it is precisely the reliability of your word that’s now in doubt.

We understand that under normal circumstances you’d be leary to reveal certain trade secrets in your methodology, but you need to grasp that unless you do this, you’ll have no trade to protect at all.

Only once you throw the doors completely open to that final poll will researchers truly be in a position to assess what went wrong where. Only then will we understand specifically why it is that you showed such strikingly lower figures for undecided or “won’t answer” voters in your sample compared to your competition. Only then will we be able to assess whether the assumptions that led into your projection were reasonable or not.

You can console yourself with the knowledge that in taking up this radical path you’ll be making a major contribution to the transparency of an industry that has been pathologically opaque in our country for far too long. It may be unfair that Consultores 21 has to carry the weight of making that first step all on its shoulders. But them’s the breaks: you screwed up big time. Now nothing short of radical openness will help make up the credibility gap that’s opened up.

con sinceros animos de colaborar,
Quico

87 thoughts on “An Open Letter to Consultores 21

  1. I hate to say I told you so, but I always discarded C21 as a legitimate predictor of results for the simple reason that Comando Venezuela was their client. Period.

    Like

  2. It’s not gonna happen. They’ll probably hire another company, probably from abroad, to do the postmortem examination. Thats’ what Datanalisis did to check their results. You know, let the people that actually know how to do a poll evaluate their work and find out what went wrong.

    Like

  3. Oil tanks go on fire, refineries “explode”, bridges collapse, electricity does it’s stuff, money disappears and polls fail.
    That’s life in Venezuela, get over it.

    Like

  4. Good Luck with that, Quico.
    The last thing anyone would do is open their methodology, flawed as it may be, to complete strangers and to their competition.

    More likely, as A. Barreda says, a foreign pollster, after signing more NDA’s than you can shake a stick at, will do the post mortem.

    Nice thought, and all, sin sarcasmo.

    Like

    • I’m under no illusions. I know that, in anything like normal circumstances, my request would be laughed out of the room. But C21 just hit an iceberg. All I’m doing is telling them to launch the lifeboats right now to see if they can salvage something.

      Like

      • But you come across as some young punk who does not know how business functions, when you write this open letter to C21 and demand that they “open their books” to you, and by presumption, to the general, or your public-at-large.

        Not good.

        Like

        • Sorry, in normal countries this is standard operating procedures for pollsters. Take it away, Iñaki,

          En otros países (EEUU, Reino Unido, México, por ejemplo) las encuestadoras se han unido en asociaciones para establecer entre otras cosas unas reglas de funcionamiento. Han escrito, por ejemplo, principios de divulgación para las encuestas destinadas al dominio público. Estos principios establecen los requerimientos de divulgación de las encuestas. Veamos por ejemplo a lo que se le obliga a publicar a las encuestadoras estadounidenses.

          “Nivel 1 de divulgación: Todos los reportes de encuestas realizados por algún miembro para publicación deben incluir la siguiente información:

            Quien patrocina el estudio
            Que empresa realizo el trabajo de campo
            Fechas de la entrevista
            Método de muestreo
            Población muestreada
            Tamaño de la muestra
            Tamaño de la sub-muestra, en caso de no usar toda la muestra
            Margen de error de muestreo
            Tipo de contacto con encuestados
            Fraseo y orden completo de las preguntas en las que se basa el estudio
            Porcentaje de resultados en todas las preguntas

          I’m just trying to leverage C21’s credibility death spiral into a way to nudge Venezuelan polling practices closer to the international norm.

          Like

          • Watch the level of disclosure in a standard Brazilian poll, for instance: http://pesqele.tse.jus.br/pesqele/publico/pesquisa/Pesquisa/visualizacaoPublica.action?id=10909

            Yup, those two links at the bottom are the complete questionnaire and the detailed information on sampling points.

            What I’m asking for is extraordinary only here, because our pollsters treat their data the way Giordani treats Fonden project disclosure – o, lo que es lo mismo, por el tercermundismo imperante.

            Like

            • Yes, it’s good to see the disclosure (especially pdf’s of sample questionnaire) from a Brazilian pollster. The problem with Venezuelan pollsters is that they’ve been given, for so long, carte blanche by journos and bloggers alike, you being one of the pom-pom cheerleaders. Pollsters could do no wrong. Commissioned? Schmissioned. Who cares who funds the poll? That and other inconsistencies were your playbook, my friend, for years, as you toyed with the polling results to find some charmed aha! moment.

              Now all of a sudden you get on the bandwagon to demand what you never wanted to realize, before, what you should have demanded before?

              Sorry Quico. But this post just comes across as un pataleo. Or to use that unfortunate phrase: tarde piaste, pajarito.

              Like

        • ‘Punk’ being the primary adjective, presumably? Although with acid-tongued Syd who regularly goes for the jugular repeatedly, who knows?

          Like

      • But you blindly used the C21 data just to back up your owen political preferences. You did it here – you did on the live interview that I heard. Perhaps you need to be less naive when all the other pollsters were going the other way and even Bank of America said Chávez would win by double digits in information sent to its investors.

        Sure C21 look like a bunch of pollster hacks but you also look stupid for being so subjective in youir choices. Still, better luck in 2018. One day you will get something right. Many are still waiting since September 2002. You bet on red and black comes up….and vice versa….you have won once in 15 attenpts. Not very good but good enough for IHT and Foreign Policy Mag. who could not even place venezuela on a map.

        Like

  5. Well, unless they “open their (old) methodology” and show that they have made the necessary changes, they will have zero credibility in future elections. C21 predicts election gains in the upcoming regional elections? Who will even bother to write that?

    Like

    • To make gains in the regional elections the opposition will have to win 6 states which looks unlikely. There is no guarantee that Capriles will win Miranda since chavismo won 16 out of 21 munciplaities in that state and my guess is that the opposition vote will be much less on December 16th as all the people near where I live are completely demoralized here in Chacao.

      Like

      • And what, Chavez is going to run for governor in 22 states?

        Perjhaps you should move to Anzoategui, so you can vote for Aristobulo!

        Like

          • It’s still too early to tell. Early days yet.

            One the one hand the opposition voters are probably somewhat demoralized, but hopefully will realize it’s kind of do or die. If Chavismo ends up with a great majority of governorships our country will certainly go down the toilet a lot faster.

            On the other, I don’t think Chavistas will come out in the same numbers as they did for Chavez. Isn’t it “street wisdom” that it’s not Chavez, it’s the ones around him that suck?

            So who does he send out to be elected in the Finger Primary? The ones around him. With the exception of Vielma Mora, who by all lights seems to have done a decent job a SENIAT, the rest are either incompetent or scary enough to cause doubt amongst many voters, Chavistas included.

            It’s one thing to vote for the Pope, another to listen to your parish priest.

            And while we’re on the subject, please stop being sarcastic and let’s try to have a decent discussion that does not include name calling, yes?

            I’ll go first, Arturo.

            Like

  6. So Quico, even if they did this, you would give them a second chance? For me, they are long gone. They need to leave the room right now.

    Like

      • Maybe you could start with yourself, by making an act of contrition, and facing up with honesty to the fact that you bought into the polling results, without demanding more. For years. C’mon Quico. You need it. Auto-análizate, as my Dad would say.

        Like

        • Funny, not two weeks ago I was taking shit for refusing to dismiss Datanalisis’s polls, for arguing there was no basis for assuming they were wrong, and for noting that it was equally likely that they were right as that C21 was right.

          My line was entirely consistent – una vaina muy rara pasaba, with one of our two best pollsters clearly getting it very wrong. I was careful to be agnostic on the question of which one it might be. And I fought with Juan again and again when he assumed it was Datanalisis.

          I’m glad I took the line I took.

          But I think ultimately the one who has it right is Iñaki. This entire psychodrama developed as a result of absurd levels of pollster opacity. THAT needs to be addressed.

          Like

          • Who cares what any of you think? Most of the arguments here are made by people who don’t live in Venezuela and have no intentions of living in Venezuela. Sure, you may visit the auld sod periodically, but the ‘millions’ of Venezuelans alluded to in this post don’t give a rat’s ass who you are, or what you think. Chavez won, Capriles lost…. Venezuelans – including the ‘oppo’ – have the government they richly deserve. Now go cry in your beer.

            Like

              • I don’t read your blog. You’re too self-absorbed in gazing – and marveling – at your own navel. But one nof your regular commenters sent me the link this morning under the Subject: Stupid is as stupid does – which, admittedly, is unfair. I would have titled the email ‘Quico spanks his monkey in a mango tree.’

                Like

              • You don’t read the blog, and yet you know this: “Although with acid-tongued Syd who regularly goes for the jugular repeatedly, who knows?” For someone who never reads us, you sure know a lot about our commenters.

                Like

              • Bare minimum indeed, but there you are, busily commenting on everyone’s comments. Better pour yourself a fresh stein of brew to continue the boo-hoo-hooing. Ten years of self-absorbed but essentially irrelevant navelgazing must have caused one heckuva torticolis.

                Like

              • “I don’t read your blog,” says he who then opines on the regular nature of another’s acid-tongue, “repeatedly”.

                What B.S. Buh-bye, Carlos.

                Like

            • Now Juan cristobal weighs in. Two peas in a peanut. Syd is (in)famous on the Venezuelan blogosphere. She spares no one, takes no prisoners anywhere. Kudos to Syd. I wish she’d start her own political blog because her prose – albeit over the top at times – is superior to anything I’ve read on your blog, which I do look at periodically whenever friends send me emails urging me to read dis und dat. But I stand corrected, not one but two monkeys getting spanked in a mango tree.

              Like

              • Oh pull-eze. Spare me the laurels. Punk. They’re silly. Lots of people here have stronger and better opinions than me. Suggest you start your own blog. Then let us know how you’re doing, from time to time.

                Like

            • i do read caracas chronicles regularly. but only to read the occasional comment written by someone not with the opposition. it’s the only time truth is spoken on this website.

              Like

  7. None sense!
    @FT You could also take seriously any other poll, but your wishful thinking just let you see what you wanted, i.e. a Capriles’ victory.
    Now it’s time to kill the messenger that sang the song everyone in the opposition wanted to hear. It’s like a Rolling Stones’ Concert in which they don’t sing “Sympathy for the devil”. Isn’t it?
    You just discharge whatever doesn’t work, that’s it. Next elections, which are round the corner, just be careful with your choices.
    De pana.

    Like

  8. What about your culpability in knowingly publishing polls which had been exposed as unreliable

    here

    http://venezuelanalysis.com/analysis/7308

    here

    http://venezuelanalysis.com/analysis/7311

    and here

    http://www.guardian.co.uk/commentisfree/2012/oct/05/venezuela-elections-voting-trustworthy-polling

    You chose to rely on obviously dodgy polls because they told you what you wanted to hear and reinforced your now obviously flawed political analysis.

    As one of the above articles puts it: “Of the 18 polls carried out in September, 14 had Hugo Chavez ahead and the average lead across all the polls was 12%”. The actual result was just a single point off.

    Yet you chose to ignore the ‘poll of polls’ and the mountain of evidence that suggested that Consultores were not to be trusted. You are in the same “reputational hole” as them.

    Like

    • That hurt man!
      He will ignore you anyway, good thing he doesn’t filter comments.
      BTW I’m not on your side, that Chavez won doesn’t mean he is the rightful leader you try to portray.
      But fair enough, in this one you are deadly right

      Like

        • Well, first of all I am not Chavista. Just because I was clear Chavez was going to win just make me less delusional than others in the blog.
          Second you should answer Calvin looking straight at his face. The guy is right, there were 18 polls and, as far as I remember, just this one was giving Capriles a short margin win. Then the question of whether to believe the other extreme, i.e. GISXXI (strangely accurate by the way), is not a question, it’s a choice.
          I think that Datanalisis are the guys, at least as a reference.
          Not to see a massive amount of polls giving Chavez a lead was not only delusional, was stupid.
          Don’t bother to read Venezuelanalysis. They are at the same schizophrenic level of Globovision and some ladies I heard crying on radio programs Monday morning.

          Like

        • I wouldn’t say I personally “trust” any particular pollster. But when the average of 18 polls puts Chavez 12% ahead, you need some pretty compelling reasons to ignore that.

          Like

  9. Welcome to the world of useless open letters!

    Now you know how I felt when I wrote that letter to MUD asking them to distance themselves from the PP govt in Spain. You know they are never going to listen to you, but you know you gotta write.

    Like

    • I actually think if they have any kind of self-preservation instinct, C21 will release the data from their final poll, just as a matter of the incentives they face. They really do face a kind of existential crisis over this.

      Like

      • You are assuming people are rational actors. Sometimes people are. Not always.

        To me, there was a really small cost in distancing ourselves from the PP. We never did, and one of the attacks against Capriles was precisely that he would bring “El Paquetazo” in a similar way. In any case, that would not have been decisive, but I remain convinced it was one thing we could have done preemptively to weaken their arguments.

        Like

        • Well, we’ve been through this on email. I always thought the evidence of them getting close to the PP in the first place was weak – a stray remark in a one-off speech by J. Borges. Storm, meet tea-cup.

          Like

          • People, please. The PP is one of Spain’s largest party. This is not some radical fringe group we’re talking about. Show a bit of tolerance toward the right wing.

            Like

            • Juan!
              The guys that went on support the Iraq invasion are just Spain´s largest party? No issues about succession during Franco’s rule, i.e. Aznar belonging to falangists’ youth movement, and then of course, the creation of a real state bubble that burst in the PSOE’s face (I agree the PSOE is not better than the PP)?
              Well done mate!
              They are not right wing, they are francoists!

              Like

  10. Why is all and everybody mum about the – now already proven – miscounts and cheatings? Maybe the encuestadores were right and the count was wrong?

    Like

      • I don’t see why you’d say they are crazy. it is possible that a significant majority of the population prefers Chavez but don’t you think they’re crazy too? The fact is, it appears quite easy for someone with knowledge of the voting machine (like would be the case with Smartmatic) to manipulate results.

        See http://www.eecs.umich.edu/eecs/about/EECSNews/EECSNews12.pdf

        One excerpt of what the professor did to a voting machine in India:
        They published a video in which they were able to demonstrate two types of attacks against a real Indian electronic machine. One attack involved replacing a small part of the machine with a look-alike component that can be silently instructed to steal a percentage of the votes in favor of a chosen candidate. These instructions can be sent wirelessly from a mobile phone.

        Like

        • The machines printed paper vouchers. The vouchers for 54% of the machines were hand counted by volunteers from both sides. I will personally mail you a $100 bill if you show me evidence of any hand-count that does not match the machine count.

          Until such evidence emerges, allegations of this type are deeply irresponsible and destructive.

          Like

  11. Mayke, jeez, yeah, Nelson… Although i think they have already written about him and other “journalists” like him.

    I don’t think this is THE END for C21. As it is common here, they will lay low for a while, do some product market research or whatever. All will be forgotten in a few months. It tends to be that way here. We dont hold people accountable. Sometimes that can be a blessing (second chances and all that) but more often than not it’s a curse.

    Like

    • He is predicting now that Fidel is connected to an artificial lung and he is going to be disconnected “soon”. 80 years old Fidel dying, what a surprise!
      What it upsets me is the fact that he was one of the biggest campaigners of Capriles’ victory. Also, he spent months saying Chavez was about to die and then in his last runrunes this week, he says we (he) don’t know if Chavez is sick. CDTM!

      Like

      • Well Nelson is a bit of a case in point to why I don’t think anything will happen to C21. Nelson, no matter his many “pelones”, still has his column, his twitter followers and his radio show.

        Like

        • As they say in show bizniz, even bad news is good news. Meaning, it doesn’t matter what Bocaranda says. For he’s crafted ‘news’ as entertainment.

          Like

  12. I agree with you Quico. They need to fully disclose what they did. Actually, it’s the only option they have if they want to remain in business. They may have made a bad methodological choice, and in an industry so deeply rooted in the academic world that’s not necessarily a bad thing. But they’d do themselves a favor to discover what it was, learn from it and teach the industry -and whoever is interested- how they got it wrong. Whether that would buy them back some credibility with the general public is a different story, but at least they would do what any professional firm would do (and what’s actually customary in the academic arena, where they will ultimately be judged) . There’s really nothing “secret” in election polling methodology. If a poll claims to be scientifically conducted -and IF IT IS PUBLICLY RELEASED BY THE FIRM CONDUCTING IT- it has to be willing to disclose all aspects of its methodology to whomever requests them. That’s actually a guideline from AAPOR, WAPOR, CASRO and ESOMAR (the major industry entitities that self-regulate the survey research industry around the world). So, if they want to continue to have the blessing of these organizations they need to do that.
    Now, I noticed C21 letter is not addressed to the general public but its clients and friends. It seems they’re implying those polls where proprietary info of their clients. But I don’t think they can sustain that claim because I believe Luis Christiansen himelf presented results in public. Plus, even if their polls got leaked, they should know better.
    Now, if C21 did publicly release its polls itself, they need to say who paid for them. If their client is really the MUD I think they’ll be in trouble with the general public. They could still make a case with academia that they made a bad methodological choice and save their honor there, but I think in the public domain their name will always have and asterisk next to it, much like Barry Bonds’ record-breaking homerun ball.

    Like

  13. Psychologists have a word for this…I think it’s called displacement. Anyway, we all got it. Better a polling company than your pet or dentist.

    Like

  14. Derecho al pataleo: faltaría más. ¿Que apostaste a esta encuestadora y no a otra de las tropecientas que como setas florecen en Venezuela? Bueno y qué.
    But I do distinctly remember you arguing against predictive values attached to any poll whatsoever. And I applauded you for that.
    So what happened to you on the way to the Forum?
    C’mon. You were wrong? Supéralo. But please stop pointing at the guy next door. Hey, look, he got it wrong, not me!
    Childish stuff. And yes, way beneath your brains.

    Like

        • Arturo why do you bother with las moscas aqui? Do you not have important work to do consolidating the victory of los hijos de la patria? Why toy with us when you could be raising the consciousness of the peasantry? Why do you care what we think? Is it not more heroic to just go away now? Have the enemies of el pueblo not been vanquished? Are you not wasting your immense talents on us?

          Like

          • Reading between the words I think you are saying that you do not like the truth being thrust at you. Knowing that your illusion has been burst is not easy to deal with.

            Like

  15. This open letter, sound more like I want to put the blame in someone else rather than assume my own fault. If the credibility of C21 is shattered your credibility is broken beyond repair. Nobody pointed a gun at you to believe and take for granted the results given by C21.

    I mean come on!! you are demanding the company to disclose their methodology way past the elections??… are you fuc…g kidding me???. Have you lost your fuc..g mind??. I mean what’s the point??. seriously guys. You are in total delusion.

    Others bloggers have taken a more humble stance and assumed their own fault, by predicting a victory of Capriles. But you rather to blame someone else.

    You and Juan, and many people chose to believe in the results given by C21, because their polls yielded the results you wanted to believe. period.

    For me the discussion of whether or not the polls followed the methodology and if their predictions were relatively close to the final results given by CNE is POINTLESS!!! since this was not a normal election, with normal rules and the circumstances are very different.

    Your main mistake, same as Capriles, was to believe that this election was like any other democratic election in some other democratic country.

    Like

Comments are closed.