Policy Evaluation as Subversive Activity

Mision Robinson has a lot of work on its hands…

My new piece on that FP Blog takes one more look at that 2006 research into Misión Robinson. I’m totally fascinated by that paper, and only wish I’d seen it much sooner. The contrast between its plodding, sober, evidence-based appraisal of Misión Robinson and chavismo’s breathless propaganda lies about it strikes me as all anyone should need to know about the thoroughgoing rot in Venezuela’s public sphere.

At first I thought Juan’s theory was a little hokey, but more I more I think he’s right: the basic problem with our public sphere is that nobody understands the concept of Opportunity Cost.  Without a healthy grasp of the fact that different legitimate policy priorities have to compete for the same, finite set of state resources, reasoned debate about public policy really can’t take place.

That new (to me) research on Misión Robinson is a beautiful case in point because, in the original paper, Daniel Ortega and Francisco Rodríguez go there:

One way to evaluate the program’s returns is by comparing our estimated program impacts with the program’s official expenditures. According to the Ministry of Finance, Misión Robinson has received an investment of 80 billion Bolívares (US$50 million). This amount may well substantially understate Robinson expenditures, since it excludes a number of off-budget expenditures on the program. Even if one attributes all of the reduction in illiteracy observed between the first semester of 2003 and the second semester of 2005 to Robinson, the estimated cost would be $536 per pupil who learned to read. In contrast, a recent study by UNESCO of 29 international adult literacy programmes estimated the  average cost per successful learner to be $47 in sub-Saharan Africa, $30 in Asia, and $61 in Latin America. Under a more conservative – yet still optimistic – estimate of program success, namely that the total number of people who become literate through the program was only 48,327, then the cost per newly literate person would be much higher, at US$1035.

This, of course, is territory the government steadfastly refuses to cover. Instead, chavismo would have us turn the debate into an idiot controversy pitting good, righteous people who want to teach illiterate adults how to read against a mythical race of strawmen who oppose that for some reason.

Such aggressively dumbed down polemics are all you’re left with when the debate happens in a policy evaluation vacuum. Without systematic evaluation, it’s not possible to think give a serious answer to such basic questions as “was Misión Robinson worth it?”

No explicit targets, no ongoing monitoring, no systematic evaluation – these are the hallmarks of chavista social policy design.

In such circumstances determining how effective a misión is becomes a matter of religious conviction: an automatic extension of political commitments that has only the faintest link with that misión’s performance. Even the religious nomenclature seems designed to take the program out of the realm of cold, calculable Weberian rational bureaucracy and into the sphere of redemption via faith. To measure, to evaluate, to monitor such a program is subvert its essential logic within the chavista cult of personality. Misión Robinson works because Chávez is righteous.

Chávez willed it, after all. And Chávez is good. Whatever he wills is good. Who would want to risk running an evaluation exercise that puts that into question!?

25 thoughts on “Policy Evaluation as Subversive Activity

  1. We still live under the false premise that Venezuela is a wealthy country, and that we can spend money foolishly because tomorrow we can dig a hole in the ground and strike it rich.

    Our GDP per capita is $12,700, most of it coming from the oil industry. That’s not exactly a stellar performance. As a matter of fact we are far and away from any a rich country by any standard.

    We need to come to terms with the fact that we are not rich and explain that to people as clear as possible. We cannot keep acting like nouveaux riches, wasting money (in F1) like there is no tomorrow and then borrowing money to keep a life style that we cannot afford anymore.

    Only after making that clear to people, only then we’ll be able to talk about opportunity cost and blood, sweat and tears. No more lies about how wealthy Venezuela is.


    • A. Barreda, I see this argument that “we are not rich” used often, and I think there is a semantic issue in that endless debate. When people think that Venezuela is rich, they are usually referring to the fact that Venezuela has oil, which makes it richer, way richer, than if it didn’t have oil. Just because the oil has not helped in making Venezuela a nation of greater GDP per capita than other nations does not take from it that it is rich compared to the same Venezuela without oil.

      My personal view of it is that Venezuela *is* rich because, by simply distributing the cash from that oil, it would have all its citizens out of poverty, five times over. Regardless of GDP per capita, that, to me, is rich.


      • I think your last statement is not so obvious. The barrel is around 100$ and Venezuela produces around 2.5 million barrels a day, so in a year we get 100×2.5×10^6×365$ = 90 billion $ approximately, from selling oil. Divide that by around 30 million inhabitants and you get only around 3000$ for every citizen per year. That would certainly be helpful to anyone, but would hardly put them out of poverty, although a family of four would get 12000$, which is already something. However, you have to consider that not all of the money can be given away like that. A lot of the money has to be invested back in PDVSA, and some would necessarily have to be kept by government in the form of taxes, in order to pay for a few basic things any government has to take care of no matter how libertarian it might be. So at the end, if the money were to be distributed to the citizens in Norwegian style, I don’t think anyone would get much.


        • Getashrink, even using your 3000 $/yr estimate per person, that comes to 8$/day per person which is 4 times the poverty line, defined at 2$/day.

          As to the money that PDVSA needs, if you went all out with my proposal, PDVSA would be private, and PDVSA would have to pay the government for any oil it extracts. What it does with that money from that point on would be its decision in maximizing profits from it. The government’s only role is in trying to maximize revenue from selling the oil to any company that would want to extract it, at fair competitive market prices.

          As to what the government gets to keep from the sales of oil, 100% would be via taxation. The citizens would spend it on goods and services that pay taxes, and oil companies would pay taxes. From these taxes the government would pay whatever it is to what you are referring that it needs to.

          Note, that not giving people the money, in Venezuela’s case, is a regressive retention taxation with the poorest Venezuelan falling in the 100% taxation bracket and the richest falling in the 0% taxation bracket. This is because of every 30million dollars of oil sold, $1 belongs to the poorest and $1 belongs to the richest. So, anything the government spends the oil money on, say PDVSA investment, or higher education, it is being paid, regressively, more by the poorest than by the richest.

          Think about it, 8 $/day per person would make a huge difference to a majority of Venezuelans, not to mention Venezuela making headlines worldwide as having eliminated poverty 4 times over. Did I mention it reactivates the market, lowers interest rates, and wins votes, too.


          • Fair enough, although that poverty line being at 2$ a day seems a bit too low to me. In any case, I’m not againts the idea of distributing the money from oil. If it has worked in Norway it can work elsewhere.


            • I didn’t make up the 2$/day, by the way. That’s a global standard, of sorts, with critical being at half that. In Venezuela, however, they reached a comparable number by establishing critical poverty at the amount of money required to pay a minimally balanced local diet, and that came to just slightly over 1 $/day. Then they simply established that double that would define non critical poverty which came to slightly over 2 $/day.

              Glad to know you’re on board with the distribution.


          • I am not famliar with the World Bank definition, but in our case the $2 thresshold would cover food and nothing else. And more like survival stuff, not healthy nourishment.

            Trying to live in Venezuela with just $30 a month is nothing but realismo magico. Is you want to raise that to 60$ a month, it’s simply ludicrous. Finding a room for rent for $60 a month in any city in Venezuela is mission impossible. Unless you’re willing to live in a slum, of course.

            Additionally, there is a big part of the picture missing in your back-of-the-envelope calculations. You’re using the GDP per capita as an estimate of the income, but as Get mentioned, you’d first have to deduct a huge chunk of money that belongs to PDVSA and some more money to pay for government spending.

            Even using the scheme of UCT/CCT, I’m not sure that your Regular Joe will get much of the 8 dollars you were mentioning. Even if you decide not to build a new power plant, highway or school – all things that we need desperately – we need to mantain those that already exist, and that requires a lot of money. And if you’re willing to make those services private, I guess you’ll have to add that to the expenses of Regular Joe, and from my experience, not even the current $250 a month minimum wage would be enough to cover all that…


            • A. Barreda,
              “The World Bank defines extreme poverty as living on less than US$1.25 (PPP) per day, and moderate poverty as less than $2 or $5 a day ” http://en.wikipedia.org/wiki/Poverty
              As I mentioned earlier, those numbers are comparable to the numbers Venezuela reached on its own, and they they address a minimally balanced local diet, and double that for critical, and non critical poverty, respectively.

              “Trying to live in Venezuela with just $30 a month…” Again, not my numbers. They are the global and local definitions by which UN and Venezuela are measuring the millenium goals being met. I agree that life with just 60 $/mo is still harsh, but you must agree that whatever the income –even zero– , adding 60 $/mo would mean the world to the majority of Venezuelans.

              “You’re using the GDP per capita as an estimate of the income, …” No, you were. I only replied using the terms you brought up, and you still missed the point: If you have two identical nations and one has oil, the other doesn’t, the one can be considered richer than the other. I was merely pointing out that *regardless of GDP* Venezuela can be considered rich *because it has oil*. It’s a rich bonus, whether we waste it or not.

              “…you’d first have to deduct a huge chunk of money that belongs to PDVSA and some more money to pay for government spending.” As I replied to Getashrink, if PDVSA were private, then the government would charge PDVSA for the oil that PDVSA extracts, then be done with oil decisions. PDVSA would go off and resell, or refine the oil for money, and do whatever oil companies do with their monies. A big chunk usually goes into reinvestment, but that would be PDVSA’s problem, not the governments.

              As I also replied to Getashrink, the government would take the money it got from the sale of oil to PDVSA and any other oil extractor at fair, competitive market prices, and distribute it, thus ending poverty *by global standard definition*, then the citizens would spend it, in turn getting taxed back in the hands of government for the government spending you mention, but not before reactivating the market and lowering bank interest rates.

              So, no, I don’t have to “first” deduct a huge chunk, I can deduct it *after* poverty is eliminated. It’s the same net economic effect, but better distribution, less corruption, less mismanagement.

              “I’m not sure that your Regular Joe will get much of the 8 dollars” I am. The barrel of crude is selling for 110. Even assuming that an extracting company would only pay 50$/barrel, at 2.5 million barrels per day, divided by 30 million Venezuelans, would still give us over 4$/day per Venezuelan, the amount Regular Joe would receive.

              “Even if you decide not to build a new power plant, highway or school…” Why would I decide that? As I have pointed out, repeatedly in the past, the money for that would still be there. Money doesn’t disappear just because it’s distributed. It flows right back up the market, much more efficiently than it trickles down, I may add, into the hands of government via taxation of the most successful, progressively. With that money, the government builds the new power plant, highway, or school, and maintain the old ones, all desperately needed, but not at the cost of having people below the poverty and critical poverty line, which you yourself admit is too low to even consider reasonable living.

              “And if you’re willing to make those services private…” As many as would make sense to do so, getting those things off of government hands so that government can better handle its already full plate.

              “not even the current $250 a month minimum wage would be enough to cover all that…” Maybe, but wouldn’t $250+$120 be better than just $250? That’s almost 50% increase, and for a family of 4, it’s 200% increase.

              I think it’s you missing a big chunk of the picture.


              • Yes, the poverty threshold is an international standard, but I’m still not buying it. $60 a month is hardly enough to buy food in any major city in Venezuela much less put a roof over your head.
                No doubt that $60 a month would be a great help for many persons in Venezuela. But only if that’s additional income. If you have only $60 a month, you’ll be able to survive and nothing else. With some luck the person will have the chance to build a cardboard house and that’s it.
                Yes, I use the GDP as a reference, but it was Get who made the back-of-the-envelope calculation. Still, I believe that oil revenue is a mirage of wealth. Yes, it’s supposed to be worth millions, but what about the impending economic crisis? Most likely the prices will fall? How much? Can you predict that it will not fall to $10/barrel?
                Besides, you’re ignoring the “dutch disease” related to the oil industry. Yes, oil should be a bonus, but that’s not the case. Venezuela is not Norway. Venezuela exports oil and nothing else.
                The $90 billions are gross domestic product and not the net gain. PDVSA will keep a chunk of that for itself some part of the oil revenue not only for investment, but also to pay salaries and keep its operation running. A private owned PDVSA doesn’t change that. So, the net income of the state will always be much smaller than the aforementioned $90 billions.
                I have already mentioned before that I support the UCT/CCT program. However, I cannot imagine how that could solve the many critical problems that we are facing right now with our decaying infrastructure. Not all the money in our economy flows back to the government. Some of it goes to the private investors that import or produce the things that people buy. How much revenue will the government get from that? 20%? 40%? Is that enough to solve all our infrastructure problems? Should we just accept that electricity is too bourgeois for us and not invest on that?
                Once again, you’re assuming that the $60/month are in addition to some imaginary income that sometimes is not even there. So:
                1) Mere $250/month is not enough to live in any major city of Venezuela, so I would consider that our poverty line in our major cities. $60/month is extreme poverty.
                2) Oil revenue is nothing but a mirage of wealth. If the prices sink drastically, we’d poor. We should act according to the worst case scenario. No more idiotic optimism


              • “$60 a month is hardly enough to buy food in any major city in Venezuela much less put a roof over your head.” One of the predictable results of cash distribution would be decongestion of the cities, precisely because many people would go to where the $60 go a longer way.

                “No doubt that $60 a month would be a great help for many persons in Venezuela. But only if that’s additional income.” You keep implying that $60 would only help those with income, but how can it not be even greater help for those with no income? You mention $60 is barely enough for survival –agreed– so how do you justify implying that we should not even give them enough to survive? In my book surviving with a cardboard house is better than not surviving.

                “Most likely the prices will fall? How much? Can you predict that it will not fall to $10/barrel?” What I can predict is that even at 10$/barrel, Venezuela would be much richer than an identical nation with zero barrels. Remember that I’m not arguing your claim that Venezuela isn’t rich, in your sense of the word; I’m arguing that it is rich in a relative sense of the word when compared to an identical nation without oil. Venezuela is rich in that sense, and it has all the *potential* to be rich in your sense, too, if the riches were better spent.

                “A private owned PDVSA doesn’t change that.” Yes, it does, because the money *from the oil* as opposed to *from PDVSA’s value added* does not enter PDVSA. A private PDVSA would have oil as a cost. It would buy the oil from Venezuela, as would other buyers. They would then go make money doing what they do. It’s from THAT money that PDVSA would have to take chunks out to invest, pay salaries, and keep operations running. Not from the oil money, but from their sales. Venezuela would be receiving bids from PDVSA adn other buyers for its oil and selling it at the best possible price. That money would be distributed to the citizens. That money is more than enough to get and keep all Venezuelans out of poverty, and has been enough since the barrel was at $20. The government would get a chunk of PDVSA’s money and Venezuelans’s money, via taxation. Your claim of 20% or 40% is unrealistically low if you follow the math. Taking a look at income tax, the more money moves the more income gets reported, the more money gets taxed. Income tax is not based on a percentage of new money entering the hands of those above a threshold; it’s a percentage of the total movement of money in those hands. The more movement the more taxation. Cash distribution is causes turbo movement in the market. So income taxes would be much greater than what you imply. Sales taxes, too. Because so much money would be deposited in banks, these would have to lower interest rates, which makes property values go up until contruction cathces up with new demand which also increases property taxes. All very progressive, and all desireable for the economy.

                On the other side of things, the government would have lower spenditures on the social side because the poverty is now at zero. You try to make it seem like the money distributed is lost. It isn’t. It saves from having to set up food programs, increase pensions, and such other such social issues. Money is not destroyed; it keeps changing hands, the more the better.

                So, again, you would get your electricity, shools, highways, AND we’d have all Venezuelans above survival situation.

                “you’re assuming that the $60/month are in addition to some imaginary income that sometimes is not even there.”

                No, I’m not. I’m assuming that Z income plus 60 $/mo is better than just Z, and that most certainly 60 $/mo is better than nothing, which is your alternative.

                1) You are referring to your definition of “living in poverty” versus the poverty measure which I’m using. My argument still stands in that even if the money is not enough, it is better than not having it, and it will provide incentive to leave the cities which will also redistribute jobs as well as get city jobs to be better paying while making the city more liveable.

                2) Oil is just a bonus wealth. No mirage. It exists. It’s there. And it’s better than not having it. UCT is not idiotic optimism; it’s system optimization. There is no more efficient way to get the economic engine going at full speed with as little corruption and mismanagement as possible. You seem caught by the Lord of the Oil Syndrome, not wishing to let go. Consider, it’s not ours to decide what to do with other people’s money. And the oil money belongs to all Venezuelans. What you are suggesting is that there be a retention tax whereby we take the same amount from the poorest as the richest Venezuelan to pay for the new electricity generators. That’s regressive, having the poor pay a bigger percentage of those investments than the rich. Now, that’s rich!


            • “Unless you’re willing to live in a slum, of course.” That one line of yours ran chills up my spine. Are your proposals only for those who don’t live in the slums?! Is chavez right?


              • “Are your proposals only for those who don’t live in the slums?”
                My statement is pretty simple: Life in a place with no basic services such as police patrol, waste disposal, water drainage like the slums is not a place anybody should live. Or are you actually willing to live there? Don’t be silly and stop reading between the lines things that are not there…


              • Not reading between lines, just trying to see where your logic leads, which where you placed that comment led to believe those who live in slums didn’t count in considering the improvement 60 $/mo would have in people’s lives.


  2. “We cannot keep acting like nouveaux riches, wasting money (in F1) like there is no tomorrow and then borrowing money to keep a life style that we cannot afford anymore.”

    Now, think about it: How many people actually behave this way in Venezuela? Are we thrifty people, or big spenders? What’s our approach towards earning and spending? Do we save for tomorrow, or learn to invest wisely?

    “With no explicit targets, no ongoing monitoring, no systematic evaluation, determining how effective a misión is becomes a matter of religious conviction: an automatic extension of political commitments that has only the faintest link with that misión’s performance.”

    Again, think about it: To what extent our governments actually engage in responsible spending, across the board (including municipalities and state administrations)? And who holds them accountable, and how? To what extent are we interested in actually holding the government accountable for its expenses, policies, or even for its behavior?


    I don’t know. I’m in a bit of a pessimistic mood lately, but I don’t think explaining the concept of opportunity cost to Venezuelans would be that easy. It wasn’t easy when we went absolutely broke (late 80s, 1990s), much less now. Plus, our overall approach to political authority is delegative: Whoever wins office can do whatever he or she wants as long as he is a competent repartidor. The emphasis is on particularistic benefits, and not on public goods…


    • I believe we never got over our oil addiction of the late 70’s. We went to rehab in the nineties because we were broke, but as soon as we got some money back we relapsed. Big money, big spending. And not only the government is doing that, but also the regular folks were bitten by the bug. The Eudomar Santos’ philosophy rules our everyday: “como vaya viniendo, vamos viendo”.
      The petrodollars are nothing but an ilussion of wealth that can hurt us in the end, as it already happened in the 80’s and 90’s. With another global economic crisis just around the corner, our reckless attitude towards money will be our doom. I just hope we will be able to endure it when it comes.


  3. A lot of chavistas think that as long as some of the money reaches poor people then that is a win compared to none. The idea of a non-corrupt government which efficiently allocating and administering resources is alien to them. They assume that the only government possible in Venezuela in the short and medium term is a wasteful government. Of course it would still make sense to talk about less wasteful vs more wasteful, however, from my experience many chavistas are not prone to digest quantifiable comparisons between the 4th republic and the 5th “republic”. And the government is actively destroying whatever statistical data which could be comparable to 4th republic data. This connects with the chavista idea of the 5th republic being something else than the 4th, something incomparable, and they have partly succeeded in spreading this idea. 5th is good 4th is bad, forget about the numbers, forget about comparisons.

    Communism differs sharply from “regular” politics in that it no longer cares about specific policy goals. If you are building a staircase to heaven then whatever earthly goals and measurements become irrelevant. In Cuba (and in Venezuela more and more), public communication about policy is focused on non-measurable issues like independence, imperialism, patriotismo, burguesia vs pueblo, deportes. It is gradually accepted that statistics are only interesting as a means of showing the government’s success and goodness. Policy evaluation goes out of fashion. Compare http://www.bbc.co.uk/news/politics/ to http://www.vtv.gov.ve/index.php/noticias

    To confront this, it could possibly work to tell people that what really matters is there standard of life not talking paja all day every day. What good is Chavez if your standard of life is gradually declining in most areas? I think one should make commercial spots interviewing Cubans about there daily struggle and lack of freedom to sabotage the chavista mythology. When people believe more in there leader than their standard of life we have lost. Another kind of campaign could be something like “Ya estoy harto de politicos en camisa roja que dicen ser mi amigo mientras roban la plata. Prefiero un politico que actua en lugar de hablar”.


  4. Opportunity costs and “Who would want to risk running an evaluation exercise that puts that into question!?”

    The first is a recurring theme of ages. For example, Red Cross’s 80% overhead is often used as an example. Businesses are measured against bottom lines of cold, hard cash. That’s how they know they are doing well, and that’s how they compare to see if they are doing well against others. So For-Profit businesses have a simple Measure of Success. With Non-For-Profits the bottom line stops serving as a measure. If the purpose is to save lives, then, as is often expressed, saving one is enough to make it all worth it. When Red Cross reports that they saved 80 lives in a year, or that they provided 2000 beds, is that a lot, a little? The question really should be, would a competing ssorC deR have doing 90 lives and 2200 beds? So, in courses for those wishing to run a non profit organization, what is recommended is to have a well-defined, cold, hard measure of success.

    But chavez not only has managed to reduce the number of people wanting to evaluate his programs by making it a personal risk to question the “good” of his programs, to which you point in the post, and also taken it further to prevented any competing activity for comparisons as in Red Cross’s case, he has prevented media from making it near impossible for information regarding any remaining negative evaluations to reach the majority of his voters.

    Está blindado, pues.


    • “Because if the government had invested a fraction of the effort it devoted to publicizing Misión Robinson’s supposed achievements to optimizing the program’s efficiency, Venezuela would have eradicated illiteracy long ago. ”

      If they had distributed 20% of the oil revenues to the citizens, poverty would have been eradicated, too.


  5. If it takes a thousand bucks to teach someone to read it is still worth it. What is depressing is the apparent lack of results at any cost. I remember reading in a barrio adentro report a while back that illiteracy had been eliminated. This is so patently wrong that the conclusion to be drawn is that accountability has been eliminated. What is the Chief Auditor doing? (rhetorical question)


    • What if it costs 900 bucks to pay for dialysis to keep a Kidney patient alive but you don’t have it because you spent $1,000 for a literacy program you could’ve gotten for $100?

      Still worth it?


      • Point taken. I guess to my non economist mind, the economic and social costs of illiteracy make what is being spent – inefficient though it might be- still seem like a drop in the bucket…if it actually worked. But you are right to point out the practical effect of this kind of waste. Could be spent on other necessities. Depressing.


    • See…the concept of opportunity cost is very elusive!!!

      Please, elaborate a bit on “it is still worth it”. In what sense it’s still worth it? And don’t come with some lame argument about human dignity or any crap like that.


      • You missed the point where I agreed with our host. Having said that, your comment raises an interesting question as to how you monetize the cost of illiteracy. I’m not talking about human dignity mush either. I taught adult literacy. One of the lessons we had was how to navigate public transport ( ie read a map). That would get people out of their four square blocks to…productive jobs. But again, I have no disagreement with FT so spare me the spanking amigo!


    • See it from this point of view, would you send your child to a school that charges a tuition ten times higher than a good school and in top of that gives him no education? The road to hell is paved with good intentions and chavismo pretty much sums up that. State funds are not a bottomless pit and cost-benefit analysis are necessary even for the best intentions, and is not only here in Venezuela, I think that the situation in Canada is related, I want my tuition low regardless of the effect on the overall economy and quality of the learning.


  6. Opportunity Costs is the concept behind the finite resource allocation problem that drives decision making both in government and in industry.

    And having and applying a consistent decision framework aligned with your strategic goals is what distinguishes a competent leadership vs. an incompetent one.

    And – porque me sale del forro – is NOT a valid decision framework aligned with strategic goals.

    No mistery there …

    Ask any competent economist.


Comments are closed.