Wednesday, February 13, 2008

Measurement

Just after talking about the euphemistic use of "economy" yesterday, I found an even better one here:

"Nearly all Italians drink bottled water rather than the piped stuff. The industry is worth an estimated 3.2bn euros (£2.38bn) a year to the Italian economy."

It would be very refreshing if they'd just say "GDP", since that's what they mean. That wouldn't make it any less understandable either, because "economy" is equally vacuous. Let's play the show and tell game again: what does the quotation mean?

It can't mean that "if no bottled water was sold, people would spend 3.2bn euros less" - I'm sure they'd find another way to spend it. It can't mean "worth 3.2bn euros a year to the Italian resource allocation", because that's not a sentence. It can't mean that "Italian workers/producers would get 3.2bn less in wages/payments a year", because I'm sure that they could do something else besides produce bottled water.

My best guess is "the Italian bottled water industry makes sales worth 3.2bn every year". Why, oh why, can't the reporter simply say that? It's not remotely the same thing as any of the other suggestions I just made, yet I guess they're all technically possibilities if we read "economy" as "system of production and consumption" or something like that. If I want to be really obnoxious I could ask whether the reporter has measured every consequence of the hypothetical disappearance of the Italian bottled water industry to come up with his figure.

More to the point, let's forget about the absurdity of the quotation in itself and ask why the "worth" of any effect on the "economy" measured in money? This screams a confusion of metric and quality, a cardinal sin of positive science; even if I could get an accurate figure for the effect of something on "Gross Domestic Product", I still think "worth" is too loaded a term.

A big chunk of the gulf between theoretical economics and empirical testing of real-world relationships is the metrics we use. Our abstractions work (or can, or should work) in a world where we measure outcomes agnostically: if you care about this. Theoretical economists can play in imaginary worlds all day, exploring the "relationships" between fundamentally unmeasurable things under their assumptions. On the other hand, some imaginary concept like "utility" is singularly useless if we want to actually talk about the real world. Empirical economists must deal with this problem somehow: if you want to talk about the effect of this measurable thing on that measurable thing you must explicitly ignore the intangible (like, perhaps, satisfaction).

Then what conclusions can we draw? This affects that, but not how "good" it is. This is, again, the reason why economics can never be a technocratic prescription of what "should" be done; we simply have no real-world metric to answer the question, and our theoretical metrics are unobservable. It's the power and beauty of the science - we don't have the answers. Is someone pretending to? Just for fun, I Googled "what's wrong with GDP". When our metrics are the sole determinant of policy, of course the metric - and, by extension, economics - comes under intense attack.

Now that's all well and good until we get to economics teaching, practice and discussion which ignores this important conclusion. I don't deny the challenge of constant vigilance to make sure student, reader, researcher know that we're dealing with only what we can measure, but nothing short of a commitment to acknowledge the limitations of measurement at every turn will be enough to dispel the notion that the science of economics can tell us what to do.

No comments: