Posts tagged ‘theory’

The better the question, the worse the answer

Justin Wolfers wrote recently about the level of interaction between economics and other social sciences.  In particular, he wonders why economic work is not well represented in a list of the books most cited in social science research.  It’s a good question: I find many of the tools and techniques developed by economists are useful in my works studying political phenomena, and I do cite economic research.

One particularly thoughtful commenter on Wolfers’ post notes that economics combines the controversy of addressing everyday issues with the general inaccessibility of chemistry.  This conflict may make some people resist the conclusions of economists, ie. strong prior + incomprehensible evidence = small amount of updating.  The comment continues:

However, the inaccessibility of economics does not merely arise through inadvertence. As many jokes attest, economists are not merely unsentimental, they are ANTI-sentimental. An economist will often revel in the opportunity to rub people’s noses in the conclusion that their pre-conceptions are fluffy-headed poppycock. To many people (including some economists, I fear), economics appears to be less a social science than a religion, revealing to a chosen few the mighty counter-intuitive truths by which to pass judgment on a sinful world.

If I’m an accomplished scholar in another discipline, to what extent am I open and receptive to this kind of intellectual upbraiding?

Another commenter notes

My guess is at least part of this effect is reciprocity. Economists are famously bad when it comes to citing relevant research from the other social sciences. And I am not even talking about humanistic research: experiments from social psychology and statistical work from sociology are often ignored when economists do related (or even nearly identical) work. There is a widely held perception in the other social sciences that economists are, at best, disinterested in having interdisciplinary conversations–or, at worst, regularly tolerate cross-disciplinary plagiarism. My guess is that a culture of arrogance is the ultimate cause.

These are legitimate concerns, and there are reasonable rejoinders, but I think there is a more compelling explanation than “economists are a$$holes.”

When we develop a theory, we throw away some of the details about the world in order to make the theory simple enough to understand.  This abstraction is an art: there are no hard rules and too few guidelines on how to make useful theories.

Suppose that there is only one choice: how much detail to discard.  One can (grossly over-)generalize by saying that economists throw away more detail, ignoring things like irrational behavior, while other social scientists throw away less detail.  By simplifying more, the economists gain the ability to apply very technical tools to generate results that are very reliable given the assumptions they made.  By simplifying less, other social scientists are left with questions that more closely resemble reality but are harder to analyze.

Internal validity versus external validity.  Tractability versus verisimilitude.  We’ve heard this song and dance before.  The implied question is something like Heisenberg’s Uncertainty Principle:

Hypothesis:  (time to analyze) * (problems with applicability) >= (some constant)

Does this hold?  Actually, I am optimistic that it doesn’t.  Regardless, we all face choices when trying tell a story that explains something.

All Theorists are Normative (or run that risk)

A recent exchange at the excellent Cheap Talk focused on how the uselessness of the United States’ recent promise not to nuke other states who comply with the Nuclear Non-Proliferation Treaty (NPT).  Sandeep Baliga writes

This is an attempt to use a carrot and stick strategy to incentivize countries not to pursue nuclear weapons.  But is it any different from the old strategy of “ambiguity” where all options are left on the table and nothing is clarified?  Elementary game theory suggests the answer is “No”.

We are left with the conclusion that a game theoretic analysis of the Nuclear Posture Review says it seems little different from the old policy of ambiguity.

Baliga raises a simple, clear, and important question:  When we say we are not going to nuke you, what keeps us from doing it anyway?  “[T]he words of the [new policy] are just that – words.”  Baliga seems to imply that there is no reason to make such a promise.

Suppose that the assumptions of the “elementary game theory” employed are all correct.  They are standard assumptions and they have been employed correctly; therefore, our administration has wasted its time.

Let’s look at it from the other direction.  Whether or not the folks who make our policies have studied game theory, are bright enough to look back and notice which things tend to work and which things tend not to. We may have only 6+ decades of experience with nuclear politics, but we have many, many more years of experience with the role of cheap talk in diplomacy.  Apparently, our policymakers seem to think that cheap talk can work, at least enough to be worth the effort of making a statement.  Therefore, our administration did not waste its time.

Barring a logical error, one of three things must be true.

  1. The assumptions of the game theoretic analysis are correct (or close enough to being so,) so the administration is wrong and can learn from the theory.
  2. The assumptions of the game theoretic analysis are wrong (or not close enough,) the administration is correct, and theorists need to update their model.
  3. The assumptions of the model are wrong, but the conclusion is correct and the administration is still wrong.

Setting aside case (3), on to the central question:  Is the theorist being positive, describing the world, or normative, telling us how it should be?  (Both are important, useful roles, and formal theorists can, and I believe should, speak up on normative issues whenever science can help anchor moral choices.)  In case (1) the theorist is describing the world, providing information.  In case (2) the theorist is incorrect, but still earnestly trying to describe the world.  In either case, he is being positive.  However, by asserting or implying that case (2) is not under consideration, he is taking a normative position:  not about the conclusion (that in contexts like this cheap talk is useless) but about the assumptions.  He is saying, for example, that it should be the case that we can ignore audience costs.  Unfortunately, this assumption and other similar ones turn out not to be tenable even in theory.  More importantly, I agree with the administration that promises like this can have a real, positive effect; given this, the assumptions of the model must not be correct.

In my view, it comes down to this:  When a formal theorist derives a behavioral prediction that does not coincide with what people actually do, maybe the model is correct and we can learn from the model, or maybe the model is wrong and the modeler should learn from the world.  Perhaps one’s goal (understanding policy or shaping it) should drive that decision.

If you are confident that your theory explains the relevant situation very well, go ahead and use it to make recommendations.  Just remember Cromwell’s Rule:

I beseech you, in the bowels of Christ, think it possible that you may be mistaken.