Posts tagged ‘wrong question’

The better the question, the worse the answer

Justin Wolfers wrote recently about the level of interaction between economics and other social sciences.  In particular, he wonders why economic work is not well represented in a list of the books most cited in social science research.  It’s a good question: I find many of the tools and techniques developed by economists are useful in my works studying political phenomena, and I do cite economic research.

One particularly thoughtful commenter on Wolfers’ post notes that economics combines the controversy of addressing everyday issues with the general inaccessibility of chemistry.  This conflict may make some people resist the conclusions of economists, ie. strong prior + incomprehensible evidence = small amount of updating.  The comment continues:

However, the inaccessibility of economics does not merely arise through inadvertence. As many jokes attest, economists are not merely unsentimental, they are ANTI-sentimental. An economist will often revel in the opportunity to rub people’s noses in the conclusion that their pre-conceptions are fluffy-headed poppycock. To many people (including some economists, I fear), economics appears to be less a social science than a religion, revealing to a chosen few the mighty counter-intuitive truths by which to pass judgment on a sinful world.

If I’m an accomplished scholar in another discipline, to what extent am I open and receptive to this kind of intellectual upbraiding?

Another commenter notes

My guess is at least part of this effect is reciprocity. Economists are famously bad when it comes to citing relevant research from the other social sciences. And I am not even talking about humanistic research: experiments from social psychology and statistical work from sociology are often ignored when economists do related (or even nearly identical) work. There is a widely held perception in the other social sciences that economists are, at best, disinterested in having interdisciplinary conversations–or, at worst, regularly tolerate cross-disciplinary plagiarism. My guess is that a culture of arrogance is the ultimate cause.

These are legitimate concerns, and there are reasonable rejoinders, but I think there is a more compelling explanation than “economists are a$$holes.”

When we develop a theory, we throw away some of the details about the world in order to make the theory simple enough to understand.  This abstraction is an art: there are no hard rules and too few guidelines on how to make useful theories.

Suppose that there is only one choice: how much detail to discard.  One can (grossly over-)generalize by saying that economists throw away more detail, ignoring things like irrational behavior, while other social scientists throw away less detail.  By simplifying more, the economists gain the ability to apply very technical tools to generate results that are very reliable given the assumptions they made.  By simplifying less, other social scientists are left with questions that more closely resemble reality but are harder to analyze.

Internal validity versus external validity.  Tractability versus verisimilitude.  We’ve heard this song and dance before.  The implied question is something like Heisenberg’s Uncertainty Principle:

Hypothesis:  (time to analyze) * (problems with applicability) >= (some constant)

Does this hold?  Actually, I am optimistic that it doesn’t.  Regardless, we all face choices when trying tell a story that explains something.

You’re Asking the Wrong Question, Fortunately

Today I got up, finishing a decision I started last night about how much to sleep before today.  I will choose my attire to fit the weather and strike the right tone in the classes I will teach. I will go to work and spend the day at work making optimal decisions about how to allocate my time and effort considering my immediate goals, teaching effectively and  preparing for an experiment, and longer term goals like getting along with my peers and building my tenure packet.  I will come home along a route that balances safety, convenience, fuel economy, and curiosity.  I will talk with my wife, play with my daughter, read to my son, all with an eye toward building both their individual lives and my relationships with them.  I may make a few allocation decisions about improving our house or saving for retirement.  I will decide whether to work out tomorrow morning, then  begin the decision about how much to sleep before tomorrow.

I don’t know if I’ll make the right decisions. I cannot know, even afterward, whether I did.  It turns out that it would take a very long time to check.  Constantinos Daskalakis has shown that it would take a computer as complex as the entire universe longer than the lifetime of the universe to solve for Nash equilibria of many common games, like how to invest for retirement. There all I am trying to do is optimize money.  When I read to my son I am doing something harder:  I am trying to influence his education, his confidence, his happiness, and my relationship with him, all without being so bored that I fall asleep.  Most of the decisions we make all day are too hard to solve.

Of course, I can simplify each of these decisions to make them tractable. Any simplifying assumptions I make are wrong, and therefore I am answering the wrong question.  I can make specific, certainly wrong assumptions, I can approximate a bunch of decision characteristics by combining them into one or more stochastic elements in a simpler game.  Either way, I am limiting my rationality, or really acknowledging that my rationality is bounded. When I make one of my decisions I am not answering the question before me but rather a different, simpler, but wrong question.

You might object.  “Once we give up rationality, we give up prediction.  There are no limits to how we can be irrational.”  How does the joke go?  “The difference between genius and stupidity is that genius has its limits.”  There is no limit on the answers we can get if we allow our models of human behavior to admit irrational behavior.  If you allow for heuristics, anything could happen, so we know nothing!

True, anything could happen, but we can still note that some things happen more than other things.  Data can guide us.  We can examine which ways of simplifying are likely and which are unlikely.  Assigning likelihood statements to different behaviors is the statistician’s strategy.  Some steps along this route are taken by quantal response theory. However, there is still much work to be done.

Acknowledging that when we model we always lose essential details, George Box said “All models are false but some models are useful.”  At some point we all have to decide to answer the wrong question, whether in the economy or just in deciding when to go to bed.  Otherwise we wouldn’t be able to get up in the morning.