GitHub is superior to Subversion in notable ways, but that’s not our topic here. GitHub does make it easy to read source code directly from the site as plain text. Here’s an example of an address for a bit of code I use almost daily to give me a clean R session.

  https://raw.github.com/shaptonstahl/R/master/Decruft/Decruft.R

Anyone see the problem? Two things: (1) The URL for the plain text version of code hosted on GitHub is reached via a secure connection, and (2) R can’t source via https without the use of an external library. I’m a big fan of R’s external libraries, but it doesn’t fit the purpose of the code. This code usually sits at the top of just about every .R file I write:

  ##  Start fresh!
  source("http://address.of/Decruft.R")

Isn’t that pretty? Short, sweet, easy to remember. This is how I used to do business. Unfortunately, this is the most concise way that I could find for doing it with GitHub:

  ### Fugly code
  if( !any("devtools" == installed.packages()[,"Package"] ) install.packages("devtools")
  library(devtools)
  source("https://github.com/crikey/thats/as/long/as/the/code/Im/sourcing.R")

I checked the Google and such but nobody seemed to be asking precisely what I was: how can I read code stored on GitHub in plain text using http, not https?  We will not be discussing how long it took me to come up with a satisfactory solution. Let’s just say it took long enough that I really don’t want anyone else to have to go through it.

Here’s my solution:

1. Have a Web server running PHP that allows you to create and use .htaccess files.
2. Choose a URL for the stem of where your code will appear to be.
3. Use .htaccess to point 404 Not Found errors to a custom error page.
4. The PHP-based error page uses https to get the live file from GitHub and feeds it to the person requesting it.

It could be worse. I decided that I would request pages from (nonexistent) subfolders of http://www.haptonstahl.org/R/ in order to read code stored under https://raw.github.com/shaptonstahl/R/. So I put two little files in the document root of my site. The first is named .htaccess and contains this:

  ErrorDocument 404 /404.php

The other file is the 404.php file mentioned in .htaccess. You can download the PHP file here. This is a copy of the actual one I am using. Now to get a clean R session I just type the following:

  source("http://www.haptonstahl.org/R/Decruft/Decruft.R")

Easy peasy. Perhaps the best part is that I’m done. I never have to update or modify this if I want to source other public code in that GitHub repository. For example, without changes I can source

  http://www.haptonstahl.org/R/RoundBoundsNicely/RoundBoundsNicely.R

to get

  https://raw.github.com/shaptonstahl/R/master/RoundBoundsNicely/RoundBoundsNicely.R

Lesson to take home:

1. Putting code you reuse up on teh webz makes it easy for you to use it over and over instead of writing it over and over.
2. GitHub rocks, now even more since I can source my R code from the live GitHub versions.
3. Safety first, kids!

Those in society’s minority who did well in math courses are “shocked” at the suggestion that we change the typical math curriculum.  The teaching may be “dismal” but algebra is a “foundation stone” in developing critical thinking skills.  “It teaches one how to think.”  It’s a little amusing but mostly disheartening to see folks who claim to support more challenging math standards fall back on strawman arguments, condescension, sarcasm and, my favorite, math errors in their arguments.

Those in society’s majority who did poorly in math tended to respond with relief at the suggestion of dropping algebra, although there are a few PMSD (post-mathematics stress disorder) victims whose career paths were altered by failing math and who still carry the associated baggage and resentment.

Let’s set aside the hysterics (“We are breeding a nation of morons“) and give both sides of this debate a fair shake, shall we?

Arguments in Favor of Eliminating Algebra as Courses Required for All

We definitely teach too much algebra and do so mindlessly, without considering whether it’s useful. As a teacher of math courses from arithmetic through calculus at a community college, I fought the losing fight to remove useless topics from the curriculum. For example, Cramer’s rule is a relic from the days before computers and is as practical as a slide rule, but trying to remove it from the required topic list elicited resistance and deep resentment from many of my fellow faculty.  Hacker’s suggestion that we reconsider the requirement for so much symbolic manipulation is sensible.

While teaching algebra I tried to limit my syllabi to (1) the topics that would be used in later courses, (2) topics that might be useful outside the classroom, and (3) some examples of true beauty.  I emphasized (1) but snuck in some of (2) and (3).  Still, I did teach some material that would only be useful in later math courses and never in any kind of applied setting.  We could easily cut more topics by curtailing the length of the required math sequence, at least for the math subjects taught.

Algebra is not the only way to teach disciplined thinking. One can teach precisely the same thinking skills while removing the abstraction that makes math seem useless and difficult to many students.  One idea (not mine): we could integrate math beyond middle school into the science curriculum and use applications as motivation.  Then there’s no need to learn how to apply math to story problems; the stories are the original problems.  This also prevents folks from running around with “hammers” looking for “nails”.  Unfortunately I am not sure how to get there from here; teachers of other subjects would have to cover the needed material and that would require revamping the way teachers are taught.

We don’t need to learn algebra to develop our intuition about rates of change, interest rates, probability, statistics, and other topics that typically follow algebra.  There are ways (videos, interactive widgets, simulation using simple programming) to develop the intuition of calculus — often the only calculus needed in a job like medicine — without approaching it the rigorous, analytic, symbol-pushing way we typically do.  Even for those who eventually will need algebra, we can teach more advanced symbol manipulation skills later as needed.

Nobody (well, almost nobody) is saying that learning algebra has no value whatsoever.  However, as long as we have limited resources the pertinent question is, does algebra give us more benefit than spending that time elsewhere?  I suggest that programming, statistics, and finance are better uses of most students’ time.  Programming is how the nearly countless computers in our lives work; a basic understanding of how they do their magic would be invaluable.  Statistics are essential for making sense out of the sea of information around us.  Finance is challenging and vital for artists and engineers alike as long as they want to buy a home or save for retirement.  If we remove the symbol-pushing exercises of algebra and replace that class time with simple programming, statistics, and finance, we’ll gain more than we’ll lose.

Arguments in Favor of Keeping Algebra a Courses Required for All (with occasional rejoinders)

Let’s be more specific about “algebra”.  A first course (“Algebra I”) often includes basic linear algebra (lines, graphing them, solving systems of linear equations, and matrices) plus evaluating polynomial functions, graphing quadratic functions, and solving single quadratic equations.  A second course (“Algebra II”) builds on this with lots of factoring polynomials, exponential and logarithmic functions, quadratic inequalities and other algebraic prep for calculus.

An Algebra I course like this is incredibly useful.  The concepts generalize to every science, from physics to political science.  With this foundation a student can learn the intuition behind calculus, statistics, and other tools that are useful to have seen but usually not useful to retain. I strongly recommend keeping Algebra I part of the core curriculum.  With moderate resources, this material can be covered in middle school for most if not all students.

Algebra II is where common responses to “Why am I learning this?” jump the tracks.

• “It builds your brain like exercising build muscle.”  I used this one regularly; it’s true, but only a half truth.  Programming, statistics and finance can do the same with the added bonus of being unquestionably practical.
• “It helps students understand where more advanced math comes from.”  Playing with simulations is even better for most students in understanding why more advanced math techniques work the way they do.
• “It teaches structured thinking.”  Programming is even better for that and it is easier for students to see what why structure matters.  Mess up the structure and programs do odd things.

Algebra II is full of topics that you don’t need in order to understand the intuition of common useful advanced ideas; you need Algebra II when you will try to master more advanced ideas.  I strongly recommend making Algebra II something that fewer students take.

Known Unknown

My recommendation to remove Algebra II from the universal curriculum is contingent on at least one assumption: Students who need more algebra will have sufficient time to learn it later.  To see if this is true we would need to take average to bright students interested in technical fields and wait until college to teach them algebra.  This is not common. Interestingly, I have some experience that is close to this:  teaching math to political science graduate students.  While there are a few notable exceptions, most political science undergraduates take very little math.  Graduate students work very hard to learn the math necessary for advanced statistics and game theory, and generally they succeed.  This evidence is circumstantial but shifts the burden of proof onto those who might argue college is too late to learn algebra.

Bottom Line: Question Mathematical Authority

Hacker thoughtfully asked a good question: are we teaching what we should be teaching?  One cannot decry the educational establishment as ossified but resist any attempt to change for the better.  Hacker may be throwing out the baby.  My ideas might not be the best.  What are your ideas?  Let’s discuss it.

* This blogger is an award-winning teacher of math, statistics, and programming at the high-school through graduate levels, holds graduate degrees in math and political science, and works in the defense industry as a data scientist.

Patil takes up the necessary and generally thankless task of writing a “big-think piece”.  It’s necessary because, with all the recent talk about Data Scientists, it would be easy for some to see “Data Science” as a recent entry in the long history of business fads.  Business fads tend to indulge in one of two sins: oversimplifying, or being so general that anything counts.  Both are sins to which Data Science advocates are susceptible.  An advocate of Data Science might oversimplify by giving a recipe or IT shopping list for doing Data Science; Patil avoids this triteness by emphasizing the wide diversity of the good teams he’s built or worked with.  Alternatively, one could sin by making anything count as Data Science.  Commenter “Verbose” anticipates this problem with his erudite critique of BDST: “Same technical data analysis, new bullshit name“.  Patil avoids this and provides a blueprint for others to do the same.

When discussing the etymology of “Data Scientist” Patil writes that “Research Scientist” would not be as appropriate a term for this profession:

“Research scientist” was a reasonable job title used by companies like Sun, HP, Xerox, Yahoo, and IBM. However, we felt that most research scientists worked on projects that were futuristic and abstract, and the work was done in labs that were isolated from the product development teams. It might take years for lab research to affect key products, if it ever did. Instead, the focus of our teams was to work on data applications that would have an immediate and massive impact on the business. (emphasis added) The term that seemed to fit best was data scientist: those who use both data and science to create something new.

Elsewhere Patil provides a solid definition of Data Scientist, but this paragraph encapsulates the concept just as well:  A Data Scientist uses data and science to have an immediate and massive impact on the business.  Just moving data around?  Not data science.  Have an impact in the vague future?  Not data science.  Improving entirely on the margins?  Not (all of) data science.  Holding the Data Scientist’s feet to the fire — asking “How does this immediately and massively impact our business?” — provides accountability and hence focus for the team.

Writing a “big-think piece” is also a thankless task: the breadth of the topic means that it’s easy for critics to find something to criticize as being presented too simply.  This ignores the contribution of providing a view of the new discipline from space, showing all of it as a piece, and showing (if briefly) how the disparate parts fit together.  I was glad for a look at a map for this road I’m traveling.

Overall BDST is short, shorter than I would have liked.  (I’m glad that Patil is sharing more of his experience through other venues.)  The advice Patil gives about Data Science, forging teams, and hiring Data Scientists seems both specific and useful; I’ll post again as I have occasion to use his advice.

Recommendation: “Building Data Science Teams” is short, but with enough good ideas as to be required for anyone in business intelligence, internal data analysis, or applied computational modeling and prediction.

In theory, a panel takes place in a room where 3-300 (median = 10) people watch three to five papers get presented by their authors.  Then a discussant, who reads the papers in advance, comments on the papers both to draw connections among them and to stimulate conversation among the attendees.  Then the audience asks questions and offers feedback to the authors.  The whole panel takes about 1 3/4 hours.

Although panels comprise most of the scheduled events at a conference, they are not the best reason for scientists to attend conferences and they are far from the most rewarding part of a conference.  Panels are often poorly attended.  The papers in a panel often have very little to do with each other.  The discussant may not receive the papers until days or moments before the panel, if at all, and even so the comments may focus more on typography than on big ideas.

Panels are a party game. They are an excuse to get smart people, who are interested in similar things, together in a room talking. Put a bunch of clever folks together and strange, wonderful, unpredictable things happen.  A conference is mass planned serendipity.

The largest conference benefits to my research have happened when I have not been at panels: between panels, skipping panels, into the evening and the night. It’s the networking, but not “networking” in the Machiavelian, sales-person sense.  It’s the comment on my paper that someone was a little too shy to offer in front of everyone, the comment that helps me recast the paper so it will place higher.  It’s running into the same person at three panels and finally discovering we would love to work together on some research.  It’s the dinner outing that leads to an invited talk or an interview.  It’s the shared coffee followed up with a Facebook friending that leads to a new real friendship.

All of this is made possible by panels, but it’s not the direct result of the panels.  So when someone tells me they didn’t go to a lot of panels, I understand that they probably got a lot of professional good out of the conference.

Also, I had a lot of fun at the Space Needle.

It is regular sport for Bayesians to criticize frequentist confidence intervals as unintuitive, usually misinterpreted, and based on what are usually unjustifiable assumptions.  There are good reasons for this:  they are unintuitive, usually misinterpreted, and based on what are usually unjustifiable assumptions.

Confidence intervals take into account the uncertainty we have in trying to describe a population given that we only observe a random sample.  Maybe our sample is representative of the population, maybe not.  The smaller the sample, the more likely it is that, through random variation, we got a sample that suggests a relationship (causes us to reject the null hypothesis) when really there is no relationship (the null is true.)  If we take 100 random samples of the same size and for each construct a (different) confidence interval, how many will contain the true value of the parameter?  We expect that 95 of those constructed intervals will contain the true value.

We don’t have 100 samples of the same size, so this is not the question we want to answer.  Here’s what we want to know:  What is the probability that the true value is in the interval?

There are a variety of very good, sound reasons why the frequentist approach does not make sense. I understand the difference between believing that the true parameter is “fixed and known to God” (per the frequentist assumption) and a random variable (the Bayesian assumption.)  I agree completely that a confidence interval answers the wrong question.

It doesn’t matter.

For large samples and given the regularity conditions of maximum likelihood estimators, the marginal posterior distribution for a single parameter is approximately normal with a mean equal to the MLE and standard deviation equal to the standard error.  Under these conditions, maximum likelihood and Bayesian estimators give you the same inferences.

Bayesian researcher perspective: Suppose I want a credible interval for a parameter where I have a lot of data and the model is well-behaved but I don’t have convenient code for drawing posteriors.  I could estimate the MLE and use the resulting confidence interval as an approximate credible interval. My friend and colleague Jeff Gill calls this the “lazy Bayesian” approach.  Lazy, efficient, tomato, tomahto.

Bayesian consumer perspective: Suppose you are reading an article where there is plenty of data and a well-behaved model, but the author provides frequentist confidence intervals.  You can treat them as approximate credible intervals.  Easy peasy.

Pragmatist perspective: As long as the conditions are met, you can go on “misinterpreting” confidence intervals.

This “lazy Bayesian” approach is not limited to making simple inferences about single parameters.  King, Tomz, and Wittenberg explain how to generate draws from the approximate posterior (as opposed to approximate draws from the actual posterior via MCMC) and make any inference a Bayesian can using the output from maximum likelihood estimation.

As a Bayesian, I advocate strongly for increasing the use of Bayesian methods.  However, we should be careful to avoid overselling the advantages.

Bayesian estimation addresses the “small $latex n$ problem”

This is right and wrong.

Maximum likelihood estimators (MLEs) leverage large $latex n$ in two ways.

1. Making inferences about parameters.
2. Checking model fit.

The goal of #1 is to answer questions like “Is $latex \beta>0$?” or “What is the shortest interval that has a 95% change of containing $latex \beta$?”  MLEs let us take a stab at answering* this. A large number of observations means we can apply the Central Limit Theorem, which means we can use the standard errors to test simple hypotheses and build confidence intervals.  The goal of #2 is to justify our choice of model after the fact.  Presumably we had a good, substantively-informed theory to justify our choice of model, but it’s nice to be able to say afterward “See? The models fits well, so we weren’t crazy to choose this model.”

How many is a “small” number of observations?  It’s difficult to say.  Long (1997) gives some guidance about this, suggesting that 100 is a minimal number for an MLE, and that one needs at least 10 observations per parameter in the model.  However, we never really know if this is enough to be able to trust our inferences and fit.

What happens if we have too few observations?  Both #1 and #2 become unreliable.  We have too little data to assume that the Central Limit Theorem has “kicked in,” so our point estimates have more uncertainty, which means our standard errors are too small**, and therefore any inferences are unreliable.  Our tests for model fit are similarly starved for information, so any post-hoc model justification will be difficult.

What does Bayes fix?  Bayes estimators are finite-data estimators:  more data gives more accurate estimates, but the measures of uncertainty of those estimates are reliable regardless of the amount of data.  This means that inferences are reliable regardless of the size of the data set.  Want to know if $latex \beta>0$ but our $\latex n$ is small?  No problem for a Bayesian.

What about model fit?  Bayesians have the same tools as frequentists for checking model fit and they also have the numerical and graphical analysis of posterior predictive distributions.  As with inferences about parameters, more data is more information and thus is preferable.  However, any tests of model fit are still reliable.  As a purely practical matter, if we have a very small data set we probably will not be able to conclude anything about model fit from the data.  This puts the burden back on the substantive/theoretical argument for the model form.  If we understand the data-generating process very well, great; if not, then this part of our argument will need extra scrutiny.

Bottom line: Bayesian estimators don’t create more information.  However, they do let us correctly identify how sure we are about the inferences we draw.  That’s a clear improvement.  Other Bayesians aren’t helping by overselling it.

===

* Don’t get saucy with me about how frequentist confidence intervals (CIs) either contain or don’t contain $latex \beta$, implying that it’s meaningless to talk about the probability of $latex \beta$ being in the interval. CIs are better than commonly described — details in my next post.

** In theory our standard errors could be too small, too large, or correct.  We can appropriately account for this additional uncertainty by increasing the size of our standard errors (think “put a confidence interval around our confidence interval”) except we can’t know how much to increase them.  See “reasoning, circular”.

The way to be sure that a statement about politics is true is to make a formal argument: explicitly state the assumptions and use rigorous logic to create a 100% valid argument from assumptions to conclusions.  Euclidian geometry was a popular touchstone of Truth^TM for two thousand years…until Gauss proved that one of Euclid’s assumptions was arbitrary.

Oddly enough, most of the mathematicians I know do not assert that math reveals Truth^TM.  For a set of statements to be true, they must not disagree.  Gödel’s showed us that we cannot be sure that math does not contradict itself.  Accepting this means abandoning the claim that 100% truth can be generated analytically or using any logical system. For many mathematicians (not including probabilists or statisticians) once something is uncertain, all bets are off.

Analytic formal modelers know this, but assert that deduction (assumptions + logic => conclusion)  is better than statistical induction (theory + data => conclusion).  Let’s take a look at this analytically, shall we?  Consider this statement that is true simply as an application of the definition of conditional probability:

$latex \text{Pr}(\text{conclusion is true})=\underbrace{\text{Pr}(\text{conclusion true}|\text{assumptions})}_{\text{chain of reasoning}}\cdot\text{Pr}(\text{assumptions are true})$

Analytic modelers use a deductive chain of reasoning, which has a (conditional) probability of being true that approaches 1, so the probability of the conclusion being true is essentially the same as the the probability that the assumptions are true.  One critique of this approach is that all assumptions, if pushed far enough, are false.  However, that attitude underestimates the usefulness of analytic results.  If the assumptions are close to being true and the chain of reasoning is “well-behaved” (robust to small errors in inputs) then the conclusions will be close to true.

Computational (including agent-based) modelers start with clear, explicit assumptions but use an inductive chain of reasoning which is therefore less likely to be conditionally true.  However, the original assumptions are often much more realistic:

If the assumptions made in a computational model are more likely to be true (or nearly true) and the chain of reasoning is strong, then the conclusions are more likely to be true than those “proven” using an analytic formal model.

Of course, that’s a tall order to fill.  Your mileage may vary.  Whether analytic or computational approaches is more appropriate depends on the specific assumptions being made and the strength of the induction used in the computational approach.  This is a trade off that should be made mindfully, not dogmatically.

Three nights ago I had a dream in which I was being assaulted. One of the two offenders in my dream was someone from my division on the ship, someone I have not seen in over fifteen years. I woke up wondering what that was all about, but I gave it little thought as I went on through my day, my happy, productive day. Everyone has weird bad dreams sometimes. Civilians just have ones with less gray paint.

When I was on the ship I lived in, essentially, a locker room. My 38 roommates and I slept in 13 triple-stacked bunks in a berthing space the size of my current living and family rooms. Before the Navy I hated locker rooms. I wasn’t frail, but I was a geek in high school and no athlete. I hated the inevitable comparisons. In the Navy I hated showering next to a 198-lb functionally retarded man who could bench press 500 lbs and who, because he had joined many years before and outranked me, treated me like a child. I hated the regular joking about sailors being gay, complete with accompanying grabbing of each others’ bodies. Yesterday when changing at the gym I was completely comfortable in the locker room, thinking about the class I was going to teach a little later.

Here’s something weird: I never heard about any crime on the ship more serious than a regular, playground-style fight. I never heard about a sexual assault. I never saw any sign of any actual homosexuals. (My gaydar wasn’t completely oblivious, as I had been around the “family” a lot in my previous college years.) I never saw anyone shoot or get shot in the line of duty or otherwise. I never really faced an enemy. I never had to deal with a fellow sailor who was obviously a “bad guy”, someone who would, in the boring movie version of my life, be seen as any kind of villain.

There are a lot of people I know now who I like, but don’t trust. There were some people I served with whom I really didn’t like, but I trust them, and would still, with my life. And afterward it wouldn’t change how I feel about them.

Three guys I played poker with died in a plane crash. When you deploy 10,000 sailors and soldiers on metal ships with guns, bombs, torpedoes, fuel, and high voltage equipment on indifferent seas for an indefinite length of time, some don’t come back. It doesn’t matter whether it’s “peacetime” or not. They were on a E-2C Hawkeye reconnaissance plane coming in for a landing on the carrier when they got waived off. They started to gain altitude for another pass, but stalled out right after clearing the flight deck. A carrier has four screws. We never found a piece of the plane larger than a surfboard. I don’t remember their names.

My ship was in the Adriatic Sea enforcing the no-fly zone over the former Republic of Yugoslavia. We left to go home, relieved by another AEGIS cruiser who was, like us, fully provisioned and armed with Tomahawk missiles. We didn’t even make it home before we found out that the relieving ship got to fire some of her missiles. Lucky bastards! We had trained and trained — we were ready! — but we never got to fire a shot. Of course, nobody wants to kill people. Even then I knew I was lucky not to have to live with the burden of having killed.

What I didn’t realize is that, in training, in making myself mentally and emotionally ready to pull the trigger, I had already lost something. When we hurt others, we are diminished. Soldiers who are willing to do this, are willing to diminish themselves, do diminish themselves. The sacrifice is admirable, but it is a sacrifice just to prepare to kill. It does change you to prepare for war. Some changes can be more obvious, some are less so.

I am fine. I am healthy, physically and mentally. But I did not serve in combat. Many, many men and women are serving in combat now or have served in combat before. My experience was approximately 1.4 million times easier than their experience. My experience was also at least a thousand times easier than that of my wife and every other military spouse. Preparing for war takes your mind off the dangers involved; if it didn’t few people could follow through. Staying at home waiting for a loved one to return provides no such distraction. Thank you, my precious wife Dawn, for your sacrifice.

These days, I am living the dream, living the good life, the sweet retreat. I think of my father who helped identify targets for bombers during Vietnam, who now is a retired minister. I look at my children, and I hope they will serve others. If called, and am confident they will respond and serve with honor. However, I dearly hope they will never have to carry a weapon. I wish that nobody did, but some people must.

The price of freedom is eternal vigilance.

– Thomas Jefferson

One definition of insanity is doing the same thing over and over and expecting different results.

The quote was misattributed to Einstein, and it’s often misattributed to Ben Franklin, Mark Twain, and others. Various folks take issue with this pop-culture “definition” in different ways.  As a recovering mathematician, I’ve repeated it myself, and it used to make some sense to me.

However, as I’ve tried to move from brittle assumptions (knife-edge mixed equilibria, trembling hand-perfection) in my models to more robust assumptions (quantal response equilibria, non-parametric tests) this truism has rung less true.

Suppose you hit a subject with a treatment.

A physical scientist tends to have objects of study that have simple internal states.  The aggregate systems are complex, but the smallest observable parts of the system are very simple.  The simplicity of the set of states means that I have a good chance of being able to choose a sample from those states that is sufficiently random so that there is little or no correlation between states and my variables of interest.  No correlation means that we get unbiased estimates of the parameters for models connecting my variables of interest.  I might run a physical experiment, say, n times to assure that we have a good sample and to compensate for measurement uncertainty.  I don’t really expect running it another n times to show me anything new; that would be crazy.

A social scientist has objects of study that are vastly more complex. A person has far more possible internal states than an atom, a mole of gas, or a cell.  More germane, we can’t easily be sure of sampling from those internal states randomly, so we constantly face the threat of internal states being correlated with our variables of interest, which leads to bias if not statistically corrected.

Can we just set n higher?  No. If we had a better understanding of the internal states of a person, we might be better able to sample randomly from those states.  At least for now, however, we do not grok the data-generating processes within people as well as we do those in chemistry.  Samples would have to astronomically large to overcome the selection bias in our samples of the internal states of a person.

I’ve heard this “definition” many, many times.  Each time before I kept my mouth shut.  This time I had to speak out.  Keeping silent would have driven me crazy.

If you categorize political ideology into two camps, you are implicitly using a one-dimensional model, a model that puts everyone on a single line from left to right.

If you think there are two basic issues — how much the government should influence the economy and how much the government should influence social issues — then you are using a two-dimensional model, a model that locates each person on a plane.

So here’s a question:  How many dimensions are there in political ideology? This question is a lot like the question “How long is the coast of Britain?” (alt): the answer depends on how you’re going to use the answer.

If you want to explain variance in the US House and Senate, one dimension is plenty in most cases, and two dimensions is sufficient the rest of the time. If you are looking at an entire (two-year) Congress or a longer period, one dimension will do a good job most of the time

If you are trying to explain a single bill — or each of a set of single bills — the story is more complicated. On a given bill, some legislators vote for ideological reasons, some for procedural reasons, some because of interest groups or constituent pressure, some because of bribes, some because they are taking cues from other legislators, and some just make mistakes. When you are interested in distinguishing among legislators based on aggregate voting, few dimensions are needed. When you want to explain the path a bill takes, more dimensions may be required.

This has implications in three kinds of research we conduct based on the analysis of roll call votes:

1. Theorizing about ideology:  When building theories about ideology, we cannot blindly assume that ideology is one-dimensional.  Arguably there are one-dimensional policy areas, legislators, and specific bills, and the one-dimensional case is worth considering, but it is not anywhere near universal.
2. Ideology as an explained variable in empirical work:  How do parties, interest groups, constituent pressure, and other factors affect votes? When exploring factors that affect ideology — using statistical models where ideal points on the “left-hand side” — we should not ignore that there may be multiple dimensions to explain.
3. Ideology as an explanatory variable: How does ideology affect interactions with the president, the courts, and the bureaucracy? Even scholars of institutions other than Congress should remember that when ideology is on the right-hand side of the statistical model we might want to include more than one dimension.

To indulge in the snowclone, dimensionality matters when considering ideology. Make a mindful choice when modeling it.