New methods are key to developing new hypotheses and testing old ones. It would be difficult to imagine social sciences today without regression models, or social network analysis, or online ethnographies. Each has been developed and used to expand what we think that we know about the social world.

Black and white picture of a Victor snap trap
Image by pepperberryfarm via Flickr, Creative Commons BY-NC-ND 2.0

But social scientists, especially junior researchers, too often fail to justify their new methods. They demonstrate their creativity and document the often considerable work they put into developing their new approach. But scholars should take care that they're not just building a better mousetrap.

Ralph Waldo Emerson purportedly coined the phrase, "Build a better mousetrap, and the world will beat a path to your door." Americans took the advice, whether actually from Emerson or not, to heart. Jack Hope documented more than 4,400 patents for mousetraps have been granted by the US Patent Office since 1838. It became a singular challenge to build the best machine to eradicate Mus musculus from the lives of Homo sapiens.

For all of Americans' industriousness, however, the mousetrap has not really changed in 120 years. The simple snap trap has served people well. It's economical to manufacture and straightforward to operate. Entice the unsuspecting mouse with some cheese or peanut butter on the bait pedal, and then wait. After the trap has sprung its mortal blow, throw out the mouse corpse and trap and all. Repeat with new traps, which cost less than a dollar a piece, until the whole lot are gone and the bait rests untouched.

Spring traps provide an inelegant solution, but one that requires little effort to work. The same could be said of many of our standard methods: they're inelegant, but they work. For that reason, we need to be careful about methods that purport to fix the inelegance of current methods. While we should always be striving for precision in our scientific endeavors, new methods introduce costs that must be weighed against potential benefits. For example, new methods can make comparisons to existing literature difficult. The incomparability lowers reproducibility and can slow the progress of science in a field. New methods can also introduce substantial learning curves, and are only valuable if scientists who invest in mastering the method lead to substantial new insights into the problems of their fields.

With the exception of a very small number of general purpose methods, the method is not the important contribution that we make. We should measure the value of methods based on the substantive problems we can answer using new or improving existing methods. Therefore, I would encourage authors of methods papers---especially graduate students and junior faculty---to answer this question honestly: does this new method either help us ask new questions in the field or lead us to new, and better answers to old questions?

Some of the problem comes from our training. We too often teach graduate students to look for violations of our methods as technical problems rather than substantive ones. It's not unreasonable for students to deduce that technical solutions on their own represent progress in the field. In our methods classes, it would help to focus on how authors link methodological problems to substantive interpretations. We should also teach the trade-offs inherent to changing methods would improve students' intuition about sociological contributions.

Our methods, like spring traps, aren't perfect. But many solutions, like the 4,000 new mousetrap patents, don't do much better (and might make things worse). Before proposing a new method, consider what substantive knowledge we gain by its use. Explain to readers how the method makes our knowledge more precise, and how that precision helps us ask or answer new questions. If you do that, I promise the world will beat a path to your door.1


  1. Okay maybe not, but your method will get picked up and used by others in the field. 

Pingbacks

Pingbacks are closed.

Comments

Comments are closed.