Transcript
The Tyrannyof Methodologiesin InformationSystems Research 1 R. Brent Gallupe Queen’s University 1 This paper is based on a Distinguished Speaker Address for theInformation Systems Division at the Administrative SciencesAssociation of Canada Conference Abstract A preoccupation with research methodologies isnegatively affecting the quality of information systems(IS) research. Current IS research seems moreconcerned with “how” the research is conducted rather than “what” research is conducted and “why”.This “tyranny of methodologies in IS research” ishaving a number of effects on IS research including influencing the problems that are chosen for study,affecting the papers that are selected for publication,and generally influencing the direction of the entirefield. This paper explores this phenomenon. The paper describes the context and outlines thedimensions of the problem. It compares the“methodologies problem” in the IS field with the use of methodologies in other management disciplines.Finally, it proposes a number of approaches to helpthe IS discipline take a more balanced perspective interms of the research methodologies it uses. ACM Categories: H.0, K.0 Keywords: Information Systems Research,Research Methodologies Introduction Over a number of years, I have become increasinglyconcerned about what I consider to be a growingproblem with information systems (IS) research. Ibelieve that current IS research is more concernedwith “how” the research is conducted rather than“what” research is conducted and “why”. I call thisproblem “The Tyranny of Methodologies in ISResearch.” The basic premise of this paper is thatresearch in the IS field is increasingly being driven bymethodologies rather than importantmanagerial/organizational problems. In my view, thisis weakening the field and subjecting our research toincreasing criticism from both inside and outside thefield.I would like to start with a story. All names andlocations are fictitious but the situations are real. Owen Reid was an MIS PhD student at a prestigious, mid-western U.S. University a few years back. The University had one of the most widely recognized IS research programs in theworld. He was excited at the prospect of conducting research that would extend our knowledge about the use of informationtechnology and systems in organizations. He had come into the doctoral program with a number of ideas about what he would like to study. He had worked in IS shops and as an IT consultant for anumber of years. As his IS research programbegan, he soon realized that at this university, The DATA BASE for Advances in Information Systems20Volume 38, Number 3, August 2007 there were “acceptable ways of doing IS research” and there were “other ways of doing IS research.” It became clear to him that most of theIS faculty believed in one research methodology for IS research and that they were very skilled inthis methodology. Indeed most of the researchcoming out of this program used that methodology. He had not realized this beforecoming into the program, but as he becameimmersed in the research culture of this program,he knew that he would have to pick a research problem that fit the dominant IS methodology at the school. This is not what he had expected.Other problems that would definitely be better researched using other methodologies were moreimportant to him. He had to make a decisionwhether to stay in the program and complete a“less risky” thesis on a problem he wasn’t that interested in but was “methodology friendly,” or switch schools and programs, being very careful to find out about the “acceptable researchmethodologies” at prospective schools before hecommitted to anything. This story illustrates a number of things, but the mostimportant is that methodologies can and do have amajor influence and effect on the research that isconducted in IS. When this influence is too strong, Ibelieve it can have a negative effect on thedevelopment of a field of knowledge, particularly anemerging field such as Information Systems.The rest of the paper proceeds as follows. First, I willdefine some terms and concepts that will help framethe context of the problem. Next, I will describe theproblem as I see it, by exploring its many dimensions.Finally, I will suggest some approaches to solve to theproblem. Terms and Definitions Tyranny: When I use the word “tyranny”, I am usingthe word for effect. The Merriam-Webster Dictionarystates that the word “tyranny” comes from the 14 th Century Latin “tyrannus” or “tyrant”(Merriam-Webster,2005). Tyranny is defined as “oppressive power” or “arigorous condition by some outside agency or force”.Clearly, the notion of tyranny has a negativeconnotation. The way that I am using the word in thispaper is as “oppressive power” or “unwarranteddomination.” It is my perception that methodologiesare exerting unwarranted domination over IS researchand it is my belief that we must be aware that this ishappening and act accordingly.I find it interesting that in recent years, the word“tyranny” seems to be experiencing greater use,particularly in business and management. On whatmight be considered the “negative side”, I found titlessuch as “The Tyranny of the Bottom-Line” (Estes,1996) and “The Tyranny of Change” (McKendall,1993). But I also found titles such as “In Praise of Corporate Tyranny” (Pogue, 2000) and “BenevolentTyranny” (Bloomer, 2000) which implies a slight“positive” tone to the term.In summary, I have chosen the word “tyranny” withsome care. It reflects the strength of my view thatcurrently, research methodologies have greater influence than they should have in terms of what getsresearched and what gets published in IS. It alsoreflects a slight ambiguity about its use.Methodologies: There are two accepted meanings of the term “methodology.” The American HeritageDictionary defines “methodologies” (using the plural)as either “bodies of methods, practices, procedures,and rules used by those who work in a discipline or engage in an inquiry”, or “the studies or theoreticalanalyses of working methods” (American HeritageDictionary, 2000). However, Checkland (1981)defines methodologies as “collections of problem-solving methods governed by a set of principles and acommon philosophy for solving targeted problems”. Inthis paper, I adopt Checkland’s view of methodologiesand think of them as groups of methods sharing acommon philosophy and set of principles.In the IS field, one of its characteristics is itsassociation with its reference disciplines. From thesedisciplines have come many research methods thatcan be grouped into categories of methodologies. For example, quantitative methodologies mean a varietyof methodologies that are primarily “numbers” based.For each quantitative methodology, a number of methods are possible. So methodologies in this caseare simply referring to the groups of methods we useto study IS phenomena.IS Research: Finally, what is “IS Research?” This isan interesting question that has intrigued scholars inmany disciplines (including IS) for over 25 years.Some say it is “applied computer science”. Others sayit is “technically-oriented applied psychology.” Using amarketing term, what is the “IS research” brand?What is IS research known for? How do we know ISresearch when we see it? After 25 to 30 years, we stilldo not have good answers to these questions. MostIS scholars would probably agree with a definition thatsounds something like “the study of the planning,design, development, use and management of information systems in organizations.” This is a “big”definition. I think we may have our referencedisciplines to thank for that! Yet as we all know,particularly PhD students studying for their comprehensive exams, there is a lot of IS research. Itseems to many scholars that IS research is “all over The DATA BASE for Advances in Information Systems21Volume 38, Number 3, August 2007 the map.” Again the question, what is IS research? Or more appropriately, what is not IS research? I thinktoday, with the ubiquity of computers in organizations,you would be hard pressed to find a research journalin any management field that didn’t publish someresearch paper on the use of computers in their field.However, irrespective of all this, let’s say that weknow what IS research is, wherever it is found, andthat we are focusing on the methodologies used tostudy the phenomena in this area.To summarize and to help clarify the scope of thispaper, I am talking about IS research, not research inother fields. I am talking about methodologies, thatinclude the methods and approaches we use to studyIS phenomena. I am arguing that thesemethodologies impose a tyranny that may be doingharm to the field. The Problem Let me now describe the problem in detail. Let meprovide evidence to support my position and describewhat I see as the dimensions of the problem. First, Iwould like to tell another story that illustrates theproblem. Amanda Sayers had wanted to be ananthropologist. The work of Margaret Mead had inspired her to complete her undergraduatedegree and her PhD degree in anthropology at two of the leading universities in her field in theworld. Unfortunately for her, there were very few jobs for anthropologists. After a year or two of trying to find a research/faculty position in her field, she gave up and took a job as a technical writer at a large computer manufacturer. Being smart and ambitious, she quickly rose in theorganization. She also realized that many of theapproaches and methods she had been taught asan anthropologist could be applied to the study of information technology in modern organizations.She applied for a junior faculty position ininformation systems at a small university in Australia and based on her IS experience and her PhD she was given the position. She immediately set out to apply her anthropology training to IS research problems. She published her first two papers in top IS journals. These papers described ethnographic approaches to studying IS problems. Almost immediately she became the IS expert in the use of these methods. Researchersshe had never heard of asked her for adviceabout how to use them. She became anassociate editor on a prestigious journal in IS just to make decisions on papers using thosetechniques. She became increasing uncomfortable with her role as a “methodology gatekeeper.” Why was she the one to do this? How did she gain so much power over what got published using those methods? Were her standards too high or too low? She knew that some people, behind her back, were already calling her “power hungry” and why should she bethe one making decisions about what gets published and what doesn’t? This story represents another aspect of the problemas I see it. All research methodologies have their “disciples,” that is, their human proponents. Thesepeople can gain substantial influence and power over a field of research, whether they want to or not,particularly a young and growing field such as IS. Thismay be distorting the research that is being publishedboth in quantity and quality. Amanda is in anuncomfortable position. She is beginning to be viewedas a “research tyrant”, something that she does notwant to be.The problem is that the IS field’s currentpreoccupation with methodologies is taking awayfrom its contribution to knowledge on many fronts.This is not to say that IS researchers should not beconcerned about methodologies. They should. It is just that when the “how” (as in methodologies)becomes more important than the “what” or the “why”,then we have something we need to think about. Asnoted earlier, this is not a new problem for IS. ISscholars since the early days of the field have beenconcerned about research methodologies and their relative importance. It seems that about every tenyears or so, as technology changes and as newproblems emerge, the issues related to methodologyraise their heads again and demand renewedattention.What evidence do I have that methodologies aredominating the practice of IS research? The evidenceis not in numbers of papers published using anyparticular methodology. That would be too easy. Theevidence is found in what IS researchers are sayingand writing about research in our field.A Senior Editor of MIS Quarterly recently made thefollowing statement at an IS Conference. He claimedthat the “battle was over” in terms of researchmethodologies in IS. What he was referring to werethe trials and tribulations of trying to get qualitative,methodology-based research published in top tier IS journals. His claim was that qualitative methodologieshad finally won out. I believe this claim is one piece of evidence that points to the preoccupation withmethodologies that our field currently holds. The DATA BASE for Advances in Information Systems22Volume 38, Number 3, August 2007 An editor of Accounting and Organizations, in arecent MIS Quarterly volume, lamented about “themethodological bar” (being) too high in IS (Robey,2001). His concern is that we appear to be rejectingpromising ideas for the sake of methodology.On a recent trip to the University of Houston to talk ISresearch with two or three top researchers in the ISfield, the discussion moved on to methodologies in ISresearch and again concern was raised about“methodology police” and what appears to be thesubjective nature of methodological standards.At a recent Hawaii International Conference onSystems Sciences, one of the leading authors andresearchers in use of communications technologies,in a session on GSS, complained that not muchlaboratory experimentation had been published lately.It seemed to this person that unless “you were doinga case study or ethnographic research, you couldn’tget it published”. Everyone laughed and nodded.At a recent International Conference on InformationSystems, at a meeting of the Editorial Board of Communications of AIS, one of the senior editorsstated he didn’t want CAIS to be a “methodology-based” journal like some other journals in IS. Hewanted CAIS to be a forum for new ideas.In recent discussions with IS PhD students inCanada, US, and New Zealand, I found them asking“what is more important, the methodology or theproblem?” “What is chosen first, the methodology or the problem?” “Does methodology drive the choice of theory?I could continue to build my case aboutmethodologies in IS research but I think you get thepicture. When you start to look at an issue through acertain lens, all of a sudden you start to see thingsyou hadn’t noticed before. The evidence is in whatpeople are saying at meetings and conferences andin small group settings. The evidence is in whatpeople are writing about and where they arepublishing what they write.So this is the way I am seeing IS research right now,“Problem follows Method”, or “Methodology is King”.Let me try and show you some of the dimensions of the problem as I see it. The Dimensions of the Problem The tyranny of methodologies has a number of dimensions. Here are the ones I see as mostimportant.Different methodologies: First, IS is characterized bymany different methodologies. Just about all thesehave been obtained from our reference disciplines.This means that many methodologies and their proponents are vying for limited publication space.This is putting stress on both researchers and the journals themselves. For the researchers, it maymean “re-tooling” in different methodologies. For journals, it means having people on editorial boardsthat are knowledgeable in the use of thosemethodologies.Varying levels of appreciation for differentmethodologies: Second, as researchers we are all aproduct of our backgrounds and experience. We havebeen trained in different ways. We have had varyingexposure to the different research methodologies thatmake up our research “tool set”. As such we havedifferent levels of appreciation for differentmethodologies. For example, I think I have a rather low level of appreciation for hermeneutics, but arather high level of appreciation for experimentation.Therefore, I am more likely to not appreciate a studyusing hermeneutics. We all have our biases andthese seem to come to the fore with methodologies.The implication here is that we “push” themethodologies we know and “discount” the ones wedon’t.Different standards with different methodologies: Third, another interesting phenomenon is the so-called “different standards with differentmethodologies.” Some researchers believe that themethodologies they use have higher or “harsher”standards than methodologies they don’t use. Theyfeel it is more difficult to publish their work becausethe standards are higher. We have all looked at apaper published in one of our top journals using amethodology we know relatively little about andwonder how that piece of work ever got published,when my rigorously designed and implemented studywith a well established methodology and clear standards, got rejected. The perceived differentstandards for different methodologies means thatproponents of a methodology need to defend their chosen methodology from criticisms from others,others who may not appreciate the standards thathave been developed for their methodology.Different philosophical bases: Fourth, and this is adifficult one, the methodologies may come fromdifferent philosophical bases. For example,experimentation comes from a positivist, scientifictradition, while ethnography comes from aninterpretivist, sociological tradition. The basicassumptions on which some methodologies are builtare fundamentally different. What this means is that“methodology can turn into ideology” and that there isonly “one right way” to think about, and to conductresearch. The DATA BASE for Advances in Information Systems23Volume 38, Number 3, August 2007
