9 Generating Good Research Questions

Generating Good Research Questions

From broad research topic to research questions

Once you have a broad research idea, you need to use it to generate one or more empirically testable research questions, that is, questions expressed in terms of a single variable or relationship between variables. One way to do this is to look closely at the discussion section in recent research articles on the topic. This is the last major section of the article, in which the researchers summarize their results, interpret them in the context of past research, and suggest directions for future research. These suggestions often take the form of specific research questions, which you can then try to answer with additional research. This can be a good strategy because it is likely that the suggested questions have already been identified as interesting and important by experienced researchers.

But you may also want to generate your own research questions. If you have some specific behavior or a specific broad topic you are interested in (e.g. manager effectiveness), you will often find that prior researchers have already done research on a this topic. Based on this prior research, you may identify variables that interest you (e.g. the effectiveness of a particular management style), and to subsequently think about how that variable is related to other variables. One way to do this is to ask yourself the following series of more general questions and write down all the answers you can think of.

• What are some possible causes of the behavior or characteristic? (independent variables: which other factors might make this management style more or less effective?)
• What are some possible effects of the behavior or characteristic? (dependent variables: what possible consequences can the effectiveness of this management style have? e.g. related to firm performance)
• Are there indirect causes or effects of this behavior or characteristic (mediation: maybe a management style increases employee productivity because it increases  their motivation which, in turn, increases their productivity).
• What types of people might exhibit more or less of the behavior or characteristic? (moderators: is this management style always equally effective for every employee? if not, which employee characteristics causes it to be more/less effective?)
• What types of situations might elicit more or less of the behavior or characteristic? (moderators: is this management style always equally effective in every situation? If not, which situational characteristics cause it to be more/less effective?)

In general, each answer you write down can be conceptualized as a second (or third!) variable, suggesting a question about a relationship. If you were interested in influencer effectiveness for example, it might occur to you that a possible cause of this might be how many followers this influencer has, or how often they upload new content. Or it might occur to you that influencer effectiveness may not just depend on the amount of followers, but also the type of content they produce: do they appear authentic? Does their content feel ‘forced’? This approach should allow you to generate many different empirically testable questions about almost any behavior or characteristic.

New ideas and building on existing work

If through this process you generate a question that has never been studied scientifically—which again is something that you will learn in your literature review—then it might be interesting and worth pursuing. But what if you find that it has been studied scientifically? Although novice researchers often want to give up and move on to a new question at this point, this is not necessarily a good strategy. For one thing, the fact that the question has been studied scientifically and the research published suggests that it is of interest to the scientific community, suggesting you’re on the right track. For another, the question can almost certainly be refined so that its answer will still contribute something new to the research literature. Again, asking yourself a series of more general questions about the relationship is a good strategy.

• Are there other ways to define and measure the variables? Could this change the results? Why?
• Are there types of people for whom the relationship might be stronger or weaker? Why?
• Are there situations in which the relationship might be stronger or weaker—including situations with practical importance? Why?

Evaluating Research Questions

Researchers often generate many more research questions than they ever attempt to answer. This means they must have some way of evaluating the research questions they generate so that they can choose which ones to pursue. In this section, we consider two criteria for evaluating research questions: the interestingness of the question and the feasibility of answering it.

Interestingness

How often do people tie their shoes? Do people feel pain when you punch them in the jaw? Are women more likely to wear makeup than men? Do people prefer vanilla or chocolate ice cream? Although it would be a fairly simple matter to design a study and collect data to answer these questions, you probably would not want to because they are not interesting. We are not talking here about whether a research question is interesting to us personally but whether it is interesting to people more generally and, especially, to the scientific community. But what makes a research question interesting in this sense? Here we look at three factors that affect the interestingness of a research question: the answer is in doubt, the answer fills a gap in the research literature, and the answer has important practical implications.

First, a research question is interesting to the extent that its answer is in doubt. Obviously, questions that have been answered by scientific research are no longer interesting as the subject of new empirical research. But the fact that a question has not been answered by scientific research does not necessarily make it interesting. There has to be some reasonable chance that the answer to the question will be something that we did not already know. But how can you assess this before actually collecting data? One approach is to try to think of reasons to expect different answers to the question—especially ones that seem to conflict with common sense. If you can think of reasons to expect at least two different answers, then the question might be interesting. If you can think of reasons to expect only one answer, then it probably is not. The question of whether women are more talkative than men is interesting because there are reasons to expect both answers. The existence of the stereotype itself suggests the answer could be yes, but the fact that women’s and men’s verbal abilities are fairly similar suggests the answer could be no. The question of whether people feel pain when you punch them in the jaw is not interesting because there is absolutely no reason to think that the answer could be anything other than a resounding yes.

A second important factor to consider when deciding if a research question is interesting is whether answering it will fill a gap in the research literature. Again, this means in part that the question has not already been answered by scientific research. Note that this should be considered rather broadly: even though your exact constructs or variables may not have been studied before, expectations might be very obvious based on research on related constructs. Filling a gap in the research literature also means that the question is in some sense a natural one for people who are familiar with the research literature. For example, the question of whether taking lecture notes by hand can help improve students’ exam performance would be likely to occur to anyone who was familiar with research on note taking and the ineffectiveness of shallow processing on learning.

A final factor to consider when deciding whether a research question is interesting is whether its answer has important practical implications. Again, the question of whether taking notes by hand improves learning has important implications for education, including classroom policies concerning technology use. The question of whether cell phone use impairs driving is interesting because it is relevant to the personal safety of everyone who travels by car and to the debate over whether cell phone use should be restricted by law. Much business research is relevant for managers, employees, consultants or policymakers.

Feasibility

A second important criterion for evaluating research questions is the feasibility of successfully answering them. There are many factors that affect feasibility, including time, money, equipment and materials, technical knowledge and skill, and access to research participants. Clearly, researchers need to take these factors into account so that they do not waste time and effort pursuing research that they cannot complete successfully.

Looking through a sample of professional journals in business will reveal many studies that are complicated and difficult to carry out. These include longitudinal designs in which participants are tracked over many years, neuromarketing studies in which participants’ brain activity is measured while they carry out various tasks, and complex non-experimental studies involving several variables and complicated statistical analyses. Keep in mind, though, that such research tends to be carried out by teams of highly trained researchers whose work is often supported in part by government and private grants. Also, keep in mind that research does not have to be complicated or difficult to produce interesting and important results. Looking through a sample of professional journals will also reveal studies that are relatively simple and easy to carry out—perhaps involving a convenience sample of university students and a paper-and-pencil task.

A final point here is that it is generally good practice to use methods that have already been used successfully by other researchers. For example, if you want to manipulate people’s moods to make some of them happy, it would be a good idea to use one of the many approaches that have been used successfully by other researchers (e.g., paying them a compliment). This is good not only for the sake of feasibility—the approach is “tried and true”—but also because it provides greater continuity with previous research. This makes it easier to compare your results with those of other researchers and to understand the implications of their research for yours, and vice versa.