How will you measure the independent, dependent, and moderating variables?  For some studies, it is important to identify a way to measure the change in the variables. For quantitative deductive studies non-parametric measurement instruments lead to non-parametric statistical analyses which are acceptable but not as powerful as the parametric statistical analyses. For qualitative studies the measurement might be qualitative descriptions based on the responses to the questions in an interview guide. For mixed method quasi-deductive studies the measurement plan might include both qualitative descriptions and quantitative measures (e.g., survey questions with scales, performance measures such as financial performance). The measurement plan should be consistent with the overall approach identified in the previous step and the conceptual framework and research questions. Remember - we “measure” variables and “analyze” relationships.

Data Collection Plan

How will you collect the data using the instruments and measurement approaches identified in the measurement plan?  The data collection plan includes a sampling plan.  Be explicit about the nature of your population and sample, the kinds of data gathering instruments you will use, and the circumstances under which the data are likely to be gathered. Include any specific activities that you would propose to reduce bias and increase validity.  The type and level of data that is collected will determine the analysis options that are available.

Triangulation of sources and methods is a common technique to mitigate bias and enhance validity.

Click on diagram for a larger PDF view.

Sampling

The main sampling difference for each methodology (qualitative and quantitative) is based primarily on the purpose of the research. If the purpose is to deductively “test” a specific hypothesis then a random sample that is sufficiently large to represent the population is the desired sampling approach.  Read more...

Example - Chad McAllister’s [Prospectus Version]

The population to be explored includes users who are involved in specifying requirements for IS and developers who create information systems.  An opportunity sample will be used consisting of three to four companies that meet the following criteria:

1. Sufficient size to create NGT groups of users and developers.

2. Publicly traded company performing in the top 49% of their industry group (a measure of success determined by the stock market).

3. Each company will be from a different industry to obtain a broader perspective.

NGT will be used to generate the factors from both user and developer groups.  NGT is a consensus generating technique often used in small groups of six to 12 participants.  NGT ensures that each participant has a voice in the group and that all participants are equal in their contributions.

The size of the sample is based on the work of Griffin and Hauser (1993) and related research, which determined that 20-30 participants can express 90% or more of the factors.  Consequently, three to four NGT small groups of users are expected to capture at least 90% of the factors users believe contribute to misunderstanding requirements.  The same is true for developers, resulting in three to four user-developer pairs of NGT small groups. 

The NGT is expected to involve two hours with each small group.  NGT itself is a bias-limiting approach that helps to reduce the influence of the facilitator.  As part of the approach, a standard introduction will be used for each group that describes the ground rules and the approach.  Also, selecting pairs of small groups from the  same company improves validity by implicitly controlling variables that may impact differences in factors between users and developers, such as different software development methodologies, definitions of quality, and the like.

After collecting the factors from users and developers via NGT, two aggregated lists will be created—one for users and the other for developers.  Two web-based survey instruments will be used to weight the importance of the factors.  One will contain the user factors and users will be asked to complete the survey.  The other will contain developer factors and developers will be asked to complete the survey. The survey participants will be the same individuals who participated in the NGT small groups.

Align and Integrate

1. The data collection methods (measurement, sampling, and data collection plan) should provide the data needed by the data analysis phase.

2. The data collection methods should be derived from and consistent with the overall approach.

3. As with all the components of the research methodology the data collection methods should be appropriate for the variables, relationships, context, etc. identified in the conceptual framework.