What distinguishes an excellent market research survey? There is no easy solution to this issue, and it is the combination of many factors that makes for a perfect piece of study. Indeed, market research is a balancing act in which the researcher often needs to cope with judgments that have contradictory effects. The researcher must balance the different features to guarantee that much of what is gained on the swings is not lost on the roundabouts.
After all, this is what the research process is all about, and it entails a concerted effort to reduce mistakes, even though errors can never be totally eliminated. Many people focus on sample concerns but ignore questionnaire issues, which are typically a greater cause of inaccuracy. This essay examines two major sample concerns before moving on to two additional issues in questionnaire design that are sometimes overlooked.
Concerns About Sampling
When requested to do research, one of the first concerns that usually arises is the question of sample size. In sample size calculations, at least two factors must be underlined. First, are the sample findings to be generalized to the full population? This seems to be a simple query, and a surprised ‘of course’ response is anticipated. The corollary is, what degree of precision and certainty do you want this to occur? The problem is that these are researcher or management choices, and on this premise, a wide variety of sample sizes may be justified! In reality, statistical concerns for determining sample size are sometimes of minor relevance.
Perhaps a more pertinent question for sample size determination is, “What is the planned application of the findings?” Will they be utilized to make a key choice, as part of a public relations campaign, or just as a ‘nice to know’ situation? Only the first goal is likely to need a representative sample. However, regardless of sample size, a random sample should be taken to allow for statistical analysis of the data. Even if it is not generalizable, any random sample may be statistically analyzed. Furthermore, there is nothing wrong with results from a sample that is not generalizable as long as the conclusions are explicitly limited by the sample’s limits.
A second factor to consider when determining sample size is the sort of analysis performed on the data gathered to answer your research question or questions. The sample size may severely restrict the kind of analysis performed. To guarantee the robustness of the study, various research concerns or models would need different statistical analysis techniques and sample size requirements. A multiple-stage model containing constructions or conceptions impacting another concept or concepts affecting another concept would, for example, need the use of a specific analytical approach such as structural equation modelling.
Such models are not implausible and reflect the circumstance experienced while attempting to understand consumer decision-making processes. They are often best addressed using structural equation modelling, with its own sample size and needs independent of generalizability difficulties. Indeed, big sample size may result in an incorrect response since the statistical approach may be oversensitive to large samples. This may seem hard if all of one’s research has been based on percentages. However, proper research entails more than just percentages. Indeed, percentage type analysis and conclusions are often the consequence of a choice made by the researcher about the data collecting utilized in the questionnaire used.
Questionnaire Development Pitfalls
There is no question that sample size is important in the research process, but there is much more to be worried about than sample size difficulties. Let me begin by eliminating one misconception that always makes me shudder. This is the notion that anybody can sit down and compose an effective questionnaire. Of course, anybody may create a questionnaire, and there is nothing prohibiting anyone from utilizing it. However, the focus of my statement is on a good questionnaire. Because this is not the place to get into the complexities of questionnaire creation, I will highlight two elements to demonstrate how often it is ignored.
I’ll begin by describing how one attempts to capture a notion while constructing a questionnaire. When doing market or management research, a researcher often attempts to capture something that lives in the respondent’s head, or “black box,” as it is sometimes referred to. It is a simple task, and it is seldom accomplished by merely asking a single question. A good comparison is a lecturer attempting to capture students understanding of a topic. Normally, a lecturer would strive to assess student understanding by asking many questions to capture the knowledge notion.
If you ask a student only one question, you can strike on an area that the student just did not study. However, if one asks a few questions regarding various parts of knowledge that the student was expected to have mastered and the answers are dry, one is much safer forming inferences about the student’s understanding. A similar multi-question technique is beneficial to follow in questionnaire construction to capture notions such as service quality, loyalty, and so on that are similarly residing in the respondent’s black box. As a result, a fundamental idea in questionnaire design is for the researcher to ask a battery of questions to capture the targeted concepts.
A related problem is the kind of data gathered by each question in a questionnaire, often known as a research instrument. It is frequently wise to avoid yes/no questions since they are often straightforward, poor at capturing constructs, and restrict what statistical analysis and testing can or cannot be performed. Scales should be used wherever feasible since they allow for greater variation capture, broader statistical analysis and testing, and ultimately more relevant results.
A second critical feature of questionnaires is the problem of the research instrument’s validity and reliability. This component is often overlooked while being a substantial cause of inaccuracy. Throughout, I have used the words research instrument and questionnaires interchangeably since questionnaires are our tools in the social sciences and management research, and they are analogous to thermometers, spectrometers, and other instruments used in the physical sciences.
Our surveys, like these devices, must fulfil two fundamental requirements: they must be valid and trustworthy. Using the thermometer example, the user of such an instrument would want to be certain that it gives the same reading when used among healthy persons, and therefore the device may be regarded as dependable.
Furthermore, you want to ensure that it is accurate and measuring what it is designed to measure, which is plainly temperature. Although this element is not always so obvious, the fact that a thermometer measures temperature is self-evident. Using the prior example of measuring student knowledge, it might be argued that the instrument utilized in tests and examinations is collecting the student’s recall capacity rather than knowledge.
This is a genuine validity issue. Validity and reliability look straightforward in the instance of the thermometer, but less so in examinations to capture student knowledge and much less so when the researcher is attempting to capture marketing or management notions. Concepts like service quality, value, contentment, loyalty, and so on are difficult to conceptualize and translate into surveys (termed operationalization). Testing for validity and reliability is often restricted in much commercial research, yet it is necessary if useful results and management decision-making directions are to be suggested.
Validity and reliability may be statistically assessed and endorsed if they fall within accepted boundaries. Indeed, legitimacy is a matter of degree rather than absolute. Some surveys are more trustworthy than others. However, many are not legitimate or dependable, making any inferences from the data questionable.
Creating a solid questionnaire to capture an idea, whether it’s brand personality, service quality, or one of the many other concepts we use in marketing and management, may frequently take an entire PhD dissertation. It necessitates following a hard procedure comprising many data collecting exercises that allow for testing of the questions used to capture the idea of interest and testing and retesting the instrument’s dependability and other elements of its validity.
I’ve seen many surveys attempting to capture constructs by asking a few ‘off-the-cuff’ questions. Such an approach is scarcely a firm foundation for robust and useful data collecting, analysis, and management decision-making recommendations. This essay has attempted to illustrate some of the subtleties often overlooked and may lead to significant inaccuracy. Many people are too concerned with sample size difficulties and pay little attention to questionnaire flaws, which are often a greater cause of the mistake.
Thorough research is surely difficult. However, when confronted with what seems to be overwhelming complexity, resolve is essential to resist reverting to previous questionable behaviours. If one is going to spend time, money, or both on research, this is an invitation to utilize proven and tested instruments and put in some effort to do analysis that goes beyond providing a few percentages. Survey research, when done correctly, is a vital instrument that may give helpful inputs to management’s decision-making process.