Questionnaire Design
Rohit Vishal Kumar
For Circulation to MBA 3rd Year Marketing Specialisation Class of 2003

August 2002

Abstract This article deals with the practicle problem of making a questionnaire. It points out the pitfalls, the common errors, methods and industry practises etc. Do not expect that your first questionnaire will come out trumps... Questionnaire design takes a lot of hard work, and work ... and re work before you can start mastering the arcane art. Use the article as a guide to what to do and what not to do, and ... start your work. I am sure you will not only understand the design of questionnaire better but also be able to design better questionnaire. As the outcome of the research is the report — I have included in Appendix A the outline that needs to be followed when writing a research report. The sample questionnaire used in the article is presented in Appendix B

1

Introduction

Every stage of marketing research is important but the most important stage is the designing of questionnaire since if the questionnaire design is faulty then no amount of clever interviewing, analysis and interpretations can provide meaningful answers. Questionnaire is the basic research tool and can be defined as collection of a formalised set of questions — drawn up with the research problem in mind — used for obtaining information from the respondent for finding solutions to the research problem. The various steps in questionnaire design can be classified as follows:(i) Identifying constructs to be measured (ii) Preparing the questionnaire flow (iii) Deciding the type of questions (iv) Wording and writing the questions (v) Piloting the Questionnaire (vi) Administering the questionnaire. The problems of analysis and reporting is outside the scope of this article and shall be discussed in details. However, to provide a standard structure to the report, a report outline is provided in Appendix A (see page 18)

1

c Rohit Vishal Kumar 2002

2

2

Construct Identification

The first and the most basic step in designing a questionnaire is to list down the specific information required to find the answers to the marketing problem. Many people assume that once they have understood the problem they can start making the questionnaire. Understanding the problem is easy; however identifying the constructs (or what are the set of questions that need to be asked) is hard. Constructs can be defined as the set of attributes that needs to be measured to provide meaningful answers to the question in context. Take the parable of blind men and the elephant. The man who got hold of the tail described it as a long, thin and hairy animal. The man who got hold of a leg defined it as round and thick animal. The person who touched the ears defined elephant as huge, thin and fan-type animal. All were correct — but all were wrong. They were correct because the construct they measured (or rather felt) were parts of the whole elephant but without any idea of what constitutes an elephant they gave the answers to the best of their ability. Similar is the problem faced by the market researcher. Unless he has a very clear cut idea of what constitutes the research problem (the elephant) he will not be able to develop measures and features to define the whole elephant. Take the question “identify market structure”. For an economist the problem is extremely easy - if there is only one firm in the market then the market is monopoly; if there are two firms, then the market structure is duopoly; if there are 3-8 firms present then there is oligopoly and if more than 8 firms are present then we have competition. However, as soon as the question lands into the hands of the market researcher - the complexities begin to emerge. The company asking for market structure information obviously knows — with a fair degree of accuracy — the number of companies that are present in the market and they want more than just the number of companies as the answer. So what should a market researcher try to measure? Should he measure the total turnover? Should he measure the degree of price elasticity or should he develop completely new constructs to capture essence of the question. The thing to keep in mind is that in marketing research, the questions that need to be asked are extremely sensitive and dependent on the formulation of the problem and the choice of the respondent segment. Questions (and consequently the constructs) about reasons for buying an air-conditioner would differ greatly from buying an air cooler. Even though both the products perform the same service of ‘providing relief from heat’ the buying decision drivers are distinctly different in both the cases. Developing constructs - or what should be measured and how - requires time, patience and a good grasp of the marketing research problem in hand. Before jumping into identifying the constructs, it may be well worth to

c Rohit Vishal Kumar 2002

3

spend some time on understanding the various aspects of the problem, the product in hand, the company, the competitors and other areas of interest. After the search is over, the next step is to “search-through” the available information to identify: (i) What are the measures that are currently in use – either defined in some other study and/or based on common conception of the market (ii) Why were these measures defined – was it to provide the same kind of answers that the current study is looking for or was it for a completely different purpose. (iii) How are the measures defined – what are the factors that have been used to define these measures (iv) Can these measures be used in the current study? (v) What do we need to modify in these measures to achieve the research objectives.

3

Questionnaire Flow

The second stage is to prepare the questionnaire flow. The technique of “Flow Chart” — borrowed from information technology — comes in extremely handy. The questionnaire flowchart is a powerful tool - which graphically (or otherwise) outlines the sequence in which the questions need to be asked. It is a comprehensive material detailing not only the sequence in which the questions would be asked but also the constructs and scales used for measuring the various attributes under study. The questionnaire flowchart needs to take care of the following three main processes — sequencing, routing and skipping — of the questionnaire design.

3.1

Sequencing

Sequencing details the order in which the questions would be asked to the respondent.In order to make the questionnaire effective and to ensure quality to the responses received, a researcher needs to pay a lot of attention in preparing the questionnaire. A proper sequence of questions considerably reduces the chance of individual questions being misunderstood. The question sequence should be clear and smooth moving, meaning that the relation of one question to another is readily apparent to the respondent. If necessary, the questionnaire may be segmented into sections — each section dealing with a specific area of investigation. Although questionnaire design depends on the problem in hand, a broad outline of the segment of a questionnaire can be identified as follows: 1. Control Information In market research studies it becomes necessary to have certain information that help identify the respondent and provide a means of rechecking whether correct sampling procedures have been followed by the investigator or not. It generally constitutes of (i) Questionnaire Serial Number (ii) The Ward / Area Number (iii)

c Rohit Vishal Kumar 2002

4

The Starting Point Address / Number (iv) The Name of the Respondent (v) The contact information - address, phone number etc. - of the respondent (vi) The Investigator’s Name and Code (vii) Other Field Control Information as necessary. 2. Introduction This forms the start of any questionnaire. It is a small paragraph — in which the investigator introduces himself to the respondent and solicits time from the respondent to participate in the interview process. The normal wordings of an introduction — which have almost become standardised across market research industry — is as follows: “Good ...... (MENTION AS APPROPRIATE). I am ...... (SAY YOUR NAME) from ......(SAY COMPANY’S NAME) - a leading market research organisation of the country. From time to time we conduct studies on a variety of product and services. Currently, we are doing a survey about ...... 1 and we shall be grateful if you could spare us some time to answer the questions. We assure you that as per the norms of marketing research your answers would be kept strictly confidential and only reported to the client on an aggregate basis. Thank you very much for your cooperation” 3. Eligibility Questions These are put at the beginning of the questionnaire to allow investigator to quickly establish the eligibility of the respondent. For example, if our target audience consists of “individuals who have completed Post Graduation (PG) before 1999”, it would be worthwhile to check the following — (a) Highest education level of the respondent and (b) year of completing the highest education level — at the start of the interview. This will allow us to establish whether the person is eligible for the interview or not. Otherwise after the interview is over, we may discover that the respondent is currently studying in PG or has passed out of PG in 1999 — thereby invalidating the interview; and leading to a waste of time and money. 4. Warm Up Questions The set of questions that follow the Eligibility Questions are of particular importance because they a likely to influence the attitude of the respondent and establish the degree of cooperation with the respondent. These questions are framed with the following objectives in mind: (i) easy to answer (ii) establish the respondents cooperation (iii) arouse respondents interest and (iv) lead up to the main questions.
This blank is usually filled up with the generic product category. For example if the survey is for distemper paints then we can say “... a survey about paints”
1

c Rohit Vishal Kumar 2002

5

5. Main Body This consists of a set of questions, which have been designed to elicit the desired information pertaining to the research problem in hand. Care should be taken to maintain the sequence of the questions within the Main Body. Questions which are easier to answer should be asked first, followed by questions that become progressively harder to answer. Two specific advantages result from using this strategy: (i) The respondent feels ‘morally’ bound to respond to the remaining questions and (ii) The respondent is more focused thereby provides better answers. However, opponents of this approach claim that it is best to ask the harder to answer questions first, because the respondent is ’fresh and eager’ to provide answers. As the interview progresses ’interview fatigue’ sets in and distort the respondents answers. The researchers task is to determine whether he would like to use the “easy-to-hard” or “hard-to-easy” approach while designing the questionnaire. If the number of areas to be covered is more, it is again worthwhile to break up the main body into several sections. For example, if we are trying to find out about the consumer perception, overall opinion and price reactions for a certain product; the main body of questionnaire could be broken up into three distinct parts - (i) Overall Opinion (ii) Consumer Perception and (iii) Reactions to Price. It may also be worthwhile to have a small Section Introduction — which helps the respondent to re-orient himself to the new section. “Section Introduction” should be kept as small as possible - big enough to reorient the respondent but small enough not to reveal anything. For example, the following could well serve as the section introduction for section III of our hypothetical study ”Let us now talk about price”. 6. Classification Questions These set of questions are used to classify and segment the respondent and usually cover the following five areas (i) Age (ii) Sex (iii) Monthly Household Income (iv) Whether the respondent is the Chief Wage Earner(CWE) or not (v) Highest Education Level of the CWE and the respondent and (vi) Occupation of the CWE and the respondent2 . There may be many more classification questions, but the six listed above are extremely important and should always be included because they constitute the basic. demographic information.
If the CWE/Respondent has retired then we ask about the occupation just prior to the retirement
2

c Rohit Vishal Kumar 2002

6

3.2

Routing & Skipping

Routing refers to the questionnaire sequence that would be followed by the person administering the questionnaire based on certain conditions being fulfilled. Skipping, a terminology interconnected with routing, refers to the system of not asking certain questions depending on the answer to the previous question. Take a look at the - Sample Questionnaire 1 (see page 20) - which shows a hypothetical questionnaire. Line numbers have been appended to the questionnaire at the extreme left for better clarity. The questionnaire will be used to identify and elucidate further principles of question design. Line numbers 03-04 provide an example of routing instruction, whereas line numbers 28-31 are examples of skipping instruction.

4

Types of Questions:

Essentially all marketing research questions can be grouped into two broad heads -Open Ended and Close Ended.

4.1

Close Ended Questions

In most of the cases, an investigator who has enough familiarity with the subject — or with a degree of secondary research and using logic — can build up a list of possible responses expected from the respondent with a high degree of accuracy. Take a look at the hypothetical questionnaire. Assuming that the respondent owns a mid size car, the possible response to the Q 2 can be well covered by the list of cars given in the sample questionnaire. Under such a circumstances it may be worthwhile to use the given list of mid sized cars throughout the survey to provide standard set of responses. Using a pre-standardised set of responses allows all the investigators to use a uniform recording style and in the process minimize influence of the investigator bias on the responses. Questions which carry with them a list of preselected responses are called closed ended questions. Qn. 1, 2 and 3 in ‘Sample Questionnaire 1’ are examples of close ended questions. Closed ended questions can be further divided into two parts - Single Code and Multi Code. The list of preselected responses attached with a single-code question are mutually exclusive of each other. Or in other words only one valid response is possible for a single code question. Take a look at Qn. 1 of the ‘Sample Questionnaire 1’. The person contacted can either own a car (Response: Yes) or he does not own a car (Response: No). No other response is possible. In the case of multi-code questions, the list of preselected response is not mutually exclusive. Or in other words, more than one valid responses are possible. Take a look at Qn 2 of the ‘Sample Questionnaire 1’. In the best

c Rohit Vishal Kumar 2002

7

case scenario, the respondent can recall all the thirteen car names which we have listed, thereby providing us with thirteen valid responses. In the worst case scenario, the respondent may be able to recall the name of only one car - namely the car he owns. In the normal course of events, most respondent will recall between 1 and 13 names which can be easily captured in the questionnaire. Framing a multi-code question carries with it certain problems - (i) How to capture more number of responses than allowed for and (ii) Which question(s) should be denominated as single or multi-code. Let us deal with the first problem. The respondent may, in fact, recall more than 13 car names that we have provided for. To capture the “extra responses” — which is information over and above desired by us — provision should be built in the questionnaire. This is done by providing the category Others. The researcher has to take a call as to the number of Others to include in a question. It is common practise to provide for 5-6 Others space and save time. This approach may work well if the numbers of Other responses generated are low. However, if the number of Other responses is large; analysis, tabulation and interpretation becomes problematic. Take a look at Q 3. of our sample questionnaire. Common sense tells us that a respondent may very well own more than one car, and if such is the case then Q 3 will fail to capture all the cars owned by the various respondents. One way of tackling the problem would be to break up Q 3 into two distinct parts as follows...... Q 3a. Which all mid sized cars do you own? MULTI CODE Q 3b. ASK ONLY IF MORE THAN ONE CAR OWNED ELSE CODE THE RESPONSE IN Q 3A AS THE RESPONSE IN Q 3B AND GOTO Q 4 Considering everything, which car do you use most often? SINGLE CODE ...... and continue the interview based on the car coded in Q 3b. The second method would be to use the question as given in our hypothetical questionnaire, but provide instructions to the investigators — that if the respondent owns more than one car then the car used most frequently should be recorded as the answer in Q 3. It is the researchers job to determine whether to designate a particular close ended question as multi code or single code. Practise, understanding of the research problem, the constructs desired, time, length of the questionnaire et. al. go into determining whether the question is to be a single or multi-code. The best maxim to follow in this case - “when in doubt, go out and ask a few possible respondents”

c Rohit Vishal Kumar 2002

8

4.2

Open Ended Questions

Open ended questions essentially are used to elicit a free response from the respondents. At times it is not possible to anticipate the set of possible responses for a given question. Questions which try to elicit qualitative aspects — like moods, fears, emotions, ethos, cultural influence etc. — generate different responses amongst different respondent. Making these sort of questions as close ended will not allow us to capture the full range of responses. Side by side there are questions for which the set of possible responses can be anticipated - but the list is so huge that it becomes unwieldy to administer and/or include into the questionnaire. Under such a situation it may be prudent to leave the question as an open ended question — as we have done with our Q 4 in the sample question. However care should be taken to limit the number of open ended questions in a quantitative exercise. Analysis of open-ended question require special care and understanding - specially when it has been generated as a part of a quantitative exercise. If the number of open ended questions are relatively large — then it may be worthwhile to re-investigate the research problem. Either the researcher has not understood the problem in hand or he is using quantitative study where a qualitative study is required.

5

Writing the questionnaire

After the questionnaire flowchart is complete and the type of questions decided on, the next task of writing out the questionnaire begins. However, before we start writing down the questions we should remember the fact that questionnaire is instrument through which the respondent is made to reveal his/her knowledge, attitudes and perceptions, behaviours, likes and dislikes etc. In other words - the respondent is doing a favour to the researcher by revealing these information and as such maximum care should be taken to avoid generating any stress for the respondent. The do’s and don’t’s of framing a question are given in section 5.3.2 (see page 10).

5.1

Establishing Eligibility

The most basic task of the investigator is to ensure that the questionnaire is being administered to the relevant respondent. This should be done by establishing eligibility of the respondent as early as possible - to avoid wastage of the investigator’s time. The criteria that a respondent should match to be eligible for the interview are known as eligibility criteria. If, for example, the target segment is owners of car then it would be much better to establish ownership of the car at the earliest. Line no 04 of the example questionnaire eliminates those respondents who do not own a car.

c Rohit Vishal Kumar 2002

9

5.2

Contact & Main Questionnaire

In case the number of eligibility criterion to match are more that one — it may be worthwhile to split the questionnaires into two parts. The first part, called the contact questionnaire, contains all the questions that are needed to establish eligibility of the respondent for the main interview. The main questionnaire, on the other hand, contains the questions — answers to which will provide solutions to the problem at hand. Listing questionnaire also perform a very important task. They help in maintaining the randomness in any marketing research process. For example, assume that we need to identifying the percentage of car owners in a particular locality. For the sake of simplicity let us further assume that we decide to base the answer on a sample 30 residents of a particular PIN code zone. One of the methods of solving the problem would be to approach 30 people whom we think will own a car. Suppose the investigation reveals that 27 people own a car. Would that imply that 90% of the people own a car? Statistically speaking the inference ‘that 90% own a car’ cannot be drawn as there was not well identified sampling procedure used to conduct the study. Another method would be to administer a contact questionnaire to 30 respondents of the locality in such a manner that the investigator knocks on every 5th door from a given starting location. If the resident owns a car then the main questionnaire is administered; otherwise the investigator moves off to the next house after skipping 5 houses. Now suppose you get a result that 21 residents own a car. The result ’that 70% of the people own a car’ would be admissible statistically3 because in doing so we have inadvertently followed the steps of systematic random sampling.

5.3
5.3.1

Phrasing the Question
Problem of word sequence

The most vital task is to word the question correctly - because even a small misspelling or change in the position of words can change the meaning of the question that is being asked. Take a look at the following examples: Q 1a. Agree or Disagree - “The government should actively support social security measures” Q 1b. Agree or Disagree - “The government could actively support social security measures” The above two questions differ on a single word — should and could. But the difference is enough to change the meaning of the sentence. In the strictly speaking, the inference cannot be drawn - but for illustration purpose we can take the leeway
3

c Rohit Vishal Kumar 2002

10

former case an assertion is being made and in the latter case a possibility is being explored. Majority of the respondent would say yes to the former than to the latter. Q 2a. Agree or Disagree - “Students should study hard before the exams” Q 2b. Agree or Disagree - “Students should study hard only before the exams” The above two questions differ on the inclusion or exclusion of the word ‘only’. Asked to students almost 100% would agree with the first statement whereas almost everyone would disagree with the second statement; in spite of the fact that a large majority of students actually follow the behaviour pattern highlighted in Q 2b. The reason can be found in human psychology. Human beings do not want to admit their bad traits and therefore will automatically disagree with the second statement. Q 3a. Comment - “Should we dissolve our difference with Pakistan” Q 3b. Comment - “We should dissolve our difference with Pakistan” Question 3a and 3b contains the same words — but differ in the arrangement of words and in the process change the meaning of the sentences. Q 3b would generate a vehement NO from a large segment of Indian citizens, whereas Q 3a would generate a lot of debate without giving a concrete answer. The above three examples should be enough to convince a researcher about the importance of wording a question. A wrongly worded question, a misspelling, a change in the word sequence — can change the whole complexion of the problem and negate the whole research exercise. Extreme care should be taken to word the questions. Care should also be taken to avoid certain other practises - which are detailed below. 5.3.2 Don’t’s of framing a question

Avoid unfamiliar words Words which are not familiar to the respondent, jargons, difficult words should be avoided as much as possible. For example “Do you think that CORBA model is better than COM?”. Respondents unfamiliar with Information Technology will have no idea as to what is CORBA or what is COM. Even people working in IT industry may find it difficult to answer the question as CORBA and COM are specialised technologies which are not used on a day to day basis.

c Rohit Vishal Kumar 2002

11

But yes, if the sample is of highly educated specialist in a particular field; then the jargons specific to that industry may be used — as it provides a standardised reference frame. Avoid many things in one question A question like “What to you think of liberalisation, privatisation and globalisation?” asks the respondent to evaluate three things — liberalisation, privatisation and globalisation — at one go. Even hardened economist will have problems in answering the question; forget the man on the street. It would be better if the question was broken up into three distinct questions. Breaking up would allow the respondent to focus on one question at a time and thereby provide a more lucid and meaningful response. Avoid asking complicated questions: Take a look at the following question: What would you think would you rather have in the way of lather, a low level of lather which would give less cleaning power but would be easier to rinse away, or a high level of lather which would give more cleaning power but would be harder to rinse away?” — a question actually asked in a survey. Chances are you would have to repeat the question more than 5 times just to make the respondent understand what is being asked. If it becomes necessary to ask such a complicated question, it would be better to break up the question into smaller individual parts. Alternatively visual clues4 can be provided. For the above question we can design a card as follows: Concept A B Amount of Lather LOW HIGH Power of Shaving LESS MORE Ease of Rinse EASY HARD

With the cue card being shown to the respondent, the corresponding question becomes much easier “Do you prefer concept A or do you prefer concept B?”. The cue card also allows the respondent to keep focus on the complicated concept and thereby provide more meaningful answer. Avoid double negatives Another very common practice is to use double negatives in a question. Negation is an extremely powerful tool in mathematics, statistics or other sciences but for a respondent it poses a double hurdle. Consider “Would you rather not use a non-medicated shampoo?” First the respondent has to figure out what is a medicated shampoo. Then
4 Known as ‘cue cards’ or simply ‘show cards’ in marketing research terminology. Use of cue cards is also suggested for multi code questions

c Rohit Vishal Kumar 2002

12

he has to figure out the term non-medicated shampoo. Then he has to figure out the reasons for using a non-medicated shampoo. Finally he has to negate the reasons to give a meaningful answer to your question. It would be much better to ask ”Would you prefer to use a cosmetic shampoo?” - Simple, direct and easy to answer. Avoid abstract concepts Introducing abstraction provides an easy way of passing the burden of response to the respondent. However in doing so a researcher inadvertently builds a base for non response which may prove fatal at the time of analysis. Consider “What do you think of the state-ofthe-art production facility of XYZ limited located in Maharajnagar?” First level of abstraction — what is the meaning of state-of-the-art? Is using up to date technologies “state-of-the-art” or does it mean use of extremely advanced technologies? Second level of abstraction — What is the state-ofthe-art technology in production of products manufactured by XYZ Ltd? Third level of abstraction — Does XYZ Ltd. has a production facility at Maharajnagar. Furthermore I have never visited the production facilities of XYZ Ltd at Maharajnagar, so how do I provide you the answer? Again it would be better to use cue cards. However, cue cards may not provide the desired answer in the above case; because the normal man on the street will have no idea as to what constitutes the state-of-the-art technology in production of the product. It may be better to ask for the opinion of the respondent. Or even better to drop such questions. Avoid vague concepts Another quick method of passing the buck on to the respondent is to ask him vague concepts. “Do you think that your house is the right sort of house for your family?” What is meant by the term “Right Sort”. Do you mean to say whether my house is comfortable to live in? Or do you mean that every member of my family has a room of his own? Or do you mean that ventilation is proper? Or are you asking me about the architecture of the house? The above question also shows the sloppiness in construction of a construct. Probably the client said that we want to make the ‘right sort’ of house for our customer; and the researcher without bothering to figure out what ‘right sort’ means passed on the burden of interpretation on to the respondent. A number of different attributes may contribute in making a house the ‘right sort’ of house. The level of comfort, roominess of the house, layout of the house, quality of construction, quality of neighbourhood, quality and quantity of essential utilities like water, electricity etc. and value for money. It is the researchers job to drill down and find out what does a phrase actually means. Once the meaning is clear, it becomes easy to frame the constructs and provide meaningful answers. A composite index composed of the 8 attributes listed above would be a better indicator of ‘right

c Rohit Vishal Kumar 2002

13

Table 1: Recalling Product Purchase Product Super FMCG5 FMCG Low Cost CDG6 Medium Cost CDG High Cost CDG Low Priced Assets High Priced Assets Example Cigarettes Soaps Cassettes Walkman Television Cars Home Time frame 1-7 days 7-30 days 1-3 Month 3-12 Months 1-3 Years 3-5 Years > 5 years

sort’ of house than just the phrase ‘right sort’. Avoid Futuristic Question Another variation on the same theme is asking the respondent impossible to answer questions. Consider the question “How long do you think will your current tennis racket will last?”. The tennis racket may break the next day or it may last me a lifetime. There is no way in which a meaningful answer can be provided. Impossible to answer question forces the respondent to fall back on the arcane art of prediction — as the respondent has no reference point to go by. Reframing the question to “How long did your previous tennis racket last?” will provide more meaningful answer; whose analysis can and do provide good indicators of the life of the racket. Avoid mathematical concepts Mathematical concepts in the questionnaire may also tend to make the question impossible to answer or vague. Questions with the term — on an average — are repeatedly asked in various marketing research exercises. Like “On an average how much did you spend on clothing last year?. In doing so we are forcing the respondent to evaluate his past behaviour and then apply the arithmetic concept of mean to provide a meaningful answer. Even if the technique works with an erudite urban respondent; responses received from semi literate respondents would be of no practical use because of the inherent vagueness of the response. Responses like “I guess around 1000 - 2000 Rs. or thereabouts” are common. The investigator is forced to judge the veracity on the spot. Should he record Rs. 1000 or should he record Rs. 2000 or should he record the average i.e. Rs. 1500 or should he record any arbitrary figure between Rs. 1000 and Rs. 2000? If such a question needs to be asked, then it would be better to stick to a clearly defined time frame and to divide the question into distinct parts. The above question may be broken into three parts (i) which all articles of clothing did you purchase last month and (ii) what was the amount

c Rohit Vishal Kumar 2002

14

of quantity of such article purchased and (iii) what was the price of such item(s). An argument can be raised that ‘last year’ is also a well defined time frame. Admitted, but can you recollect which all clothing did you purchase in the last one year? If no, then why expect the guy on the street to remember. A rule of thumb for deciding time frame is given in 1 (see page 13). Avoid generalisation Related to the above is the problem of generalising. Research exercises tend to ask questions which relate to the general pattern of behaviour and attitudes. Given the fact that modern methods of marketing rely heavily on behavioral aspects the questions on behaviour and attitudes tend to make their way into the questionnaire. “What do you usually do after returning home from office?” or “What do you do on holidays?” may not provide the behaviourial pattern that the marketer may be looking for. If the behavioral pattern is fixed (or relatively fixed) then the chances of identifying the correct behavioral pattern increases. But if opposite is the case — then the chances of identifying the correct behavioral pattern decreases and chances of impressions creeping in increases. The former question has a higher chance of being answered correctly than the latter. In such a situation it may be preferable to ask the behavioral pattern question with a fixed time frame which provided an anchor to the respondent and allows him to increase the accuracy of the response by having a pre-identified frame of reference. Or in other words — it may be better to ask “what did you do on last Sunday?.”

6
6.1

Final Steps
Piloting the Questionnaire

After designing the questionnaire comes the most vital task of piloting the questionnaire. Piloting refers to the technique of administering a few questionnaires to a small set of target respondent with a view of finding out whether the questionnaire is doing it’s job or not. Special attention is placed on wording and figuring out whether it is conveying the correct meaning or not. If a vernacular translation has been done, then care should be taken that the vernacular version of the question asks the same thing as the English one. Translation goof-ups can completely jeopardise a research study. Another area that should be looked into is whether the multicode questions are generation the responses within the precoded response set or not. If majority of the respondent provide responses that get coded as Others then probably the response set is inaccurate and may need a lot of reworking. Attention should also be paid to the constructs. A few analysis should be carried out to see whether the constructs are providing the desired in-

c Rohit Vishal Kumar 2002

15

formation or not. If latter is the case, then constructs would need to be reworked. Time taken to administer the questionnaire should also be carefully monitored - both in the aggregate sense and for individual questions. If individual questions take a long time to be answered, then the reasons for the delay should be identified and rectified. If the overall time taken for the interview is more than 60 minutes, then chances of getting accurate responses towards the end decrease dramatically. Respondents are overcome with ‘interview fatigue’ and they just want to get over with it. Also, long questionnaire tends to bring down an interviewer productivity - as the number of interviews that can be done in a working day or 8 hours fall. To boost their productivity, interviewer may well resort to unfair means to fill in their daily quota of questionnaire. In such a case the questionnaire may need to be reworked to bring down the interviewing time. On the other hand if time cannot be cut down, then a shuffling of sections may be required to average out bias due to fatigue. Suppose, that a long questionnaire has two sections - A and B. It would be better to administer sequence A-B to 50% of the respondent and the sequence B-A to the remaining. After all the changes, suggestions etc have been incorporated, it is recommended that the piloting exercise be carried out again, and again - until and unless the researcher feels confident of the questionnaire and all the problems have been ironed out. Piloting is crucial if the research is carried out on a large scale and/or across the country — as because once the green light has been given enormous amount of time, money and energy is wasted to rectify an error that went unnoticed. Some researchers tend conduct pilot exercise on approximately 10% of the total sample size. However the size of the pilot is not so critical — but the process is. A pilot exercise conducted fairly and squarely will reveal lots of problems — rectification of which will make the whole research exercise more richer.

6.2

Administering The Questionnaire

After the final alterations and re-piloting, the questionnaire is ready to be administered. In case the interviews are to be carried out by investigators - or persons other than the researcher - a detailed guideline7 should also be prepared and sent along with the questionnaire. This guideline takes up each question and explains in detail (i) how the question needs to be asked (ii) how the responses are to be taken down (iii) how to handle unexpected responses etc. The briefing note also clarifies in detail the sampling procedure, target respondent, and other logistical schedules - like dispatch, last date of completion etc. The main aim of the guide is to uniformity in respect of all salient points in the study
7

Also called the ‘Questionnaire Briefing Note’

c Rohit Vishal Kumar 2002

16

It must be remembered by the person administering the questionnaire that interviewing is an art and one learns it only by experience. However if the following points are kept in mind then eliciting the desired information becomes much easier: (i) Approach must be informal and friendly. the interviewer should greet the respondent and explain the purpose of the interview (ii) Proper rapport should be established between the interviewer and the respondent; people are motivated to communicate when the atmosphere is favourable (iii) Interviewer must develop the art of listening and must show interest, respect and curiosity towards the responses of the respondent. However he should not loose sight of the fact that he has to fill up the questionnaire, and therefore should be able to guide the communication without interfering, interrupting and without giving offense.

7

Summary

In sum, designing questionnaires is an art form of the highest level. It involves al lot of things starting with the researchers ability to research and understand the problem, drawing up the questionnaire and finally administering it. Questionnaire design cannot be learned in a day and neither can a questionnaire can be drawn up at the drop of the hat. To develop an effective questionnaire the researcher has to do a lot of work and has to keep in mind the following salient points: (i) The researcher must be clear about the various aspect of the research problem he is trying to solve — because the research problem forms the basis on which the questionnaire is drawn (ii) Appropriate forms of question depend on the nature of information that is being sought, the target audience in question and the kind of analysis intended. Questions should be simple, easy to understand and must be constructed with a view of their forming a logical part of a well thought out tabulation plan (iii) Rough draft(s) of the questionnaire should be prepared, giving due thought to appropriate sequencing, routing and scheduling of questions. Drafts should be thoroughly re-examined and revised repeatedly until and unless the researcher is confident that the questionnaire is going to provide him the answers he is looking for (iv) Questions should be worded with care. Special emphasis should be laid on keeping the questions simple and straight-forward, so that the respondents do not have any difficulty in understanding and answering the question (v) Pilot study should be undertaken for pre-testing the questionnaire. The questionnaire should be edited in the light of findings of the pilot study. Designing questionnaires requires patience. It requires the ability to analytically analyse problems, devise measures to measure the problem. It requires the ability to play with words, to understand the fact that what is communicated is not always what is decoded. To look through the initial

c Rohit Vishal Kumar 2002

17

stage and figure out what type of analysis to do and therefore what kinds of questions to include. To work, rework, revise and re-draft or even junk, the questionnaires based on the outcomes of the pilot study and the ability to suppress own opinions and listen to the ranting and ravings of - on hand the client and on the other the respondent; for the sake of the research study. In the end - one must remember that “a well done questionnaire, is like completing eighty percent of the research study”
This document can be obtained from: Rohit Vishal Kumar Lecturer - (Marketing & Market Research) Indian Institute of Social Welfare & Business Management Management House, College Square (West) Calcutta - 700 073, W.B. India Phone: (91-33) 241-8694 / 8695 Ext. 406 Email: rvkumar@iiswbm.edu. Final Print on: August 20, 2002

c Rohit Vishal Kumar 2002

18

A

Report Outline

Any printed work, be it a book or an article, can be broken up into three distinct parts - the front matter, the main matter and the end matter. The layout below shows the outline of a research report and the position of different elements of the report w.r.t. each other. 1. Front Matter (a) Title Page (b) Table of Contents (c) List of Tables (d) List of Figures (e) Acknowledgement 2. Main Matter (a) Executive Summary (b) Nature of the Study (c) Research Design (d) Research Methodology (e) Analysis of Data (f) Presentation of Findings (g) Conclusion (h) Recommendations 3. End Matter (a) Elaboration of Special Techniques 8 (b) Questionnaire(s) Used (c) Bibliography (d) Index 9

Things to Remember:
• The items Conclusion and Recommendations may be combined into one chapter if so desired. However, for large reports, it is better to separate the two.
8 May be dropped if so desired. But it provides a useful way of relegating to the appendix detailed technical stuff — thereby preventing the report from becoming too technical 9 If possible, an effort should be made to provide an index to the report. It provides a quick way for the reader to find any desired area or topic

c Rohit Vishal Kumar 2002

19

• Reports should always be prepared on A-4 paper. A margin of at least 1.5 inch should be allowed on the left hand of the paper and of at least 0.5 inch at the right hand of the paper. There should also be 1.0 inch margins at the top and bottom. Reports should preferably produced using a word processor or a typewriter. All the typing should be done using a line spacing of 1.5 (or 2.0) on one side of the page only. • Special Note: Students working on Microsoft Word (or other such word processing software) should refrain from typing the whole report in one go and in one document. This is to prevent any loss of data - as larger the file, more computer memory is required for processing it, thereby increasing the chances of a crash. It may be worthwhile to break up the report into several small files and then use the “Insert -> File” option to create the full report before printing.

c Rohit Vishal Kumar 2002

20

B
Line No. 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39

Sample Questionnaire
Sample Questionnaire: Survey amongst car owners Q 1. Do you own a car? YES 1 NO 2

IF 1 CODED ASK QUESTION 2-10. IF 2 CODED TERMINATE INTERVIEW Q 2. Could you tell me the names of all the mid size that you are aware of? (MULTI-CODE POSSIBLE) Q 3. Could you tell me which make of car you own? (SINGLE CODE) Q2 Q3 --Maruti 800 01 01 Maruti Esteem 02 02 Maruti 1000 03 03 Maruti Zen 04 04 Maruti Alto 05 05 Maruti Van 06 06 Maruti Wagon R 07 07 Hyundai Santro 08 08 Daewoo Matiz 09 09 Fiat Uno 10 10 Ambassador 11 11 Premier 118 12 12 Premier Padmini 13 13 Others ________ 14 14 IF IF IF IF 01 08 11 14 - 07 CODED ASK QUESTION 4a. - 10 CODED ASK QUESTION 4b. - 13 CODED ASK QUESTION 4c. CODED TERMINATE INTERVIEW.

Q 4a. You said that you own a Maruti Car. What in your opinion distinguishes ______ (INSERT RESPONSE FROM Q 3 ABOVE) from other Maruti cars? PROBE Anything Else? PROBE Anything Else? RECORD VERBATIM _____________________________________ [ ][ ] _____________________________________ [ ][ ] _____________________________________ [ ][ ]

c Rohit Vishal Kumar 2002

21

Contents
1 Introduction 2 Construct Identification 3 Questionnaire Flow 3.1 Sequencing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Routing & Skipping . . . . . . . . . . . . . . . . . . . . . . . 4 Types of Questions: 4.1 Close Ended Questions . . . . . . . . . . . . . . . . . . . . . . 4.2 Open Ended Questions . . . . . . . . . . . . . . . . . . . . . . 5 Writing the questionnaire 5.1 Establishing Eligibility . . . . . . . . 5.2 Contact & Main Questionnaire . . . 5.3 Phrasing the Question . . . . . . . . 5.3.1 Problem of word sequence . . 5.3.2 Don’t’s of framing a question 1 2 3 3 6 6 6 8 8 8 9 9 9 10

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

6 Final Steps 14 6.1 Piloting the Questionnaire . . . . . . . . . . . . . . . . . . . . 14 6.2 Administering The Questionnaire . . . . . . . . . . . . . . . . 15 7 Summary A Report Outline B Sample Questionnaire 16 18 20