Programme name | Management | Module title | Survey Research Management | Module code | MN3121 | Local Resource Centre (where relevant) | School of Management | Date Submitted | 13/12/2013 | Word count (to include everything except the references and appendices) | 2108 |

Nonprobability sampling in management research

ESS has made a survey created to measure attitudes cross-nationally in Europe, using probability sampling. Measuring an attitude across countries is a tough job, but to successfully apply the methods of probability sampling too, seems close to impossible.
This essay will look at the sample-problems that this survey faces, and how a non-probability sample can be successfully integrated.
Before starting to analyse the survey, I would like to briefly explain what a sample is, and the main differences between the two sampling techniques.
First of all the objective of most surveys or research projects is to obtain information about the parameters of a population. To do this a sample is collected representing a subgroup of the population selected for participation in the project. The sample characteristics are used to “make inferences about the population parameters”. (Malhotra, 2010: 370) Meaning that you by selecting a small representation of the population can tell something about the whole population.
Non-probability sampling can be defined briefly as “Sampling techniques that do not use chance selection procedures, but rather rely on personal judgement of the researcher” (Malhotra and Birks, 2000, 358) An example of this would be a person who choices people on the street to take part in a survey by using his personal judgement.
There are different types of non-probability sampling, the most common are: Convenience -, quota -, snowball - and judgmental - sampling.
The convenience sample is the most accessible of the samples, by being quick, convenient and less expensive. Quota sampling ensures that every sector in a sampling design is filled. Snowball sampling addresses the problem ensuring an adequate sample of hard-to-find people. Finally judgemental sampling is a form of convenience sampling in which the population elements are selected based on the judgement of the researcher. This sample is a useful approach when investigating the potential of a new product or new merchandise. (Malhotra, 2010: 379)
Non-probability sampling is a good way of getting a bigger respond rate its quick, convenient and much cheaper than probability sampling. However, there is a chance of self-selection and no guarantee that the findings are creditable since the credibility of findings relies in the measure of the sample. (Smith and Jackson, 2012: 229)

Probability sampling can be defined as “a sampling technique in which each element in the population has a known and equal probability of selection”. (Malhotra, 2010: 382) This means that everyone in a population has the same statistically chance of being selected.
The four most common known types of probability samples are:
Simple random sample, systematic sample, stratified sample and multistage cluster sampling.
With simple random sampling each unit of the population has an equal chance of selection. In a systematic sample you select units directly from the sampling frame. (Bell and Bryman, 2011: 180) Stratified sampling, involves a process of stratification, followed by random selection. This method is used to trace the differences in the parameters of the subgroups within a population. (Sekaran and Bougie, 2013: 249) In multi-stage cluster sampling, the primary sampling unit is not the units of the population but groupings of those units. This method is very useful when doing a national sample, as it saves time and costs. (Bryman and Bell, 2011: 182)
The basic advantages of probability methods are that the sample provides estimates, which are essentially unbiased and have measurable precision. (Sekaran and Bougie, 2013: 251) Meaning that probability methods in general have few mistakes and are very precise.

The choice between non-probability and probability samples should be based on considerations such as the nature of the research. (Malhotra, 2010: 390) As an example: Are we dealing with a marketing test (exploratory) or a national projection survey (conclusive)? In a marketing test we would prefer a non-probability sampling, as projections to the populations are usually not needed, and time and money are scarce. However we would in contrast use a probability sample when dealing with a national projection survey, as we would want the survey to be as presentable towards the population as possible.
Another consideration depends on the variability in the population. (Malhotra, 2010: 390)
A more heterogeneous population would favour the use of probability sampling, as it would be more important to secure a representative sample. Consider as an example math test scores. Apply these tests to a general population, and the scores would be treated as homogenous, and non-probability could be used. However if want to apply the test to school dropouts and graduates also, the population will show up as heterogeneous, and here probability sampling would be favoured.
Another important factor is the degree of non-sampling errors vs. sampling errors. (Malhotra, 2010: 390) If we want a low margin of sampling errors, we will use a probability sampling, as it allows us to calculate the margin of errors and the level of confidence in our survey. We can’t do this with a non-probability sample, as the relationship between the sample and the population is unknown. However here we can examine the potential non-sample error, which can involve: mistakes in the sample frame, misinterpreted questions, poor analysis and basically any other human error.
Finally it is important to consider the statistical aspect. For some research projects, highly accurate estimates of population characteristics are required and here probability is favoured as it targets the elimination of selection bias. (Malhotra, 2010: 380)
However probability sampling is sophisticated and requires trained researchers. It generally costs more and takes longer than non-probability sampling. That is why many marketing research projects prefer non-probability sampling, as it’s easy and saves time and expenses. (Malhotra, 2010: 380)

Enable to properly understand non-probability sampling and when to use it, I have chosen to focus on a survey created to measure attitudes cross-nationally in Europe.
Measuring an attitude is a tough job in it self, to measure it across countries seems to be close to impossible. Surely there has to be some challenges regarding the research design, and the reliability of this survey. The survey is built on a random possibility sample, which raises questions about how the samples have been contained and what bias it faces?
The data has been received by face-to-face interviews, in the different countries. The survey had a maximum number of respondents per interviewer in order to reduce the impact of the interviewers variability of the results. (Jowell, Kaase, Fitzgerald and Eva, 2007: 16) However there still might be bias in the method in which the data has been conducted. The respondent might have answered differently based on the structure, phrasing of the questions asked. Also the respondents might have felt uneasy about the anonymity of their responses when they interacted with the interviewer. (Sekaran and Bougie, 2013: 124) The questions should also have the same meaning to all respondents, to ensure variations vary in the difference of answers and not interpretation of the questions. In this survey differences in interpretation could be present due to first: first language, mode of expression, levels of education, cultural differences etc. (Hoffmeyer-Zlotnik, Harkness, 2005: 31)
Another important factor of bias is the measurement of social attitudes cross nationally. Measuring social attitudes and values are risky, as different cultures and countries might have different perceptions of what attitude and values are, this can be due to social structures, legal systems, language, politics, economics and cultures etc. Even though the results might be reliable from a random probability perspective, it is still possible that the measurement of attitude doesn’t correlate in the different countries, as the definition of attitude is different. (Hoffmeyer-Zlotnik, Harkness, 2005: 33)
Another question the survey raises is how the samples have been made? Not all European countries have a reliable list of the individual members of the country. It would therefor be difficult to get the right sample in the different countries.
It can be seen in the sample design that the survey varied from a “simple random sampling size” in Denmark to a “four-stage stratified clustered design” in Ukraine. However because the sample methods are different it doesn’t mean that the results are not reliable, however it shows us how complicated and expensive the process of a probability sample is. In the process of being equivalent and reliable, it becomes even more expensive, complicated and time consuming. To do a random probability sample across nations, measuring attitude, would be an impossible job for a student, as the process stated above requires, time, academic expertise and money. However an alternative would be to make a non-probability sample.
As we are measuring “attitude” in different countries, it could be appropriate to make a quota-sample, as it would target all the different subgroups in each country, making sure that the people chosen for the survey gives us a true value of the population parameter estimated. Information about the stratification of the populations in the different countries could be obtained from public sources, as an example; national surveys and censuses based on probability samples research. Once we have that data we can start forming our categories in which we want to place the population. In our example we might divide the countries in terms of; gender, social class, age and religion etc. Finally we could spare time and money by creating an online survey instead of conducting face-to-face interviews. Lastly it would be a good idea to repeat the survey in three stages, to ensure our measures are reliable.
This seems like a straightforward method, however it to faces some problems that can make the survey non-representable. First of all we have to ask our selves if certain type of people had a greater chance to become respondents? How was the survey participants distributed compared to the distribution in the population? It’s also important to know about the willingness of the people that took part in our survey. How many of the people who got invitations answered? If we received a high degree of unwillingness, it may indicate that those who in fact responded may be unlike the majority of the population. Finally when doing an online survey about attitude, it is possible that the questions could have been misinterpreted. Having explained these implications of the use of non-probability sample. We can start comparing them to the probability sample, carried out in the survey.
The survey used a random sample as it wanted the data to be as valid as possible, however even though the survey states that it has used the correct sampling frame for all the countries stated, it still admits that there are non-response bias in the collected data. (Jowell, Rogers and Eva: 132) Technical speaking, when using a probability sample, there should be 0% mistakes. Even though the survey contains very little mistakes it still has bias that questions its results and the use of the random possibility sample. In the survey, it could have been useful to use a non-probability sample, in countries such as Ukraine, where the population is widely dispersed, and non-sampling errors likely to occur. A quota sample could by the use of judgement allow greater control over the sampling process, by targeting the precise categories in the population. This would make it easier to conduct and maybe even more precise compared to the four-stage stratified clustered design used in the survey, making the survey a lot cheaper, less time consuming and lead to a higher response rate.
However one should not forget that even though non-probability sampling is cheaper, it is entirely based on the researchers judgement. If the researcher is not a qualified researcher and has not spent enough time to select the right people for the sample, the research will be useless. ” (Buckingham and Saunders 2008: 124)

The conclusion in this essay is that probability samples are the most representative samples; however creating a representative sample without any form for bias is close to impossible. It takes a long time and requires a lot of resources enable for it to be equivalent and reliable. However the non-probability sample is a good alternative way for students to create a sample, as it’s cheaper and easier to construct. However as this sample relies completely on the judgement of the researcher, the results should be critically examined in order to make a representable measure of the population.

Word count: 2108

Reference List:

Bryman, A and Bell, E (2011) Business Research Methods 3rd edition: Oxford University press.

Buckingham, A and Saunders, P (2008) The Survey Methods Workbook: Polity Press,

Easterby-Smith, M., Thorpe, R and Jackson, P (2012) Management research 4th edition: Sage, Cambridge.

Hoffmeyer-Zlotnik, J and Harkness, J (2005) Methodological Aspects In Cross-National Research: Zuma, Mannheim.

Jowell, R., Roberts, C., Fitzgerald and Gillian, E (2007) Measuring Attitudes Cross-Nationally: Lessons from the European Social Survey: Sage Publication.

Malhotra, N (2010) Marketing Research: An applied orientation: Pearson.

Sekaran, U and Bougie, R (2013) Research Methods for business 6th edition: Wiley.

--------------------------------------------
[ 1 ]. Questions need to have variation to allow for the full range of answers to be tested correctly.
[ 2 ]. Ensuring our results are stable and consistent
[ 3 ]. Notice that we can’t make any statements about bias, variance, as bias is a statistical term for “expected value” and therefor can only be used in a statistical context, such as in a random sample.
[ 4 ]. Ensuring that the survey measures what it claims to measure.