uminary ReviResearch Definitions and Study Guide for Luminary Comps HC3 - 2013

Ideology | Oversimplified explanations for social behavior that focus on one or a few variables to explain a social phenomenon. Ideologies are resistant to change and become entwined with moral, instead of empirical arguments. Begins and ends with facts. | Social Theory | A systematic, detailed means of explaining why a social phenomenon exists that recognizes the influences of a multitude of factors, is subject to change, and avoids moral arguments in favor of empirical arguments. Begins with a question and ends with facts. | Concepts or Constructs | Abstract terms people use to describe reality. People may share general definitions or understandings of a concept, but because they are abstract, the specific interpretation of these concepts among many people may differ. An abstraction. Cannot be measured directly. | Dimensions | A theoretical term used to describe clusters of concepts that together represent a broader, more complex concept. | Conceptual Definition | Is the working or “dictionary type” definition a researcher uses for a concept. It tends to be nonspecific | Operational Definition | Is used to define something (e.g. a variable, term, or object) in terms of a process (or set of validation tests) needed to determine its existence, duration, and quantity. | Measurement | The process of observing concepts, as indicated by their operational definitions, and assigning some type of score or meaning to responses. | Nominal Measure | Low level of measurement, answer choices are exhaustive as well as mutually exclusive. (Exhaustive means there is an answer choice for everyone, and mutually exclusive means the participant selects one and only one choice unless otherwise instructed). Associating a name with something. No ordering or rank, just a name. (e.g., numbers on a football jersey, or yes = 1 and no = 2). (Nominal is Latin for number). | Ordinal Measure | Level of measurement, answer choices are ranked, exhaustive and mutually exclusive. Doesn’t suggest how much better or different the number are. Just gives the order or ranking. The rankings are not assigned relative weights. (e.g., like cancer stages: stage 2 cancer isn’t twice as bad as stage 1) (horse race: win, place or show doesn’t imply intervals, only ranking). | Interval Measure | Level of measurement, answer choices are exhaustive, mutually exclusive, and have quantifiable distances between categories (evenly spaced), but does not have a true zero point (any zero value is arbitrarily assigned). (e.g., degrees on a thermometer). Formerly called equal interval. | Ratio Measure | High level of measurement, answer choices are exhaustive, mutually exclusive, have quantifiable distances between categories, and have a true zero point. (e.g., on a salary scale, one person might make $10,000, but someone with a zero salary makes absolutely no money. Zero is a true value of" nothing.") | Variable | An element, feature, or factor that is liable to vary or change. A means of measuring a concept where the answer choices can vary. | Latent Variable | Variable that does not manifest itself physically. Not directly observable. (e.g., intelligence, personality, knowledge, athleticism). | Observed Variable | (Also known as an indicator): something that can be measured directly. (e.g., a ruler can measure your height, a scale can measure your weight, an IQ test can be an indicator of your intelligence). | Independent Variable | It is the variable that produces the change in the dependent variable. It is the “cause” in a causal relationship. | Dependent Variable | The value of the dependent variable “depends” or is influenced by the independent variable. It is “the effect” in a causal relationship. | Validity | Refers to the accuracy of the research. Does the test measure what it claims to measure? A valid indicator may be used as a predictor. | Internal Validity | Refers to whether some factor other than the independent variable produced the observed change in the dependent variable. Threat to internal validity include: history effects, maturation, testing, instrumentation, selection bias, contamination, and experimental mortality. | External Validity | When researchers are unsure whether cause/effect relationships will hold for other groups, different settings, or different times, they are concerned with external validity. Threats are: whether it is applicable to other uses, testing effects, selection bias, reactivity or Hawthorne effect. | History Effects | Type of threat to internal validity. Events external to the research, such as news or natural disasters, that can affect the results obtained. | Maturation Effects | Type of threat to internal validity. Participant’s behavior changes due to the simple act of aging or the length of the study. This is particular to longitudinal studies. | Instrumentation Effects | Type of threat to internal validity. Occurs when the measure used to observe concepts change during the course of the study. This can happen due to changes in concept definition or question wording. | Contamination Effect | Type of threat to internal validity. Occurs when members of the experimental or control group learn from each other and this knowledge affects their behavior I the posttest. | Experimental Mortality | Type of threat to internal validity. Is a common threat in longitudinal studies and occurs when members of an original sample drop out of a study as time passes. | Testing Effects | Type of threat to internal and external validity. It occurs when the administration of a pretest sensitizes participants and therefore affects their answer at a later test point. | Selection Bias | Threat to internal and external validity. It occurs when researchers draw samples that are not representative of the population about which researchers hope to make conclusions. | Reactivity or Hawthorne Effect | Type of threat to external validity. It occurs when research participants behave atypically because they know they are being observed by another person. (Western Electric: Hawthorne Plant in Chicago). People in the study try to please the researcher. | Face Validity | The most basic validity test that indicates whether the measure appears valid “on the face of it.” In other words, determining whether the measure seems to make logical sense as an indicator of a concept. “Does it appear to measure what it claims to measure?” (Dr. Ree’s test: would you go to court on evidence of face validity: No). (example in book: # of drinks per week as a test of someone becoming an alcoholic). | Content Validity | A validity test for whether the measure covers the full range, or all of the dimensions, of a concept’s meanings. “Is the content (what makes up the measure) appropriate to the task of measurement?” It is still somewhat subjective, but often it is all we have. (Example in book: ADi score for drinkers). | Criterion Validity | A type of validity that compares a researcher’s chosen measure to some external criteria—whether it is another measure widely accepted in that particular field or a more directly observable criterion. “Can I predict some other measure with my measure?” Typically, we correlate two measures. (e.g., r = .55, p < .05). (Example in book: ADi score & 2 friends as references OR ADi score and AAIS score). Two types: concurrent validity and predictive validity. | Construct Validity | When researchers can show that their measure is related to other measures specified in a theory. “Does it actually measure the construct it is designed to measure?” This is usually found after a number of studies. When two things have a high degree of correlation there is a good chance a single construct is influencing them (e.g., weight and height have a 0.7 degree of correlation; and they are influenced by a common construct: body mass). Two types: convergent validity and discriminant validity | Reliability | Is established when studies or measures produce consistent results in a variety of settings, times, or circumstances. | Temporal Reliability | This is reliability across time; in other words, will the measure produce the same results if measured at different time periods? | Representative Reliability | This reliability addresses whether an indicator produces the same results across different groups of people. | Population | A broad grouping of people from which researchers select a sample. In quantitative research, the sample is used to generalize finding to this larger group, the population. | Sample | A subset of a population that researchers use in order to study that population. | Elements | The units in a study, what I am collecting information about. They can be people (mother) or groups of people (families). | Sampling Frame | A list of all the possible elements from which your sample will be drawn. | Sampling Unit | Group of elements, usually in decreasing size, which researchers select in multistage sampling. When researchers draw only one sample, the sampling equals the element. | Sampling Error | The variability in results that arises as a result of simply taking a sample from a population. | Confidence Levels | How sure researchers are that their findings are not produced by chance or random error. | Confidence Intervals | The range of values within which researchers think the true population parameter lies. | Probability Sampling | Each member, or element, of the population has an equal chance (or probability) of being selected into the study, and because of this, researchers can estimate issues such as sampling error and confidence levels. Also defines simple random sample. | Central Limit Theorem | When a large number of samples of sufficiently large size are drawn, the means of those samples will distribute themselves along the normal curve, with the midpoint of this curve approaching the true population parameter. | Simple Random Sampling | Uses a table of random numbers to select each element or sampling unit. Each sampling unit has an equal probability of being selected. | Systematic Random Sampling | Uses a table of random numbers to select the first element or sampling unit and then selects every kth element or unit base on a calculated sampling interval. | Stratified Random Sampling | Before sampling selection, divides broad sample into subgroups called strata to improve the likelihood of adequate representation of these subgroups in the sample. Then uses either simple random or systematic sampling to select individual elements separately from the different strata. | Cluster Sampling | Addresses very large populations by randomly selecting sampling units, called clusters, of decreasing size until reaching sampling units from which manageable sampling frames of the desired elements can be obtained. | Nonprobability Sampling | In these samples, people do not have an equal, or known, probability of selection; therefore the findings cannot necessarily be generalized to a broader population. | Accidental Sampling | Simply selects people who happen to be “in the right place at the right time.” | Purposive Sampling | Selects elements that, based on the researchers expertise, seem like they would be appropriate for study of the research topic; however, these people are not selected using a table of random numbers. It is not a random sample and generalization is problematic. | Quota Sampling | Like the stratified random sampling, the researcher tries to make sure that the proportion of subgroups in the sample are the same as those in the population; however the elements are not selected using a table of random numbers. | Snowball Sampling | Sample elements are obtained by word-of-mouth or recommended by other sample elements. As participants recommend other participants for the study, the sample size grows or snowballs. It is not a random sample and generalization is problematic | Causation | Indicates that one event is the result of the occurrence of the other event; i.e. there is a causal relationship between the two events. This is also referred to as cause and effect or causality. | Time order relationship: | A change in the independent variable, the cause, has to clearly precede a change in the dependent variable, the effect. A must come before B. | Covariance (empirical association): | If there is going to be a causal relationship between two variable, a change in one variable (empirically, the independent variable) has to be associated with a change in the other variable (the dependent variable). Sometimes people refer to this criterion as a correlation, in that there has to be a correlation between the two variables. Essentially, correlation and association are the same idea; a correlation is really one statistical means of measuring an association. | Nonspurious relationship | Establishing that the observed relationship between two variables is not due to (or caused by) a third variable. The book calls these rival causal factors. | Research | The systematic investigation into and study of materials and sources in order to establish facts and reach new conclusions. The four purposes of research are: exploration, description, explanation, and evaluation. | Experiment (scientific) | To manipulate a situation (cause) and see if it changes the outcome of a behavior (effect). Scientific experiments are the best research designs for establishing causality because they are the easiest to control the three criteria for causality (time order relationship, covariance, and nonspuriousness. | Experimental Design | Is the design of any information-gathering exercises where variation is present, whether under the full control of the experimenter or not. In statistics, these terms are usually used for controlled experiments. Formal planned experimentation is often used in evaluating physical objects, chemical formulations, structures, components, and materials. | Hypothesis | A statement that makes a causal connection between concepts that will be tested with empirical observations (research) | Null Hypothesis | Refers to a general or default position: that there is no relationship between two measured variables, or that a treatment (intervention) has no effect. | Experimental Group | The group that experiences the treatment or intervention in an experiment. | Control Group | The group that does not receive a treatment or intervention in an experiment. It serves as a comparison to the treatment group. | Treatment | The intervention, which usually is the value of the independent variable. | Pretest | The first measure of the dependent variable, it happens before the experimental treatment. | Posttest | The second measure of the dependent variable, it happens after the experimental treatment. | Random Assignment | A condition in true experiments where the researcher randomly decides which subjects will be in treatment and control group. Each subject has an equal and known probability in being in any particular group. | True Experiment | This type of experiment has three general characteristics: it involves random assignment, has an experimental and a control group, and it commonly involves a pretest and a posttest. | Classic Experiment/Pretest, Posttest, Control Group Design | This type of experiment involves random assignment, has an experimental and a control group, and it involves a pretest and a posttest. Where R stands for random assignment, O is the observation collected at times 1 and 2, and X is the treatment.R O1 X O2R O1 O2 | Posttest-Only Control Group Design | This is the classic experiment minus the pretest. This step is omitted when researchers are concerned that a pretest would bias their results. There could be time constrains or concerns that the participants will “learn” elements of the treatment and affect the posttest results.R X OR O | Solomon Four-Group Design | This is also a true experiment and it is designed to combine the classic experiment with the posttest-only control group design to address the strengths and weaknesses of each of the individual designs.R O1 X O2R O1 O2R X O2R O2 | Non- Experiments | Designs that lack random assignment or that are shortcuts to gather information are called non-experiments, pre-experimental design or quasi-experimental designs. | One-Shot Case Study | This non-experimental design has only one group, one treatment, and only a posttest.X O | One-Group Pretest-Posttest Design | This non-experimental design has only one group, one treatment, a pretest and a posttest.O1 X O2 | Static Group Comparison | This non-experimental design has two groups but does not use random assignment, has one treatment, and a posttest.X O O | Time-Series Design | Research designs that collect data over long time intervals - before, during, and after an intervention, program implementation, or significant event. Basic notation: O X OMultiple observation notation: O O O X O O O | Multiple Time-Series Design | Research designs that collect data over long time intervals - before, during, and after an intervention, program implementation, or significant event. It includes a comparison group that is intended as a control group, however is very difficult to find a “true” control situation equivalent to the ones in a true experiment.O O O X O O OO O O O O O | Nonequivalent Control Group Design | In this non-experimental design there is no random assignment to groups, there is a pretest, a posttest, may have several treatments, and has a control group.O X1 OO X2 OO O | Ethical Issues in Research | The recognition that ethical guidelines, to some degree, specify what “ought” to happen in the course of social science research. Behavior that is ethical in one context may not be ethical in another. | Institutional Review Boards (IRB’s) | Review committee that is responsible for making sure that the benefits of its research projects cover any potential costs to the participants and that the procedures used in the research methodology include adequate safeguards to protect the identity, safety and general well-being of participants. | Informed Consent | Telling potential research subjects any and all information about the study that might influence their decision whether to participate. To achieve informed consent, researchers have to make sure they specify: The purpose of the research; any potential risks or harms the participants may experience; any benefits the participants may experience; the procedures used in the research; who is funding the research; incentives/payments for participations (if any); that participation in the research is voluntary; and that confidentiality or anonymity is guaranteed. | Anonymity | Guarantee given to a research subject (respondent in a survey) that the research will have no means of matching the respondent’s answers to his or her identity. | Confidentiality | An ethical issue where the researcher knows the identity of the respondent, but guarantees that he or she will not reveal it in any publication of the research. | Survey Research | Survey research is a method often used to assess thoughts, opinions, and feelings. Survey research can be specific and limited, or it can have more global, widespread goals. A survey consists of a predetermined set of questions that is given to a sample. | Descriptive surveys | Describes the population (e.g., 54% of San Antonio’s population is Hispanic, and 35% of them are of voting age.). They offer a snapshot of a current situation or condition. They often describe the respondent’s experiences and views. | Opinion surveys | Ask people’s opinions or beliefs (e.g., do you think the President is doing a good job?). | Self report of behavior | Ask people to report on their behavior (e.g., do you drink alcohol? Did you vote in the most recent election?). NOTE: recollective questions: people are better at reporting recent behavior than past behavior. | Push survey | Intended to get people to change their minds. | Survey Construction | Question wording is very important in survey construction; a researcher must be conscious of the audience and must avoid the following: confusing or vague wording, biased questions, leading questions, assuming prior knowledge on behalf of the participants, double-barreled questions, and response set patterns. Also, a researcher must be sure answer categories are mutually exclusive as well as exhaustive. | Cover letter | The letter included with a mailed survey. It contains the general information about the survey and research study that is necessary to obtain informed consent and encourage people to participate. | Leading Questions | Sometimes researchers intentionally or unintentionally word questions so as to elicit a specific response (e.g., should we pay even more taxes to assist the poor?). Leading questions can also occur when we try to get people to agree (e.g., should taxes be increased to assist poor families in giving better care to their children who are victims of their birth?). | Ambiguity In Words | Surveys should avoid using words that are capable of being understood in different ways by different respondents. | Vague Or Confusing Words | Can be vague or confusing if, 1) the questions are unnecessarily long or wordy, 2) researchers use terms that are highly subjective or indefinite and can therefore have different meaning to different people (e., regularly, sometimes, etc.), 3) researchers do not take into account the characteristics of their respondents, and 4) researchers use double negatives in wording their questions. | Biased Questions | Questions can be biased in a number of ways: 1) when a specific behavior is implied no matter how the respondent answers (e.g., do you still drink alcohol?), 2) when the question is worded in such a way as to elicit a certain response (e.g., do you believe it is okay for a man to beat his wife?), 3) when prestige is evoked (e.g., experts feel that Americans use too much gasoline. What is your opinion?), and 4) when categories do not reflect the full range of possible meanings. | Double-Barreled Questions | Single questions that actually ask two or more questions within them even though there is only one answer. This makes it impossible for a researcher to interpret someone’s answer (e.g., do you think a person convicted of child molestation should be sent to prison and forced to undergo counseling once he is released?). (NOTE: every instance of the word and in a question does not necessarily indicate a double-barreled question (e.g.,…a person who is beaten and robbed…). | Assuming Prior Knowledge | Do not ask questions that may involve knowledge that the average person may not know (e.g., do you think the government is spending too much money on the military?), 2) do not ask questions people will never be able to remember (e.g., how old were you when you were toilet trained?), 3) make sure the question makes sense to respondents (e.g., how many hours do you generally work in a week, not how many hours did you get paid for last year?). | Exhaustive | Researchers must be sure there is an answer choice for everyone. | Mutually Exclusive | Categories for which respondents can select one and only one response, unless otherwise directed. | Avoid Response Set | Response set occurs when all survey questions are written in the same direction, meaning that agreement or disagreement to all the questions shows the same view of the hypothesis. To avoid response set patterns, researchers need to word survey questions where positive answers to some questions and negative answers to other questions would illustrate the same opinion regarding the research topic. That is, strongly agree answers to some questions and strongly disagree answer to some other questions reflect the same opinion regarding the topic. | Open Vs. Closed Ended Questions | Open-ended questions have no response options presented for the respondents to select therefore they are “open” to write any responses they want. Coding and finding patterns among open-ended questions can be difficult. Closed-ended questions have specific answer categories for respondents to select. Open-ended questions tend to be a more expensive way to survey. The can be used to develop closed-ended questions later. | Avoid Threatening Questions | Those who find a question threatening are less likely to admit performing the action of that question. When respondents try to portray a positive image of themselves with regard to certain social behavior, researchers call this social desirability bias, which can affect the survey results. | Yes/No Vs. Scaled Questions | Some questions lend themselves to a yes/no answer while others are better suited for a scale or degree of agreement or disagreement. Of the yes/no options, circling the answer choice is seen as the most advantageous | Matrix Questions | Are efficient ways of presenting several items that share the same answer choices. The answer choices are listed once across the top of a matrix and the questions sharing those choices are listed down the page. Then, the respondent checks or circles a corresponding space or number for each choice that is lined up under the response categories listed at the top of the question set. | Order of questions | The order in which questions appear on a survey influences how respondents will react to the survey and whether they will continue with it, also how they may answer some of the questions. The first question should be answerable by everyone. Generally speaking, the survey questions should be organized into broad thematic categories. | Filter Questions | The first question in a series of questions that determines whether a respondent will continue to the next question or skip some questions and resume the survey at a later question. | Contingency Question | The questions after the filter question are called contingency questions, because they are only completed by those respondents who do not skip the questions following the filter question. | Survey Appearance | If a survey is attractive, clearly presented and easy to understand, respondents are more likely to complete it and interviewers are more likely to be able to follow it. Basic guidelines: 1) it should be easy to identify questions and answer choices, 2) there should be more space between different questions than between a particular question and its corresponding answer choices, and 3) answer choices look clearer if they are presented vertically and 4) questions and their corresponding answer choices should all appear on the same page. | Probes | When an interviewer asks further questions that try to guide the respondent to a certain level of information (doesn’t mean answering in a specific way....just another level of information). | Types of Surveys and Characteristics | Issue | Mail | Face to Face | Telephone | Electronic | Response Rate | Questionable | High | Moderate | Questionable | Cost | Cheap | Expensive | Cheap/Moderate | Cheap/Costly | Ease of Collection | Easy | Moderate/Hard | Easy | Easy/Moderate | Matching Data | Tedious | Manual/Tedious | Manual/Tedious | Easy/Automatic | Use of Probes | No | Yes | No | No | Compliance (who answers) | No | Yes | No | No | | | Quantitative Research | Research aimed at trying to find statistical patterns or trends in a population based on the observation of a sample. In this type of research, the researcher is the expert, the research is hypothesis and theory driven, concepts are clearly defined, there are systematic steps to facilitate replication, the data are primarily in the form of numbers, and the analysis involves statistical tests, usually for significance and association | Qualitative Research | Research aimed at an in-depth understanding of a social issue. It focuses on the nuances of behavior or experiences in a small group. In this type of research the participant is the expert, the research is driven by broad research questions, the focus is not on concept definition but on the identification of themes, there are no systematic research steps, replication is not a strong goal, the data is primarily in the form of paraphrased or quoted participants explanations, and the analysis is in the form of general patterns or trends. Any numerical analysis is descriptive with no detailed statistical test. | Content Analysis | Content analysis is utilized in the social sciences for studying the content of communication; it makes inferences about the message or the audience of the message and can be applied to examine any piece of writing or occurrence of recorded communication. It uses quantitative methods such as looking at word frequencies, space measurements, time counts, and keyword frequencies. | Meta-Analysis | Method focused on contrasting and combining results from different studies, to identify patterns among study results, sources of disagreement among those results, or other relationships that may come to light in the context of multiple studies. It identifies a common measure of effect size, and aims at providing a weighted average output. | Statistical Significance | Statistical significance is a statistical assessment of whether observations reflect a pattern rather than just chance. When used in statistics, the word significant does not mean important or meaningful, as it does in everyday speech. Significance deals with the p value. The choice of significance level is somewhat arbitrary, but for many applications, a level of 5% or a p < .05 is chosen by convention. | p-value | The p-value refers to the probability that the results of your sample were arrived at by chance. In scientific studies, p values are used to determine whether a null hypothesis formulated before the performance of the study is to be accepted or rejected. Common p values are p < .05 and p < .01 | T-Test | Statistical test of the difference of means for two groups. The dependent variable is continuous, the independent variable is categorical and there are only two groups. | ANOVA | Analysis of Variance is used when there are multiple categorical independent variables but only one continuous dependent variable. (uses the f test) | ANCOVA | Analysis of Covariance Answers the question: are the means different? Multiple independent variables, at least one is continuous, and there is one continuous dependent variable (uses the f test). | MANOVA | Multivariate Analysis of Variance is used when there are multiple categorical independent variables, as well as multiple independent variables. (does not use the f test. Uses other kinds) | MANCOVA | Multivariate Analysis of Covariance Multiple independent variables, at least one is continuous, and multiple dependent variables (does not use the f test. Uses other kinds) | Correlation | Determines the relationship between two variables obtained from the same set of cases. It can range from -1.00 through 0.00 to +1.00. It is the ratio of how much one (standardized) variable’s changes coincide with the changes in the other (standardized) variables. Both the independent and dependent variables are continuous. Correlation is a measure of association and indicates how strong and which direction exists in the relationship (related term: Pearson product moment correlation or the correlation coefficient) | Post Hoc Analysis | After the researcher finds significance in relationships (e.g., from MANOVA/MANCOVA), this analysis allows the researcher to find out why “significances” showed up in the original tests. | Factor Analysis | Is a search for the latent variables that create the observed variables. Factor analysis is the name of a series of techniques introduced by Charles Spearman in the early 20th Century. (“the search for latent variables”). Factor Analysis is the name of a series of techniques used in this process. | Exploratory Factor Analysis | Says this latent factor is related to this observed variable. The researcher explores data and tries to determine relationships between latent and observable variables. The problem is, although you get an answer, you don’t know if the answer is accurate. | Confirmatory Factor Analysis | The researcher determines what is to be studied and gives his/her theory about which latent variables affect which observable variables. (e.g., I state that these are the latent factors which affect X and that is what I investigate). |