Chapter 1

Introduction to
Educational Research

LEARNING OBJECTIVES
To be able to
Explain the importance of educational research.
List at least five areas of educational research.
Explain the difference between basic and applied research.
Describe evaluation research, action research, and critical theory research.
Discuss the different sources of knowledge.

Visit the study site for an interactive concept map.

Explain the scientific approach to knowledge generation.
Explain how to determine the quality of a theory or explanation.
List the five objectives of educational research and provide an example of each.

2

RESEARCH IN REAL LIFE Research Aids Decision Making
In June 2002, New York Governor George Pataki signed a state law giving Mayor
Michael Bloomberg control of New York City’s public school system. Most observers agree that this is a school system desperately in need of reform. The 1,100 schools within this system educate 1.1 million kids. However, using the word educate would seem to be somewhat of a misnomer because only about half of the city’s public school students finish high school in 4 years. Only 40 percent of third- through eighth-grade students score at an acceptable level in reading, and only 34 percent do so in math.
About 100 of the 1,100 schools are classified by the state as failing, and another
300 are almost as bad. Clearly, something needs to be done.
While campaigning for mayor, Michael Bloomberg had many ideas, one of which was to establish an unpaid Board of Education that functioned like a corporate board, providing fiscal oversight and expertise. This idea was approved by new legislation and was a radical departure from the old Board of Education, which was responsible for day-to-day management decisions, including even routine contracting and procurement decisions.
Bloomberg needs to do a lot more than just reconstitute the Board of Education because there is no single panacea for all of the problems facing the New York City school system. There is no shortage of ideas to assist Bloomberg in this process. Coles
(2002) wrote an article in the City Journal giving his opinion as to what should be done. He stated that Bloomberg should choose a chancellor from outside the system so that he or she would not be constrained by existing relationships or vested interests. A uniform core curriculum should be established that focuses on basic skills, particularly in the elementary and middle schools. Social promotion should end. Finally, the best teachers should be rewarded, contended Coles, because fully 40 percent of the city’s teachers failed the basic teacher certification test.
Given differing opinions about what should be done with a school system such as New York City’s, which ideas do you think should be implemented? Which ones would provide the best return on capital expenditures and best help students? Obviously, there are many differing philosophies and many differing opinions. However, we contend that policymakers will benefit if they examine the findings of educational research studies comparing the outcomes resulting from implementing different ideas and approaches.
This will help eliminate personal bias and vested interests in particular approaches by providing strong evidence of what really works best. In short, research provides an effective and evidentiary way to help sort out and resolve differing ideas and opinions on educational issues. Perhaps our most important goal in writing this book is to convince you that it is important and helpful to add the examination and conduct of research to your list of ingredients to use when making decisions about education.

W

elcome to the world of educational research! Research has been conducted in virtually every area in the field of education.In fact,the research techniques described in this book are used all over the world to help people in many fields advance their knowledge and solve problems.The search for better and better answers to important questions will probably always continue. In this book, we discuss the way in which research is conducted in an attempt to provide answers to important questions.We hope you will enjoy learning about research, and we hope it opens up new ways of thinking for you.
As you read this book,you will learn how to think about research,how to evaluate the quality of published research reports,and how to conduct research on your own.In a sense,you will
3

4

PA R T I : I N T R O D U C T I O N

also be learning a new language, the language of researchers, because researchers use a specialized language or jargon.But remember,don’t be afraid of new words.The words used in this book have definitions that represent ideas you can understand,and you have been learning new words and ideas all of your life.On the lighter side,perhaps you can use some of the new words to impress your friends. In sum, we welcome you to the world of research and hope that you will enjoy it. Because this is likely to be a required course for you, we begin by discussing a few reasons for taking a course on educational research methods.

WHY STUDY EDUCATIONAL RESEARCH?
You might have asked,“Why do I have to take a class on educational research?”First of all,research can be more interesting than you might think. We hope that in time you will find the material and the ways of thinking both interesting and beneficial.Second,throughout this book,you will be learning critical thinking skills.Rather than assuming that what is written in a book or what someone says is “fact” or undeniable “truth,” you can use the techniques that you will learn for evaluating arguments.In all cases,the question is one of evidence.As a start,we suggest that you take the word proof and eliminate it from your vocabulary this semester or quarter when you talk about research results. Proof exists in the realms of mathematics and deductive logic, but in science and research, the best we can do is to provide evidence. Sometimes the evidence is very convincing; at other times,it might not be.You must use your critical thinking skills to judge the available evidence on any given topic. These critical thinking skills will be helpful in your studies and professional work as long as you live.Learning about research methods should help sharpen your critical thinking skills.
Another important reason to study research is to help you better understand discussions of research you hear and see in the media, such as television and radio, on the Internet, or at professional meetings. Examples of research in our society abound. For example, when you watch a television program, what comes in between those short segments of actual programming? Commercials! Do you ever wonder about those “research studies” that claim to “prove” that one laundry detergent is better than another? As you know, the purpose of commercials is to influence your buying behavior.Advertisers spend millions of dollars each year on marketing research to understand your thinking and behavior. If you watch a sporting event, you will likely see commercials for beer, cars, trucks, food, and tennis shoes. If you watch soap operas in the afternoon, you are likely to see very different commercials. The reason for this variation is that advertisers generally know who is watching what programs at which times. The actual commercials are developed to appeal to viewers’ ways of thinking about what is fun, exciting, and important.And did you know that every major presidential candidate has a research consultant who tries to identify the most effective ways to get your vote and win the election? The point is that other people study you all the time and in this book you will learn about the techniques they use.Understanding these techniques should help you be more aware of their efforts.
You will learn here that not all research is created equal. That is, some research studies are more defensible than others.You will learn how to ask the right questions about research studies, and you will find out when to put confidence in a set of research findings.You will learn to ask questions such as these:Was the study an experiment,or was it a nonexperimental research study? Were control groups included in the design? Did the researcher randomly assign participants to the different comparison groups? How did the researchers control for the influence of extraneous variables? How were the participants in the research selected? Did the researcher use techniques that help reduce the effects of human bias?

C h a p t e r 1: I n t ro d u c t i o n t o E d u c a t i o n a l R e s e a r c h

One day you might need to examine the research on a topic and make an informed judgment about what course of action to take or to recommend to someone else. Therefore, it is important that you understand how to review and evaluate research. Understanding research terminology, the characteristics of the different types of research, and how research can be designed to provide solid evidence will allow you to evaluate research results critically and make informed decisions based on research literatures.A research literature is the set of published research studies on a particular topic. A fundamental point to remember is that you should always place more confidence in a research finding when several different researchers in different places and settings have found the same result.You should never treat a single research study as the final word on any topic.
On a practical level,understanding research techniques might even help you in your career as a student and as a professional teacher, counselor, or coach. Perhaps one day you will be asked to write a proposal to obtain a grant or conduct a research study on your own. If you study the contents of this book, you will learn how to design and conduct a defensible study, and you will learn about the different sections in a research grant proposal.You will learn how to construct a questionnaire and how to write a proposal. Furthermore, if you look at the bibliographies in the books you use in your other education courses, you will see that many of these references are research studies.After learning about research,you will be able to go back and evaluate the research studies on which your textbooks are based. In other words, you will not have to accept something as true just because someone said it was true. You might find that an article with what you believe to be a questionable finding is based on highly questionable research strategies.

1.1 Why should we study educational research?

Research literature Set of published research studies on a particular topic

REVIEW
QUESTION

AREAS OF EDUCATIONAL RESEARCH
To give you a feel for educational research, let’s look at some of the areas of research in education. In Table 1.1 you will find a list of the major divisions and the special interest areas in the
American Educational Research Association (AERA). (The AERA website is at http://aera.net.)
The AERA is the largest and most prestigious research association in the field of education, and it has approximately 25,000 members. It is composed of university professors from all areas of education; governmental employees; teachers; and professionals from educational think tanks, consulting firms, and testing companies. Each year, approximately 11,000 of these members and many nonmembers attend a national conference sponsored by the AERA, where many attendees present the results of their latest research.
You can see in Table 1.1 that education is a broad field that includes many different research areas. Do you see any areas of research in Table 1.1 that seem especially interesting? If you are writing a research paper, you might pick one of these areas as your starting point. The areas of research listed in Table 1.1 are still fairly general, however. To see the specific areas and topics of current interest to educational researchers, go to the library and browse through the education journals.

5

6

PA R T I : I N T R O D U C T I O N

TABLE 1.1 Divisions and Special Interest Groups in the American Educational Research Association*
Major Divisions in the AERA
Division A: Administration, Organization, & Leadership
Division B: Curriculum Studies
Division C: Learning & Instruction
Division D: Measurement & Research Methodology
Division E: Counseling & Human Development
Division F: History & Historiography

Division G: Social Context of Education
Division H: Research, Evaluation, & Assessment in Schools
Division I: Education in the Professions
Division J: Postsecondary Education
Division K: Teaching & Teacher Education
Division L: Educational Policy & Politics

Special Interest Groups in the AERA (called SIGs)
Academic Audit Research in Teacher Education
Action Research
Adolescence and Youth Development
Adult Literacy and Adult Education
Adulthood and Aging
Advanced Studies of National Databases
Advanced Technologies for Learning
Applied Research in Virtual Environments for Learning
Arts and Inquiry in the Visual and Performing Arts in Education
Arts and Learning
Arts-Based Educational Research
Associates for Research on Private Education
Bilingual Education Research
Biographical and Documentary Research
Brain, Neurosciences, and Education
Business Education & Computer Information Systems Research
Career and Technical Education
Career Development
Caribbean and African Studies in Education
Catholic Education
Chaos & Complexity Theories
Charter School Research and Evaluation
Classroom Assessment
Classroom Management
Classroom Observation
Cognition and Assessment
Communication of Research
Computer and Internet Applications in Education
Conflict Resolution and Violence Prevention
Confluent Education
Confucianism, Taoism, and Education
Constructivist Theory, Research, and Practice
Cooperative Learning: Theory, Research, and Practice
Critical Educators for Social Justice
Critical Examination of Race, Ethnicity, Class, and
Gender in Education
Critical Issues in Curriculum and Cultural Studies
Critical Perspectives on Early Childhood Education
Cultural Historical Research
Democratic Citizenship in Education
Design and Technology
Disability Studies in Education

Districts in Research and Reform
Doctoral Education across the Disciplines
Early Education and Child Development
Ecological and Environmental Education
Education and Philanthropy
Education and Student Development in Cities
Education, Health, and Human Services Linkages
Educational Change
Educational Statisticians
Faculty Teaching, Evaluation, and Development
Family and Consumer Sciences
Family, School, Community Partnerships
Fiscal Issues, Policy, and Education Finance
Foucault and Contemporary Theory in Education
Grassroots Community & Youth Organizing for
Education Reform
Hierarchical Linear Modeling
Hispanic Research Issues
Holistic Education
Inclusion & Accommodation in Educational Assessment
Indigenous Peoples of the Americas
Indigenous Peoples of the Pacific
Informal Learning Environments Research
Instructional Technology
International Studies
Invitational Learning
Ivan Illich
John Dewey Society
Language and Social Processes
Large Scale Assessment
Law and Education
Leadership for School Improvement
Leadership for Social Justice
Learning and Teaching in Educational Leadership
Learning Environments
Learning Sciences
Literature
Lives of Teachers
Longitudinal Studies
Marxian Analysis of Society, Schools, and Education
Measurement Services
Media, Culture, and Curriculum

C h a p t e r 1: I n t ro d u c t i o n t o E d u c a t i o n a l R e s e a r c h
Mentorship and Mentoring Practices
Middle-Level Education Research
Mixed Methods Research
Moral Development and Education
Motivation in Education
Multicultural/Multiethnic Education: Research, Theory, and Practice
Multiple Intelligences: Theory and Practice
Multiple Linear Regression: The General Linear Model
Music Education
NAEP Studies
Narrative Research
Online Teaching and Learning
Organizational Theory
Out-of-School Time
Paulo Freire, Critical Pedagogy, and Emancipation
Peace Education
Philosophical Studies in Education
Politics of Education
Portfolios and Reflection in Teaching and Teacher Education
Postcolonial Studies and Education
Problem-Based Education
Professional Development School Research
Professional Licensure and Certification
Professors of Educational Research
Qualitative Research
Queer Studies
Rasch Measurement
Religion and Education
Research, Education, Information, and School Libraries
Research Focus on Black Education
Research Focus on Education and Sport
Research in Global Child Advocacy
Research in Mathematics Education
Research in Reading and Literacy
Research in Social Studies Education
Research on Evaluation
Research on Giftedness, Creativity, and Talent
Research on Learning and Instruction in Physical Education
Research on Teacher Induction
Research on the Education of Asian and Pacific Americans
Research on the Education of Deaf Persons
Research on the Superintendency

7

Research on Women and Education
Research Use
Rural Education
Safe Schools and Communities
School Choice
School Community, Climate, and Culture
School Effectiveness and School Improvement
School Indicators, Profiles, and Accountability
School Turnaround and Reform
School/University Collaborative Research
Science Teaching and Learning
Second Language Research
Self-Study of Teacher Education Practices
Semiotics in Education
Service-Learning & Experiential Education
Social and Emotional Learning
Society of Professors of Education
Sociology of Education
Special Education Research
Spirituality & Education
State and Regional Educational Research Associations
Stress and Coping in Education
Structural Equation Modeling
Studying and Self-Regulated Learning
Supervision and Instructional Leadership
Survey Research in Education
Systems Thinking in Education
Talent Development of Students Placed at Risk
Teacher as Researcher
Teacher’s Work/Teachers Unions
Teaching Educational Psychology
Teaching History
Technology as an Agent of Change in Teaching and Learning
Technology, Instruction, Cognition, & Learning
Test Validity Research and Evaluation
Tracking and Detracking
Urban Learning, Teaching, and Research
Vocabulary
Workplace Learning
Writing and Literacies

*For more information about any of these divisions or special interest groups, go to the AERA website at http://aera.net.

EXAMPLES OF EDUCATIONAL RESEARCH
The majority of journal articles in education include an abstract on the front page of the article. An abstract is a brief summary of what is included in the article. We have reproduced the abstracts of several research articles here so that you can get a feel for what is done in an

Abstract
Brief summary of what is in an article

8

PA R T I : I N T R O D U C T I O N

See study site for journal articles.

actual research study. Abstracts are helpful because they are short and include the main ideas of the research study. You can often decide whether you want to read a journal article by first reading its abstract. We recommend that you read some full-length research articles as soon as possible so that you will see some full examples of educational research. Throughout this book, we will be putting an icon in the margin telling you to go to the companion website to examine a relevant journal article. You can see the journal article icon right now in the margin. The next time you see it, it will be referring you to a full-length article to download at your convenience.
Now, just read the following abstracts and see if you can determine (a) the purpose of the study,
(b) how the researchers studied the phenomenon, and (c) what the major results were.
I. The Development of a Goal to Become a Teacher, by Paul A. Schutz (University of Georgia),
Kristen C. Croder (University of Georgia), and Victoria E. White (University of North Carolina at Greensboro), 2001, from Journal of Educational Psychology, 93(2), pp. 299–308.
The purpose of this project was to investigate how the goal of becoming a teacher emerges.
The study used interviews to develop goal histories for 8 preservice teachers. There tended to be 4 sources of influence for their goal to become a teacher: (a) family influences, (b) teacher influences, (c) peer influences, and (d) teaching experiences. The categories developed from the interviews to describe the types of influences those sources provided were (a) suggesting that the person become a teacher, (b) encouraging the person to become a teacher, (c) modeling teacher behavior, (d) exposing the person to teaching experiences, and (e) discouraging the person from becoming a teacher. In addition, influences such as critical incidents, emotions, and social-historical factors, such as the status and pay of teachers, were prominent in the goal histories of the participants. Finally, the results of the study are discussed within the context of goals and self-directed behavior.
II. Getting Tough? The Impact of High School Graduation Exams, by Brian A. Jacob at John F.
Kennedy School of Government, Harvard University, 2001, from Educational Evaluation and
Policy Analysis, 23(3), pp. 99–121.
The impact of high school graduation exams on student achievement and dropout rates is examined. Using data from the National Educational Longitudinal Survey (NELS), this analysis is able to control for prior student achievement and a variety of other student, school, and state characteristics. It was found that graduation tests have no significant impact on 12th-grade math or reading achievement. These results are robust with a variety of specification checks.
Although graduation tests have no appreciable effect on the probability of dropping out for the average student, they increase the probability of dropping out among the lowest ability students.
These results suggest that policymakers would be well advised to rethink current test policies.
III. Giving Voice to High School Students: Pressure and Boredom, Ya Know What I’m Saying? by
Edwin Farrell, George Peguero, Rashed Lindsey, and Ronald White, 1988, from American
Education Research Journal, 25(4), pp. 489–502.
The concerns of students identified as at-risk of dropping out of school in an urban setting were studied using innovative ethnographic methods. Students from the subject population were hired to act as collaborators rather than informants and to collect taped dialogues between themselves and their peers. As collaborators, they also participated in the analysis of data and contributed to identifying the research questions of the inquiry. Data indicated that pressure and boredom were most often mentioned as negative factors in the lives of the students, with pressure emanating from social forces outside of school but contributing to boredom inside.

C h a p t e r 1: I n t ro d u c t i o n t o E d u c a t i o n a l R e s e a r c h

9

GENERAL KINDS OF RESEARCH
In this section we introduce you to some of the general kinds of research conducted by educational researchers (see Table 1.2). Although these general research types can overlap at times, they have different purposes and are intended for different audiences.
TABLE 1.2 Summary of General Kinds of Research
Kind of Research

Key Characteristics

Basic research

Focuses on generating fundamental knowledge.

Applied research

Focuses on real-world questions and applications.

Evaluation research

Focuses on determining the worth, merit, or quality of intervention programs.

Action research

Focuses on solving local problems that practitioners face.

Orientational research

Focuses on reducing inequality and giving voice to the disadvantaged.

Basic and Applied Research
Research studies can be placed along a continuum with the words basic research at one end and the words applied research at the other end. The word mixed can be placed in the center to represent research that has characteristics of both basic and applied research. Basic research and applied research are typically conducted by researchers at universities. Basic research and applied research are also conducted by researchers working for think tanks, corporations, government agencies, and foundations. The primary outlet for basic and applied research is academic and professional research journals.
Basic research is aimed at generating fundamental knowledge and theoretical understanding about basic human and other natural processes. An example of basic research is a study examining the effect of priming in memory. Priming is “an enhancement of the processing of a stimulus as a function of prior exposure” (Anderson, 1995, p. 459). Assume that a researcher asks you to name a fruit and you say, “Pineapple.” Then on the second trial, the researcher either asks you to name another type of fruit or asks you to name a type of dog. Which response do you think you could provide more quickly? It turns out that research participants could name another type of fruit faster than they could name a type of dog when they were asked to name a type of fruit first
(Loftus, cited in Anderson). The naming of the fruit on the first trial primed the research participants’ mental processing to name another fruit. It is believed that priming operates because the first exposure activates the complex of neurons in long-term memory, where the concept is being stored. Basic research is usually conducted by using the most rigorous research methods (e.g., experimental) under tightly controlled laboratory conditions. The primary audience includes the other researchers in the research area. The key purpose of basic research is to develop a solid foundation of reliable and fundamental knowledge and theory on which future research can be built.
At the other end of the continuum is applied research. Applied research is focused on answering real-world, practical questions to provide relatively immediate solutions. Topics for applied research are often driven by current problems in education and by policymakers’ concerns.Applied research is often conducted in more natural settings (i.e., more realistic or real-world settings) than basic research. An applied research study might focus on the effects of retaining low-performing

Basic research
Research aimed at generating fundamental knowledge and theoretical understanding about basic human and other natural processes Applied research
Research focused on answering practical questions to provide relatively immediate solutions

10

PA R T I : I N T R O D U C T I O N

elementary school students in their present grade level or on the relative effectiveness of two approaches to counseling (e.g., behavior therapy versus cognitive therapy). In the former, the results would potentially have practical implications for education policy; in the latter, the results would potentially have implications for practicing counselors. The primary audiences for applied research are other applied researchers (who read the results in educational research journals) as well as policymakers, directors, and managers of programs who also read research journals.Applied research often leads to the development of interventions and programs aimed at improving societal conditions, which leads us to the next type of research.

Evaluation Research
See Journal Article 1.1 on the study site.
Evaluation
Determining the worth, merit, or quality of an evaluation object
Formative
evaluation
Evaluation focused on improving the evaluation object
Summative
evaluation
Evaluation focused on determining the overall effectiveness and usefulness of the evaluation object

When interventions and social or educational programs aimed at improving various conditions are implemented, evaluation research is often carried out to determine how well the programs work in real-world settings and to show how they might be improved. Evaluation research, or, more simply, evaluation, specifically involves determining the worth, merit, or quality of an evaluation object, such as an educational program. Evaluation requires evaluators to make value judgments about evaluation objects (e.g., Program XYZ is a good program, and it should be continued; Program ABC is a bad program, and it should be discontinued). An evaluation object (also called the evaluand) is the thing being evaluated: a program, a person, or a product (Guba & Lincoln, 1981; Scriven, 1967; Worthen, Sanders, & Fitzpatrick, 1997). An educational program might be an afterschool program for students with behavioral problems or a new curriculum at school. A person might be your new school district superintendent. A product might be a new textbook or a new piece of equipment that a school is considering purchasing.
Evaluation traditionally is subdivided into two types according to the purpose of the evaluation. When the primary purpose of an evaluation is to lead to judgments about how a program can be improved, it is called a formative evaluation. Formative evaluation information helps program developers and support staff design, implement, and improve their program so that it works well. When the primary purpose of an evaluation is to lead to judgments about whether a program is effective and whether it should be continued, it is called a summative evaluation. Summative evaluation information is important for policymakers and others who commission programs when they make funding decisions and when they have to make choices about which competing programs will be supported and which will be eliminated.
It is currently popular to divide evaluation into five areas or types (e.g., Rossi, Lipsey, &
Freeman, 2004), each of which is based on a fundamental evaluation question:
1. Needs assessment: Is there a need for this type of program?
2. Theory assessment: Is this program conceptualized in a way that it should work?
3. Implementation assessment: Was this program implemented properly and according to the program plan?
4. Impact assessment: Did this program have an impact on its intended targets?
5. Efficiency assessment: Is this program cost-effective?

As you can see, evaluation can provide important information to educators. On the basis of the evidence collected and the recommendations made, program evaluators provide an important voice in decision making about educational and other social programs.

C h a p t e r 1: I n t ro d u c t i o n t o E d u c a t i o n a l R e s e a r c h

11

Action Research
Action research is focused on solving specific problems that local practitioners face in their schools and communities (Lewin, 1946; Stringer, 1996). It views your classroom or other work environment as a place to conduct research. Also, action research is based on the idea that having the “researcher attitude” is helpful in dealing with your complex and changing environments; this attitude involves continuously identifying new problems that you want to work on and trying new actions to see what improves the situation. Action research thoroughly integrates theory and research with practice, and we hope all of our readers will take the attitude of the “action researcher” as they go about their professional careers.
In action research, practitioners in the local settings design and conduct the research study.
Action research follows the research principles that are explained in depth in this book. A key difference between action research and basic and applied research is that rather than being conducted by academic researchers interested in the development of generalizable practices and theory to be published in academic journals, action research is focused more on local practice and local solutions; it is more participatory; and it is conducted by teachers, administrators, counselors, coaches, and other educational professionals (sometimes in collaboration with university-based researchers) trying to solve their very specific problems. Some examples are a principal studying teacher burnout and dissatisfaction in a local school context, a group of teachers studying classroom discipline problems in their unique classrooms, teachers and administrators studying the lack of parental involvement with their school’s PTA, and a teacher studying a particular problem child in a particular classroom.
To carry out an action research project, you would need to diagnose the specific problem you are facing and conduct a thorough literature review to see if a useful answer already exists.
If the literature doesn’t suffice, then you will need to plan and carry out your own research study
(i.e., collect data to help answer your question) in your environment with your students or clients.
When you finish interpreting your results,you can implement the changes needed to help solve your local problem.
Action research is a never-ending process because most problems are not fully solved through a single research study. Many school districts have departments that are specifically set up to conduct research about the local schools. Also, many teachers and administrators now attempt to collect data about the problems they face, and they learn from their past attempts at problem solving. Action research is a state of mind. It requires a positive attitude toward research, where the practitioner (teacher, counselor, coach) keeps up-to-date on published research and tries new practices and observes the results. Action research makes research local, helping solve problems in your particular situation or context. It requires a commitment to lifelong individual and organizational learning and to reading published research as well as conducting local research. Your actions and practices will be based on these results, but the process of learning through research will continue because the present is always your new starting point.
It is especially useful when you and many of your coworkers are all interested in conducting local research; brainstorming, critiquing, and improving each other’s work; and forming an action research culture. If the results of your action research offer promising solutions, be sure to share your findings with others, including other practitioners in your local place of work, as well as making presentations at local, regional, or even national conferences. You also should consider publishing your findings in newsletters and journals. Sharing action research results with the larger educational research community will add to the larger body of knowledge.

Action research
Applied research focused on solving practitioners’ local problems See Journal Article 1.2 on the study site.

12

PA R T I : I N T R O D U C T I O N

Orientational Research
Orientational
research
Research explicitly done for the purpose of advancing an ideological position or orientation

See Journal Article 1.3 on the study site.

The last general type of research, called orientational research, focuses on collecting information to help a researcher advance a specific ideological or political position or orientation that he or she believes will improve some part of our society (e.g., Sandoval, 2000; Smith, 2008).
Orientational research also focuses on “giving voice” and increased power to the disadvantaged in society. Orientational researchers are concerned about such issues as social discrimination and the inequitable distribution of power and wealth in society. Although all orientational researchers are concerned with reducing inequality of some form, there are several variants of orientational research. The most common areas of focus are class stratification (i.e., income and wealth inequality), gender inequality, racial and ethnic inequality, sexual orientation inequality, and international inequality (i.e., rich and poor nations).
All researchers are ideological to some degree (e.g., in their selection of their research topics, in the recommendations they make), but orientational researchers make their ideology and political agendas very explicit. Orientational research is sometimes called critical theory research
(Anyon, 2009). This is appropriate because these researchers often are critical of “mainstream research,” which they argue supports the current power structure in society. If orientational research sounds interesting, you will find a wealth of information on the web (using search terms such as critical theory, ethnic studies, feminism, postcolonialism, and queer theory).

1.2 What are the definitions of the five general kinds of research?
1.3 Why is it important that both basic and applied research be done?

REVIEW
QUESTIONS

1.4 What is the difference between formative and summative evaluation?
1.5 What is the key question associated with each of the following forms of evaluation: needs assessment, theory assessment, implementation assessment, impact assessment, and efficiency assessment? SOURCES OF KNOWLEDGE

Epistemology
The theory of knowledge and its justification Take a moment now to consider how you have learned about the world around you. Try to identify the source or sources of one of your particular beliefs (e.g., parents, friends, books, tradition, culture, thinking, experiences). For example, consider your political party identification (i.e., Democrat, Republican, Independent, or something else). Political scientists have shown that college students’ party identification can often be predicted by their parents’ party identification. How does your party identification compare with that of your parents?
Obviously, there are many additional influences that affect party identification. Can you identify some of them?
In this section, we examine the primary ways in which people relate to the world and how they generate knowledge. The study of knowledge, including its nature, how it is gained or generated, how it is warranted, and the standards that are used to judge its adequacy, is known as epistemology. Epistemology sometimes is called the “theory of knowledge.” We group the sources of knowledge into the primary areas discussed in the field of epistemology. C h a p t e r 1: I n t ro d u c t i o n t o E d u c a t i o n a l R e s e a r c h

13

Experience
Empiricism is the idea that all knowledge comes from experience. We learn by observing, and when we observe, we rely on our sense perception. Each day of our lives, we look, feel, hear, smell, and taste so that we can understand our surroundings. According to the philosophical doctrine of empiricism, what we observe with our senses is said to be true. John Locke (1632–
1704), a proponent of this idea, said that our mind at birth is a tabula rasa, a blank slate ready to be written on by our environment. Throughout our lives, our slate is filled up with knowledge based on our experiences. The statement “I know the car is blue because I saw it” is an example of an empirical statement: a statement based on observation, experiment, or experience. Empirical is a fancy word meaning “based on observation, experiment, or experience.”
The word empirical denotes that a statement is capable of being verified or disproved by observation, experiment, or experience. In the next paragraph, we try to trace some of the sources of experiences you might have had during your lifetime.
Throughout our lives, we participate in and learn about the world around us. We interact with people and generate our personal knowledge. In the beginning, we are born at a certain time, in a certain place, into a specific family that uses a specific language. When we are young, our family is the most important source of our knowledge, our attitudes, and our values. As we grow older, other people and social institutions around us—including our peers, our religion, our schools (and libraries), our economy, our government, and the various media we are exposed to or seek out—influence us more and more. We learn the customs, beliefs, and traditions of the people around us. As we learn “how things are,” we construct our personal knowledge and viewpoints about our worlds. Over time, many of our actions and beliefs become automatic and unquestioned.

Empiricism
The idea that knowledge comes from experience
Empirical
statement
A statement based on observation, experiment, or experience Reasoning
Rationalism is the philosophical idea that reason is the primary source of knowledge. One famous rationalist philosopher was René Descartes (1596–1650). Reason involves thinking about something and developing an understanding of it through reasoning. In its strong form, rationalism means that many truths are knowable independent of observation. In its weaker form, rationalism simply refers to our use of reason in developing understandings about the world. Deductive reasoning and inductive reasoning are the two major kinds of reasoning.
Deductive reasoning is the process of drawing a conclusion that is necessarily true if the premises are true. One form of deductive reasoning is the syllogism. Here is an example:
Major Premise: All schoolteachers are mortal.
Minor Premise: John is a schoolteacher.
Conclusion: Therefore, John is mortal.

According to this deductive argument, John necessarily is a mortal. Keep in mind, however, that reasoning like this depends on the validity of the premises. Just try replacing the word mortal with the word Martian; you then conclude that John is a Martian. Deductive reasoning is useful as we reason about things in our world, but we must always make sure that our premises are true, and we must use valid argument forms. We need to be careful about what we assume when we draw our conclusions.

Rationalism
The philosophical idea that reason is the primary source of knowledge

Deductive reasoning The process of drawing a conclusion that is necessarily true if the premises are true 14

PA R T I : I N T R O D U C T I O N

Inductive reasoning The process of drawing a conclusion that is
“probably” true
Probabilistic
Stating what is likely to occur, not what will necessarily occur

Problem of induction The future might not resemble the past REVIEW
QUESTIONS

Inductive reasoning is the form of reasoning in which the premises “provide good reasons, but not conclusive reasons to accept the conclusion” (Salmon, 2007, p. 79). We engage in inductive reasoning frequently in our everyday lives when we observe many specific instances of some phenomenon and draw conclusions about it. For example, you have certainly observed all of your life that the sun appears every morning (except on cloudy days). On the basis of your observations, you probably feel comfortable in concluding that the sun will make its appearance again tomorrow (if it is not cloudy). In this case, you are indeed likely to be correct. But notice that, when you use inductive reasoning, you are using a probabilistic form of reasoning. That is, you are stating what is likely to occur, not what will necessarily occur. Because of this, you are taking a risk (albeit a very small risk in this case) because induction involves making conclusions that go beyond the evidence in the premises (e.g., going from some to more, from the examined to the unexamined, from the observed to the unobserved). This is not necessarily a problem, but you should be aware that it could be one if you expect certainty about your conclusions.
The famous philosopher named David Hume (1711–1776) pointed out what is called the problem of induction: Although something might have happened many times in the past, it is still possible that it will not happen in the future. In short, the future might not resemble the past. Let’s say that every cat you have ever seen had a tail. Using inductive reasoning, you might be led to conclude that all cats have tails. You can see the problem here: One day you might run across a Manx cat, which has no tail. The point is that inductive reasoning is useful in helping us come up with useful conclusions, predictions, and generalizations about the world; however, we must remember that we have not proven these to be true. Induction only provides statements of probability.

1.6 What are the different sources of knowledge? Which ones are especially important for educational research?
1.7 What is the key difference between inductive reasoning and deductive reasoning?

THE SCIENTIFIC APPROACH
TO KNOWLEDGE GENERATION

Science
An approach for the generation of knowledge Although the word science has become a hot button or loaded word in some circles, the root of the word is the Latin scientia, which simply means “knowledge.” We define science in this book in a way that is inclusive of the different approaches to educational research. We define it as an approach for the generation of knowledge that holds empirical data in high regard and follows certain norms and practices that developed over time because of their usefulness. Many of these norms and effective practices are explained in this book.
Science includes any systematic or carefully done actions that are carried out to answer research questions or meet other needs of a developing research domain (e.g., describing things, exploring, experimenting, explaining, predicting). Science often involves the application of a scientific method; however, as philosophers and historians of science have pointed out, science includes many methods and activities that are carried out by researchers as they attempt to generate scientific knowledge. Science does not accept at face value taken-for-granted knowledge

C h a p t e r 1: I n t ro d u c t i o n t o E d u c a t i o n a l R e s e a r c h

15

(i.e., things that we assume to be true); instead, it is used to uncover and justify descriptions and explanations of people, groups, and the world around us. In this book, we generally treat the term science (as just defined) and the term research as synonyms.

Dynamics of Science
Over time, science results in an accumulation of specific findings, theories, and other knowledge. In this sense, science is said to be progressive. When researchers conduct new research studies, they try to build on and extend current research theories and results. Sir Isaac Newton expressed it well when he said, “We stand on the shoulders of giants.” Newton’s point was that researchers do not and cannot start completely from scratch, and Newton knew that he was no exception to this rule. In short, researchers usually build on past findings and understandings.
At the same time, science is dynamic and open to new ideas and theories that show promise.
Different researchers approach research differently, and they often describe, explain, and interpret things in different though often complementary ways. New ideas emerge. As new ideas are generated and evidence is obtained, results are presented at conferences and are published in monographs, books, and journals so that other members of the research community can examine them. Before findings are published in journals, the studies are usually evaluated by a group of experts, called referees, to make sure there are no major flaws and that the procedures are defensible. Researchers are usually required to report exactly how they conducted their research so that other researchers can evaluate the procedures or even replicate the study. Once published, research findings are openly discussed and are critically evaluated by members of the research community. Overall, we can say that science is a never-ending process that includes rational thinking, the reliance on empirical observation, constant peer evaluation and critique, and—very importantly—active creativity and attempts at discovery.

Basic Assumptions of Science
Educational researchers must make a few general assumptions so that they can go about their daily business of doing research. Most practicing researchers do not think much about these philosophical assumptions as they carry out their daily research activities; nonetheless, it is helpful to examine some of them. The most common assumptions are summarized in Table 1.3.
First, at the most basic level, educational researchers assume that there is a world that can be studied. In education, this includes studying many phenomena that are internal to people
(e.g., attitudes, values, beliefs, lived experiences), as well as many broader phenomena or institutions that are either connected to people or external to them (e.g., schools, cultures, and physical environments). Educational researchers study how the following factors relate to educational issues: psychological factors (e.g., characteristics of individuals and individuallevel phenomena), social psychological factors (e.g., examining how individuals interact and relate to one another and how groups and individuals affect one another), and sociological factors (e.g., examining how groups form and change; documenting the characteristics of groups; studying intergroup relations; and studying group-level phenomena, such as cultural, social, political, familial, and economic institutions).
Second, researchers assume that part of the world is unique, part of the world is regular or patterned or predictable, and much of the world is dynamic (i.e., changing) and complex (e.g., involving many pieces or factors). One important task of educational research is

Psychological factors Individual-level factors Social psychological factors
Factors relating individuals to other individuals and to social groups
Sociological
factors
Group- and societylevel factors

16

PA R T I : I N T R O D U C T I O N

TABLE 1.3 Summary of Common Assumptions Made by Educational Researchers
1. There is a world that can be studied. This can include studying the inner worlds of individuals.
2. Some of the world is unique, some of it is regular or patterned or predictable, and much of it is dynamic and complex. 3. The unique, the regular, and the complex in the world all can be examined and studied by researchers.
4. Researchers should try to follow certain agreed-on norms and practices.
5. It is possible to distinguish between more and less plausible claims and between good and poor research.
6. Science cannot provide answers to all questions.

to document the stories and experiences of particular people and groups. Another important task is to identify the predictable part of the world in order to generate findings that will apply to more than one person, group, kind of person, context, or situation. As you can imagine, it would be very difficult if we had to conduct research on every single individual! To see an example of regularity in the world, the next time you go to your research class, note the seats that you and a few people around you are sitting in. When your class meets again, see whether you and the others you observed sit in the same seats as during the previous meeting. You will probably notice that many of the people sit in the same seats. Why is this? This happens because humans are to some degree predictable. Understanding the predictable part of the world allows researchers to generalize and apply their findings beyond the people and places used in their particular studies.
Third, the unique, the regular, and the complex in the world can be examined and studied by researchers. In other words, “discoverability” exists in our world (i.e., it is possible to document the unique, discover the regularity in human behavior, and, in time, better understand many of the complexities of human behavior). This does not mean that the task of discovering the nature of educational phenomena is simple. For example, although significant progress has been made, we still do not know all of the causes of many learning disabilities. Research must continue, and over time, we hope to find more and more pieces to the puzzles we are trying to solve. One day we hope we will be able to solve many educational problems.
The fourth assumption is that researchers should follow certain agreed-on norms and practices. A few of these are the selection of educational and social problems in need of attention, collection of empirical data, open discussion of findings, integrity, honesty, competence, systematic inquiry, empathic neutrality and respect toward research participants, a healthy skepticism toward results and explanations, a sense of curiosity and openness to discovery, the active search for negative evidence (e.g., instances that do not fit your emerging or current explanation of a phenomenon), the careful examination of alternative explanations for your findings, and an adherence to the principle of evidence. One of this book’s authors (Johnson) likes to tell his students that a researcher is a lot like the slogan on Missouri’s license plates: “The Show Me
State.” If you have a claim to make, then “show me the evidence, please!” A good researcher tries to collect and assemble high-quality evidence and expects other researchers to do the same.
Obviously, it is all but impossible for a researcher to follow fully all of the ideals listed here.
Furthermore, because science is a human activity, it is also affected by social and power relationships among researchers and society (Kuhn, 1962; Lincoln & Guba, 2000). That’s why it is so important that researchers strive to follow the norms we have listed.

C h a p t e r 1: I n t ro d u c t i o n t o E d u c a t i o n a l R e s e a r c h

17

The fifth assumption is that it is possible to distinguish between more and less plausible claims and between good and poor research. For example, through empirical research, we can choose between competing theories by determining which theory best fits the data. We can also judge the quality of a research study by examining the research strategies used and the evidence that is provided for each of the conclusions drawn by a researcher. We say that highquality research is more trustworthy or more valid than low-quality research. We will explain throughout this textbook how to identify and carry out research that is trustworthy, valid, credible, and, therefore, defensible.
The sixth assumption made by researchers is that science cannot provide answers to all questions. For example, science cannot answer philosophical questions such as what the meaning of life is, what virtue is, or what beauty is. Science cannot settle issues of which position is morally correct (e.g., human cloning versus no human cloning; pro-choice versus pro-life in the abortion debate) or politically correct (e.g., Republican or Democrat) and cannot explain ideas such as the difference between good and evil in the world or the veracity of claims about the existence of life after death. As you can see, many important questions simply lie outside the domain of science and empirical research.

Scientific Methods
Science is not a perfectly orderly process (Kuhn, 1962). It is a dynamic process that includes countless activities. However, several of the key features of science are (1) making empirical observations, (2) generating and testing hypotheses (predictions or educated guesses), (3) generating or constructing and testing or justifying theories (explanations or explanatory systems), and (4) attempting to predict and influence the world to make it a better place to live (American Association for the Advancement of Science, 1990). Although the conduct of research is clearly not a perfectly orderly process and is composed of many activities, it still is helpful to start with some commonly used scientific methods.
We distinguish two major scientific methods here: the exploratory method and the confirmatory method. (Several additional methods are listed under Research Exercise 3 at the end of this chapter.) Although both of these methods use empirical data, their purpose is different.
The basic exploratory method includes three steps. First, the researcher starts by making observations. Second, the researcher studies the observations and searches for patterns (i.e., a statement of what is occurring). Third, the researcher makes a tentative conclusion or a generalization about the pattern or how some aspect of the world operates. The basic confirmatory method also includes three steps. First, the researcher states a hypothesis, which is frequently based on existing theory (i.e., currently available scientific explanations). Second, the researcher collects data to be used to test the hypothesis empirically. Third, the researcher decides tentatively to accept or reject the hypothesis on the basis of the data.
The exploratory method can be thought of as a bottom-up approach because it emphasizes starting with particular data and observations and discovering what is occurring more generally (i.e., movement from data to patterns to theory). This exploratory method is sometimes called the inductive method because it moves from the “particular to the general.” On the other hand, the confirmatory method can be thought of as a top-down approach because it emphasizes the process of starting with a general theory and testing it with particular data (i.e., movement from theory to hypothesis to data). This confirmatory method is sometimes called the deductive method because it moves from the “general to the particular.”

Hypothesis
A prediction or educated guess
Theory
An explanation or explanatory system that discusses how a phenomenon operates and why it operates as it does
Exploratory
method
A bottom-up or theory-generation approach to research Confirmatory method A top-down or theory-testing approach to research 18

PA R T I : I N T R O D U C T I O N

Quantitative researcher A researcher who focuses on testing theories and hypotheses using quantitative data to see if they are confirmed or not
Qualitative
researcher
A researcher who focuses on exploration, description, and sometimes the generation and construction of theories using qualitative data

FIGURE 1.1

The exploratory method is the theory-generation approach: It follows a “logic of discovery” that says to look at your world and try to generate ideas and construct theories about how it operates. The confirmatory method is the traditional theory-testing approach: It follows a
“logic of justification” that says always to test your theories and hypotheses with new data to see if they are justified. New knowledge is generated using the exploratory or inductive method, and this tentative knowledge is tested or justified using the confirmatory or deductive method.
The bottom line is this: The exploratory scientific method focuses on theory discovery, generation, and construction, and the confirmatory scientific method focuses on theory testing or justification. Although we have talked about two separate scientific methods (the exploratory method and the confirmatory method), it is important to understand that researchers use both of these methods in practice. As you can see in Figure 1.1, the use of the methods follows a cyclical process. One researcher might focus on the theory-testing process, and another researcher might focus on theory generation, but both researchers will usually go through the full cycle many, many times as they think about and carry out their research programs over time. In fact, quantitative researchers (i.e., educational researchers who like “hard” quantitative data such as standardized tests and focus on hypothesis testing) and qualitative researchers (i.e., educational researchers who like to explore educational issues using qualitative data such as open-ended interviews that provide data based on the participants’ perspectives and their actual words) both go through the full research cycle, but they emphasize different parts. Quantitative researchers emphasize movement from theory to hypotheses to data to conclusions (i.e., the
“logic of justification”), and qualitative researchers emphasize movement directly from observations and data to descriptions and patterns and, sometimes, to theory generation (i.e., the
“logic of discovery”).

The research wheel

Theory

Patterns,
Descriptions

Exploratory

Confirmatory

Hypotheses,
Predictions

Observations
Data

Theory
The exploratory and confirmatory methods both involve the concept of theory (i.e., explanation). The term theory as used in this book most simply refers to an explanation or an explanatory system that discusses how a phenomenon operates and why it operates as it does. Theory often refers to a generalization or set of generalizations that are used systematically to explain some phenomenon. In other words, a well-developed theory explains how something operates

C h a p t e r 1: I n t ro d u c t i o n t o E d u c a t i o n a l R e s e a r c h

in general (i.e., for many people), and it enables one to move beyond the findings of any single research study. Using a well-developed theory, you should be able to explain a phenomenon, make sense of it, and be able to make useful predictions. When you need to judge the quality of a theory or explanation, you should try to answer the nine questions listed in Table 1.4. We now define and briefly elaborate on the criterion of falsifiability and the rule of parsimony.
Sir Karl Popper (1902–1994), who was one of the most famous philosophers of science of the 20th century, contended that the most important criterion used to judge theories is the criterion of falsifiability (Popper, 1965, 1974, 1934/1985). The criterion of falsifiability is “the property of a statement or theory that it is capable of being refuted by experience” (Blackburn,
1994, p. 135). If someone said, “I don’t care what the results of my research study are because
I’m going to conclude that my theory is supported, no matter what,” then that person would obviously not be doing the kind of research that could ever reject or falsify a theory. There must be two sorts of possible outcomes for empirical research: (a) outcomes that would support the theory (that would “confirm” the theory) and (b) outcomes that would not support the theory
(that would “not confirm” the theory and over many tests would be used to reject or falsify the theory). Then you conduct your research to find out which type of outcome occurs. In practice, researchers do not give up on promising theories based on single negative tests, but if a theory fails many times,then the theory will be abandoned. The criterion of falsifiability also says that we should not selectively search for confirming evidence for our beliefs and explanations and then stop with that so-called evidence. Good researchers carefully search for and examine any negative evidence that operates against their beliefs, research conclusions, and theoretical explanations. Another criterion for evaluating theories is called the rule of parsimony. A theory is parsimonious when it is simple, concise, and succinct. If two competing theories explain and predict a phenomenon equally well, then the more parsimonious theory is to be preferred according to the rule of parsimony. In other words, simple theories are preferred over highly complex ones, other things being equal.
Now let’s briefly examine an educational theory to give you an idea of what a relatively well-developed theory looks like.According to expectation theory, teachers’ expectations about their students affect their behavior toward their students, which in turn affects their students’ behavior. The theory is based on the self-fulfilling prophecy (Merton, 1948). Robert Rosenthal
TABLE 1.4 How to Evaluate the Quality of a Theory or Explanation
1. Is the theory or explanation logical and coherent?
2. Is it clear and parsimonious?
3. Does it fit the available data?
4. Does it provide testable claims?
5. Have theory-based predictions been tested and supported?
6. Has it survived numerous attempts by researchers to identify problems with it or to falsify it?
7. Does it work better than competing or rival theories or explanations?
8. Is it general enough to apply to more than one place, situation, or person?
9. Can practitioners use it to control or influence things in the world (e.g., a good theory of teaching helps teachers to influence student learning positively; a good theory of counseling helps counselors to influence their clients’ mental health positively)?

19

Criterion of falsifiability The property that statements and theories should be testable and refutable Rule of parsimony
Preferring the most simple theory that works 20

PA R T I : I N T R O D U C T I O N

and Lenore Jacobson (1968) studied the effects of teachers’ expectations and found that students whom teachers expected to perform well had higher increases in IQ than other students. These authors labeled this the Pygmalion effect. Rosenthal also found that “those children in whom intellectual growth was expected were described as having a significantly better chance of becoming successful in the future, as significantly more interesting, curious, and happy” (Rosenthal, 1991, p. 6). Students who had IQ increases but had not been expected to have increases by the teachers were not viewed more favorably by the teachers.
These results suggest that teacher expectations can sometimes affect student performance.
Note, however, that recent research has suggested that the power of expectations is not as great as had originally been concluded (Goldenberg, 1992). Nonetheless, the theory of expectations is a useful idea.
There are many theories in education. A few are attribution theory, constructivism, labeling theory, Kohlberg’s theory of moral development, operant conditioning, proximal development, rational emotive therapy, site-based management, situated learning, and social learning theory. If you want to find out more about any of these theories, just go to the library
(or, using your computer, go to www.eric.ed.gov) and conduct a search using ERIC or one of the other computerized search tools, which are discussed in Chapter 3. You can also find nice descriptions of many educational and psychological theories at http://tip.psychology
.org/theories.html.
Keep in mind as you read research articles that you will not always find the word theory in the article because often a well-developed or explicit theory will not be available to the researcher, or the researcher might not have a fancy name for his or her theory. In this case, you can view the authors’ explanations of their findings as the theory. Remember that some theories are highly developed and others are very brief or not well developed. When we use the word theory in this book, you might replace it with the word explanation until you get used to the idea that theory most simply means “explanation.”

The Principle of Evidence
Many beginning students believe that science and research are processes in which researchers constantly prove what is true.You might be surprised to learn that researchers rarely use the word prove when discussing their research findings. In fact, as we mentioned earlier, we recommend that you eliminate the word prove from your vocabulary when you are talking about research because most researchers hold knowledge to be ultimately tentative (Phillips &
Burbules, 2000; Shadish, Cook, & Campbell, 2002). They recognize that principles that are believed to be true today might change eventually; some of today’s findings will later be found to be partially true or even patently false. What we obtain in research is scientific “evidence.” It is essential that you understand this idea. An important educational methodologist, the late Fred
Kerlinger (1986), made this point very clearly:
The interpretation of research data culminates in conditional probabilistic statements of the
“If p, then q” kind. We enrich such statements by qualifying them in some such way as: If p, then q, under conditions r, s, and t. Let us flatly assert that nothing can be “proved” scientifically.
All one can do is to bring evidence to bear that such-and-such a proposition is true. Proof is a deductive matter, and experimental methods of inquiry are not methods of proof [emphasis added]. (p. 145)

C h a p t e r 1: I n t ro d u c t i o n t o E d u c a t i o n a l R e s e a r c h

21

Here is the way the American Association for the Advancement of Science (1990) put it:
Science is a process for producing knowledge. The process depends on making careful observations of phenomena and on inventing theories for making sense out of those observations. Change in knowledge is inevitable because new observations may challenge prevailing theories. No matter how well one theory explains a set of observations, it is possible that another theory may fit just as well or better, or may fit a still wider range of observations. In science, the testing and improving and occasional discarding of theories, whether new or old, go on all the time. (p. 2)

As you learn more about research, keep these points in mind. It is also important to understand that you should never place too much weight on a single research study. Replication by other researchers (i.e., research examining the same variables with different people and in different ways) should make you more confident about a research finding because the resulting evidence is much stronger. But even in the face of replication, strong evidence rather than proof is all that is obtained because we always leave open the possibility that future researchers will come up with new theories and new conclusions.
Whenever you are tempted to use the word prove, stop and think and remind yourself about the fundamental nature of educational research. For now, whenever you want to use the word proof, just use the word evidence instead. Sometimes I (Johnson) like to tell my students that proof is what television commercials claim for their products’ performance, but in research the best we can do is to obtain evidence. During a presidential election in the 1990s, a campaign manager kept a slogan posted in the campaign office that read,“It’s the economy, stupid!” to keep the staff focused on the economic performance of the current administration as the primary campaign issue. In research our slogan goes like this: “It’s about evidence, not proof!” We call this idea the principle of evidence.

1.8

Explain why researchers do not use the word proof when they write up the results of their research in journal articles.

Principle of evidence The philosophical idea that empirical research provides evidence, not proof

Describe the two forms of the scientific method and explain why both are important.

1.9

Replication
Research examining the same variables with different people 1.10 What criteria can you use to determine the quality of a theory or an explanation?

REVIEW
QUESTIONS

1.11 What does the principle of evidence state?

OBJECTIVES OF EDUCATIONAL RESEARCH
Discussions of science and empirical research often focus on the importance of explanation.
However, several additional objectives are also important if the field of educational research is to continue to operate effectively and to progress. The first objective is exploration, or attempting to learn about and generate ideas about phenomena. Exploration is especially important in the early phases of research because researchers must generate ideas about phenomena before additional research can progress. To determine whether exploration was the objective of a particular research study, answer the following questions:
1. Were the researchers studying a phenomenon or some aspect of a phenomenon about which little was previously known?

Exploration
Attempting to generate ideas about phenomena

22

PA R T I : I N T R O D U C T I O N

2. Did the researchers choose to ignore previous research or explanations so that they could study a phenomenon without any preconceived notions?
3. Were the researchers trying to “discover” important factors or “generate” new ideas for further research? Description
Attempting to describe the characteristics of a phenomenon If you answer yes to any of these questions, then the researchers were probably operating in the exploratory mode of research.
As is implied in the second and third questions, exploration does not always have to be done in the early phases of research. Sometimes researchers might want to enter the field without fixed or preconceived notions about what they are studying so that they can explore a phenomenon in a new way and so that they can avoid being biased or blinded by previous findings or theories. The article mentioned earlier in this chapter (in the section “Examples of
Educational Research”) entitled “Giving Voice to High School Students” was exploratory because the researchers tried to uncover what at-risk students thought was important in their lives, why the students acted in the ways they did, and how the students viewed various formal and informal groups (e.g., teachers). The researchers tried to describe the at-risk adolescents’ beliefs and circumstances to explain why they acted as they did. One finding was that some at-risk students formed subcultures that were in conflict with the teachers’ culture; that is, the groups differed on such criteria as values, beliefs, and activities that were considered appropriate. These differences made it difficult for the teachers and the students to communicate, which resulted in student apathy and boredom in the classroom. For another example in which the objective was exploratory, you can reread the abstract of the article mentioned in the same section entitled
“The Development of a Goal to Become a Teacher.”
Exploration sometimes is focused on describing the nature of something that previously was unknown; it also is used when the researcher tries to understand the specifics of some phenomenon or some situation to develop tentative hypotheses or generalizations about it.
Exploration is similar to basic descriptive activities in that it often includes description. However, attempts are also frequently made in exploratory research to generate preliminary explanations or theories about how and why a phenomenon operates as it does.
The second objective is description, or attempting to describe the characteristics of a phenomenon. To determine whether description was the main objective of a particular research study, answer the following questions:
1. Were the researchers primarily describing a phenomenon?
2. Were the researchers documenting the characteristics of some phenomenon?

Description is one of the most basic activities in research. It might simply involve observing a phenomenon and recording what one sees. For example, a seasoned teacher might observe the behavior of a student teacher and take notes. At other times, description might rely on the use of quantitative measuring instruments such as standardized tests. For example, a researcher might want to measure the intangible construct called intelligence quotient, or IQ. To do this, the researcher must rely on some type of test that has been constructed specifically for this purpose. At other times, description might involve reporting attitudes and opinions about certain issues. For an example, see the September 1996 issue of Phi Delta Kappan, which reports national attitudes toward education each year. The study is conducted by the Gallup Organization and

C h a p t e r 1: I n t ro d u c t i o n t o E d u c a t i o n a l R e s e a r c h

23

is commissioned by the education honor society Phi Delta Kappa (1996). Two questions and their responses are shown in Table 1.5.
TABLE 1.5 Items From Phi Delta Kappa/Gallup Poll (September 1996)
Question: Would you favor or oppose a requirement for high school graduation that all students in the local public schools perform some kind of community service?
National Totals %

No Children in
School %

Public School
Parents %

Nonpublic School
Parents %

Favor

66

66

67

75

Oppose

32

32

32

25

2

2

1

*

Don’t know

Question: Just your impression, do you think that the national dropout rate of students in high school is higher today than it was twenty-five years ago, lower today, or about the same as it was twenty-five years ago?
National Totals %

No Children in
School %

Public School
Parents %

Nonpublic School
Parents %

Higher

64

62

66

73

Lower

15

15

15

8

About the same

18

19

17

16

3

4

2

3

Don’t know
*Less than one half of 1 percent

The third objective is explanation, or attempting to show how and why a phenomenon operates as it does. According to many writers, this is the key purpose of science. To determine whether explanation was the primary objective of a particular research study, answer the following questions:
1. Were the researchers trying to develop a theory about a phenomenon to explain how and why it operates as it does?
2. Were the researchers trying to explain how certain phenomena operate by identifying the factors that produce change in them? More specifically, were the researchers studying cause-andeffect relationships?

If the answer to either of these questions is yes, then the researchers’ primary objective is probably explanation. The objective of the majority of educational research is explanation.
An example of a research study focusing on explanation is a study entitled “Are Effects of Small
Classes Cumulative?” by Nye, Hedges, and Konstantopoulos (2001). In that study, the researchers were interested in determining the effect of class size on student performance.
They found that smaller classes in Grades 1 through 3 resulted in improved reading and mathematics achievement scores and that the effect continues to occur over time. The study used a strong experimental design that provides relatively solid evidence about cause and effect.
In a study like this, the cause (i.e., smaller class sizes) is used to explain the effect (i.e., improved achievement scores). For another example in which the objective was explanation,

Explanation
Attempting to show how and why a phenomenon operates as it does

24

PA R T I : I N T R O D U C T I O N

Prediction
Attempting to predict or forecast a phenomenon Influence
Attempting to apply research to make certain outcomes occur

see the article mentioned earlier (in the section “Examples of Educational Research”) entitled “Getting Tough? The Impact of High School Graduation Exams.”
The fourth objective is prediction, or attempting to predict or forecast a phenomenon. To determine whether prediction was the primary objective of a particular research study, answer the following question: Did the researchers conduct the research so that they could predict or forecast some event in the future? A researcher is able to make a prediction when certain information that is known in advance can be used to determine what will happen at a later point in time. Sometimes predictions can also be made from research studies in which the primary focus is on explanation. That is, when researchers determine cause-and-effect operations (explanations), they can use this information to form predictions.
One research study in which the focus was on prediction was conducted by Fuertes,
Sedlacek, and Liu (1994). These researchers conducted a 10-year research study and found that
Asian American university students’ academic performance and retention could be predicted by using the Scholastic Assessment Test (SAT) and another instrument called the Noncognitive
Questionnaire. The strongest predictor of the students’ GPAs was their SAT math scores. Other useful predictors (from the Noncognitive Questionnaire) were community service, realistic selfappraisal, academic self-concept, nontraditional knowledge, and handling racism. The strongest predictors of enrollment (i.e., retention) were self-concept, realistic self-appraisal, and
SAT math score.
The fifth objective is called control or influence, or attempting to apply research to make certain outcomes occur. This objective refers to the application of research knowledge rather than the generation of research knowledge. It refers to the application of previous research to control various aspects of the world. Here you should ask the following questions: Were the researchers applying research knowledge to make something useful happen in the world?
Were the researchers checking a “demonstration program” to see if it works in practice? The ultimate objective of most social, behavioral, and educational research is improvement of the world or social betterment. Therefore, influence is important. For teachers, influence involves things like helping students learn more than they previously knew, helping children with special needs, and preventing negative outcomes such as dropping out of school or disruptive behavior in the classroom. For counselors, influence might involve helping clients overcome psychological problems such as depression, personality disorders, and dysfunctional behaviors.
As you work through this book and learn about the different methods of research, you will be learning more about these objectives. At this point, you should be able to examine a research article and determine what the researcher’s objectives were. Don’t be surprised if there appears to be more than one objective. That is not at all uncommon. You should also be aware that researchers often use the terms descriptive research, exploratory research, explanatory research, and predictive research. When they do this, they are simply describing the primary objective of the research.

REVIEW

1.12 What are the five main objectives of science? (Hint: The first letters form the acronym EDEPI.)

QUESTIONS

1.13 Why is each of the five main objectives of science important?

C h a p t e r 1: I n t ro d u c t i o n t o E d u c a t i o n a l R e s e a r c h

25

OVERVIEW OF BOOK
We have organized your textbook to follow the general steps involved in the research process.
In Part I we introduce you to the kinds of educational research and the process and assumptions of research. In Part II we show how to come up with a research idea and how to plan a research study. In Part III we introduce some concepts required to design and conduct a good study. In Part IV we discuss the major methods of research. In Part V we show how to analyze data resulting from a research study. In Part VI we explain how to write a research manuscript. To master the material fully, you will need to take advantage of some of the application exercises provided in the book and on the companion website because they will give you some practice applying the material. As you start to review for exams, you can test your overall knowledge of the material by answering the multiple-choice questions on the companion website (the answers are included) and by answering the study questions. You can also print the definitions of the terms given in the chapters. Don’t look at the answers until you have stated your own answers, then compare and identify your areas of strength and weakness. Use the concept maps on the companion website to keep what you learn organized in terms of the big picture.
We also strongly recommend that you read some examples of published research to see full-length examples of how research is done. Throughout the text we provide references to many published research articles that you can examine. Furthermore, we have provided downloadable copies of 60 journal articles on the companion website that you can print and read and discuss in class.You can start right now by going to the companion website and printing and reading the article entitled “Gifted Dropouts: The Who and the Why.” Reading or carefully examining this article will give you a concrete example of educational research.
Our practical conclusion for this chapter is clear: Anyone can learn the material in this book if he or she works hard at it, and that means that you can do it! We hope to show you that learning about research can actually be fun. Good luck, and don’t forget to use the many learning tools that are available at the companion website to make your learning experience easier and more productive. See Journal Article 1.4 on the study site.

SUMMARY
It is important that educators and counselors be research literate because of the importance of research in education and our society. By learning about research, you will be able to find published research articles that are relevant for your profession, evaluate those research articles, and propose and conduct research studies on your own if the need ever arises in your career (e.g., perhaps one day your principal or manager will ask you to conduct a survey or to write a grant proposal).
Educational researchers generate evidence about educational phenomena by collecting empirical data and using the exploratory and confirmatory scientific methods. We also explained that five general objectives of research are to explore, to describe, to explain, to predict, and to influence or control things in our world. When reading research articles, you should determine the primary objective researchers had when they conducted their research studies. In the next chapter, we will finish our introduction to educational research by describing the key features of the three major research paradigms: quantitative research, qualitative research, and mixed research.

26

PA R T I : I N T R O D U C T I O N

KEY TERMS abstract (p. 7) action research (p. 11) applied research (p. 9) basic research (p. 9) confirmatory or deductive method
(p. 17) criterion of falsifiability (p. 19) deductive reasoning (p. 13) description (p. 22) empirical statement (p. 13) empiricism (p. 12) epistemology (p. 13) evaluation (p. 10)

explanation (p. 23) exploration (p. 21) exploratory or inductive method
(p. 17) formative evaluation (p. 10) hypothesis (p. 17) inductive reasoning (p. 14) influence (p. 24) orientational research (p. 12) prediction (p. 24) principle of evidence (p. 21) probabilistic (p. 14) problem of induction (p. 14)

psychological factors (p. 15) qualitative researcher (p. 18) quantitative researcher (p. 18) rationalism (p. 13) replication (p. 21) research literature (p. 5) rule of parsimony (p. 19) science (p. 14) social psychological factors (p. 15) sociological factors (p. 15) summative evaluation (p. 10) theory (p. 17)

DISCUSSION QUESTIONS
1. Which of the following do you think is the most important kind of research: basic, applied, evaluation, action, or critical theory research? Why?
2. Why is it asserted in this chapter that one does not obtain necessary or final proof in educational research?
3. How does the presentation of exploratory and confirmatory scientific methods fit with your prior understanding of the methods of scientific research?
4. What is a research finding that you have heard (e.g., on the news or in another class) and wondered about?

RESEARCH EXERCISES
1. We have put a Research Methods Questionnaire on the book’s companion website under Chapter 1 bonus materials. Fill it out and test your prior knowledge about research methods.
2. Go to the Internet and search for more information on some of the terms that you found most interesting in the chapter. For example, you might want to search for more material on critical theory, inductive reasoning, program evaluation, or epistemology.
3. In this chapter we distinguished between the exploratory and confirmatory methods of science. As we mentioned, however, researchers use many approaches to gain knowledge. As an exercise, go to the web, find the needed information, and summarize (in a two-page paper) one of these scientific methods: inference to best explanation, Mill’s methods, abductive reasoning, analogical reasoning, deductive-nomothetic model, hypothetico-deductive model, inductive methods, or deductive methods. 4. Take a moment, right now, to examine what is available at the companion website that goes with this book. Here are some of the many features you will find: lectures, concept maps, answers to the review questions, quizzes, web resources, chapter supplements, and more. If you think of something else that will help you learn the material in your book, please e-mail us and let us know because we are always adding new features to the companion website. Our e-mail addresses are bjohnson@usouthal.edu and lchriste@usouthal.edu.

C h a p t e r 1: I n t ro d u c t i o n t o E d u c a t i o n a l R e s e a r c h

27

ACTION RESEARCH JOURNALING
Action research (defined on page 11) tries to solve real-world problems. It is about applying research ideas in your world to make it better. It is not about there being a single correct type of research or there being one right way to reach your valued goals. Rather, action research involves thinking creatively, ethically, “scientifically,” and personally about how you can apply ideas and insights that you come up with in your life, especially in your work life. To help facilitate your growth, we will be asking you to work on your personal action research journal as you think about the chapter materials. We expect that you also will be planning and conducting a small action research project this semester or quarter, although that is not necessary to benefit from this 20-chapter exercise. At the end of each chapter, we will provide a few thought questions to get you started and keep you progressing. Later, you will be able to come back to your journal and reconsider your insights, apply them again, and build on them as you continue your lifelong learning and action research.
Here is a list of a few action research ideas to help you get started in thinking about an action research project that you would like to do:
•
•
•
•
•
•
•
•
•
•
•

How can I increase my students’ intrinsic motivation to read?
How can I increase my students’ self-efficacy for giving in-class presentations?
How can I get students in social cliques (“insiders”) to care for and respect other students (“outsiders”)?
What are teachers and administrators’ views of the characteristics of a “good teacher” and how can these be merged?
How can I get students to increase their care for and interaction with students who have special needs?
How can I get my students more engaged in mathematics (or reading or history)?
How can I tailor my class so that low- and high-achieving students are progressing?
How can I increase parental participation in students’ homework?
How can I increase staff participation in school activities?
How can I improve the school culture at my school?
What are students’, teachers’, parents’, and administrators’ views about the purposes of school, and how can I increase communication among these groups?

Insight: Action research is prescriptive. Action researchers are interested in science as an instrument to help practitioners. You might say that action researchers are very “practical scientists.”
1. How might you start viewing science broadly (e.g., as “action science”) so that it can be useful in your everyday life and professional practice?

See Journal Article 1.5 on the study site.

2. What insights did you have as you read this chapter? (If you didn’t have any, try to think of some now!)
3. What do scientists do? What do practitioners do? Action researchers attempt to connect science and practice. How might you use science to help your work practices, and how might you use your work practices and experiences to inform the scientist part of your multidimensional self? (Note: Action researchers view science and practice as continually feeding off of one another, and they advocate a practical science.)

RELEVANT INTERNET SITES
Educational research http://www.aera.net Program evaluation http://www.eval.org and http://www.wmich.edu/evalctr/

Action research http://coe.westga.edu/arsig/ Philosophy of science links http://www.pitt.edu/~pittcntr/About/links.htm 28

PA R T I : I N T R O D U C T I O N

STUDENT STUDY SITE
Visit the Student Study Site at www.sagepub.com/bjohnson4e/ for these additional learning tools:
Video Links
Self Quizzes
E-Flashcards
Learning Objectives

Full-text SAGE journal articles
Interactive Concept Maps
Web Resources
SPSS Exercises and Data Set

RECOMMENDED READING
Davidson, E. J. (2005). Evaluation methodology basics:
The nuts and bolts of sound evaluation. Thousand Oaks,
CA: Sage.
Harding, S. (Ed.). (2004). The feminist standpoint theory reader.
New York, NY: Routledge.
Kirkpatrick, D. L. (2005). Evaluating training programs: The four levels. San Francisco, CA: Berrett-Koehler.

Phillips, D. C., & Burbules, N. C. (2000). Postpositivism and educational research. New York, NY: Rowman & Littlefield.
Reason, P., & Bradbury, H. (Eds.). (2006). Handbook of action research: Concise paperback edition. Thousand Oaks, CA:
Sage.
Sandoval, C. (2000). Methodology of the oppressed. Minneapolis:
University of Minnesota Press.