Example of the research design part of a thesis (Education)

Example 2

RESEARCH METHODOLOGY

In the previous chapter, the effects of IGI promoting discovery-learning, procedure of IGI, and the effectiveness of IGI on SLA and motivation are discussed. From the insights, parameters for designing materials, tests, and questionnaire are drawn out. Chapter Three is divided into five sections. Research design and subjects and program description sections provide general information about the study. The next three parts, materials, instruments, and procedures of the research are described in details.

3.1. Research Design

This study is carried out with a one group pre-test and post-test design. In this experimental research, the implementation of discovery-learning through IGI (independent variable) will be monitored, and acquisition about English tenses and learners’ motivation (dependent variables) will be measured.

The grammar points in this research are English tenses. A pre-test and a post-test are designed to be the same. An anonymous questionnaire on motivation is delivered after the treatment instruction.

3.2. Subjects and Program Description

The researcher, who is also the teacher, carries out the study in six classes in Course 28 and Course 29 at CFL in 2006. The experimental course, Level-Seven Grammar Course, is the first of three courses for Level C (low-advanced-English level). The grammar course is offered to the students for three 45-minute periods on each occasion during nine weeks.

The students are randomly chosen and can be the representatives for the research. The total number of the students was 227, but the authentic number of participants for the present study was 133 because some students did not satisfy the conditions of data analysis (see part 3.4 of this thesis). The participants were from 14 to 26 years old, including 71 female and 62 male non-English majors. Most of them were undergraduate students of Cantho University; some were high school and junior high school students in Cantho city. The students in the course came from three sources. Some of them have passed the exam of Level B (intermediate-English level) at CTU; some have gotten the Certificate of Level B at other centers of foreign languages; some have just taken the placement test of CFL. Their educational background of general knowledge and English was varied. This is also the primary reason why the research had only one experimental group.

3.3. Materials

Based on the parameters concluded from the part Procedure of Implicit Grammar Instruction (part 2.3 of this thesis), and the first four chapters of the main course book Mosaic One: A Content-Based Grammar (Werner, 1996), the detailed lesson plans have been designed (see Appendix one). To promote discovery-learning in the treatment, three stages of tasks must be concentrated. First, noticing and awareness of form are designed in Setting the Context. Next, exercises in Thinking about Meaning and Use are discovery-based, and they encourage learners to use their grammatical knowledge and English skills to express their remarks and finish the tasks. Finally, the three parts Practicing, Using What You’ve Learned, and Taking a Step Beyond are production tasks that move from controlled to free language activities. Besides, Review Exercises is a consolidation part of the materials. It gives students a chance to practice test-designed exercises.

The texts serve as the handouts for the learners in the treatment course of the research. The materials consist of seven topics and three review exercises (see Appendix 1). Beside Mosaic One: A Content-Based Grammar (Werner, 1996), some activities and exercises in the materials are collected and adapted from the following books:

1) Grammar Practice for Upper Intermediate Students (Walker & Elsworth, 2000)

2) Grammar Sense 3 (Bland, 2003)

3) Grammar Sense 3 – Workbook (Blackwell & Davy, 2003)

4) Nitty Gritty Grammar - Sentence Essentials for Writers (Young & Strauch, 1998)

5) Oxford Practice Grammar (Eastwood, J., 1999)

6) The Anti-Grammar Grammar Book - A Teacher’s Resource Book Of Discovery Activities For Grammar Teaching (Hall and Stepheard, 1991)

7) Understanding and Using English Grammar – Workbook (Azar & Azar, 1990)

The students receive the handout of each lesson at the beginning of the class meeting. In general, each topic discusses in one class meeting (three 45-minute periods). There are five sections in each topic. Here are the detailed descriptions of each part.

Setting the Context presents a brief passage that highlights specific structures covered in that section. “The text is used not only for comprehension and for setting up a topic for use later in the unit, but also to make the students familiar with the new language before teaching it later” (Harmer, 1987, p.30). It can be used to introduce key vocabulary and to make sure the students have a general understanding of the content. This part is very necessary to cover in class because it provides the context for students to notice and raise awareness of the language form, and a tool for them to discover the “rules” underlined. In Stern’s (1992) opinion, “put whatever grammatical feature is being taught into a meaningful context of practical use so that the meaning is never in doubt” (p. 144). Besides, “they see the language in its proper (authentic) context and are then led to understand how it works” (Harmer, 1987, p.36).

Thinking about Meaning and Use is the main section that conveys the discovery-based approach. This part serves not only as language input for students to expose but also as opportunities for them to activate their own grammatical knowledge of language. All tasks encourage students to analyze how we use the target structure and consolidate students’ understanding of all aspects of the structure. Students are helped to involve and use their reasoning processes to discuss about the target structure in order to find out rules. This part will be “a good reinforcement to what the students have been learning” (Harmer, 1987, p. 37) and encourage them to be actively involved in the acquisition of grammatical knowledge about of English verb tenses.

Practicing includes two controlled kinds of exercises of the target structures that are selected from a variety source of grammar practice books. One is supplying the correct verb tenses in a passage or a dialogue, and the other is error analysis. Both kinds of exercises are aimed for recognition and reproduction. Specially, in the later type of exercises students are asked not only to judge but also to correct the language while in Mosaic One: A Content-Based Grammar these exercises are only for error judgment. Most exercises can be used for either oral or written work; they can be done as a class, in pairs, in small groups, or as homework.

Using What You’ve Learned activities are chosen from Mosaic One: A Content-Based Grammar. In this textbook, there are many activities. From our empirical experience, however, we cannot cover all because of time limitation. Moreover, we have to adapt them to meet students’ needs and to suit Vietnamese situations. Thus, our adapted activities of realistic and motivating contexts offers students more time and opportunities to use target structures and vocabulary in more natural, personalized communication, to apply their skills to relevant, real-world situations. This kind of communicative activities engages students in active learning and appeals to a wide range of students’ abilities and interests, allowing them to adjust their perceptions of the language input they have received.

Taking A Step Beyond is a new section in A Content-Based Grammar Course - Level 7. The activities in this part provide students with suggestions of looking for the target structures in authentic materials from magazines, newspapers, Internet, or other sources. Sharing and discussing these materials with other students will help to reinforce and contextualize the linguistic items covered. They become a bridge between the controlled, structured language of the classroom and real-life language outside the classroom.

Review Exercises appear after two topics. These exercises are carefully selected from a rich source of grammar books. They highlight structures covered in the two topics, recycle previously studied material, and integrate it with newly practiced material. There are three kinds of exercises in this part: 15 multiple-choice questions, a passage or a dialogue to supply the correct form or (modal) verbs, and 10 sentences for error analysis. These review sections can be used in class, for homework, or as the basic for quizzes. This part serves as preparation for students to take grammar tests in the Certificate Level C exam at the Center of Foreign Languages, Cantho University. Through repeated exposure to and practice with this standardized test format, students will gain more confidence and experience less anxiety in actual test-taking situations.

To ensure these lessons comprehensible to the subjects, a pilot study was randomly conducted with 36 students in Course 27 at CFL in 2006. All of the tasks have been fully tried and tested. The results of this pilot study shows that all texts are appropriate to the level, fully comprehensible for the students, and suitable to the current study.

3.4. Research Instruments

The present research applies both qualitative and quantitative approaches. There are two research instruments: the scores from the pre-test and the post-test and the learners’ responses to the questionnaire. The first instrument will measure learners’ acquisition on English tenses. The second is designed to explore learners’ motivation in learning grammar after implicit grammar instruction with the implementation of the discovery technique.

3.4.1. Tests on Acquisition About English Tenses

Designing Tests: From the discussion in the above part about Implicit Grammar Instruction and Second Language Acquisition (part 2.4 of this thesis), the achievement test on English tenses will be recognition and judgment tasks. The task types selected for this project are the most popular kinds of grammar tests. Some parts of the test follow the format of the test for Level C certificate at CFL. Furthermore, these frames exert a very high degree of control over the specific form of the limited production responses. The test questions are efficient at measuring the learning of the objectives in the course program.

The pre-test and the post-test of the study are the same, but the part order in the two tests is changed to distract wise-test students. The tests are a kind of discrete point test types focused on only the uses of English verb tenses. They include five parts. Here is the description of the test parts in the pre-test (see Appendix 2).

The first and the second part are multiple-choice recognition tasks that are similar to the CFL test format. The first task consists of 26 incomplete sentence stems followed by four multiple-choice options for completing the sentences. The questions are selected and adapted from three sources:

1) Mosaic One: A Content-Based Grammar (Werner, 1996)

2) Understanding and Using English Grammar – Workbook (Azar & Azar, 1990)

3) Teaching Outline and Supplementary Activities (2005)

The test type in the second section is the same as the first task, but it is presented in an incomplete passage with 14 blanks. This part is adapted from the Review Exercise 4 on page 141 of Mosaic One: A Content-Based Grammar (Werner, 1996). The test takers must understand the context, then complete it with the correct answers from the four given multiple-choice options for each blank. Test takers’ ability of recognizing the correct uses of verb tenses is evaluated in these two sections although some items in this part test only verb forms (see Table 3.1).

Part 3 and Part 4 of the pre-test are designed to check learners’ recognition of tense meanings in discrete sentences. Students have to fully understand the meanings of English tenses to fulfill the tasks effectively. Part 3 consists of ten sentences that ask learners whether the paraphrase sentences are true or false. In Part 4, there are two options for each of ten sentences, and test takers have to choose one. The two parts examine learners’ understanding the meanings of English tenses. They are selected from exercises in Grammar Express – For Self-Study and Classroom Use (Fuchs & Bonner, 2002).

The last part is an error analysis exercise – a kind of judgment task as a means of measuring learners’ internalized knowledge (Coppieters 1987, Masny & d’Anglejan 1985, Schachter 1989 cited in Renou, 2001). In this part, test takers are asked to identify what is wrong with the use of verb tenses in ten discrete sentences. The sentences are selected for two books: Mosaic One: A Content-Based Grammar (Werner, 1996) and TOEFL Grammar Workbook (Lim, et al., 1992). This is an open-ended task with no indication of which words or phrases are potentially incorrect. Then they have to find and correct typical mistakes that students make when they use English tenses. Not only test takers’ ability of recognition and judgment of the right structures but also their capacity of reproduction are required in this task. Thus, students need to use their grammatical knowledge to analyze and then apply in the task. However, some sentences just aim to learners’ attention to verb forms.

Table 3.1: Grammatical Dimensions Tested in the Pre-Test

|Grammatical Dimensions |Items in the Pre-test |
|Form |1, 6, 11, 16, 21, 25, 61, 63, 66, 67, 69 |
|Meaning |41 – 60 |
|Use |Remaining items |

Scoring Tests: The perfect score on the tests is 80 points. For the recognition production task (Parts 1-4), one point is awarded to each correct answer, and no point for incorrect answers, for a total of 60 points. For the judgment task, the maximum score for each sentence is two points. One point is given for correctly identifying the error. Two points are given for identifying and correcting the error. This part has 20 points in total.

Scorer reliability is built into the test by having responses scored objectively with scoring key. The answer key in parts 1-4 is carefully checked, so test takers have only one choice, and there is high reliability. In part 5, if test takers recognize the errors correctly and produce grammatical correct responses that are acceptable and are not included in the scoring key, the key might need to be revised after trailing.

Piloting Tests: After designing the tests, we have to “test the test” (Harmer, 1987, p. 59). The test draft was shown to two English teachers at CFL. They carefully read and gave suggestions for improvements. Then the revised draft was piloted in a class at CFL of over thirty students with the same level. After that, some students taking the piloting test were interviewed to identify any unclear or ambiguous instructions. The test length, allotted time (60 minutes), and test instructions were appropriate and clear for all students. Finally, the results from the try-out in class were analyzed. The results were good. There was no problem of the input. Shortly, the test procedure and the test content seem appropriate to the purpose of the present study: measuring learners’ acquisition of English tenses. We have the authentic tests for the study (see Appendix 2).

3.4.2. Questionnaire on Motivation

As mentioned in the part Implicit Grammar Instruction and Learners’ Motivation (part 2.5 of this thesis), Watanabe’s motivation questionnaire focusing on value and expectancy components is adapted to use in the current study. However, Watanabe’s questionnaire aims to general language learning, so it was modified to focus on only grammar learning motivation. The adapted questionnaire is composed of three factors: value components (intrinsic motivation and task value), motivational strength, and expectancy components (expectancy and language aptitude).

A draft of adapted questionnaire was designed with 20 closed-item questions in five rating points (strongly disagree, disagree, neutral, agree, and strongly agree). One of the reasons for the choice of this kind of instrument is that the instrument may offer a high reliability and it may be easy to be quantified and analyzed and cost the least time to collect and analyze (Mackey & Gass, 2005). This draft has been piloted in two classes of 57 students with the same setting and background. The Cronbach’s Alpha of the pilot is .59. Five questions have been deleted to improve the total reliability of the inventory slightly to α = .74. With the alpha, there is no need to reconstruct the items in the inventory. The questionnaire used for the authentic study has 15 questions relating to motivation in learning grammar, especially with IGI promoting discovery-learning.

Table 3.2: Items in authentic questionnaire on motivation in learning grammar through IGI

|Clusters |Sub-clusters |Items |
|Value components |Intrinsic motivation |1, 7, 11 (R), 4 |
| |Task value |8, 2, 5, 9, 15 |
|Motivational strength | |14 (R), 12 |
|Expectancy components |Expectancy |3, 10, 6 (R) |
| |Language aptitude |13 |

For the detailed questionnaire and description of the adapted questionnaire, see Appendix 3.

3.5. Procedures

This section presents three procedures of this empirical research: treatment instruction procedure, test administration procedure, and questionnaire administration procedure.

Treatment Instruction Procedure: The study is conducted during regular class time. The students are not informed about the research. All students receive an equal amount of IGI on English verb tenses from the same instructor, who is also the researcher.

The Level-Seven Grammar Course has nine class meetings. Each meeting per week lasts for 135 minutes. The pre-test, seven topics about English tenses and three review exercises are discussed during eight weeks. The post-test, the questionnaire, and some remarks and feedback of the course occur in the last meeting occasion. Moreover, three review exercises are set as homework; first because the time in class is limited and because students need time to absorb the knowledge they recieve. In class, students have to check answers together and exchange opinions. The teacher only takes part in the discussion whenever the students ask for an explanation or when they cannot come to a final conclusion.

Test Administration Procedure: The participants complete a pretest at the beginning of the course. The teacher gives some explanation about the purpose of the test as a diagnostic test before the course and does not say about the purpose of the research. Post-testing occurs after the instructional treatments. The two tests take place during the timetabled class hours. During the testing time, students do not permit to use dictionaries, exchange information to each other, or ask for help from anyone. All the tests are collected and kept well. The researcher is the first rater, and another teacher is asked to check for the second time to avoid any missed answer. The pre-test do not give back to the students until the end of the course. The photocopy of the two tests is given to the students at the tenth week – an extra meeting of the course. Students might check their scores and give any feedback.

Questionnaire Administration Procedure: The questionnaire is administered to students for about 20 minutes after the post-test. This questionnaire is anonymous, but the students’ code is written on the top of the page before it is delivered to them. Students are first given a brief verbal explanation of the purpose of the survey, followed by specific clarification of the each question. All of the completed questionnaires are collected immediately at the end of the class. The students fill out the questionnaire without a time limit. None of the students denies participating. The responses of the questionnaire keep confidential. All the items are checked carefully for the missing items. Therefore, there is no missing item in the questionnaire data.