PLANNING AND CONSTRUCTING ACHIEVEMENT TESTS

BY
DR UCHENNA UDEANI
DEPARTMENT OF SCIENCE AND TECHNOLOGY EDUCATION
UNIVERSITY OF LAGOS
AKOKA, YABA, LAGOS.

PAPER PRESENTED AT THE TRAINING PROGRAMME ON IMPROVING THE ASSESSMENT AND EVALUATION SKILLS OF SCHOOL TEACHERS
11TH -15TH APRIL 2011

1.0 INTRODUCTION The amount of effort involved in constructing an educational test varies with the type of test and the purpose for which it is intended. Most classroom teachers probably spend relatively little or no time in preparing essay or short answer tests for measuring students’ progress in a unit of instruction. The procedures employed in constructing a test also vary with the type of test and the aims of the users. Whatever the kind of test or the goals of the users may be, some content planning is necessary before the items comprising the test are written. 2.0 PURPOSES AND USES OF TESTS The primary purpose of test is to assess behaviour, mental abilities and other personal characteristics in order to assist in making judgements, predictions and decisions about students. More specifically tests are used to * Retain or dismiss, promote and rotate student in education and training programmes. * Evaluate cognitive changes due to educational and other interventions. * Classify and place people in educational and employing contexts. * Counsel and guide individuals for educational, vocational and personal counselling. * Diagnosis of learning difficulties. This typically occurs in an educational setting where the immediate aim is to spot areas in which additional instruction needs to be given or in which teaching methods may need to be altered. * Test as teaching aids: * Motivating students. * Adjustment of teaching to student abilities. * Teaching while testing. 3.0 CLASSIFICATION OF TESTS Tests are classified according to the use to which it would be put into. One method of classifying tests is the dichotomy standardised versus non-standardardized. A standardized test has fixed directions for administration and scoring having been constructed by professional test makers and administered for a representative sample of examinees from the population for whom the test is intended. Various types of converted scores or norms may be computed from the raw test scores of this sample group (standardisation group) of examinees these norms serve as a basis for interpreting the scores of subsequent examinees. More common than published standardised tests are non-standardised classroom tests which are usually constructed in an informal manner by school teachers. Tests are also classified as individual versus group. An individual test such as the Binet-Simon Intelligence Scale (all intelligence tests) is administered to one examinee at a time. A group test can be administered simultaneously to many examinees. The other dichotomy is speed versus power tests. A pure speed test consists of many easy items, but the time limits are very strict and almost no one finishes in the allotted time. The time limits on a power test are ample for most examinees, but the test contains more difficult items than a speed test. A fourth classification is objective versus non objective. This is concerned with the method of scoring the test. An objective test has been fixed, precise scoring standards and can be scored by anybody. Scoring essay tests is on the other hand quick subjective and different scorers may obtain different results. A fifth classification of tests according to content or process is cognitive versus affective. Cognitive tests which attempt to quantify the processes and products of mental activity may be classified as measures of achievement and aptitude. An achievement test which assesses knowledge of some academic subject or occupation focuses on the examinee’s past behaviour (what he or she has already learnt or accomplished). An aptitude test focuses on future behaviour that is what a person is capable of learning with appropriate training. Achievement and aptitude however are not separate entities; what a person has accomplished in the past is usually a fairly good indicator of what can be expected of that person in the future. Affective tests are designed to assess interests, attitudes, values, motives, temperament traits and other non cognitive characteristics of personality. Various techniques such as behavioural observation, paper and pencil inventories and projective pictures, have been designed for these purposes. 4.0 PLANNING A TEST Achievement Tests More attention has been devoted to procedures for constructing scholastic achievement tests. This is understandable when we realise that achievement tests are more widely administered than any other type of psychological assessment instrument. Despite the wide spread use of achievement tests, classroom teachers, who presumably are well acquainted with their subject matter typically devote insufficient time to evaluating student progress Used effectively, however, the results of testing not only evaluate and motivate students, but also provide information to teachers, school administrators and parents concerning the extent to which the specific educational objectives have been attained. The following questions must be answered by planners of classroom achievement tests 1) What are the topics and materials on which the students should be tested 2) What behaviour objectives are they expected to achieve 3) What item and test format layouts should be used 4) How many items should the test contain 5) When, where and how should the test be given 6) What kinds of questions should be constructed 7) How the completed test papers should be scored and evaluated. 5.0 TAXONOMIES OF EDUCATIONAL OBJECTIVES The preparation of a test to measure institutional objectives is most effective when the behaviours to be assessed are clearly defined and at the onset. Since the mid-1950’s, much attention has been given to formal, standard systems of classifying the cognitive, affective and psychomotor objectives of instruction. The most popular taxonomy of educational objective is the one by Bloom (1956). Bloom categorised educational objectives into three major categories-cognitive, affective and psychomotor domains. For the purposes of this lecture we shall only use the cognitive domain. Illustrative Outline of Blooms Cognitive Objectives 1) Knowledge: involves the recall of specific facts. Sample verbs in knowledge items are define, identify, list and name. 2) Comprehension: means understanding the meaning and or purpose of something. Sample verbs in comprehension items are convert, explain and summarise. 3) Application: involves the use of information and ideas in new situations. Sample verbs in application items are compute, determine and solve. 4) Analysis: is breaking down something to reveal its structure and the interrelationship among its parts. Sample verbs are analyse, differentiate, distinguish and relate. 5) Synthesis: is combining various elements or parts into a structural whole. Sample verbs are design, devise, formulate and plan. 6) Evaluation: is making a judgement based on reasoning. Sample verbs are critique, evaluate and judge. TABLE OF SPECIFICATIONS Most test designers do not adhere rigidly to a formal taxonomy in specifying the objectives to be measured. It is helpful in test planning to construct a two-way Table of Specifications. In preparing such a table, the behavioural objectives to be assessed are listed as row headings and the content objectives as column headings. Then the descriptions of specific items falling under the appropriate row and column headings are written in the body (cells) for the table. Table I: A table of specification for a 40-item test in Biology for SSIII | Cognitive Objectives | | Content | 1 | 2 | 3 | 4 | 5 | 6 | Total | Cell structure | 2 | 2 | 3 | 1 | 1 | 1 | 10 | Cell function | 3 | 2 | 2 | 1 | 1 | 1 | 10 | Organs | 2 | 2 | 3 | 1 | 1 | 1 | 10 | Systems | 2 | 2 | 2 | 1 | 1 | 1 | 9 | Taxonomy | 3 | 2 | 2 | 2 | 1 | 1 | 11 | Evolution | 2 | 2 | 2 | 2 | 1 | 1 | 10 | Total | 14 | 12 | 14 | 8 | 6 | 6 | 60 | Key: 1- Knowledge 2- Comprehension 3- Application 4- Analysis 5- Synthesis 6- Evaluation Table 2: A table of specifications for a 60-item test in Biology for SSIII (abridged) | Cognitive Objectives | Total | | Content | 1 | 2 | 3 | | | Cell structure | 4 | 3 | 3 | 10 | | Cell function | 5 | 2 | 3 | 10 | | Organs | 4 | 3 | 3 | 10 | | Systems | 4 | 2 | 3 | 9 | | Taxonomy | 5 | 2 | 4 | 11 | | Evolution | 4 | 2 | 4 | 10 | | Total | 26 | 14 | 20 | 60 | | Key: 1- Knowledge and comprehension 2- Application 3- Analysis, Synthesis and Evaluation Preparing Test Items We shall now turn to the important issue of what types of items to use on a test, Diagram I shows Types of Test Forms. Most teacher-made tests are achievement test items and are of two broad types namely: a) Essay-short answer
Objective test b) Free-response type of test

Essay test. Essay-Long answer
Teacher-made tests

Arrangement type

Completion type
Structured-response type

True/False type

Matching type

Multiple-choice

In the objective test, possible answers to the question are incorporated with the question and the student is required to select the correct answer. In the essay, the students are required to supply the answer. Each of these broad categories may take a number of different forms. The Objective Type The objective test derives its name from the fact that the marking is done with a standard key and is thereby objective. The main concept is an objective item is that the student is provided with a specific problem and is given a limited number of choices from which to select a wanted answer. The main features of the objective item are: 1) Each student chooses his answers for each question from a limited list provided by the test maker (teacher) who also has a predetermined key of desired answers. 2) Each student is given an opportunity to answer a large number of questions 3) Each student functions in a completely structured situation. Types of Objective test Items 1) Short Answer/Completion Types: A short-answer item is a supply-type task on which examinees are required to complete or fill in one or more blanks of an incomplete statement with the correct words or phrases or to give a brief answer to a question. They are among the easiest items to construct, requiring that examinees supply the correct answer rather than simply recognising it. Although short-answer items are especially useful for assessing knowledge of terminology, they have serious limitations * They are not appropriate for measuring complex instructional objectives * There may be more than one correct answer, scoring is not always entirely objective. The following guidelines should be followed in constructing short-answer items: 1) Questions are preferable to incomplete statements. 2) If an incomplete statement is used, word it so that the blank comes at the end. 3) Avoid multiple blanks in the same item especially if they make the meaning of the task unclear. 2) True/False Items: * One of the simplest types of test items to construct. * True-false items can be written and read quickly and so they permit a broad sampling of content. * A notorious shortcoming of these items is that they are often concerned with trivial information or are constructed by lifting statements verbatim from textbooks. * Consequently, they are said to encourage rote memorisation on the part of the examinees. It also encourages guessing. In defence of the true-false items, Ebel (1970) maintained that the extent of students’ command of a particular area of knowledge is indicated by their success in judging the truth or falsity or proposition related to it. Ebel argued that such propositions are expressions of verbal knowledge which he viewed as the essence of educational achievement. Suggestions in constructing true-false test items * Make the statements relatively short and unqualifiedly true or false. * Avoid negatively stated items, especially those containing double negatives. * Avoid ambiguous and tricky items. * On opinion statements, cite the source of authority. Matching Items A set of response options is matched to a set of stimulus options. The task on a matching item is to indicate which response options are paired with which stimulus options. Matching is usually one to one (one response per stimulus) but it may well be one too many, many to one or many to many. Matching items are easy to construct and cover the material more efficiently than other types of items, but they usually measure only the rote memory of facts. In addition, the necessity of making the options homogeneous (all options of the same kind) limits the type of material that can be fitted into a matching framework. Some guidelines for constructing matching items are the following: * Place the stimulus and response options in a clear, logical column format, with the stimulus options in the right column. * Number the stimulus options successively and place letters (a, b, c and so on) before the response options * Place the entire item on a single page Multiple- Choice Items Multiple-choice items are the most versatile of all objective test items in that they can be used to measure both simple and complex learning objectives at all classes and in all subject matter areas. Although constructing an answer to an essay item may demand greater organisational skill than answering a multiple-choice item, responding correctly to a well prepared multiple choice item requires good discriminating ability and not merely skill in recognising or recalling the correct answer. Among the short comings of multiple choice items are that * Good ones are difficult to construct * They emphasise recognition rather than the recall and organisation of information. Guidelines for Writing Multiple-Choice Items * State the specific problem of the question clearly in the stem and at a reading level appropriate for the examinees, but avoid taking questions or statements verbatim from textbooks. * Place as much of the item as possible in the stem. It is inefficient to respect the same words in every option. * Four or five options are typical but good items having only two or three options can be written. * Make all options approximately equal in length, grammatically correct and appropriate in relation to the stem. However, do not let the stem give away the correct option by verbal associations or other clues. * Make all options plausible to examinees that do not know the right answer, but mark only one option correct or best. Popular misconceptions or statements that are only partially correct make good distracters. * In constructing each distracter, formulate a reason why an examinee who does not know the correct answer might select that distracter. * Avoid or at least minimise the use of negative expressions such as “not” in either the stem or options. Simply following these guidelines, which are primarily the products of logic and experience rather than research, will not ensure the construction of a good multiple choice test. The ability to write good items depends as much or more on knowledge of the subject matter of the test, understanding what students should know about the subject matter and the art or skill of asking questions. THE ESSAY TEST Essay tests are well known in Nigeria since it was the mode of testing that colonial education introduced into the continent. Their distinctive features are: 1. Each student answers a small number of questions. 2. The scripts are written in the student’s style in his own words and usually in his own handwriting. 3. Each student is considerably free to organise his own answers. This gives rise to answers with varying degrees of accuracy and completeness. 4. Very often there is a choice of questions.

TYPES OF ESSAY TEST ITEMS 1) Short Answer The pupil is required to provide written answers of a few lines in length to brief questions. Write short notes on any two of the following: a) Prof. Babs Fafunfa b) Dr. Alvan Ikoku c) Dr. J.K Aggrey 2) Long Essay The pupil is required to provide long comprehensive, written answer to two or more pages to a question. For example-write on account of the remote and immediate causes that led to the Nigerian Civil war of 1967-1970. CHARACTERISTICS OF TEST TYPES-A SUMMARY We summarise the relative strengths of Essay (free response) and Objective (structure response) tests in Table 3 | | Essay Test | Objective Test | 1 | Sampling | The candidate can only be asked to answer a very limited number of questions because he has to spend a long time writing his answers. As a result, it does not cover the subject matter being tested. | The candidate can be given a large number of questions within the time available. As a result, it is possible to have a wide coverage of the subject matter being tested. | 2 | Guessing | It is difficult for a person to get the correct answer by simply guessing | A candidate can guess and obtain the correct answer to an item | 3 | Reliability in scoring | It is difficult to get a high reader reliability. This is because no single pattern of response can be taken as the only correct one | The scoring reliability is high because it is possible to agree on one and only one correct answer | 4 | Setting and scoring test | Most of the work goes into the scoring while setting the paper is relatively easy | Setting the test is time consuming and demanding while the scoring can be done by machine | TEST FORM TO USE The question of what test form to use is usually answered on the basis of what form will best measure the particular attribute being measured. The comprehensive measurement of achievement calls for a careful combination of the several forms as appropriate to maximise the benefits derivable from their strong points. A teacher needs to be in a position to use any form to evaluate the work of his students. EXAMPLES OF TYPES OF OBJECTIVE TESTS 1. True/False A statement is presented and the pupil is then required to state whether it is true or false. For example-photosynthesis takes place in both plant and animal cells? True/false 2. Arrangement Type A number of concepts are presented. The pupil must arrange them in a particular order. For example-in what order do the following stages of development occur? Adult, egg, pupa, larva 3. Matching Type Two unequal lists of concepts are presented. The pupil must match the members of one list with those of the other. For example- The books listed on the right were written by authors on the left place the number of the author on the line before the appropriate book. Each author may only be used once. 1. Cyprian Ekwensi - Burning Grass 2. Flora Nwakuche - Arrow of God 3. Chinua Achebe - Jagua Nana 4. James Ngugi - Idu 5. Oladele Taiwo - 6. Anne Akpabot - 4. Completion Type A sentence is presented with one or more missing words, which the pupil must provide. For example- The higher you go up a mountain, the it becomes.

5. Multiple Choice A question is posed and a number of alternative answers are supplied. The pupil must select the correct answer from the alternatives provided. There are several variations of the of the general multiple choice format. The following are examples of such varieties. a) Correct answer variety Who discovered oral polio vaccine? A) Salk B) Pasteur C) Burkett D) Curi E) Waksman b) Best answer variety Which of the following is the best reason for describing polio vaccine? A) Preventive B) Preservative C) Enhance Manpower development D) Counteract disease E) Cure polio c) Incomplete statement variety The discovery of oral polio vaccine was A) Salk B) Pasteur C) Burkett D) Curi E) Waksman d) Negative variety Which one of the following is not true of a virus? A) It can live only in plant and animal cells B) It can reproduce itself C) It is composed of very large, living cells D) It can cause death