Basic test cases examples

advantages and disadvantages of objective type tests building construction objective questions
Dr.PeterCena Profile Pic
Dr.PeterCena,Swaziland,Researcher
Published Date:02-07-2017
Your Website URL(Optional)
Comment
Is This a Is This a Is This a Is This a Is This a Trick Trick Trick Trick Trick Question? Question? Question? Question? Question? A Short Guide to Writing Effective Test QuestionsIs This a Is This a Is This a Is This a Is This a Trick Trick Trick Trick Trick Question? Question? Question? Question? Question? A Short Guide to Writing Effective Test Questions Designed & Developed by: Ben Clay Kansas Curriculum Center Formatting & Text Processing by: Esperanza Root This publication was developed by the Kansas Curriculum Center with funds provided by the Kansas State Department of Education. First printing: October, 2001Table of Contents Table of Contents Table of Contents Table of Contents Table of Contents Preface ............................................................................................ i-ii Pre-Test ........................................................................................... 1-2 Generally ........................................................................................ 3-5 General Tips About Testing ............................................... 3-4 When to Use Essay or Objective Tests ............................... 4-5 Matching Learning Objectives with Test Items ...................... 5 Planning the Test .......................................................................... 6-12 Cognitive Complexity ........................................................ 6-7 Content Quality .................................................................... 8 Meaningfulness .................................................................... 8 Language Appropriateness .................................................... 9 Transfer and Generalizability ................................................ 9 Fairness .............................................................................. 10 Reliability ........................................................................... 10 How to Defeat Student Guessing ........................................ 11 General Test Taking Tips .................................................... 12 Multiple Choice Test Items ......................................................... 13-19 Section Summary................................................................ 13 Test Your Knowledge.......................................................... 14 Suggestions for Writing Multiple Choice Test Items ........ 15-16 Multiple Choice Test Taking Tips ...................................17-18 Aim for Higher Levels of Learning....................................... 19 True-False Test Items................................................................... 20-26 Section Summary................................................................ 20 Test Your Knowledge.......................................................... 21 Suggestions for Writing True-False Test Items ................. 22-23 Extreme Modifiers and Qualifiers ........................................ 23 True-False Test Taking Tips ................................................. 24 Variations in Writing True-False Test Items ..................... 24-25 Aim for Higher Levels of Learning....................................... 26 Matching Test Items .................................................................... 27-33 Section Summary................................................................ 27 Test Your Knowledge.....................................................28-29 Suggestions for Writing Matching Test Items .................. 30-31 Matching Test Taking Tips .................................................. 32 Variations for Creating Matching Tests ................................ 33Completion or Fill-in-the-Blank Test Items .................................. 34-37 Section Summary................................................................ 34 Test Your Knowledge.......................................................... 35 Suggestions for Writing Completion Test Items .............. 36-37 Completion Test Taking Tips .............................................. 37 Essay Test Items .......................................................................... 38-44 Section Summary................................................................ 38 "I'd Like to Use Essay Tests, But…" ..................................... 39 Read'Em and Weep Essay Test Items................................... 39 Test Your Knowledge.......................................................... 40 Suggestions for Writing Essay Test Items ........................ 41-42 Four-Step Process in Grading Essay Tests ............................ 43 Essay Test Taking Tips......................................................... 44 Additional Types of Test Items .....................................................45-51 Problem Solving ................................................................. 45 Using Authentic Assessments ......................................... 46-47 Grading Authentic Assessments .......................................... 48 Rubric Development ..................................................... 48-51 Etc…Etc…Etc… ........................................................................... 52-60 Purpose of Testing .............................................................. 52 Tips on Test Construction ................................................... 52 Test Layout Tips.................................................................. 52 Returning Tests and Giving Feedback ................................. 53 Alternative Testing Modes .................................................. 54 Creating Fair Tests and Testing Fairly .................................. 55 "I'd Like to Use Essay Tests, But…" ................................ 56-57 Test Administration Assignment .......................................... 58 Cognitive Domain Guide.................................................... 59 Affective Domain Guide ..................................................... 60 Bibliography ............................................................................... 61-63Preface… Preface… Preface… Preface… Preface… A notable concern of many teachers is that they frequently have the task of constructing tests but have relatively little training or Research indicates… information to rely on in this task. Is This a Trick Question? is an Teachers tend to use tests that information sourcebook for writing effective test questions. The they have prepared themselves central focus of the sourcebook’s content is derived from standards much more often than any other developed by the National Center for Research on Evaluation, Stan- type of test. (How Teaching Mat- dards, and Student Testing (CRESST). ters, NCATE, Oct. 2000) CRESST’s criteria for establishing the technical quality of a test While assessment options are di- encompasses seven areas: cognitive complexity, content quality, verse, most classroom educators meaningfulness, language appropriateness, transfer and rely on text and curriculum-em- generalizability, fairness, and reliability. Each aspect is discussed in bedded questions and tests that the sourcebook in a straight-forward, jargon-free style. are overwhelmingly classified as paper-and-pencil (National Com- Part One contains information concerning general test construction mission on Teaching and and introduces the six levels of intellectual understanding: knowl- America’s Future, 1996). edge, comprehension, application, analysis, synthesis, and evalua- tion. These levels of understanding assist in categorizing test Formal training in paper-and-pen- questions, with knowledge as the lowest level. Since teachers tend cil test construction may occur at to construct questions in the knowledge category 80% to 90% of the the preservice level (52% of the time, throughout the sourcebook are examples of or suggestions for time) or as inservice preparation developing higher order thinking skills. This supports Kansas’ (21%). A significant number of current Quality Performance Accreditation initiative which has professional educators (48%) re- established content and performance standards that cannot be port no formal training in devel- measured by low-level tests. oping, administering, scoring, and interpreting tests (Education Part Two of the information sourcebook is devoted to actual test Week, “National Survey of Public question construction. Because of the diversity of assessment School Teachers, 2000”). options, the sourcebook focuses primarily on paper-and-pencil tests, the most common type of teacher-prepared assessment. Five Students report a higher level of test item types are discussed: multiple choice, true-false, matching, test anxiety over teacher-made completion, and essay. Information covers the appropriate use of tests (64%) than over standard- each item type, advantages and disadvantages of each item type, ized tests (30%). The top three and characteristics of well written items. Suggestions for addressing reasons why: poor test construc- higher order thinking skills for each item type are also presented. tion, irrelevant or obscure mate- rial coverage, and unclear direc- This sourcebook was developed to accomplish three outcomes: tions. (NCATE, “Summary Data Teachers will know and follow appropriate principles for devel- on Teacher Effectiveness, Teacher oping and using assessment methods in their teaching, avoiding Quality, and Teacher Qualifica- common pitfalls in student assessment. tions”, 2001.) (Continued on next page…) i Teachers will be able to identify and accommodate the limitations of different informal and formal assessment methods. Teachers will gain an awareness that certain assessment ap- In Kansas… proaches can be incompatible with certain instructional goals. The Kansas Commission on Teaching and America’s Future These three outcomes directly support the standards developed by (KCTAF), chaired by Dr. Andy a joint commission established by the National Education Associa- Tompkins, Kansas Commissioner tion, the American Federation of Teachers, and the National Coun- of Education, proposes to “de- cil on Measurement in Education. The initial standards were velop higher-quality alternative identified in 1990 and revised in 1999. In May 2001, a new listing pathways to teaching” as well as was issued under the title “Standards for Teacher Competence in to “reinvent teacher preparation Educational Assessment of Students”. The first two standards and professional development.” directly reflect the outcomes of this sourcebook: As secondary and postsecondary Teachers should be skilled in choosing assessment methods institutions are exploring (out of appropriate for instructional discussion necessity mostly) alternatives to Teachers should be skilled in developing assessment methods traditional teacher recruitment, appropriate for instructional decisions. the need for training in assess- ment procedures and paper-and- While no one document can thoroughly address the needs and pencil test construction in par- concerns expressed in all of this information, this sourcebook can ticular, become more and more be a valuable resource for any teacher who is interested in measur- evident. ing outcomes of significance, tapping into higher-level thinking and problem solving skills, and constructing tests that effectively and fairly capture what a student knows. Ben Clay, Coordinator Kansas Curriculum Center iiPre-Test Pre-Test Pre-Test Pre-Test Pre-Test Two general categories of Test Item Quiz test items Circle the correct answer 1. Objective items which require students to select the T=True F=False ?=Unsure correct response from several alternatives or to supply a 1. Essay exams are easier to construct word or short phrase to answer than are objective exams. T F ? a question or complete a statement 2. Essay exams require more thorough student preparation and study time 2. Subjective or essay items than objective exams. T F ? which permit the student to organize and present an original answer. 3. Essay exams require writing skills where objective exams do not. T F ? Objective items include: multiple choice 4. Essay exams teach a person how true-false to write. T F ? matching completion 5. Essay exams are more subjective in nature than are objective exams. T F ? Subjective items include: short-answer essay 6. Objective exams encourage guess- extended-response essay problem solving ing more so than essay exams. T F ? performance test items 7. Essay exams limit the extent of content covered. T F ? Test your knowledge of 8. Essay and objective exams can be used to measure the same these two item content or ability. T F ? types by 9. Essay and objective exams are answering the both good ways to evaluate a student’s level of knowledge. T F ? following Answers on next page… questions 1Quiz Answers Quiz Answers Quiz Answers Quiz Answers Quiz Answers 1. Essay exams are easier to construct than are objective exams. TRUE Essay items are generally easier and less time consuming to construct than are most objective test items. Technically correct and content appropriate mul- tiple choice and true-false test items require an extensive amount of time to write and revise. 2. Essay exams require more thorough student preparation and study time than objective exams. ? (QUESTION MARK) According to research findings it is still undetermined whether or not essay tests require or facilitate more thorough (or even different) student study preparation. 3. Essay exams require writing skills where objective exams do not. TRUE Writing skills do affect a student’s ability to communicate the correct “fac- tual” information through an essay response. Consequently, students with good writing skills have an advantage over students who do not. 4. Essay exams teach a person how to write. FALSE Essays do not teach a student how to write but they can emphasize the importance of being able to communicate through writing. Constant use of essay tests may encourage the knowledgeable but poor writing student to improve his/ her writing ability in order to improve performance. 5. Essay exams are more subjective in nature than are objective exams. TRUE Essays are more subjective in nature due to their susceptibility to scoring influences. Different readers can rate identical responses differently, the same reader can rate the same paper differently over time, the handwriting, neatness or punctuation can unintentionally affect a paper’s grade. 6. Objective exams encourage guessing more so than essay exams. ? (QUESTION MARK) Both item types encourage some guessing. Multiple choice, true-false and matching items can be correctly answered through blind guessing, yet essay items can be responded to satisfactorily through well written bluffing. 7. Essay exams limit the extent of content covered. TRUE Due to the extent of time required to respond to an essay question, only a few essay questions can be included on a exam. A larger number of objective items can be tested in the same amount of time, covering more content. 8. Essay and objective exams can be used to measure the same content or ability. TRUE Both item types can measure similar content or learning objectives. Re- search has shown that students respond almost identically to essay and objective test items covering the same content. 9. Essay and objective exams are both good ways to evaluate a student’s level of knowledge. TRUE Both objective and essay test items are good devices for measuring student achievement. However, as seen in the previous quiz answers, there are particular measurement situations where one item type is more appropriate than the other. 2Generally… Generally… Generally… Generally… Generally… General Tips About Testing Creating a test is Length of Test one of the most In theory, the more items a test has, the more reliable it is. On a short test a few wrong answers can have a great effect on the over- challenging tasks all results. On a long test, a few wrong answers will not influence confronting an the results as much. A long test does have drawbacks. If a test is too long, and particularly if students are doing the same kind of instructor. item over and over, they may get tired and not respond accurately or seriously. If a test needs to be lengthy, divide it into sections Unfortunately, with different kinds of tasks, to maintain the student's interest. many of us have Clear, Concise Instructions had little, if any, It is necessary to give clear, concise instructions. It is useful to provide an example of a worked problem, which helps the stu- preparation in dents understand exactly what is necessary. What seems to be writing tests. clear to the writer may be unclear to someone else. Mix It Up Well constructed tests motivate It is often advantageous to mix types of items (multiple choice, students and reinforce learning. true-false, essay) on a written exam or to mix types of exams (a Well constructed tests enable performance component with a written component). Weaknesses teachers to assess the students connected with one kind of item or component or in students’ test mastery of course objectives. taking skills will be minimized. Tests also provide feedback on Test Early teaching, often showing what was It is helpful for instructors to test early in the term and consider or was not communicated clearly. discounting the first test if results are poor. Students often need a practice test to understand the format each instructor uses and an- ticipate the best way to prepare for and take particular tests. While always Test Frequently demanding, test Frequent testing helps students to avoid getting behind, provides instructors with multiple sources of information to use in comput- writing may be ing the final course grade (thus minimizing the effect of “bad days”), made easier by and gives students regular feedback. It is important to test various topics in proportion to the emphasis given in class. Students will considering the expect this practice and will study with this expectation. following Check For Accuracy suggestions for Instructors should be cautious about using tests written by others. Often, items developed by a previous instructor, a textbook pub- general test lisher, etc., can save a lot of time, but they should be checked for accuracy and appropriateness in the given course. construction. (Continued on next page…) 3General Tips About Testing (Continued from previous page) Proofread Exams On written exams, it is important to proofread exams carefully and, when possible, have another person proofread them. Tiny mis- takes, such as misnumbering the responses, can cause big prob- lems later. Collation should also be checked carefully, since miss- What makes a test ing pages can cause a great deal of trouble. good or bad? The One Wrong Answer most basic and obvious Generally, on either a written or performance test, it is wise to avoid having separate items or tasks depend upon answers or skills answer to that required in previous items or tasks. A student’s initial mistake will question is that good be perpetuated over the course of succeeding items or tasks, pe- tests measure what nalizing the student repeatedly for one error. you want to measure, Special Considerations and bad tests do not. It is important to anticipate special considerations that learning dis- abled students or non-native speakers may need. The instructor needs to anticipate special needs in advance and decide whether or not students will be allowed the use of dictionaries, extra time, separate testing sites, or other special conditions. A Little Humor Instructors have found that using a little humor or placing less dif- ficult items or tasks at the beginning of an exam can help students with test anxiety to reduce their preliminary tension and thus pro- vide a more accurate demonstration of their progress. When to Use Essay or Objective Tests When to Use Essay or Objective Tests When to Use Essay or Objective Tests When to Use Essay or Objective Tests When to Use Essay or Objective Tests Essay tests are appropriate when: It is always tempting the group to be tested is small and the test is not to be reused. to emphasize the you wish to encourage and reward the development of student parts of the course skill in writing. you are more interested in exploring the student’s attitudes than in that are easiest to measuring his/her achievement. test, rather than the parts that are Objective tests are appropriate when: the group to be tested is large and the test may be reused. important to test. highly reliable scores must be obtained as efficiently as possible. impartiality of evaluation, fairness, and freedom from possible test scoring influences are essential. (Continued on next page…) 4Conventional wisdom When to Use Essay or Objective Tests When to Use Essay or Objective Tests When to Use Essay or Objective Tests When to Use Essay or Objective Tests When to Use Essay or Objective Tests accurately portrays short- (Continued from previous page) answer and essay Either essay or objective tests can be used to: examinations as the easiest measure almost any important educational achievement to write and the most a written test can measure. difficult to grade, test understanding and ability to apply principles. particularly if they are test ability to think critically. graded well. test ability to solve problems. The matching of Matching Learning Objectives learning objective with Test Items expectations with Instructions: Below are four test item categories labeled certain item types A, B, C, and D. Following these test item categories are sample learning objectives. On the line to the left of each provides a high learning objective, place the letter of the most appropriate degree of test test item category. validity: testing A = Objective Test Item (multiple choice, true-false, matching) what is supposed B = Performance Test Item to be tested C = Essay Test Item (extended response) D= Essay Test Item (short answer) ____1. Name the parts of the human skeleton Certain item types are better ____2. Appraise a composition on the basis of suited than others for measuring its organization particular learning objectives. ____3. Demonstrate safe laboratory skills For example, learning objectives requiring the student to demon- ____4. Cite four examples of satire that Twain strate or to show, may be better uses in Huckleberry Finn measured by performance test ____5. Design a logo for a web page items, whereas objectives requir- ing the student to explain or to ____6. Describe the impact of a bull market describe may be better measured ____7. Diagnose a physical ailment by essay test items. ____8. List important mental attributes necessary To further illustrate this principle, for an athlete several sample learning objec- ____9. Categorize great American fiction writers tives and appropriate test items are provided on the right. Match ____10. Analyze the major causes of learning the most suitable test item with disabilities each of the learning objectives. 5 Answers: 1-A, 2-C, 3-B, 4-D, 5-B, 6-C, 7-B, 8-D, 9-A, 10-CPlanning the Test… Planning the Test… Planning the Test… Planning the Test… Planning the Test… Criteria for Establishing By definition no Technical Quality of a Test test can be truly 1. Cognitive Complexity objective: Standard: The test questions will focus on appropriate existing as an intellectual activity ranging from simple recall of facts to problem solving, critical thinking, and reasoning. object of fact, Cognitive complexity refers to the various levels of learning independent of that can be tested. A good test reflects the goals of the instruction. If the instructor is mainly concerned with students the mind. memorizing facts, the test should ask for simple recall of material. If the instructor is trying to develop analytic skills, a test that asks for recall is inappropriate and will cause students In general, to conclude that memorization is the instructor's true goal. test items should… Refreshing the old bloom… Assess achievement of instruc- During the 1948 convention of the American Psychological tional objectives Association, a group of educational psychologists decided it Measure important aspects of would be useful to classify different levels of understanding the subject (concepts and con- that students can achieve in a course. ceptual relations) In 1956, after extensive research on educational goals, the Accurately reflect the empha- group published its findings in a book edited by Dr. Ben- sis placed on important aspects jamin S. Bloom, a Harvard professor. Bloom’s Taxonomy of of instruction Educational Objectives lists six levels of intellectual Measure an appropriate level understanding: of student knowledge Knowledge Analysis Vary in levels of difficulty Comprehension Synthesis Application Evaluation These levels of understanding assist in categorizing test ques- Implying that one type of tions. Teachers tend to ask questions in the knowledge cat- question is automatically egory 80% to 90% of the time. These questions are not bad, objective and the other but using them all the time is. Try to utilize higher order level of questions. These questions require much more brain necessarily subjective is a power. (See the next page for a definition and sample ques- faulty assumption, since tion frames for each level of learning.) bias can occur with either Adapted from material developed by the National Center for Research on Evaluation, type of test. Standards, and Student Testing (CRESST). 6See pages 59 & 60 for Cognitive and Affective Domain Guides. 1. Cognitive Complexity (continued) Knowledge Recognizing and recalling information, Sample Question Frames including dates, events, persons, places; Who invented the…? terms, definitions; facts, principles, What is meant by…? theories; methods and procedures Where is the…? Comprehension Understanding the meaning of informa- Sample Question Frames tion, including restating (in own words); Restate in your own words…? translating from one form to another; or Convert fractions into…? interpreting, explaining, and List three reasons for…? summarizing. Application Applying general rules, methods, or Sample Question Frames principles to a new situation, including How is...an example of... ? classifying something as a specific How is...related to... ? example of a general principle or using Why is...significant? a formula to solve a problem. Analysis Identifying the organization and patterns Sample Question Frames within a system by identifying its compo- What are the parts of... ? nent parts and the relationships among Classify ...according to... the components. Outline/diagram... Synthesis Discovering/creating new connections, Sample Question Frames generalizations, patterns, or perspectives; What would you infer from... ? combining ideas to form a new whole. What ideas can you add to... ? How would you create a... ? Evaluation Using evidence and reasoned argument Sample Question Frames to judge how well a proposal would Do you agree…? accomplish a particular purpose; How would you decide about... ? resolving controversies or differences What priority would you give... ? of opinion. 7Criteria for Establishing Technical Quality of a Test (continued) To Achieve Content Quality… 2. Content Quality The first activity in planning a test Standard: The test questions will permit students to demon- is to outline the actual course con- strate their knowledge of challenging and important subject tent that the test will cover. A con- matter. venient way of accomplishing this is to take a few minutes following Some important questions need to be answered concerning each class to list on an index card the content quality of the test. What are the test specifica- the important concepts covered in tions? What skills do they indicate will be tested. How many class and in assigned reading for questions and how many areas will be covered? How many that day. These cards can then be sections will there be? What formats will be used to test? used later as a source of test items. If an instructor has focused on the War of 1812 in the major- An even more conscientious ap- ity of the class sessions and activities, this emphasis should proach would be to construct the be reflected in the test. A test that covers a much broader test items themselves after each period will be regarded as unfair by the students, even if the class. The advantage of either of instructor has told them that they are responsible for material these approaches is that the result- that has not been discussed in class. Students go by instruc- ing test is likely to be a better rep- tors' implicit values more than their stated ones. resentation of course activity. 3. Meaningfulness Standard: The test questions will be worth students’ time and students will recognize and understand their value. To Achieve Meaningfulness… "In my opinion, students should not be forced to guess what It is very easy to will be on a test, or psych-out the teacher to decide what to study. Research shows that the less able students are heavily write items which penalized by a failure to realize what is required for a test. require only rote The more able students seem to sense what the teacher wants, but the students most in need of help are likely to flounder recall but are even more painfully if they must guess what to study. nonetheless difficult because they are "The obvious solution to this problem is to give students spe- cific study questions, then draw the test from the study ques- taken from obscure tions. Sometimes this is criticized as teaching the test, as if passages (footnotes, having study questions in itself encourages a superficial ap- for instance). proach. That may be true if there are very few study ques- tions. However, if a teacher offers questions for all of the most important ideas in an assignment, then teaching the test is teaching the course." Russell A. Dewey, PhD Georgia Southern University, Statesboro, GA 8Criteria for Establishing Technical Quality of a Test (continued) Preliminary findings by the National 4. Language Appropriateness Center for Research on Evaluation, Standards, and Student Testing (CRESST): Standard: The language demands will be clear and appro- priate to the assessment tasks and to students. Results of Applying Language Evaluation Criteria to Test questions should reflect the language that is used in the Standardized Content Test Items classroom. Test items should be stated in simple, clear lan- guage, free of nonfunctional material and extraneous clues. Math and science subsections: Test items should also be free of race, ethnic, and sex bias. 67% percent of items had gen- Beyond these two qualifications, students' language back- eral vocabulary evaluated as grounds impact their performance on tests. The vocabulary uncommon or used in an (uncommon usage; nonliteral usage) and the syntax of the atypical manner; 33% of items test (atypical parts of speech; complex structures) may create had syntactic structures eval- language barriers. uated as complex or atypical in their construction. Modifications of the test for students that are limited English proficient include: assessment in the native language; text Reading comprehension: Same changes in vocabulary; modification of linguistic complex- as above for vocabulary and ity; addition of visual supports; use of glossaries in native lan- syntax; 50% of items also had guage; use of glossaries in English; linguistic modification of discourse level demands. test directions; and additional example items/tasks. To reduce frustration for good stu- dents, avoid all of these and none 5. Transfer and Generalizability of these and both a & b answers. These items are acceptable from Standard: Successful performance on the test will allow valid a theoretical standpoint, but most generalizations about achievement to be made. prepared test-takers dislike them As an example, the more subject Presentations, scenarios, projects and portfolios add dimen- matter a student knows, the easier sions to assessment that traditional testing cannot. Teachers it is to make arguments in favor can make valid generalizations about achievement more eas- of answers that the teacher might ily using authentic and performance assessments. These gen- regard as wrong. eralizations may involve instructional placement decisions, formative evaluation decisions and diagnostic decisions. Well True-false questions are the worst constructed tests—whether they are objective or performance of all in this regard. Often the oriented—allow teachers to understand what needs to be truth value of an isolated state- taught next. Teachers are also able to monitor a student’s ment is quite debatable It all learning, while instruction is underway, and can change the depends on how it is interpreted, instruction program as needed. the definition of a key term, or the context. 9Criteria for Establishing Technical Quality of a Test (continued) Five hundred secondary and 6. Fairness postsecondary students were surveyed for suggestions Standard: Student performance will be measured in a way on how an instructor could that does not give advantage to factors irrelevant to school grade fairly and accurately. learning; scoring schemes will be similarly equitable. Here are the top 10 responses. Here are a few basic rules of fairness: test questions should reflect the objectives of the unit; expectations should be clearly Consider grading based only on known by the students; each test item should present a clearly mastery of material and not on formulated task; one item should not aide in answering personalities or perceived effort. another; ample time for test completion should be allowed; and assignment of points should be determined before the Do not over emphasize grades. test is administered. Emphasize learning over grades. Grading constructively requires the instructor to provide Keep students informed of their feedback (written and/or oral) that helps the students to progress throughout the term. appreciate what they achieved and did not achieve by taking Clearly state grading policies and the test. This feedback could include the following: procedures in the syllabus and encouraging comments on a test or paper that convey respect review them with the class for what the student attempted to accomplish; praise for what during orientation. the student did accomplish and suggestions for improving performance. Avoid modifying policies during the term. 7. Reliability Provide plenty of opportunities for assessment. This will avoid Standard: Answers to test questions will be consistently unnecessary pressure and allow trusted to represent what students know. for some mistakes. The whole point of testing is to encourage learning. A good Provide some choice in format test is designed with items that are not easily guessed without or topic when assigning work. proper studying. It is possible to construct all types of test questions which are not readily guessed and therefore require Keep accurate records of grades. a student to comprehend basic factual material. Record numerical grades, rather than letter grades, when Multiple choice questions are widely scorned as multiple possible. guess questions. The solution to this problem is to design Consider allowing rewrites multiple choice items so that students who know the subject on papers. or material adequately are more likely to choose the correct alternative and students with less adequate knowledge are If many do poorly on an exam, more likely to choose a wrong alternative. (On the next page schedule an exam for the follow- are suggestions on how to defeat the TEST-WISE strategies of ing week to retest the class. students who do not study.) 10How to Defeat the Common Rules of Thumb How to Defeat the Common Rules of Thumb How to Defeat the Common Rules of Thumb How to Defeat the Common Rules of Thumb How to Defeat the Common Rules of Thumb Which Students Use to Guess Correct Answers Which Students Use to Guess Correct Answers Which Students Use to Guess Correct Answers Which Students Use to Guess Correct Answers Which Students Use to Guess Correct Answers Rule of thumb: Pick the longest answer. Way to defeat this strategy: Make sure the longest answer is right about a fifth of the time (if there are five alternatives for each question). Rule of thumb: Pick the ‘b’ alternative. Way to defeat this strategy: Make sure each answer is used the same number of times, in random order. Rule of thumb: Never pick an answer which uses the word ‘always’ or ‘never’ in it. Way to defeat this strategy: Make sure such answers are correct about a fifth of the time. Rule of thumb: If there are two answers which express opposites, pick one or the other and ignore other alternatives. Way to defeat this strategy: Sometimes offer opposites when neither is correct. Rule of thumb: If in doubt, guess. Way to minimize the impact of this strategy: Use five alternatives instead of three or four. Rule of thumb: Pick the scientific-sounding answer. Way to defeat this strategy: Use scientific sounding jargon in wrong answers Rule of thumb: Do not pick an answer which is too simple or obvious. Way to defeat this strategy: Sometimes make the simple, obvious answer the correct one. Rule of thumb: Pick a word which you remember was related to the topic. Way to defeat this strategy: When drawing up distractors (wrong answers) use terminology from the same area of the text as the right answer, but in distracters use those words incorrectly so the wrong answers are definitely wrong. 11Criteria for Establishing Technical Quality of a Test (continued) 7. Reliability (continued) "Remind, remind, remind students to Studies have shown that the grade given to an essay test depend in part upon the neatness of the handwriting. That stop and ask for seems like a poor way to assign a grade. However, if stu- directions or dents are asked to do the test on a word processor, it is hard clarification if there is to ensure that the work is original. Studies have also shown that grades for essay tests are influenced by length. If a stu- something they don’t dent rambles on, there is greater likelihood of hitting a few understand. points that the teacher is looking for. But do we want to reward verbosity? Directions are the roadmap to their final Despite all this, essay and short answer tests have many vir- destination." tues. Students need practice formulating arguments, express- ing things clearly, and integrating ideas. Nobody would ar- gue that all testing should be multiple choice. However, for teachers in many situations, a good objective test is both Suggestion… fairer and more efficient than an essay or short answer test. Encourage students to design One way to ensure reliability is to share with your students… their own test. This will help them anticipate some of the questions or information to be General Test Taking Tips included on the instructor’s exam. 1. Tell students to survey the entire test before they begin. This will help them identify which section will be quick and/or easy and which will require more time and thought. Various kinds of objective 2. Encourage students to underline important words in the directions such as list, discuss, define, etc. and essay test items are presented in the following 3. Instruct students that when they take a test, they should do the easy questions first. sections of this document. 4. Help students schedule their time by estimating the total Each kind of test item is time available compared to the number of questions on the test. They need to recognize that some types of ques- briefly described in terms tions will take longer than others. of advantages and 5. Suggest that students put a checkmark next to any ques- limitations for use. tions which they left blank and will need to come back to for completion later. General suggestions are 6. Prompt students to hold onto their test until they have also presented for the looked it over thoroughly. They should make sure they construction of each test have completed each task and have reread the entire test item variation. to verify that they have given the answers they intended. 12Multiple Choice Test Items Multiple Choice Test Items Multiple Choice Test Items Multiple Choice Test Items Multiple Choice Test Items The multiple choice item consists of the stem, which identifies the question or problem and the response alternatives or choices. "…almost any well Usually, students are asked to select the one alternative that best completes a statement or answers a question. For example, defined cognitive Item Stem: Which of the following is a chemical change? objective can be tested Response Alternatives: a. Evaporation of alcohol fairly in a multiple b. Freezing of water choice format." c. Burning of oil " d. Melting of wax Multiple choice items are considered to be among the most versa- tile of all item types. They can be used to test factual recall as well Section Summary as levels of understanding and ability to apply learning. As an example, the multiple choice item below is testing not only infor- Good for: mation recall but also the ability to use judgment in analyzing and Application, synthesis, evaluating. analysis, and evaluation levels Multiple choice tests can be used to test the ability to: 1. recall memorized information Types: 2. apply theory to routine cases Question/Right answer 3. apply theory to novel situations Incomplete statement 4. use judgment in analyzing and evaluating Best answer A. 1 only Advantages: B. 1 and 2 only Very effective C. 1, 2 and 3 only Versatile at all levels D. 1, 2, 3 and 4 " Minimum of writing for student Multiple choice items can also provide an excellent basis for post- Guessing reduced test discussion, especially if the discussion addresses why the in- Can cover broad range correct responses were wrong as well as why the correct responses of content were right. Unfortunately, multiple choice items are difficult and time consuming to construct well. They may also appear too dis- Disadvantages: criminating (picky) to students, especially when the alternatives Difficult to construct good are well constructed and are open to misinterpretation by students test items who read more into questions than is there. Difficult to come up with plausible distractors/alterna- Test your knowledge of tive responses multiple choice tests by taking the multiple choice test on the next page… 13Circle the Most Correct Answer Circle the Most Correct Answer Circle the Most Correct Answer Circle the Most Correct Answer Circle the Most Correct Answer 1. Multiple choice items provide highly 5. The right answers in multiple choice reliable test scores because: questions tend to be: A. they do not place a high degree of A. longer and more descriptive dependence on the students reading B. the same length as the wrong answers ability C. at least a paragraph long B. they place a high degree of depen- D. short dence on a teacher's writing ability C. they are a subjective measurement of student achievement 6. When guessing on a multiple choice D. they allow a wide sampling of question with numbers in the answer: content and a reduced guessing factor A. always pick the most extreme B. pick the lowest number C. pick answers in the middle range 2. You should: D. always pick C A. always decide on an answer before reading the alternatives B. always review your marked exams 7. What is the process of elimination in a C. never change an answer multiple choice question? D. always do the multiple choice items A. skipping the entire question on an exam first B. eliminating all answers with extreme modifiers C. just guessing 3. The above multiple choice item is D. eliminating the wrong answers structurally undesirable because: A. a direct question is more desirable than a incomplete statement 7. What should you not do when taking B. there is no explicit problem or a multiple choice test: information in the stem A. pay attention to patterns C. the alternatives are not all plausible B. listen to last minute instructions D. all of the above C. read each question carefully E. A & B only D. read all choices F. B & C only G. A & C only H. none of the above 8. It is unlikely that a student who is un- skilled in untangling negative statements will: 4. The above multiple choice item is A. quickly understand multiple choice undesirable because: items not written in this way A. it relies on an answer required in a B. not quickly understand multiple choice previous item items written in this way B. the stem does not supply enough C. quickly understand multiple choice information items written in this way C. eight alternatives are too many and D. not quickly understand multiple choice too confusing to the student items not written in this way D. more alternatives just encourage guessing 14 Answers: 1-D, 2-B, 3-D, 4-C, 5-A, 6-C, 7-D, 8-C