What is PAT: Reading Vocabulary Adaptive?
PAT: Reading Vocabulary Adaptive is an online computer adaptive version of a PAT: Reading Vocabulary test. In a PAT: Reading Vocabulary Adaptive test, the questions for each student are automatically selected based on the responses they have given to previous questions. Students who struggle to answer the initial questions correctly are given easier questions until they begin to have some success. Students who answer the initial questions correctly are given progressively harder questions until they start to answer some of them incorrectly.
This means PAT: Reading Vocabulary Adaptive administers tests that are tailored to each individual student. Because the tests are well targeted in terms of difficulty level, an adaptive test will generally provide a more precise result for each student in a group than using one of the existing static PAT: Reading Vocabulary tests.
Who can complete an adaptive test?
The adaptive tests are suitable for students in Year 4 to Year 10. Students should be confident using a computer to complete a test.
How long is an adaptive test?
The length of an adaptive PAT: Reading Vocabulary test is 25 questions. Students should be given 16 minutes to complete the test. Teachers are able to use their discretion if a student requires a little more time to complete the questions.
How difficult are the first questions in an adaptive test?
The adaptive tests select the first question on the basis of the student’s year level. For most students in each year level the first question would be considered as moderately easy. If a student gets the first question correct, the next question is more difficult. Conversely, if the student gets the first question incorrect, the next question is easier. The test is designed so the ‘jumps’ in difficulty are not too big.
How is achievement reported for PAT: Reading Vocabulary Adaptive?
PAT: Reading Vocabulary Adaptive tests report the same scale score and stanine information as the existing static PAT: Reading Vocabulary tests. At the end of an adaptive test the student’s achievement level is located on the PAT: Reading Vocabulary scale as a PATV scale score, and can be compared with the results of an appropriate national reference group using stanines.
Is the number of questions a student gets correct relevant in an adaptive test?
The number of questions a student has answered correctly in an adaptive test should not be used to interpret a result or make comparisons between students. The scale score is the best way to interpret how well a student has done. The adaptive tests are designed so that most students will answer about 60 percent of the questions they are administered correctly. Because the questions are targeted at a students’ achievement level, some students will be administered harder questions than others.
Some of the NZCER Marking service reports will show how many questions a student answered correctly and how many they were administered altogether. The number of questions they were administered is useful to know. Students should do 25 questions in an adaptive test. When the number administered is much lower than this, it will usually mean that the student finished the test prematurely — for instance, by closing their browser. The NZCER Marking site will still report a scale score in this case, but the score will be based on an incomplete assessment, and the margin of error associated with the score will be large. If a student has not finished the test it will be displayed as incomplete. The test status can be found in List and Student reports when viewing the reports tab.
Which reports are available on the NZCER Marking site for an adaptive test?
The NZCER Marking site produces the same suite of reports for an assessment group that has completed an adaptive test as it does for a group that has completed a static test, apart from the item reports. The item reports are not available for an adaptive test because the students in the assessment group will have completed different questions. It is possible to see how each individual student did on the particular questions they were administered. This can be done by clicking on the question links provided in each student’s Individual Report.
How often should students complete an adaptive test?
The bank of PAT: Reading Vocabulary items available for adaptive testing is large enough so that students can complete about two adaptive tests per year without having to repeat a large number of the same questions each time.
Where do the PAT: Reading Vocabulary Adaptive questions come from?
PAT: Reading Vocabulary Adaptive uses questions that are part of the existing static PAT: Reading Vocabulary tests.
Which type of PAT: Reading Vocabulary test should I use?
The adaptive tests and the static tests both have advantages. Which type you choose to use will depend on your reasons for testing and the kind of information you want to produce.
The main advantage of an adaptive test is that it tailors the content of the test to the individual student. This means that each student will be administered a test that is well-targeted to their achievement level. As well as ensuring that all students will have some success in terms of answering questions correctly, a well-targeted test also makes sure that each student’s achievement level is located on the PAT: Reading Vocabulary scale with a good level of precision. This is very difficult to achieve when we use the same static test with all students in a class. The variability between students in a typical New Zealand classroom is often quite large, meaning it is difficult to find one test form that will suit all students in the class. The adaptive PAT: Reading Vocabulary tests are also shorter (25 questions) than the static tests (40 questions).
The fact that the static tests present all students in a group with the same questions however, can sometimes be an advantage. When everyone in a group completes the same questions it becomes possible to look for patterns in the students’ responses that may expose possible weaknesses (or strengths) across a class or year level cohort.
Regardless of which type of PAT Vocabulary Test you decide to use in your school, all results will be able to be reported as scale scores and stanines. Some schools may choose to use one type of test with some students and another type with others.
How do I set up a PAT: Reading Vocabulary Adaptive test?
- The first step in setting up a PAT: Adaptive test is to add a new assessment group on the PAT: Reading Vocabulary section of the NZCER Marking website. ‘PAT Adaptive’ should be selected as the test assigned to this group. Note that the assessment group also needs to be assigned a particular year level.
- Once the assessment group has been created students can be added to it by using either the View button or the Student Tokens button. Usually, one class of students is assigned to an assessment group. However, the students assigned to an assessment group can be a subset of a class or a combination of students from different classes within the same year level.
- Once students have been assigned to the assessment group it is possible to print off the tokens (passwords) the students will need to take the test. Again, either the Student Tokens or View buttons can be used to access the token printing function.
- Students can use their individual token to access the test at www.nzceronline.org.nz.
Administering a PAT: Reading Vocabulary Adaptive test
Administering a PAT: Reading Vocabulary Adaptive test is very similar to administering a static test. The key features of adaptive and static tests are outlined in the table below.
Key features of Adaptive and Static tests
Adaptive Test | Static Test |
Each student gets their own mix of questions targeted at their achievement level. There will be detailed individual level reporting but no group item level reporting available. | All students in the group respond to the same questions. This supports item level reporting. |
Students need to answer each question before they can go on to the next question. | Students can omit a question and come back to it later. |
Students cannot change an answer once they have answered a question and pressed ‘Next’. | Students can return to a question and change their answers. |
Students answer 25 questions. | Students answer 40 questions. |
Before the test
Before the test make sure you have set up an assessment group for an adaptive test in the NZCER Marking website, assigned students to it, and printed off the student tokens (passwords) they will need to do the test (see ‘How do I set up an adaptive test?’ for details.)
Use the ‘demo’ function in the NZCER Marking site to try out an adaptive test yourself.
On the day of the test
Make sure that the assessment group’s online testing button on the NZCER marking service website is turned on (the ‘Online Testing’ button next to the name of the assessment group you are working with should be green).
Make sure each student has an appropriate device that is connected to the internet.
Set up the room so students can work independently and without interruptions. They should bring an activity, such as a book to read, if they finish early.
Starting the test
Distribute the tokens to the students. Make sure each student gets the right token.
Ask the students to open their browser, go to www.nzceronline.org.nz and enter the code printed on their token. Once the code is entered, the computer will present the student with the name of the person assigned to the token and ask if they are that person. If they are not, they should close their browser and come to you to get the correct token.
Read the on-screen instructions page with the students. Emphasise to the students that the test is designed to give them a mix of questions at their level. It is not the type of test where you can necessarily expect to get all the questions correct. The test is trying to find questions that have just the right amount of challenge for each person.
Work through the two example questions together. Make sure students are comfortable using the different item formats. Remind them that they are always looking for the best answer.
Remind students that in the test they will have to provide an answer to each question before they can go on to the next question. Once they have clicked next, they will not be able to go back and change their answer.
When you have finished working through the example questions, ask the students if they have any further questions. When everyone is ready they can click on the start button to begin.
During the test
During the test circulate around the room.You should not assist students to read or answer the questions. If a student has difficulty understanding the instructions or questions, the test may not be suitable for them.
If a student needs to leave the room they can close their browser. When they return they will need to re-enter the code on their token. This will take them back to where they were in the test when they left it.
Completing the test
Students have up to 16 minutes to complete the test. An adaptive test is 25 questions long and most students should be able to complete the test in the time provided. Use your discretion if a student needs a little more time to complete the questions.
When students have completed the test the computer will ask them to provide some feedback by answering a number of questions about their test experience. Please encourage them to complete these questions.
After the test
If a student has not finished their test, it will be displayed as incomplete on the ‘List Report’ and ‘Individual Student Report’ indicated by 2 red asterisks next to the student’s name.