What is PAT: Mathematics Adaptive?
PAT: Mathematics Adaptive is an online computer adaptive version of a PAT: Mathematics test. In a PAT: Mathematics Adaptive test the computer selects the questions for each student based on the responses they have given to previous questions in the test. Students who struggle to answer the initial questions correctly are given easier questions until they begin to have some success. On the other hand, students who do answer the initial questions correctly are given progressively harder questions until they start to answer some of them incorrectly.
In effect, PAT: Mathematics Adaptive administers tests that are tailored to each individual student. Because they are well targeted in terms of difficulty level, an adaptive test will generally provide a more precise result for each student in a group than using one of the existing static PAT: Mathematics tests. In an adaptive test, the computer also makes sure that the questions being administered cover the normal range of mathematical topics assessed by PAT: Mathematics in the right proportions (number knowledge, number strategies, algebra, geometry and measurement, and statistics).
Who can complete an adaptive test?
The adaptive tests are suitable for students in Year 4 to Year 10. Students should be confident using a computer to complete a test.
How long is an adaptive test?
The length of an adaptive test varies from about 30 to 36 questions depending on the year level of the student. Students in the earlier year levels do shorter tests than students in later year levels. Students should be given 45 minutes to complete the test. Teachers are able to use their discretion if a student requires a little more time to complete the questions.
How difficult are the first questions in an adaptive test?
The adaptive tests select the first question on the basis of the student’s year level. For most students in each year level the first question would be considered as moderately easy. If a student gets the first question correct the computer administers a more difficult question next. Conversely, if the student gets the first question incorrect the computer administers an easier question next. The computer is programmed not to make the ‘jumps’ in difficulty too big.
How is achievement reported for PAT: Mathematics Adaptive?
PAT: Mathematics Adaptive tests report the same scale score and stanine information as the existing static PAT: Mathematics tests. At the end of an adaptive test the student’s achievement level is located on the PAT: Mathematics scale as a PATM scale score, and can be compared with the results of an appropriate national reference group using stanines.
Is the number of questions a student gets correct relevant in an adaptive test?
The number of questions a student has answered correctly in an adaptive test should not be used to interpret a result or make comparisons between students. The scale score is the best way to interpret how well a student has done. The adaptive tests are designed so that most students will answer about 60 percent of the questions they are administered correctly. Because the computer selects questions targeted at a students’ achievement level, some students will be administered harder questions than others.
Some of the NZCER Marking service reports will show how many questions a student answered correctly and how many they were administered altogether. The number of questions they were administered is useful to know. Students should do between 30 and 36 questions in an adaptive test depending on their year level. When the number administered is much lower than this, it will usually mean that the student finished the test prematurely — for instance, by closing their browser. The NZCER Marking site will still report a scale score in this case, but the score will be based on an incomplete assessment, and the margin of error associated with the score will be large. If a student has not finished the test it will be displayed as incomplete. The test status can be found in List and Student reports when viewing the reports tab.
Which reports are available on the NZCER Marking site for an adaptive test?
Apart from the item reports, the NZCER Marking site produces the same suite of reports for an assessment group that has completed an adaptive test as it does for a group that has completed one of the existing static tests. The item reports are not available for an adaptive test because the students in the assessment group will have completed different questions. It is possible to see how each individual student did on the particular questions they were administered. This can be done by clicking on the question links provided in each student’s Individual Report.
How often should students complete an adaptive test?
The bank of PAT: Mathematics items available for adaptive testing is large enough so that students can complete about two adaptive tests per year without having to repeat a large number of the same questions each time.
Where do the PAT: Mathematics Adaptive questions come from?
PAT: Mathematics Adaptive draws on two sources of questions. The first is questions used in the existing static PAT: Mathematics tests. The second is a pool of new interactive items that have been developed and trialled by NZCER over the last two years. The interactive items make use of a range of item types that are possible to administer on a computer, such as hot spot, drag and drop, and sorting items.
Which type of PAT: Mathematics test should I use?
The new adaptive tests and the existing static tests both have advantages. Which type you choose to use will depend on your reasons for testing and the kind of information you want to produce.
The advantage of an adaptive test is that it tailors the content of the test to the individual student. This means that each student will be administered a test that is well-targeted to their achievement level. As well as ensuring that all students will have some success in terms of answering questions correctly, a well-targeted test also makes sure that each student’s achievement level is located on the PAT: Mathematics scale with a good level of precision. This is very difficult to achieve when we use the same static test with all students in a class. The variability between students in a typical New Zealand classroom is often quite large, meaning it is difficult to find one test form that will suit all students in the class.
The fact that the static tests present all students in a group with the same questions however, can sometimes be an advantage. When everyone in a group completes the same questions it becomes possible to look for patterns in the students’ responses that may expose possible mathematical weaknesses (or strengths) across a class or year level cohort.
Regardless of which type of PAT Mathematics test you decide to use in your school, all results will be able to be reported as scale scores and stanines. Some schools may choose to use one type of test with some students and another type with others.
How do I set up a PAT: Mathematics Adaptive test?
- The first step in setting up a PAT: Adaptive test is to add a new assessment group on the PAT: Mathematics section of the NZCER Marking website. ‘PAT Adaptive’ should be selected as the test assigned to this group. Note that the assessment group also needs to be assigned a particular year level.
- Once the assessment group has been created students can be added to it by using either the View button or the Student Tokens buttons. Usually, one class of students is assigned to an assessment group. However, the students assigned to an assessment group can be a subset of a class or a combination of students from different classes within the same year level.
- Once students have been assigned to the assessment group it is possible to print off the passwords (tokens) the students will need to take the test. Again, either the Student Tokens or View buttons can be used to access the token printing function.
- Students can use their individual token to access the test at www.nzceronline.org.nz.
Administering a PAT: Mathematics Adaptive test
Key features of adaptive and static tests
Adaptive Test | Static Test |
Each student gets their own mix of questions targeted at their achievement level. There will be detailed individual level reporting but no group item level reporting available. | All students in the group respond to the same questions. This supports item level reporting. |
| The tests include a variety of question types — such as multiple choice, hot spot, drag and drop and sorting questions. | Only multiple choice questions are used. |
| Students need to answer each question before they can go on to the next question. | Students can omit a question and come back to it later. |
| Students cannot change an answer once they have answered a question and pressed ‘Next’. | Students can return to a question and change their answers. |
Before the test
Before the test make sure you have set up an assessment group for an adaptive test in the NZCER Marking website, assigned students to it, and printed off the student tokens (passwords) they will need to do the test (see ‘How do I set up an adaptive test?’ for details.)
Use the ‘demo’ function in the NZCER marking site to try out an adaptive test yourself.
On the day of the test
Make sure that the assessment group’s online testing button on the NZCER marking service website is turned on (the ‘Online Testing’ button next to the name of the assessment group you are working with should be green).
Make sure each student has an appropriate device that is connected to the internet. They should also have access to paper and pencil to support any working they need to do.
Set up the room so students can work independently and without interruptions. They should bring paper and pencil for any working they need to do, along with an activity, such as a book to read, if they finish early.
Starting the test
- Distribute the tokens to the students. Make sure each student gets the right token.
- Ask the students to open their browser, go to www.nzceronline.org.nz and enter the code printed on their token. Once the code is entered, the computer will present the student with the name of the person assigned to the token and ask if they are that person. If they are not, they should close their browser and come to you to get the correct token.
- Read the on-screen instructions page with the students.
Emphasise to the students that the test is designed to give them a mix of questions at their level. It is not the type of test where you can necessarily expect to get all the questions correct. The computer is trying to find questions that have just the right amount of challenge for each person.
Work through the three example questions together. Make sure students are comfortable using the different item formats. Remind them that they are always looking for the best answer.
Remind students that in the test they will have to provide an answer to each question before they can go on to the next question. Once they have clicked next, they will not be able to go back and change their answer.
When you have finished working through the example questions, ask the students if they have any further questions. When everyone is ready they can click on the start button to begin.
During the test
During the test circulate around the room. If a student is struggling to read a question you can read it to them as written. However, the student should complete the question independently.
If a student needs to leave the room they can close their browser. When they return they will need to re-enter the code on their token. This will take them back to where they were in the test when they left it.
Completing the test
Students have up to 45 minutes to complete the test. An adaptive test is around 35 questions long and most students should be able to complete the test in the time provided. Use your discretion if a student needs a little more time to complete the questions.
When students have completed the test the computer will ask them to provide some feedback by answering a number of questions about their test experience. Please encourage them to complete these questions.
After the test
- Collect the token slips back from the students.
- Turn off the test in the NZCER Marking site (the ‘Online Testing’ button next to the name of the assessment group you are working with should now be red).
- If a student has not finished their test, it will be displayed as incomplete on the ‘List Report’ and ‘Individual Student Report’ indicated by 2 red asterisks next to the student’s name.