The Pacific Islands Literacy and Numeracy Assessment (PILNA) – the region’s only large-scale education assessment - is conducted by the Pacific Community’s (SPC) Educational Quality and Assessment Programme (EQAP). The assessment was carried out for a third time last year in 15 Pacific Island Countries. In mid-March, EQAP hosted the annual meeting of the PILNA Steering Committee prior to the annual meeting of its governing body, the Pacific Board for Educational Quality (PBEQ). At the committee meeting, 14 of the permanent secretaries or chief executives of the participating countries’ education ministries discussed PILNA 2018, including the reporting of the results. In this interview, EQAP director Dr MICHELLE BELISLE explains EQAP’s role in the administration of PILNA and the relevance of PILNA to the region.
1. EQAP recently held a series of important meetings, which concluded with the annual meeting of its governing board, the Pacific Board for Educational Quality (PBEQ). One of these meetings was that of the Pacific Islands Literacy and Numeracy Assessment (PILNA) Steering Committee. What is the significance of this steering committee?
The Pacific Island Literacy and Numeracy Assessment (PILNA) is a wide scale assessment that captures a snapshot of the literacy and numeracy skills of students at Years 4 and 6 in the 15 participating Pacific Island countries. It is an assessment that is owned by the member countries. The Steering Committee, made up of the Heads of Education Systems from the participating countries, provides guidance to EQAP as to the content, the focus and the reporting with respect to PILNA. The direct involvement of the steering committee, as well as other ministry officials the heads of systems designate for various roles within the PILNA programme, ensures that the interests of the Pacific Island countries and their students are well-represented and addressed at every step of the process.
2. Last year was the third time for the large-scale assessment to be conducted, and there is much interest as to the results, especially as Fiji is part of the research. How soon can the public expect to know the findings?
Yes, Fiji is part of the 2018 PILNA. More than 7000 students from Fiji took part in the assessment. This translates to about 17 per cent of the total student respondents. For the PILNA results, the report released to the public will be presented in a regional format. This regional PILNA report will be released on July 22 this year. The individual country reports will be released to the appropriate education ministries shortly after.
3. Could you explain the extent of this large-scale research and the validity of its sampling method?
The PILNA last year was conducted across 15 countries in the Pacific, in 925 schools and with more than 40,000 students. Schools and students were selected to ensure a representative sample of that country was included in the assessment, and then the country weights were calculated to ensure that the entire Pacific sample was representative of the collective participating countries. The Australian Council for Educational Research (ACER) and, most importantly, the officers from the ministries of education of each country involved supported EQAP in the sampling process. This was critical in ensuring the samples drawn in each country were truly representative and took into account the differing contexts found across the region.
4. What are some of the new insights that PILNA tried to investigate in this third round?
In this third cycle of PILNA we have, for the first time, fully implemented the coding of student results. The information from coding, which was only partially applied in the 2015 assessment, will provide much richer information about student responses, including any common misconceptions or gaps in understanding that might be leading to the responses students are providing. As well, we are also implementing a full range of contextual questionnaires for the first time. Students, teachers and head teachers who participated in the 2018 PILNA have responded to questionnaires that will provide background information about their education experience across a wide range of themes. The contextual information will support national ministries in better understanding their results.
5. How does PILNA help to improve teaching and learning in the classrooms?
One way that PILNA can support improved teaching and learning in the classroom is through the information gathered through coding. With the information about how students are responding, particularly when the response is not the expected response (correct answer), knowing what students have written is very valuable. For example, if a question is about time and the hands on a clock, knowing that a significant number of students did not get the correct answer is helpful but to know that of those who answered incorrectly, a large number were mixing up the hour hand and the minute hand is even more useful. That is direct information that can help a teacher know where students typically struggle and as a result, when teaching the time concepts, it is something a teacher can know to give attention to it.
6. At the meeting of its governing board (the Pacific Board for Educational Quality), EQAP presented the achievements and challenges in the implementation of PILNA last year. Can you share some of these to illustrate the magnitude of this research?
Because PILNA is representative of the students across the Pacific, language and geography each posed challenges. Literacy and numeracy are at the focus of the assessment and national governments make the choice as to the language for each PILNA instrument in the country. In 2018, the assessment was administered in 10 languages. This was a huge undertaking in that it was necessary to ensure high quality translation of materials in words that are familiar to students. As well, the efforts to ensure representative sampling of students meant that many of the selected schools were significantly remote from the urban areas, and this posed challenges in the distribution and collection of papers.
7. Who are the major partners for PILNA?
EQAP’s funding partners for the PILNA are the Australian Government’s Department of Foreign Affairs and Trade (DFAT), and the New Zealand Government’s Ministry of Foreign Affairs and Trade (MFAT). The countries themselves are all key partners in PILNA, and ACER is the technical partner for this research.
8. The annual board meeting has wrapped up. For EQAP, what were the main issues going into the meeting?
Going into the board meeting, we were looking, as always, for direction from the board as to EQAP’s work in the region. To that end our main topics of discussion included the qualifications work, the Pacific Regional Education Framework (PacREF), our 2019-22 Business Plan and the ways we interact with member countries to ensure we are responsive to their needs and national priorities.
9. Are you happy with the governing body’s responses to EQAP’s efforts? How would you describe the atmosphere of engagement in the room?
The board, as always, came together in a spirit of collaboration with a common goal of representing the best interests of Pacific Island students and the education systems that support those students. There was a great deal of good discussion and thoughtful deliberation throughout the meetings and that is exactly what we had hoped for.
10. What are EQAP’s key priorities for the next leg of the journey?
EQAP’s priorities now are to carry out the work that is captured in the endorsed business plan, including the reporting and dissemination of PILNA results as well as further supporting the PacREF work and the ongoing EQAP activities that have previously been directed by the board.
BACKGROUND NOTE: The Outcomes of the PILNA Steering Committee’s meeting was accepted by the Pacific Board for Educational Quality (PBEQ) at its annual meeting. The board (PBEQ) is a sub-committee of the Committee of Representatives of Governments and Administrations (CRGA), which is the governing body of the Pacific Community (SPC).
* This is the original Q&A prepared by EQAP Communications. The Fiji Times amended this article and published it in its print publication on Page 12 of its national newspaper on March 28.