In 2023, IEA launched an open call for the third round of the R&D fund and are delighted to announce the seven proposals which have been awarded funding.
The initiative, which aims to advance research and innovation for ILSAs, provides an opportunity to build and grow collaboration efforts with partners and research institutions globally. The third call was left thematically open to allow a wider range of topics to be submitted, providing the proposals were relevant to the work of IEA.
IEA has selected seven proposals—see the summaries below—at Tier one funding, with projects lasting for six months and receiving EUR 50,000.
- An Examination on the Performance of Sampling Variance Estimators in International Large-scale Assessments by Umut Atasever, IEA.
Two methodologies, the Balanced Repeated Replication and Jackknife Repeated Replication are typically used to approximate the sampling variance of complex estimators. This project will measure the performance of these methodologies under different scenarios to inform study centers and analysts in the field of ILSAs on the choice of variance estimators.
- Field Trial Sample Size for Scales by Andrés Christiansen, IEA.
This project focuses on the sample size requirements of scales produced with data from background questionnaires. The aim is to determine how different sample sizes affect factor analysis estimations, and what the minimum sample size required to produce reliable and precise model goodness-of-fit statistics would be.
- Using Large Language Models for Automatic Item Generation: Development and Validation for TIMSS Fourth Grade by Marek Muszyński, Polish Academy of Sciences.
This research aims to automatically generate fourth grade TIMSS items using large language models. The items will be validated through subject matter evaluations and field tests simulating TIMSS settings. The generated items will be compared to human made TIMSS items with respect to their measurement properties. -
Approximate Areas of Interest for Enhanced Understanding of Student Motivation and Task Interaction in IEA Assessments by Artur Pokropek, Polish Academy of Sciences.
Using a log data method based on cursor tracking, this project will devise and validate a methodology for identifying demotivated test-takers and flawed items in assessments. The approach will also be used to differentiate high-performing students from their peers and provide insights to improving educational assessment practices. -
Enhancing Cross-Cultural Comparisons in ILSAs: A Comprehensive Investigation and Guideline Development Using TIMSS 2019 and TALIS 2018 Data by Andrés Sandoval-Hernández, University of Bath.
Using alignment optimization techniques, the project aims to improve measurement equivalence in ILSAs, led by Andrés Sandoval-Hernández in collaboration with Nurullah Eryilmaz from IEA, and Diego Carrasco from MIDE-UC. -
Dimensionality and Mode Effects in PIRLS 2016 and PIRLS 2021: Evaluating the Comparability of Different Assessment Modes by Rolf Strietholt, IEA.
This project will explore how comparable the PIRLS assessment is over time. With transitions from paper-based to computer-based assessments and the introduction of non-linear tasks, alongside traditional linear materials, determinations will be made as to whether the changes in administration mode and item types impact the dimensionality of the test and comparability across cycles for trend analyses. -
Gauging the Instructional Sensitivity of Ordinal Assessment Items by Anne Taylor, Purdue University.
Looking into how differences in the implemented curriculum should/do affect students’ item responses, this project will evaluate the instructional sensitivity of conducted-response science assessment items using a statistics index and content analysis by science teachers in several nations.
Upon completion of the above projects, the outcomes—reports and/or links to journal papers— will be made available on the IEA website. View the R&D page to see the current outcomes from the first two calls.