Summary
IERI is organizing an Academy on assessment designs, and the use of Item Response Theory (IRT) and population modeling in large-scale assessments. The goal of the Academy is to familiarize participants with the topics related to assessment designs and IRT, and enable them to use commercially and freely available software to conduct appropriate IRT analysis and generate proficiency estimates or plausible values.
The Academy is targeted to participants with basic knowledge in the area of IRT and multiple imputations. The Academy will be limited to 25 participants. Participants will be accepted on a first-come, first-served basis.
Demonstration versions of the software for the hands-on training will be distributed to the participants during the Academy. Spreadsheet applications such as Excel, and statistical software such as SPSS, SAS or R will be useful, but are not required.
This Academy will cover basic topics related to Assessment Designs, Classical Test Theory, Item Response Theory and Population Modeling as they relate to large-scale assessments. It will also train participants on the use of IRT and population modeling software to conduct analysis that involve calculating item parameters and person scores. It will review advantages and disadvantages of different IRT models currently in use as well as procedures for setting up data to be analyzed with IRT software.
The topics covered will be the following:
- Basics of survey instrument design
- Test design
- Questionnaire design
- Multiple matrix sampling
- Principles of Classical Test Theory (CTT) and item analysis
- Basic principles of Item Response Theory (IRT)
- Assumptions and models
- Estimating item parameters
- Estimating person scores
- Linking scales
- Using IRT for attitude and other background scales
- Principles of population modeling
- Dealing with multiple populations
- Estimating the latent regression
- Selection and processing conditioning variables
- Imputing achievement results
- Using plausible values in the analysis
- Hands-on use of IRT software for large-scale assessments
- Examples with TAM and MIRT
This Academy is aimed towards those who want to become familiar with the principles of assessments designs used in large-scale assessments, and the psychometric methods currently used to analyze these data. To gainfully participate, participants of this Academy are expected to have knowledge of basic statistical analysis techniques.
Complete documentation, software and sample data from the publicly available databases will be distributed during the Academy and used as examples.