Item bank refinement

• Author: Castaño, Jonatan; Centeno, Clara; Jakobsone, Mara; Kluzer, Stefano; Troia, Sandra; Vuorikari, Riina; Cabrera, Marcelino; O'Keeffe, William; Zandbergs, Uldis; Clifford, Ian; Punie, Yves
• Pages: 30-33

DigCompSAT: A Self-reflection Tool for DigComp

30

Chapter 4: ITEM BANK REFINEMENT

KEY CHALLENGES

STEPS

1. Selection of items to be

removed

Decision of criteria for item elimination to be considered in th e revision of the Item

Bank, but not applied automatically

The hierarchy for item elimination

 Discrimination value of the item < 0.5

 Dimensionality, items weakly related to digital competence or specific areas

 DIF, results of differential item functioning analysis indicating large (L) or moderate

(M) impact of item score for some of the sub-groups of participants

 Difficulty index > 80%, as they are considered easy items

Removal of all 5 ‘fake’ items and 18 ‘true’ items

82 items selected for the Pilot 2 test

2. Items improvement

50 items unchanged

15 items underwent minor amendments and editing

17 items underwent more significant revisions (rephrasing, adding examples etc.)

2 answering options modified

Update the Item Bank based on Pilot 1 quantitative

and qualitative results

Cut the number of items to achieve a test-taking

duration in the 20-30 minutes range

1. Selection of items to be removed 2 3

12. Items improvement 3

1 2 3. Updating the profiency levels of the items