Final item bank features and considerations for its future use
Author | Castaño, Jonatan; Centeno, Clara; Jakobsone, Mara; Kluzer, Stefano; Troia, Sandra; Vuorikari, Riina; Cabrera, Marcelino; O'Keeffe, William; Zandbergs, Uldis; Clifford, Ian; Punie, Yves |
Pages | 47-49 |
DigCompSAT: A Self-reflection Tool for DigComp
47
Chapter 6. FINAL ITEM BANK FEATURES AND CONSIDERATIONS FOR ITS
FUTURE USE
6.1 The Item bank
The median time for test-taking in Pilot 2 was significantly shorter than in Pilot 1: 23 minutes compared to
27 minutes for the first pilot.7 The revised Item Bank therefore matches even better the desired test-taking
duration.
At the same time, we have very good results from statistical analysis. The Cronbach’s Alphavalues for the
SAT total in Pilot 2 is an excellent 0.987 and the values for the five areas are all above 0.93.
Besides, no items should be definitely removed based on the other statistical indicators from Pilot 2 results:
—none reached the 80% difficulty indicator threshold that we used in Pilot 1;
—none was below or equal to the 0.2 minimum acceptable level of the discrimination index (the lowest level
is above 0.3 for 1 item)
—and correlation analysis pointed at items that might be considered redundant, but a much larger test might
be needed to reach more stringent conclusions.
The analysis of unclear itemshas identified 2 items for which a relatively large number of respondents chose
the answer “The question is unclear to me”:
—Q48 “I am interested in understanding how a task can be broken down into steps so that it can be
automated.” (68 people who found them unclear),
—Q40 “I am keen to create new digital content by mixing and modifying existing resources. (41 people who
found them unclear).”
Interestingly, both are Attitude items and it is difficult to assess whether such high response rates reflect the
fact that respondents have flagged them unclear because they did not have a good enough level of digital
competence to understand what they meant, or because they lacked additional context/infor mation to
understand what they referred to (especially Q48 is related to 3.4 Programming competence), or because they
were worded ineffectively, or a mix of these reasons.
In any case, in the light of the above considerations, we agreed with JRC to keep th e Pilot 2 Item Bank as the
final one, but to improve the statements of Q40 and Q48 and to make some changes (different wording, adding
examples and context) to another 5 items which got a relatively high rate of “unclear to me” answers (Q5, 34,
39, 62, 78), with the aim to enhance their understandability.
The final version of the Item Bank with 82 statement and the agreed revisions can be found in Annex 13 -
Final Item Bank in English, Latvian and Spanish.
6.2 The DigCompSAT Report
The screenshot below (Figure 5.) shows an extract of the final “Digital Competences SAT report” that was
presented to users after completion of the SAT in Pilot 2. All competences and competence areas were covered
in this way. There was colour-coding indicating competences that users should focus upon with simple
associated statements. In the User Experience feedback from users, 62 felt that this was “just right” in the
level of detail provided. 19 felt there a was “a bit too much detail”, although this could be that the report
wording may have felt repetitive, especially if the user was largely on the same level across all areas or
competences. Conversely, 13 felt that it was “a little short on detail”.
7 The fact that the mean time was at 34 minutes (only slightly lower than in Pilot 1 reflects the fact that many
respondents at home (especially younger people) most likely interrupted test-taking more than once, leading
to unrealistic durations.
Get this document and AI-powered insights with a free trial of vLex and Vincent AI
Get Started for FreeUnlock full access with a free 7-day trial
Transform your legal research with vLex
-
Complete access to the largest collection of common law case law on one platform
-
Generate AI case summaries that instantly highlight key legal issues
-
Advanced search capabilities with precise filtering and sorting options
-
Comprehensive legal content with documents across 100+ jurisdictions
-
Trusted by 2 million professionals including top global firms
-
Access AI-Powered Research with Vincent AI: Natural language queries with verified citations

Unlock full access with a free 7-day trial
Transform your legal research with vLex
-
Complete access to the largest collection of common law case law on one platform
-
Generate AI case summaries that instantly highlight key legal issues
-
Advanced search capabilities with precise filtering and sorting options
-
Comprehensive legal content with documents across 100+ jurisdictions
-
Trusted by 2 million professionals including top global firms
-
Access AI-Powered Research with Vincent AI: Natural language queries with verified citations

Unlock full access with a free 7-day trial
Transform your legal research with vLex
-
Complete access to the largest collection of common law case law on one platform
-
Generate AI case summaries that instantly highlight key legal issues
-
Advanced search capabilities with precise filtering and sorting options
-
Comprehensive legal content with documents across 100+ jurisdictions
-
Trusted by 2 million professionals including top global firms
-
Access AI-Powered Research with Vincent AI: Natural language queries with verified citations

Unlock full access with a free 7-day trial
Transform your legal research with vLex
-
Complete access to the largest collection of common law case law on one platform
-
Generate AI case summaries that instantly highlight key legal issues
-
Advanced search capabilities with precise filtering and sorting options
-
Comprehensive legal content with documents across 100+ jurisdictions
-
Trusted by 2 million professionals including top global firms
-
Access AI-Powered Research with Vincent AI: Natural language queries with verified citations

Unlock full access with a free 7-day trial
Transform your legal research with vLex
-
Complete access to the largest collection of common law case law on one platform
-
Generate AI case summaries that instantly highlight key legal issues
-
Advanced search capabilities with precise filtering and sorting options
-
Comprehensive legal content with documents across 100+ jurisdictions
-
Trusted by 2 million professionals including top global firms
-
Access AI-Powered Research with Vincent AI: Natural language queries with verified citations
