Conclusions: key challenges and lessons learned

• Author: Castaño, Jonatan; Centeno, Clara; Jakobsone, Mara; Kluzer, Stefano; Troia, Sandra; Vuorikari, Riina; Cabrera, Marcelino; O'Keeffe, William; Zandbergs, Uldis; Clifford, Ian; Punie, Yves
• Pages: 50-52

DigCompSAT: A Self-reflection Tool for DigComp

50

Chapter 7. CONCLUSIONS: KEY CHALLENGES AND LESSONS LEARNED

7.1 A reliable self-reflection tool for DigComp

The DigComp self-reflection tool for digital skills at foundation, intermediate and advanced levels, as the main

product of the present research project, proved to be a reliable assessment tool for the five competence areas

and as a whole test. Based on the Cronbach’s alpha model’s results for reliability, we can say that the tool

assesses reliably digital competence in a target group having from low to advanced levels of digital skills, and

can be used modularly, as well, to assess the five competence areas.

Its soundness is based on the strong reliability properties proven by Cronbac h’s Alpha, the concise ite ms that

allow a test-time of less than 30 minutes, acros s different countries, age and educational background groups

and genders, and that can be used by beginners, as well.

It was also proven that countries with different proportions of the target group (different levels of involvement

in the digital world) can use it to assess their level of digital skills.

Summing up, the developed SAT instrument currently is able to perform three main f unctions for test takers:

— To assess existing competences based on the respondents’ self-reflection

— To identify competence gaps

— To raise awareness - motivation to develop own skills

7.1 Impact of Covid-19

The impact of the Covid-19 crisis cannot be underplayed. In the second pilot, it forced us to switch from “in-

centre” delivery to entirely remote and online. The positive side of this was that the system was tested largely

as it might be used in future, online and remotely, rather than in a classroom setting.

In general, Ibermáti ca felt that the implementation of this project (and particularly recruitment of lower

educated candidates for the DigCompSAT) was greatly affected in Spain by the circumstances arising from the

confinement caused by the pandemic Covid-19.

7.2 The importance of methodology

The applied methodology, which consisted of both quantitative and qualitative elements and involved different

input sources (questionnaires on the SAT platform, experts consultation, focus groups with end users),

contributed valuable data to the tool’s initial design, validation and development process.

The qualitative methodology was very important in this process. In particular, test respondents played a

significant role in validity checking; they shared many va luable experiences in the focus groups and online

surveys that illuminated several hidden or contradictory data in the statistical analysis. They behaved as very

self-conscious potential ‘end users’ and represented an honest and engaged approach on the assessment

instrument. Thanks to the piloting partners, participants were motivated, communicative and showed real

interest toward the assessment instrument.

From a methodological perspe ctive, the focus of the tool development process was the initial design and the

following refinement of the Item Bank, for which we gathered information and data through different research

methods. We realised that design and refinement of SAT required quite complex skills, beyond experiences with

digital ski lls and well -knowing the DigComp Framework . It also required some training experience (testing,

measuring digital skills among people), experience with the target group (digital skills of disadvantaged people),

some sociological approach (survey design), writing skills (for the statements’ texts), as well as expert

knowledge of statistical analysis.

7.3 Challenges and opportunities relating to assessing ‘Competence’

During project implementation we faced several challenges which allowed us to formulate a ‘vision’ that might

be useful for similar activities in the future.