Conclusions: key challenges and lessons learned

AuthorCastaño, Jonatan; Centeno, Clara; Jakobsone, Mara; Kluzer, Stefano; Troia, Sandra; Vuorikari, Riina; Cabrera, Marcelino; O'Keeffe, William; Zandbergs, Uldis; Clifford, Ian; Punie, Yves
DigCompSAT: A Self-reflection Tool for DigComp
7.1 A reliable self-reflection tool for DigComp
The DigComp self-reflection tool for digital skills at foundation, intermediate and advanced levels, as the main
product of the present research project, proved to be a reliable assessment tool for the five competence areas
and as a whole test. Based on the Cronbach’s alpha model’s results for reliability, we can say that the tool
assesses reliably digital competence in a target group having from low to advanced levels of digital skills, and
can be used modularly, as well, to assess the five competence areas.
Its soundness is based on the strong reliability properties proven by Cronbac h’s Alpha, the concise ite ms that
allow a test-time of less than 30 minutes, acros s different countries, age and educational background groups
and genders, and that can be used by beginners, as well.
It was also proven that countries with different proportions of the target group (different levels of involvement
in the digital world) can use it to assess their level of digital skills.
Summing up, the developed SAT instrument currently is able to perform three main f unctions for test takers:
To assess existing competences based on the respondents’ self-reflection
To identify competence gaps
To raise awareness - motivation to develop own skills
7.1 Impact of Covid-19
The impact of the Covid-19 crisis cannot be underplayed. In the second pilot, it forced us to switch from “in-
centre” delivery to entirely remote and online. The positive side of this was that the system was tested largely
as it might be used in future, online and remotely, rather than in a classroom setting.
In general, Ibermáti ca felt that the implementation of this project (and particularly recruitment of lower
educated candidates for the DigCompSAT) was greatly affected in Spain by the circumstances arising from the
confinement caused by the pandemic Covid-19.
7.2 The importance of methodology
The applied methodology, which consisted of both quantitative and qualitative elements and involved different
input sources (questionnaires on the SAT platform, experts consultation, focus groups with end users),
contributed valuable data to the tool’s initial design, validation and development process.
The qualitative methodology was very important in this process. In particular, test respondents played a
significant role in validity checking; they shared many va luable experiences in the focus groups and online
surveys that illuminated several hidden or contradictory data in the statistical analysis. They behaved as very
self-conscious potential ‘end users’ and represented an honest and engaged approach on the assessment
instrument. Thanks to the piloting partners, participants were motivated, communicative and showed real
interest toward the assessment instrument.
From a methodological perspe ctive, the focus of the tool development process was the initial design and the
following refinement of the Item Bank, for which we gathered information and data through different research
methods. We realised that design and refinement of SAT required quite complex skills, beyond experiences with
digital ski lls and well -knowing the DigComp Framework . It also required some training experience (testing,
measuring digital skills among people), experience with the target group (digital skills of disadvantaged people),
some sociological approach (survey design), writing skills (for the statements’ texts), as well as expert
knowledge of statistical analysis.
7.3 Challenges and opportunities relating to assessing ‘Competence’
During project implementation we faced several challenges which allowed us to formulate a ‘vision’ that might
be useful for similar activities in the future.

To continue reading

Request your trial

VLEX uses login cookies to provide you with a better browsing experience. If you click on 'Accept' or continue browsing this site we consider that you accept our cookie policy. ACCEPT