CS Paper Radar

The Challenge of Modeling the Complexity of Use for the Measurement of Digital Maturity in Education

Christine Michel, La{ë}titia Pierrot, Frederic Oru, Olivier Vigneau

cs.CY ·submitted 2026-11-06 09:28 UTC · v1
published

Abstract

Measurement of teachers' digital maturity can be useful in diagnosing technology adoption in education and in supporting teachers' professional development. However, there is a lack of easily usable tools in the existing literature. By investigating the most effective ways to measure teachers' digital maturity, our research aims to fill this gap. The focus of this article is on a data-driven approach to the assessment of digital maturity in education. Following a process similar to teaching analytics (data collection, data modeling, data understanding and visualization, and contextual understanding), we draw on diary data collected from 12949 French primary school teachers using a virtual learning environment (VLE) during the 2022--2023 school year. Specifically, the paper presents results related to modeling and visualizing data. To this end, we propose a critical analysis of two automatic classification techniques, one supervised rule-based and the other unsupervised (hierarchical clustering). In addition to providing different scales to measure, we have proposed a new approach to diagnose and visualize digital maturity. In this way, the method provides insight into the diversity and intensity of technology use, contributing to a better understanding of digital maturity and providing a practical tool for assessing day-to-day teaching practices.

Journal ref
Technology Enhanced Learning for Inclusive and Equitable Quality Education, Springer, pp.268-283, 2024, Lecture Notes in Computer Science, vol 15159, 978-3-031-72314-8

Closest papers

Nearest neighbours by meaning across the whole index.

New Lenses on Learning Design: Course Types and Indicators from Large-Scale Moodle Data

cs.CY cs.HC ·2026-08-14 ·Ekaterina Soroka, Niels Seidel, Marcus Specht, Kamila Misiejuk
published similarity 0.59
Abstract

Although learning design (LD) plays a central role in shaping online courses, large-scale empirical evidence on how courses are actually structured remains scarce, and existing analytics work has characterised courses through student engagement with specific learning activities rather than through indicators of the design itself. This study addresses both gaps through an analysis of learning management system (LMS) data from 606 courses at a distance university. Using LMS data, we first applied dimensionality reduction and clustering to a structural variable set, from which three distinct LD types emerged empirically: Compact, Expansive, and Interactive and statistically characterised how their design elements differ across disciplines, activity load, and interfaces. Building on this typology, we derived three LD indicators, activity-mode balance, structural clarity, and analytics readiness, each combining multiple LMS-derived variables into higher-order constructs that target a different aspect of LD. All three indicators differentiated significantly across the LD types and added pedagogically meaningful dimensions for interpretation. These contributions advance the operationalisation of LD in technology-enhanced learning research by demonstrating that higher-order LD indicators can be derived directly from routinely collected LMS data, and offer empirical insight into how LD manifests at scale within a large distance university.

0.29

Who Decides? Agency and Legitimacy in Digital Educational Systems

cs.CY ·2026-09-17 ·Eriam Schaffter, Ahmed Bounekkar
publishedposition similarity 0.55
Abstract

Digital Educational Systems (DES) enable millions of individuals to learn or to acquire new skills. Their promises vary from one platform to another: some grant genuine degrees, others pledge rapid progress, drawing for this purpose on a range of techniques, from the recommendation of learning resources through complex models to a plain course textbook uploaded onto a learning management system. In this article, we argue that categorizing DES along two dimensions allows to map differences in the way these DES structurally conceptualise the learning process. These underlying dimensions are at once dimensions of the student model and mechanisms built in the DES's designs; we identify them as agency and legitimacy. Indeed, DES are, on the one hand, anchored in an institutionalised regime that grants them legitimacy and, in turn, legitimises the learning undertaken on these platforms; on the other hand, they make design choices as to the share of decision left to the learner in the learning process. We map a curated sample of common DES onto a plane composed of these two dimensions translated into indicators. This allows us to highlight underpopulated regions of the plane and to envisage how a DES might regulate these two dimensions algorithmically, in the interest of learning.

0.43

What Do Visualization Instructors Want Students to Learn? Introducing a Concept Inventory for Visualization Design

cs.HC ·2026-08-20 ·Medina Lamkin, Heer Patel, Sayamindu Dasgupta, Leilani Battle
code/data similarity 0.53
Abstract · 11 pages, 5 figures, submitted to VIS 2026

The term "visualization design" encompasses multiple concepts and skills that go well beyond current assessments of graphical perception and visualization literacy. In the context of education, what exactly should a student be able to do if they "know" visualization design? To answer this question, we draw on existing methodology from the field of education to propose a concept inventory for visualization design, i.e., a theoretical model capturing the most important concepts and skills commonly associated with visualization design. We initially draft the concept inventory using a qualitative analysis of course objectives from visualization course syllabi. Then, we iteratively refine the concept inventory by soliciting feedback from instructors through semi-structured interviews. Based on our experiences in developing the concept inventory, we reflect on open questions and future research directions in visualization education, such as developing assessments for visualization design (similar to those for visualization literacy) and providing automated assistance for learning and teaching visualization design. Our supplemental materials are available at https://osf.io/dma3f/.

0.26

Decreasing Digital Distraction in College Students: Associated Online Learning Strategies Identified by Unsupervised Data Mining Approaches

cs.CY cs.HC ·2026-09-03 ·Hui Shi, Ran Bi, Xi Lin, Yan Dai
publishedcode/data similarity 0.51
Abstract · 51 pages. Accepted manuscript. Published in Journal of Educational Computing Research. Data and code: https://osf.io/p85s3/overview

The proliferation of digital tools in education offers numerous benefits but also introduces significant challenges, notably digital distractions that hinder academic performance, especially in online learning contexts. This study employed unsupervised data mining techniques, specifically association rule mining and clustering analysis, to identify effective learning strategies associated with lower levels of digital distractions among college students. Data from 530 participants revealed that self-regulated learning strategies (i.e., goal setting, environment structuring, and time management) co-occurred most consistently with lower digital distractions. Additionally, learner-instructor and learner-content engagement strategies, as well as technical competencies, also tended to appear in the same profiles as lower distraction. Interestingly, reliance on peer help-seeking and learner-learner engagement strategies appeared less often in those lower distraction profiles. These findings offer actionable implications for educators to design targeted interventions that foster focused and productive online learning environments.

0.51

Decoding the Dashboard: Data Comics to Support Students' Understanding of Learning Analytics Visualisations

cs.HC ·2026-09-17 ·Mikaela Elizabeth Milesi, Vanessa Echeverria, Lixiang Yan, Yueqiao Jin, Riordan Alfredo +5 more
published similarity 0.51
Abstract · Accepted to ECTEL 2026

Learning analytics dashboards (LADs) are intended to help students make sense of their learning data to support reflection and decision-making. However, their visualisations can be complex, particularly for students with low visualisation literacy. Narrative techniques, such as annotated charts and data comics, have been used to communicate insights directly, but not as supplementary materials to empower students to explore their visualisations themselves. In response, we conducted a qualitative study examining how data comics can complement LADs. We interviewed 18 nursing students and 4 of their teachers about a multimodal LAD containing visualisations with data comics explaining them. Analysis showed that data comics were clear, engaging, and helped make complex visualisations more accessible, though they must be carefully designed to avoid overwhelming students with information. The findings suggest that both students and teachers are receptive to data comics as a means of supporting the interpretability of LADs.

0.29

ClickSight: An LLM Framework for Interpreting Student Clickstreams Through Learning Strategies

cs.AI cs.CLcs.CY ·2026-11-01 ·Bahar Radmehr, Lan Tran, Ekaterina Shved, Fatma Bet{ü}l G{ü}reş, Adish Singla +1 more
code/data similarity 0.49
Abstract · 29 pages, 12 figures, 19 tables. Under review. Code: https://github.com/epfl-ml4ed/ClickSight-CEAI

Digital learning environments produce rich clickstream data that can reveal how students approach learning tasks, yet interpreting these fine-grained traces often requires substantial expert effort. We introduce ClickSight, a framework for interpreting student clickstreams through learning strategies defined in the literature. Given a student clickstream and a predefined set of learning strategies, ClickSight uses a staged LLM-based pipeline to identify strategy-relevant evidence, assess adherence to each strategy, and generate an evidence-grounded interpretation of the student's behavior. We evaluate ClickSight across two learning environments, 365 students, and 12 model backbones using an expert-grounded quality rubric and an LLM judge calibrated with human-expert ratings. Our results show that ClickSight can generate high-quality strategy-based interpretations across different models and environments, with reasoning models performing more consistently than instruction-tuned counterparts. We further find that providing external clickstream-extracted guidance can improve interpretation quality for strategies that require precise quantitative information from the trace. By connecting fine-grained clickstreams with theoretically defined learning strategies and preserving the evidence underlying each judgment, ClickSight provides a flexible approach for studying student behavior in digital learning environments.

0.40

An AI Factory for Technology Education: An Agentic Pipeline with Closed Loop Audience Feedback

cs.CY ·2026-10-22 ·Josue Hernandez, Hector R. Amado-Salvatierra, Rocael Hernandez Rizzardini
similarity 0.49
Abstract · 20 pages, 3 figures

Technology tools used for professional upskilling change on a timescale of weeks, and written guidance about them ages at the same pace. Large language models have made it inexpensive to draft a tutorial or a course outline, but most deployed generation systems still produce a piece of content once and never check it against what an audience actually does with it. This paper describes an agentic pipeline, organized as a typed workflow graph, that treats instructional content production as a manufacturing process with explicit checkpoints rather than a single generation call. Four node classes structure the graph: deterministic scripts, model steps bound to a fixed rubric, open-ended agent phases, and human approval gates, arranged along a forward production path and closed by a feedback cycle that rewrites generation rules from post level engagement telemetry. Production of content is split into two layers: a demand layer, built from video and community sources, that shapes topic selection and register, and an authority layer, built from official documentation, that every published claim must trace back to. Over five months of continuous operation, the deployed system published 26 documented guides at roughly 1.5 hours of operator time per week, and the feedback loop identified four measurable engagement effects that became binding rules for later runs, including a hook style that outperformed baseline by a factor of 1.83 in its first analysis window.

0.25

The CAPED Framework: Characterizing User Skills in Chart Authoring through the Lens of Tasks and Tools

cs.HC ·2026-10-26 ·Bongshin Lee, Zhicheng Liu, John Thompson, Chenglong Wang
published similarity 0.48
Abstract · Accepted at IEEE VIS 2026, to appear in IEEE Transactions on Visualization and Computer Graphics (TVCG)

In this paper, we introduce a new framework for characterizing the user skills required for chart authoring. Despite growing attention to authoring systems, we lack a conceptual framework that characterizes the diverse and multifaceted skills involved and helps identify opportunities for system design and development. Our initial investigation revealed that skill is a complex and elusive concept, and that existing theories on skills from other disciplines are too high-level to capture the importance of tasks and tools in data visualizations. To address this gap, we formulate the CAPED framework that characterizes skill as the ability to create a chart involving five dimensions pertinent to tasks and tools: input data, output chart, paradigm, abstraction, and expression. We demonstrate the framework's descriptive power by highlighting different skills involved in similar authoring situations and identifying the types of knowledge associated with chart authoring. We further present examples illustrating how the framework can be used to identify new research opportunities in supporting users with varying skills.

0.37