Articles

Publish at February 09 2015 Updated April 07 2022

Measuring. To control or to accompany?

What data to support learning?

Multiplying data: the learner under surveillance

Distance learning platforms tell us more about trainees than we'd dream of knowing in the classroom. What time did they start? How long did they spend on each resource? Did they come back to an activity multiple times? What score did they get? Which questions did they get wrong...

No tracking, no e-learning tells us Jérôme Bruet on the e-doceo blog. The approach is pedagogical: the data on the participants are linked to the training units (SCO - Sharable content object). They inform the trainer about the blocking points, help him to improve his learning units and to better support his learners.

Control

The pedagogical aim is noble, but the temptation of control is not far away. We find ourselves dreaming of dashboards that would give us complete information. It would be a modern, digital panopticon, like the architectural model that Bentham imagined so that a single guard could monitor hundreds of prisoners.

In her article "Counting is not understanding", Christine Vaufrey shows us that ODL platforms provide information similar to that of e-commerce platforms. We find the visited pages, the hours of frequentation, the frequency of visit of the pages, the numerical results to the evaluations. The platform provides us with an extremely detailed attendance sheet, which does not necessarily tell us what happened during the training, nor if there was learning.

More than the trainer, this information will reassure the human resources manager who will be able to convince himself that his employees have indeed "gone" to training. At the January 2015 iLearningForum in Paris, the director of a company that offers off-the-shelf online courses testified along these lines. He explained that he had alerted one of his clients to a spike in logins for the "stress management" and "resume writing" modules. The client made the connection to the buyout rumors that were circulating, and was able to adjust their internal communication.

On the other hand, we never knew if the stress management training was effective.

Is this information still relevant?

what the learner experiencesThe sheer amount of numbers gives a sense of control. But what do we really know and what is the value of this information? This is essentially the question Christine Vaufrey asks. Of course, the binary logic avoids the arbitrariness of evaluations and allows them to be processed quickly. But at the expense of more qualitative elements, which characterize learning. Learning is a continuous process where discursive reasoning, serendipity, personality, associations of ideas have an essential place.

The trainer will also regret the small clues that one perceives in face-to-face, but not at a distance. Hesitations, signs of fatigue, those moments when everyone coughs or goes to the bathroom, tell us that we need to change pace or activity... At a distance, in e-learning, we miss these signals.

More qualitative technical solutions

No matter, new companies are responding. They offer advanced technologies that allow a richer, less binary "tracking".

under surveillance

So, Knewton, often cited on Thot, offers analysis based on a richer use of statistical tools. Traditional platforms provide two types of information. They record administrative data: who is enrolled for what, and what are the characteristics of the learners, and basic engagement data: who went to what content. Knowles proposes to go beyond that and provide more pedagogical information. The platform shows what is difficult or easy, what is adapted when the learner has such and such a prerequisite... It offers a more longitudinal analysis of students. Beyond their success on a module, it makes the link with other modules followed, on the forms of interaction with the contents... Finally, Knewton also claims to highlight what helps to understand.

Based on the learner's choices, the platform adapts the activities. The questions selected on the tests will not be the same. The progression regularly adjusts to the difficulties that the learner represents. Of course, this means that we have been able to identify these difficulties and the indicators that tell us about the learner's learning style. The preparation work is particularly important since we have to build a series of "training objects" from which the platform will pick.

Mathieu Cisel introduces us to the cognitive tutors. Competence is related to a process, a pathway. Thus, medical diagnosis is not a series of independent actions. It is their succession that produces meaning. The cognitive tutor analyzes errors according to "deviation models", and provides an adapted response.

Christine Vaufrey, in the article cited above, imagines the information that could be useful to the trainer. She integrates elements about the learner: his appropriation of specialized vocabulary, the evolution of his engagement in exchanges, the importance of his contributions on the sharing sites,... Other data are more related to the training. Which themes, which aspects of the course are commented on the most? On which points do learners meet? Mooc et Cie was able to implement this data enhancement work on the Impressionism Mooc, as shown in the shared slideshow by Christine Vaufrey.

Should a platform tell us everything about the learners?

Sometimes counterproductive information

Quantitative information provided by platforms can obscure pedagogical objectives. Learners' questions are then essentially of the type: "The platform tells me that I have not completed this activity, and yet...", or "I spent an hour on the question on this forum, and the platform indicates ten minutes". Few exchanges are really about the training.

The more specific the information provided to learners about time spent and results, the more their attention is focused on these issues, at the expense of the training itself.

The first few minutes of a module should allow enrollees to understand how they are being monitored, and what information tutors will collect to follow up and validate the training.

Leave room for pedagogical dialogue

Much of our learning is done by trial and error. Dialogue between the learner and the trainer has a vital place in this approach. We use the word dialogue, not interpretation. No matter how experienced the trainer is, he or she will mostly make assumptions about the errors found. Exchanges with the students will confirm or invalidate these hypotheses.

To ask the machine to tell us everything is to dream of a system where it is no longer necessary for the learner to express his or her difficulties. To entrust adaptation to the machine is to deprive ourselves of the richness of dialogue and exchange, not only with the instructor but also with other learners.

I know all about you

Without denying the interest of these approaches and its research, it remains essential to reflect upstream of the training on the place of the person enrolled in the construction of his or her learning.

Helping the learner discover his or her learning style.

All training aims to develop a form of autonomy. Yet, some of the models presented take the learner in hand, make the best choices for them, and claim to accompany them so well that they don't even realize it... There is then a risk of reducing this capacity to learn alone, to develop one's own strategies.

Another approach is to encourage learner reflection on their strategies and their personal learning environment.

The two approaches are not antagonistic. Regardless of the information a platform produces, it remains essential to give space to participants and educational teams.

illustrations: Frédéric Duriez

Resources

"No tracking, no e-learning" Jérôme Bruet accessed February 7, 2015
http://www.e-doceo.net/blog/pas-de-tracking-pas-de-e-learning/

"The panopticon, a model of the prison world" Yohan Demeure consulted February 4, 2015
http://citizenpost.en/2015/01/le-panoptique-modele-du-monde-carceral/

Knewton, blog accessed February 7, 2015
http://www.knewton.com/blog/

"Counting is not understanding" Christine Vaufrey accessed February 7, 2015
https://christinevaufrey.wordpress.com/2015/01/19/compter-nest-pas-comprendre/

"The Memory of Moocs" Christine Vaufrey accessed February 8, 2015
http://fr.slideshare.net/christing/archivage-webpedagoina

"The Future of Education, Personalized Learning?" Alexandre Roberge accessed February 7, 2015
http://cursus.edu/article/21403/futur-education-apprentissage-personnalise/#.VNZ6_y6Pfwo

"From MOOCs to adaptive learning: what's going to get in the way" Matthieu Cisel accessed February 7, 2015
http://blog.educpros.fr/matthieu-cisel/2015/01/26/du-mooc-a-lapprentissage-adaptatif-ce-qui-va-bloquer/

From MOOC to adaptive learning: cognitive tutors Matthieu Cisel accessed February 4, 2015 http://blog.educpros.fr/matthieu-cisel/2015/01/21/du-mooc-a-lapprentissage-adaptatif-les-tuteurs-cognitifs/


See more articles by this author
payhip

Access exclusive services for free

Subscribe and receive newsletters on:

  • The lessons
  • The learning resources
  • The file of the week
  • The events
  • The technologies

In addition, index your favorite resources in your own folders and find your history of consultation.

Subscribe to the newsletter

Add to my playlists


Create a playlist

Receive our news by email

Every day, stay informed about digital learning in all its forms. Great ideas and resources. Take advantage, it's free!