Tuesday, January 31, 2012

#LAK12 Model-building in EDM

Ryan Baker's presentation on educational data mining today as part of the Learning Analytics and Knowledge MOOC helped me to better understand the model development concept in EDM. One of his slides listed several types of learner behaviors for which he developed models. When learners interacted with educational software, he was able to describe certain learner behavioral patterns (number of clicks, wait time between clicks, order of clicks, etc.) in such a way that if the data log files were analyzed for any learner, those behaviors could be spotted. He could identify when a learner was exhibiting behaviors such as:Miners' Memorial
Photo by Tim Duckett (tim_d)


  • carelessness

  • off-task activity

  • gaming the system

  • avoiding help when they needed it

  • not asking for help because they didn't need it

  • guessing

I'd be very interested in hearing more about the model development step, and how the researchers constructed meaning from a pattern of clicks.


There are two observations that I took away from the presentation.


1) Model-building is inherently value-laden. Baker alluded to this a bit when he mentioned that learner behaviors that were observed in one of his modelling tests were vastly different in the Phillipines as compared to the United States. Learner behaviors are conditioned by culture and situated in a culture. As educators, we too are the product of our culture and cannot avoid building our assumptions into every tool and system that we develop. Since these data models are used to classify learners, model builders need to approach their task with the utmost humility and care.


2) The educational data mining approach seems to be an example of behaviorism. There was much discussion about the observable behaviors of the learners but not much about the learners' internal thoughts and feelings. Baker did allude a bit to thoughts and feelings of U.S. and Phillipines students when they interacted with his Scooter software. For me, that was the most interesting part of the presentation.

3 comments:

  1. "Since these data models are used to classify learners, model builders need to approach their task with the utmost humility and care."

    Humility, care and, I would add, transparency. Transparency helps to explicit the values behind the models and to correct errors (there always are).

    ReplyDelete
  2. Hi Nancy,
    You might find this paper useful
    Baker, R. S., Corbett, A. T., & Koedinger, K. R. (2004). Detecting student misuse of intelligent tutoring systems. Paper presented at the 7th International Conference on Intelligent Tutoring Systems.
    It explains how Ryan and his colleagues went about identifying students who appeared to be 'gaming the system' in a way that correlated with low levels of learning.

    Although their analysis wasn't able to identify learners' internal thoughts, their introduction does discuss student motivation and its relation to empathy and enjoyment.

    ReplyDelete