In preparation for our ELI presentation, here are a few thoughts and resources to share with anyone interested in this topic. We will demo a section of an online module designed to present foundational concepts in Pharmakokinetics to first year PharmD students.

Caveats:

  • Students log in to the module outside normal class time with their unique ID
  • The module supports and enhances in-class interactions between students and with the instructor
  • Only students’ behavior inside the instructional module is tracked

Why use clickstream data? What are the drivers?

  • Accountability (accreditation; institutional accountability efforts)
  • Affordances of the technology (easy to implement, unobtrusive)
  • Research (scholarship of teaching and learning)

What factors raise concerns?

  • Privacy issues. Do students know that their behavior online is tracked (similar to Blackboard page tracking)?
  • The overall weight given to clickstream data as an assessment form
  • How the resulting data will be used (high-stakes decision-making regarding individual students v. assessing overall class progress at a point in time)

What opportunities are presented?

  • Improving individual and group learning outcomes by immediately identifying misconceptions and problems, and addressing them in class
  • Modifying in-class instruction and activities to meet individual and class needs at the point in time in which they arise
  • Discovering how students actually use interactive content (research)
  • Improving the instructional impact of interactive content & online learning environments based on students’ behavior, feedback, and other assessments
  • Mapping content, activities, and assessments into a connected whole rather than disparate parts
  • Co-opting a method typically used by advertising/marketing to predict who will buy something and instead using it to better understand learners’ behavior and potentially, predict how people with different learning styles will use online environments (research)
  • Providing students’ own clickstream data to help them reflect on their learning, progress, and strategies

Questions:

  • Can a student turn off tracking or is it “always on”? Can a student choose to have his or her tracking be anonymous?
  • How long is data stored? Where? What trends can reasonably be deduced over time?
  • What is the best that tracking can tell us about learning? Where are the boundaries and what are the limits?

Next steps:

  • Reports (sense-making from raw data to inform instructors, students, and designers)
  • Visualization (graphs, charts, the elusive “dashboard” of learning outcomes)

Resources:

Advertisements