Archives for posts with tag: eliannual08

What a treat to be able to exchange information and ideas related to…data! Others we met are engaged in similar efforts. Most notably, a team is working on CANSAWARE to track, mine, and present data from Sakai.

The clickstream is one <easy> way to implement an imperfect method of assessment that aligns with the technology used — but it seems that the true power comes from:

  1. the meaning individuals (faculty) or groups (faculty, students, and support staff) attribute to the data and the action that results, and
  2. the ability to display trends in behavior over time

It seems that we are on a useful path but would be wise to clearly articulate what data is tracked, why, and how it is used. Data to help students reflect on their learning (metacognition) is one of the best immediate uses of the data.

In the end, it’s about trust—-just like any instructor/student interaction.

We need to conduct experimental research in Fall 2008 classes and try to identify and publish results and best practices associated with this methodology — as pertains to education. Bring on the IRB (institutional review board) paperwork…

In preparation for our ELI presentation, here are a few thoughts and resources to share with anyone interested in this topic. We will demo a section of an online module designed to present foundational concepts in Pharmakokinetics to first year PharmD students.

Caveats:

  • Students log in to the module outside normal class time with their unique ID
  • The module supports and enhances in-class interactions between students and with the instructor
  • Only students’ behavior inside the instructional module is tracked

Why use clickstream data? What are the drivers?

  • Accountability (accreditation; institutional accountability efforts)
  • Affordances of the technology (easy to implement, unobtrusive)
  • Research (scholarship of teaching and learning)

What factors raise concerns?

  • Privacy issues. Do students know that their behavior online is tracked (similar to Blackboard page tracking)?
  • The overall weight given to clickstream data as an assessment form
  • How the resulting data will be used (high-stakes decision-making regarding individual students v. assessing overall class progress at a point in time)

What opportunities are presented?

  • Improving individual and group learning outcomes by immediately identifying misconceptions and problems, and addressing them in class
  • Modifying in-class instruction and activities to meet individual and class needs at the point in time in which they arise
  • Discovering how students actually use interactive content (research)
  • Improving the instructional impact of interactive content & online learning environments based on students’ behavior, feedback, and other assessments
  • Mapping content, activities, and assessments into a connected whole rather than disparate parts
  • Co-opting a method typically used by advertising/marketing to predict who will buy something and instead using it to better understand learners’ behavior and potentially, predict how people with different learning styles will use online environments (research)
  • Providing students’ own clickstream data to help them reflect on their learning, progress, and strategies

Questions:

  • Can a student turn off tracking or is it “always on”? Can a student choose to have his or her tracking be anonymous?
  • How long is data stored? Where? What trends can reasonably be deduced over time?
  • What is the best that tracking can tell us about learning? Where are the boundaries and what are the limits?

Next steps:

  • Reports (sense-making from raw data to inform instructors, students, and designers)
  • Visualization (graphs, charts, the elusive “dashboard” of learning outcomes)

Resources:

Attended some excellent sessions today — most notably Henry Jenkins’ keynote, “What Wikipedia Can Teach Us About New Media Literacies” and George Siemens’ presentation on “Connectivism.” More details and reflections to follow later — however, there are some good resources I’d like to remember:

Complexity, education, networks, participation & learning ecologies — loved it!