How educators can collect meaningful data in a virtual learning world
By Sarah Dunifon
For educators, data collection can be a very useful tool to gauge learner understanding and engagement. While you may be familiar with forms of assessment like standardized tests, content-specific exams, and graded projects, there are many more ways to collect data in order to improve your practice. Formal educators and informal educators alike can benefit from exploring the intersections between their practice and the space of educational evaluation and assessment.
In “normal” times (i.e., when we are not experiencing a global pandemic), educators might employ strategies like observations, formative assessments, or focus groups to understand student experience. Informal educators (like those who work in museums or aquariums) might observe how learners engage with an exhibit, taking notes on where learners spent their time, what they talked about while engaging with an activity, or what skills they exercised. Formal educators (like classroom teachers) might informally assess a class’s understanding of a new topic via a show of hands, or may read student body language to know when a student needs a little extra help.
Times have changed. Educators adapting to our “new normal” have had to pivot their practice rapidly in order to move from largely in-person instruction to remote learning experiences. Museums and other informal learning institutions are relying heavily on virtual experiences, like livestream events and lectures, virtual walk-throughs of exhibits and collections, at home art lessons, AR and VR offerings, podcasts, and engaging TikTok videos. Schools have shifted to remote or hybrid-style learning environments via platforms like Zoom.
But what does this mean for data collection? Educators can (and should!) still collect data, but need to consider how a virtual format may affect their process. For example, observations of group learning dynamics may not be possible right now, but there are plenty of digital tools available to educators to gauge interest, understandings, and impacts from their students. Educators should also consider ethics, data privacy, and informed consent. Similar to non-COVID times, if you will be collecting video recordings of students, you will need to put more thought into this than if you are sending out an anonymous survey, for example.
Data collection methods that educators may choose to use during with virtual or remote learning experiences include:
Online polls
Virtual focus groups
Virtual interviews
Social media analysis
Tracking web engagement
Digital surveys
The informal learning community has shifted their practice quickly to accommodate virtual experiences. The Visitor Studies Association (a national professional organization for informal learning in museums and institutions) has employed virtual training sessions like “Google Analytics 101: Understanding Your Visitors’ Behavior Online” and “Conducting Focus Groups In-Person and Online” to connect its constituents with new ways of working.
Evaluators themselves have risen to the challenge as evidenced by this example from the Philadelphia Museum of Art.
“Also challenged with moving to the virtual space was Kerry DiGiacomo, Director of Audience Research at the Philadelphia Museum of Art, who was asked to evaluate virtual programs, which the PMA had never implemented prior to the pandemic. Through a prototyping process, DiGiacomo’s team was able to solve a number of problems early in the design process, including pacing, allocating time for program components, and determining how to measure engagement. The team realized that participants engaged in many ways, not all “visible.” Some preferred listening and watching, while others chatted in the comment box and unmuted to speak to the group. It was a great reminder that just like in real life, online learning takes many forms.” (Essential Evaluators: Snapshot: Data Collection)
As we look towards a post-COVID world, learning institutions are already beginning to think about how they may pivot their practice back to an in-person or hybrid model. For now though, educators would be wise to continue exploring ways in which they can collect meaningful data in a digital context.
For more information on how informal learning institutions are adapting to the pandemic, see this resource from the American Alliance of Museums.
Sarah Dunifon is Founder and Principal Evaluator of Improved Insights LLC, an educational evaluation firm focused on STEM and youth-based programming. She is based in Cleveland, Ohio.