What Can We Measure in Informal STEM Learning?
By Sarah Dunifon
When you think of informal STEM learning, what do you think of? Field trips programs at the aquarium? Cooking in the kitchen with your grandfather? Listening to a science podcast?
The truth is that informal learning happens across many contexts and times. It can happen when you’re on your own or when you’re with a group. The possibilities are endless. If you haven’t seen this image before, take a moment to breathe in how vast our informal learning opportunities are in our lifetimes. As depicted in the image, while formal education takes up a structured place in our lives, informal learning is life-long, life-wide, and life-deep.
Figure 1: The LIFE Center Lifelong and Lifewide Learning Diagram
What do I mean by life-long, life-wide, and life-deep? You might hear these terms thrown around in informal learning contexts.
Life-long: informal learning occurs throughout our entire lives. As children, we might experience informal learning through play, when we taste a chunk of grass, or experiment with slime.
Life-wide: informal learning occurs in all areas of life. Even in formal education settings (think: K-12 schools), informal learning might occur when you’re investigating plants during recess or counting the number of steps you take between classrooms. Informal learning can happen in planned situations (like a visit to a museum) or in completely unexpected ways (like you catch a clip of Science Friday on the radio at the doctor’s office).
Life-deep: informal learning relates to the social, religious, and moral experiences and beliefs that we hold. How we interact with the world and our communities will influence how and what we learn.
So now that we know how and where informal learning occurs, let’s talk about what we can measure to understand the impacts of this learning. In the informal STEM learning environment, there are many different things to measure and ways to measure. You might have heard terms like formative evaluation, summative evaluation, self-efficacy, culturally-responsive evaluation, science identity, science capital, and more.
I tend to think about these things in different categories:
Ways of conducting evaluation (e.g., equitable evaluation, culturally-responsive evaluation, participatory evaluation, etc.)
Types of evaluations one could conduct (e.g., formative, summative, process/implementation, outcome, etc.)
Things we can measure (e.g., science capital, self-efficacy, processes, social-emotional learning, science identity, the strength of mentoring relationships, etc.)
If these terms are completely new to you, I encourage you to spend some time exploring informalscience.org. This is a National Science Foundation-funded website of projects, research, and evaluation resources all pertaining to informal STEM education. You can see examples of these types of evaluations, learn more about constructs like science capital, and find people who are working on informal STEM education projects.
In informal STEM education evaluation, we tend to think about these things we can measure as they relate to the programs we run. If you’ve ever created a logic model or theory of change for your programs or organization, you might recognize some of these terms. Check out the image below for how I think about what to measure.
Figure 2: Improved Insights “What Can I Measure?” Diagram
As you can see, the triangle is divided into sections. These sections do not indicate the importance of these areas hierarchically. Rather, it is used to demonstrate how I see most organizations currently collecting data. Most focus on the outputs (such as numbers served, hours of programming, etc.) over learning gains. Most do not tend to collect psychosocial outcomes (like attitudes, values, and behaviors) but totally could!
These are all representations of things we might look at with outcome evaluation. You’ll also see (represented in the circles) processes and relationships. Depending on the goals of your evaluation, some of these things might be more important to measure than others.
Okay, so by now we’ve talked about what informal learning is, how it operates, and what you might measure. The last thing I’ll leave you with is a list of some common constructs in informal STEM education, just to get you started.
Figure 3: Improved Insights Common Constructs in Informal STEM Education
As you can see, there’s a lot we can do in informal STEM learning to investigate the impacts of our work. Follow along by signing up for my newsletter to learn more about this great space.