Investing in Evaluation
Why invest in evaluation?
Typically, my clients are investing in evaluation for one of two reasons: (1) they are required to conduct an evaluation of a program as a part of grant stipulations, or (2) they’re interested in understanding their programs better and improving their practice.
No matter the motivation, the value proposition is clear.
Evaluation is vital for informal STEM organizations because it allows us to learn about our audiences and offerings, collect hard data to support our work, and more clearly tell stories of impact to our partners and supporters.
Let’s dig in to two of the most prominent benefits now:
Improving Practice
One of the most important elements of my job as an evaluator is to help organizations see the work they do more clearly. Sometimes, when we’re too close to something we fail to recognize the big picture. Or, perhaps we have some preconceived notions or early judgments on a program that are hard to shake.
Bringing an evaluator in on a project can offer a fresh perspective and clear eyes. They can point out things that would have typically gone unnoticed. These fresh perspectives can identify areas for improvement, unique strengths, curious outliers, and spaces to explore further.
One of my favorite aspects of an evaluation project is bringing the data and findings to the organization to conduct a “meaning-making session.” In that, we comb through the data together, identifying things that surprised us and that made total sense. Then, we craft a plan to address it all. The recommendations we create together give organizations a concrete plan and steps forward for improvement. A little shakeup can mean big improvements in our practice.
Evaluation and evaluative thinking is also a great way to create a culture of improvement in your organization. When we stop looking at things as “good or bad” and start looking at the nuances and complexities of our work, new opportunities open up to us. By making evaluation a priority in your programs, you introduce the idea of continuous improvement and allow everyone to take part in improving the work.
Communicating Impact
As an informal educator early on in my career, I remember looking at the programs I worked on and just knowing that they did good things for the youth we served. I saw firsthand the development of confidence, STEM literacy, leadership, and environmental content knowledge our K-12 audiences gained. But it was not until I got more involved with the evaluation process that I understood how to capture that firsthand experience and better communicate it to our funders, internal development staff, and even the education department leadership. Being able to tell the story of impact more concretely and successfully added leverage to the programs and compiled support behind them. Now, not only could I see the importance of this work - so could everyone else.
As a nonprofit leader, you know how important it is to advocate for your work. Being able to tell the story of impact for your programs and audiences is hugely important. Evaluation is a critical tool in developing and conveying that story. With more and more organizations relying on data as a key indicator of performance and success, the ability to collect, analyze, and interpret data is one of the most important tools in your kit.
How much do I invest?
Figuring out how many resources to invest in evaluation depends largely on your goals. If you’re planning to invest in evaluation for one program alone, your costs will look different than investing in a department-wide strategy or multiple sustained program evaluations.
And, if you’re working under grant guidelines, that will look different as well. The rule of thumb for large-scale governmental grants is to allocate 8-12% of the program budget per year for evaluation. So, for an award of $100,000, you’d be investing $8,000-$12,000 to evaluate the program.
Now that number is typically a little stiff for programs running without grant support, which is why it is important to identify your goals and requirements up-front, before meeting with an evaluator. Typically, evaluators are able to design a project to fit within your budget and requirements, so while it might be tempting to let them design first and scale back later, it’s definitely worth expectation-setting with a budget or budget range up-front.
Another option is to invest in an internal evaluator via a staff position. Many larger nonprofit organizations have an evaluator on staff or even develop their own evaluation departments. If you plan to do routine evaluations on most or all of your programs, this might be a good option for you. But, if you’re looking for less frequent evaluation support or do not have the resources to hire full-time, it can be more cost-effective to bring in an external evaluator instead.
A third option is to build the evaluation capacity of your current staff. Early in my career as an informal educator, I became the de-facto evaluator in my organization’s education department, simply due to my own personal interest in that area. More than ten years of informal STEM education evaluation experience later, that interest has become my lifelong career (funny how that works out, huh?). Many external evaluators offer capacity-building services for organizations looking to increase their own evaluation knowledge and skills. This might be a stand-alone series of trainings and workshops, one-on-one coaching with educators and program managers, or an add-on to an evaluation project where the evaluator provides more context for how and why they are completing the work as they complete the project.
If you’re really not sure of the scope of the work you’re looking for, reach out to an evaluator friend. Describe what you’re looking for, and ask for their rough estimate.
How do I choose what to invest in?
Deciding which program(s) to evaluate can depend on many factors, like:
Grant stipulations
Community interests
Changes in internal funding priorities
Staffing changes
And more
You might approach this question differently depending on the needs of your organization at this time.
For instance, let’s say your organization is going through a restructuring. The education department will be narrowing its scope to the most important programs - but - you’re not really sure how to determine which programs are most important and should continue. In this case, you might invest in some program evaluation to measure program effects, community interest, audience satisfaction, and other elements to determine which programs to keep and which to sunset.
Or, maybe you have a grant-funded program that stipulates evaluation as a necessary component to receiving the funding and being considered for future funding. Clearly, that’s an area to invest in.
Perhaps you have a new program you just rolled out and are so excited about. It was co-developed with the community and is an innovative take on informal STEM education that you think could really make a difference. You’d like to gauge the effectiveness of the program and ensure that it continues for years to come, past its pilot. This is definitely a project you could invest in.
The bottom line is this - investing in evaluation allows informal STEM learning organizations to take their programming to the next level, better serve their audiences, and amplify their impacts.
If you enjoyed this post, follow along with Improved Insights by signing up for our monthly newsletter. Subscribers get first access to our blog posts, as well as Improved Insights updates and our 60-Second Suggestions. Join us!