By Lindsay Lamb
We have all been there: working long hours on tight deadlines on a grant. Establishing measures, outputs and outcomes that are informative, meaningful and tied to a logic model. And inevitably, you come to one that should be easy, but often is not: “number of students served by a program.”
Recently, Andrea and I were working on a grant proposal with one of our clients and we were reminded of just how tricky this metric is – particularly during the time of COVID. Oftentimes, programs count a student if they attended one of their sessions. Some count attendance using a threshold for students meet in order to count (say 50%), while others count a student as being served by a program if a student attended a specific session (e.g., FAFSA training), and others count a student if they participated in a set number of programming hours integral to the program.
In the grand scheme of things, these all “count” (pun intended) – but are they measuring the same thing? How were these thresholds set? What does it mean if a student attends one 3-hour session and another student participating in the same program attends a 1-hour session? Where is the tipping point? It might be that attending 3 one-hour sessions spread out over a few weeks is more meaningful to students compared to one 3-hour session. It might be that some students lose focus in a 3-hour session and cannot retain all of the information they learned whereas other students might forget what they learned in-between sessions if they are spread out.
How do you know where that sweet spot is, and in terms of grant compliance does it really matter? Let’s leave the existential dilemma out of this, and all agree that these counts matter.
Now that we have that out of the way, how do you count students served by your program?
If you have any data to draw on – attendance records from each session, session duration, session type – you are way ahead! First, figure out the total number of classes/meetings/sessions that are offered each semester or year. Then determine the length of time students spend in each session. What does 100% participation look like? Now, walk back from 100% participation to set realistic targets. For example, use these data to determine the average number of sessions your students attend, the average number of hours of programming students engage in, the most common type of programming students participate in, and more.
Once you have averages you can use this information to set a threshold for participation and include expected gains in students served by your program that is more robust than simply counting bodies. Perhaps your threshold is one standard deviation above the average number of hours of programming or number of sessions, or a percent increase in number of hours of participation.
Some of you may have heard of this referred to as dosage. Having this information is extremely powerful when you tie it to your program outcomes.
For example, if you know the average number of hours in which students engage in your programming, and the type of programming they engage in, you can say things like, “Students who engaged in 8 hours of programming related to growth mindset experienced greater improvements in their academics than did their peers who engaged in fewer hours of programming.” Below is a graph from an analysis Andrea created a while back using this type of data:
This information will also help you identify the types of programming most related to positive outcomes. For example, if your program focuses on helping students achieve post-secondary success, maybe students who attended a session helping them complete the FAFSA is most directly tied to enrolling in college. Or perhaps it is attending a session with a guest speaker who attended community college that was immensely helpful.
Having the right data can help you – and your program staff – gain more insight into your program and focus your efforts on what matters most to students.
How should you track this information? Some organizations use Google Sheets, some use various survey platforms, and some use Excel. The point is to get the data and start building your database. From there, you can build dashboards in Excel – yes Excel! – and start tracking this information and make more informed decisions when it comes to including this type of information in grant proposals and grant reporting.