By Lindsay Lamb
Recently, I was in a meeting with a client presenting data from a quick survey check in. We were reviewing the data visualizations I made based on some earlier feedback. Sometimes when I share survey data, I collapse categories to show an overall level of agreement, or to indicate the overall percentage of favorable responses.
Since we were examining responses from students who were engaging in a virtual internship in the middle of a pandemic, we figured most responses would generate a negatively skewed distribution (see graph below, and if you’re interested in learning more about skewness, read this!).
Indeed, preliminary examinations of the data suggested that students’ responses were skewed such that there were few responses at the low end of the scale on our various survey questions (e.g., disagree, strongly disagree, not at all likely, etc.). We decided to examine movement in students’ responses at the top end of the distribution (e.g., strongly agree, agree, extremely likely). Based on this decision, I crated a visualization only showing the most favorable responses (presenting only the percentage of strongly agree or extremely likely responses, for example).
So, back tot the meeting.
All was going well. We were having a lively discussion of the data I presented in my one-pager when the client made a casual comment about being bummed about some of the responses. They had spent so much time pivoting their instructional material to be engaging in this new virtual environment, and only a third of responses indicated students felt the lessons were extremely interesting.
Ugh. My heart sank.
Fortunately, I knew there was more to the data. I pivoted. I had the data up in an interactive dashboard and shared my screen with our client. I recomputed responses to include extremely interesting and interesting.
“Actually, nearly all of your participants were interested in the lessons and found them relevant!”
Sure, a little over a third found the lessons extremely interesting, but the bigger story is that most students were engaged and found the content relevant to their lives. This was huge! Our client had pivoted their entire learning platform to a virtual environment, and students were engaged.
What did I learn?
Sometimes it makes sense to have a plan, but you need to adapt. We are program evaluators and there isn’t something that we love more than a good evaluation plan (except maybe a logic model!). Sometimes we can get rigid and only want to look at the data in a specific way, thinking that we will know what the data will tell us. Sometimes, however, we need to let the data tell its own story.
Don’t be afraid to pivot. Recomputing the data not only showed our agility as researchers and consultants, but also helped our client see their data in a new way. It helped our client see the value of their work and trust us in our services.
The power of one-pagers. Having a one-pager with data visualizations and short text explanations helped our client easily see the data and understand the story. It also helped us recognize where we needed to go in our analysis. Having the data in a digestible way facilitated our conversation and helped us see where we needed to go. It was empowering for me and empowering for our client.
Have your data handy. In addition to having a user-friendly one-pager, having the data accessible in a sharable format helped us see where we needed to go in our analyses. It was empowering for me and empowering for our client.