by Andrea Hutson
Evaluators: be honest: have you ever worked hard on an evaluation, found some really exciting findings, wrote an amazing report, and turned it in expecting….
- praise from program leadership!
- noticeable, positive program changes!
- maybe a promotion? (or extended contract?)
And instead, got… crickets?
Let me raise my hand the highest. It’s definitely happened to me, and one time in particular is burned into my mind.
I was in a final report meeting with a large agency. The evaluation we were reporting on was a multi-year, multi-site evaluation with mixed methods, requiring multiple staff, travel, a ton of time with SPSS and R, and a very long time writing and editing the report.
There were about 10 people in the that final meeting, including the relatively new program director (sidenote: one interesting thing about long evaluations – in some organizations, change is so constant that often the only people who are with the program from start to finish are the evaluators).
Our final report was over 100 pages, with appendices, and lots of technical detail.
After I presented all of the results from the study, the program director looked at me and smiled. “Wow! These are really interesting results!” I smiled back.
“But…”, she started… and I felt my heart sink.
Then she dropped a bomb.
“But, you know no one will ever read this.”
Oh my goodness, so painful to hear – but I knew she was right.
It’s not that the content was bad (of course it wasn’t! I wrote it!), it’s just that it was too dang long.
Decision makers don’t have time to read.
The average 30-page evaluation report takes about an hour to carefully read and process (and at over 100 pages, my report would have taken 3+ hours)
The problem is, leaders – especially the higher you get up the food chain – don’t have an hour.
There was a fantastic study of CEOs conducted in 2018 by the Harvard Business Review (https://hbr.org/2018/07/the-leaders-calendar) that explains why. (Although the report only looked at CEOs from the business world, non-profit and governmental leadership likely have very similar constraints.)
According to the study, CEOs spend the vast majority of their time (about 72%) in meetings, either with their own staff, leadership, or clients. They have very little time alone, and most of that time is in small chunks (i.e., not conducive to report reading).
Check out this time graph of one CEO’s alone time, arranged in 15 minute green blocks.
Even when we include Saturday and Sunday, this CEO only had 8 hours of one-hour or more alone time for the entire week, even though he was working a TON of hours (note that he starts working before 6 AM most days).
The study noted that the CEO spent the majority of that alone time (55%) reading and responding to emails. After we remove that time, he had only had about 3 or so hour-long chunks for all other unscheduled tasks. FOR THE WEEK.
And that’s the reason that long reports don’t get widely read.
There’s just no time!
What can you do?
Now beyond going past the existential crisis about why we write reports at all, there are a few solutions. Here are three ways you can maximize your impact.
- Don’t write a report; make a presentation instead. This method is getting more popular over time; UX Evaluators typically present results in one final PowerPoint. If you noticed above, we added on a presentation, which was the product that got us traction with the program leadership. That’s also a possibility, if a report is absolutely required (although the product we had worked for hundreds of hours on became essentially a doorstop).
- Write a brief report instead of a long one. We’ve had tremendous success with 4 or 5 page evaluation briefs. When carefully written, these briefs can include the vast majority of findings, details, and recommendations that people need, in a format that’s digestible and a relatively quick read.
- Add on a powerful one-pager. One pagers delete all but the most essential content, giving just the highest level findings from a study. The best thing about one-pagers is that they can be added on to a brief report, or even a very long one. Stephanie Evergreen has a great post on the 1-3-25 model, (https://stephanieevergreen.com/the-1-3-25-reporting-model/) in which a one-pager is tacked on to an evaluation brief which is tacked on to a longer report (but notice: the ‘longer report’ is only 25 pages).
Want to learn more about option 3?
Lindsay and I are currently working on a course on one-pagers that’s designed for anyone who wants to make more of an impact with their reports. This course will provide easy steps to turn a long report into a one page document that engages readers and packs a punch. And you’ll do it without needing graphic design skills or any fancy software. We’ll be sharing some of it on the blog, but if you’d like to be notified when the course goes live. subscribe below.