Agile Analytics, Author at Agile Analytics, LLC

Knowing when to pivot

By Lindsay Lamb

Recently, I was in a meeting with a client presenting data from a quick survey check in. We were reviewing the data visualizations I made based on some earlier feedback. Sometimes when I share survey data, I collapse categories to show an overall level of agreement, or to indicate the overall percentage of favorable responses.

Since we were examining responses from students who were engaging in a virtual internship in the middle of a pandemic, we figured most responses would generate a negatively skewed distribution (see graph below, and if you’re interested in learning more about skewness, read this!).

A negatively skewed distribution (meaning most responses are positive, or favorable)

Indeed, preliminary examinations of the data suggested that students’ responses were skewed such that there were few responses at the low end of the scale on our various survey questions (e.g., disagree, strongly disagree, not at all likely, etc.). We decided to examine movement in students’ responses at the top end of the distribution (e.g., strongly agree, agree, extremely likely). Based on this decision, I crated a visualization only showing the most favorable responses (presenting only the percentage of strongly agree or extremely likely responses, for example).

So, back tot the meeting.

All was going well. We were having a lively discussion of the data I presented in my one-pager when the client made a casual comment about being bummed about some of the responses. They had spent so much time pivoting their instructional material to be engaging in this new virtual environment, and only a third of responses indicated students felt the lessons were extremely interesting.

Slightly over a third of students’ found the lessons *extremely interesting*, just over half found the lessons or *extremely relevant* to their lives.

Ugh. My heart sank.

Fortunately, I knew there was more to the data. I pivoted. I had the data up in an interactive dashboard and shared my screen with our client. I recomputed responses to include extremely interesting and interesting.

“Actually, nearly all of your participants were interested in the lessons and found them relevant!”

Pivoting and recomputing the responses told our client a more positive story.

Sure, a little over a third found the lessons extremely interesting, but the bigger story is that most students were engaged and found the content relevant to their lives. This was huge! Our client had pivoted their entire learning platform to a virtual environment, and students were engaged.

What did I learn?

Sometimes it makes sense to have a plan, but you need to adapt. We are program evaluators and there isn’t something that we love more than a good evaluation plan (except maybe a logic model!). Sometimes we can get rigid and only want to look at the data in a specific way, thinking that we will know what the data will tell us. Sometimes, however, we need to let the data tell its own story.

Don’t be afraid to pivot. Recomputing the data not only showed our agility as researchers and consultants, but also helped our client see their data in a new way. It helped our client see the value of their work and trust us in our services.

The power of one-pagers. Having a one-pager with data visualizations and short text explanations helped our client easily see the data and understand the story. It also helped us recognize where we needed to go in our analysis. Having the data in a digestible way facilitated our conversation and helped us see where we needed to go. It was empowering for me and empowering for our client.

Have your data handy. In addition to having a user-friendly one-pager, having the data accessible in a sharable format helped us see where we needed to go in our analyses. It was empowering for me and empowering for our client.

School Goes Virtual (Again)

by Lindsay Lamb (with a few random comments by Andrea Hutson)

Due to the recent increase in COVID-19 cases nationally, it is likely that if you have a school age child, they will not go back to school in a normal manner (regularly, in person) this fall.

As I have mentioned before, I have two children that I have been doing my best to keep educated, entertained, and healthy while we have all been sheltering in place these past four months. My eldest is slated to start kindergarten this fall, and what should normally be an exciting, happy time is now riddled with fears. Going back to school in person could mean infecting teachers, our entire family getting infected, and a disruption in routine if schools are unexpectedly closed again. Virtual learning provides its own set of challenges, including me continuing to have to juggle work and school time as well as social and emotional trauma that my daughter has experienced during this time of isolation.

I am not alone. The long and short of it is that closing schools has placed an impossible burden upon families, particularly families coming from low-income communities. But given the recent spike in COVID-19 cases, what do we do? School leaders are placed in an impossible position as are teachers and families. This affects the entire community. I do not envy leaders at this time.

[Sidenote from Andrea: I just received this question from a friend – “Would you rather facilitate a PTA meeting around schools reopening in the fall, or do a trust fall into a pit of crocodiles?” The fact that the crocodiles are an appealing option shows how charged this issue is.]

Where does this leave us?

Instead of getting caught in an endless loop of ranting and venting (which is hard), let’s find some positive things we can build on as we begin to navigate virtual student learning in the Fall.

Fortunately, we have some more time to plan than we did in the spring. We can draw on the virtual teaching and learning experiences from the spring and learn from them, because chances are, no matter what our plans are in the fall, we are going to need these best practices in our arsenal until we have a COVID-19 vaccine.

What do we know about virtual learning?

1. Not all virtual learning environments are bad!

Take for example, one of our clients who is helping interns in a summer work program learn essential social and emotional skills necessary for workplace success. These students come from all different backgrounds and are eager to learn. Our client expertly pivoted their course content, creating relevant and timely material that the students are eager to learn. We are working with them to develop a list of best practices to share with other educators in the coming months.

Some research has shown that some children, particularly older children, are thriving in the online environment. Anecdotally, another program that we work with measures the anxiety and depression levels of its teen participants, and their levels of both actually went down after lockdown began.

Online education works best when it is:

high quality (obviously)
structured, with points for real connection
a mix of formats (sorry, but a 7-hour long Zoom meeting is not going to be effective)
done with older children & teenagers

2. Teach parents to be co-educators. Parent involvement is even more critical.

Parents in this time know that, even if it is unfair, they have been thrust into the role of co-teacher for their students. To the extent possible, leaning into this can produce benefits that will last long after COVID is over.

One client offers reading tutoring services to at risk students in grades K-2. Although there were many challenges with continuing their program virtually, some tutors reported a strong benefit of having the ability to connect with students’ families in a way that had not been possible before. Parents are able to watch the tutoring sessions and get a better idea of where and how their children are struggling, and are able to learn from the tutors some ways to reach their children.

Schools and organizations can help by:

Engaging with parents early in the process
Understanding that parents will have different needs and levels of availability
Providing materials to teach the parents as well as the students

3. Don’t forget Social Emotional Learning (SEL), and be creative

Virtual learning leaves out the hugs, the high fives, the social and emotional connections and learning students need now more than ever. And who else needs this support? Teachers and other school staff. Andrea recently sat in on a meeting about teacher and student SEL during this time. Some takeaways:

SEL makes a meaningful difference for both teachers and students
Teachers must fully embrace these concepts before being able to teach them to students
Time must be set aside for SEL – it can’t be ‘one more thing’ added to already full plates
SEL can fall flat for older children if not approached carefully. Many elementary schools are now teaching SEL skills, and when kids get to middle/high school, they think, “Been there, done that.” When we are teaching SEL skills to our middle and high school students, instead of trying to use the same content aimed at younger children, we need to think of how we’d teach adults SEL concepts and adapt that way.

4. We need a better way to reach students who are disengaged

Raise Your Hand Texas interviewed five Texas teachers about their teaching experiences this Spring. These teachers faced a number of challenges, and hearing their stories is inspiring. The teachers got super creative, conducting zoom dance parties, scavenger hunts, virtual field trips, and more. They show that even in the time of COVID, school can be fun and creative, and that many teachers are doing everything they can think of to engage students.

However – As these teachers described, despite their best efforts, some students just stopped participating. The official tally is about 10% statewide, but we think the ‘real’ number is higher – and it’s not evenly spread across communities. LA, for example, lost about 1/3 of their students in the spring semester [there’s a wonderful NY Times article about this issue] Anecdotally, we’ve seen some organizations talk about losing 1/2, or even 2/3 of their student participants. This is a major issue that cannot be ignored by the local, state and federal governments. If we want true equity in education, districts need a plan now for how to reach out to these families and find and connect them to the resources they need to move forward.

Which students are more likely to drop out?

Younger students, particularly in PreK, K, and 1st grade
Students from low income families
Children with poor internet access
Children who do not have an adult in the home who can supervise learning
Students with special needs

I can speak to my own experience with younger students. Both of my children (5 and almost 2) were offered short zoom gather times (~30 minutes each) throughout the spring. By the end of my daughter’s meetings, her class of 12 was down to 2-3 participants, as was my son’s. Many parents and children clearly did not feel it was worth it to continue, despite the fact that the meetings were engaging and even involved small, fun crafts. We continued because:

it brought a sense of routine to our day
it was a nice break for me
I had the luxury of a flexible job
We had reliable internet
We had multiple computers/devices
We had resources

And even despite these things, it was sometimes a struggle to make it through one 30-minute zoom meeting daily. Going all virtual this Fall seems like a challenging prospect, to put it mildly.

[Another random note from Andrea: my 4-year old’s Montessori school also did 20 minute virtual meetings daily, and that experience made me thrilled that we can sit out this school year. Good luck, Lindsay, and other parents of kindergarteners.]

What can schools do?

Schools need to gather whatever information they have from the virtual and online learning that happened in the spring as they prepare to help their students in the Fall. Did some teachers try paper packets? Was that helpful or not so much? For all students or some students? What kind of lessons were most engaging? How are students feeling socially and emotionally? How can students connect socially while remaining six feet apart, or on Zoom? How are schools addressing these needs (which likely will be huge in the fall)? What resources do parents need to help engage in their child’s learning?

The best time to get that information would probably have been at the end of the 2019-20 school year, and no doubt many districts did that. But almost all districts were scrambling, and COVID cases started to dwindle at that time, leading to hope we’d be able to operate somewhat normally this fall.

Since that’s not the case, districts and organizations serving students should take time now to conduct parent, teacher, and student focus groups/interviews/surveys and distribute any information about potential pitfalls and/or best practices to their staff members right away. Districts need a plan to involve parents. A plan to support SEL. And most importantly, a plan to reach as many students as possible.

The one-pager’s sidekick, the sidebar!

by Lindsay Lamb

Something that I have often struggled with in producing a succinct and powerful one-pager is where to put technical information, a brief description of data used in the study, analyses conducted in the study, study design, survey questions, etc.

In a typical research paper, that information is generally presented first. Most of us come from an academic background, where we learned to write reports with Introduction, then Methods, then Results (which oftentimes are full of technicalities), then, finally the Conclusion – where the actual findings and interpretation lie.

The problem? It’s BOOOOORING. You’ll lose all but the most dedicated reader with reports that are designed that way, and in a one pager, it’s a killer. The conclusions need to be front and center.

But what about that technical info about how the study works?

Sidebar to the rescue

This is where the sidebar comes in. You still need to be succinct, but it is a way to break up the layout of your one-pager and go more deeply into a specific topic that is essential to your one-pager but does not need to be part of the overall one-pager narrative.

Here is the template we’ve shared with you in the past:

Still not sure what to use a side bar for? The sidebar can also include photos, data visualizations, description of program elements, survey questions, survey response rates, quotes from participants, anything that your audience should know, but can stand alone in its own section. Here are just a couple of examples:

Here are a couple of examples of sidebars we have used in recent reports.

Here is another sidebar example using data visualizations.

Here are some key takeaways to help you become the master of the sidebar:

Put information that is necessary to understand your report in the sidebar.
Add icons, photos, or a data visualization that will break up the layout of your report.
Keep it simple! Details belong in the report (or the appendix!)

Create a One Pager In One Hour

by Lindsay Lamb

Have you ever been in a situation where someone needs a one-pager – like yesterday? You’re just sitting at your desk, and all of a sudden, your boss comes by and says, “Hey Lindsay, the superintendent [program manager, director, business analyst] needs a one-pager on that research [analysis, presentation, etc.] you’ve been working on. Can you get it to me in two hours?”

Internally, if you’re anything like me, you’re probably thinking: “AHHHHHH!!! HOW AM I GOING TO DO THIS??? THIS IS CRAZY!!!”

Externally, you remained poised and self-confident : “Of course, I’m happy to help!”

So, what can you do to make this happen? Take a deep breath and follow these simple steps.

Use what you’ve got. Before you get started, do you have a longer report, outline, or presentation? If yes, then this process just got A LOT easier. If not, don’t worry, I’m sure much of that information exists in your head. Go ahead and sketch out a rough outline either using word or on paper – whatever you need to do to get those creative juices flowing.
Use One-Pager Building Blocks: Now that you have a rough idea of what you want to say, use these four building blocks to help you get started building your report:

Use these building blocks to design your one-pager. You can even use them to segment the page (or pages as a one-pager can be two-pages front and back).

If you have a larger report, presentation, or outline, what are the main headings and/or major findings? Do you already have a description of the study? Trim that down and drop it in at the beginning or in the sidebar. If not, quickly jot down the things from your report/presentation/analysis that stand out to you. Whittle them down to the top four ideas and rank them so you know which one to add depending on how much space you have in your document. Keep in mind that when you create your report, you want to include as much white space as possible to make it easier for your audience to read you report. Don’t muddle it with too much information.

Now that you have a better idea of what you want to say, it’s time to start thinking about the layout. As a general rule, a one-pager can be broken down into three or four major sections per page: The introduction, a major finding, a visualization and a photo. Sidebars are a great way to visually break up the page and add some of this information. Here is a generic template that can give you a rough idea of what your one-pager might look like.

Here are the building blocks of a one-pager using our generic template. Remember, your one-pager can be two pages (one-page front and back):

Here’s a snapshot of our generic template. There are three sections represented here comprised of finding 1, finding 2, and a sidebar.

Have you created data visualizations? Can you use those? Can you combine visualizations (small multiples, maybe)? If not, think of ways to create data visualizations quickly. Most people respond well to percentages and simple figures. If time permits, create a visualization using that type of information. Remember, as an evaluator, you most likely won’t be the one sharing the information from this report, so it is important to ensure the data visualization can stand on its own without a lot of explanation. Don’t have time to create a visualization? Use icons or photos to break up the text and add something visually appealing for your reader.
Summarize key takeaways. What do you want you audience to remember? Are there important strategies they need to consider? Program elements that should be removed? Funding sources that should be added or eliminated? Share them, and preferably using bullets 😉

Whew. That was quick, but I hope you have found it useful. If, like me, you find yourself in tis situation, remember to use the following strategies to help you create a one-pager in just a couple of hours:

Identify the major findings
Break up the page into three or four sections. Use a sidebar to include additional information relevant to your analysis such as a description of the study, data used, research questions, project timeline, etc.
Create effective and efficient data visualizations that correspond with your major findings. Don’t have time for visualizations? That’s okay, you can use icons or photos.
Highlight the key takeaways from your analysis. What do people need to know to make informed decisions?

That’s it!

Want to learn more? Our upcoming mini-course will take you deeper into creating an effective one-pager – stay tuned!

Entering the Evaluation Job Market During a Recession: Advice for Recent Graduates

by Lindsay Lamb

I have been thinking a lot about students graduating lately. Graduating high school, college, graduate school. My daughter is graduating from Pre-K and will start Kindergarten in the fall. I have friends and family with kids graduating high school, excited to celebrate their accomplishments and start off on their adventure. Now, thanks to the pandemic, those dreams have a much different reality. There is no prom, there is no graduation, there is no party to celebrate with friends and family.

As if reading my mind, I was asked to provide recent doctoral graduates with advice they might find helpful as they enter what is sure to be a tough labor market. What could I possibly say to them?

Fortunately (maybe?) I graduated during the 2008-2010 recession. When I first entered the job market, post-doctoral funding was frozen, full-time professor positions were being transitioned into more adjunct faculty positions, the private and public sectors were beginning to enact hiring freezes.

So, back to that advice.

The prompt I got was, Based on your experience in 2008-2010 on the job market:

What strategies or resources helped you get that job offer?

I made connections. I built relationships. I put myself out there and honed some skills. I essentially created my own internship with several educational non-profits in the area. Sure, this didn’t make me money right away, but it paid off big time in the end. This is the only reason why I got my job in AISD.

I was already on the fence about an academic role. With a degree in educational psychology, I wanted to have more of an effect on students, staff, and administrators I was researching. I wanted to make a difference in students’ lives. My experiences during this time reaffirmed my beliefs: I wanted out of academia.

I started contacting educational non-profits in the Austin area to see if they needed help. Surely, I could create a survey, conduct observations, do a quick focus group, anything to help them out. I had a lot of interest, but not any pay. On the plus side, I was still enrolled in graduate school, and was getting paid to be a teaching assistant and was teaching a class at a local university so I could make it work.

Because of this work, I finally got connected with the Director of Research and Evaluation at the Austin Independent School District. Luckily for me, they had a job opening. I spent 10 amazing years there conducting meaningful program evaluations and learning from my colleagues every day.

What was the most challenging aspect of finding a job during the recession?

All of the uncertainty and frustration I faced. Several of the jobs that I applied for suddenly lost funding or were put on a hiring freeze. These unknowns exist today, and unfortunately, there isn’t anything we can do about them, but we can keep applying for jobs, learning new skills, and making new connections. Anything you can do to keep you going one day at a time.

What advice would you share with your younger self and/or graduate students who are just now looking for jobs?

Don’t give up! I know this sounds hokey, but the only way things will not work out is if you stop looking for a job, stop honing your skills, and stop building connections. Find some free (or even not free, but cheap) mini-courses on data visualizations, creating one-pagers, learn R, start a blog! Also remember that you have learned amazing skills during graduate school that can be applied to many fields, not just academic jobs. Businesses are looking for skilled workers who can analyze data, create effective data visualizations, share results in an easy to understand way. If you take a job that is below your skill level, raise it up! Bring in your skills to show that you can do more and be more, maybe you can create an opportunity for yourself by showing all of your skills.

I know this is a challenging time, but we are all in this together. As cliché as it sounds, I truly believe things will work out one way or another. It might not be exactly what you had planned, but it is how we deal with setbacks and adversity that define use. These experiences force us to look inward and grow, and In the end, I truly believe that we will come out stronger than we were before.

Good luck!

Brand Your Documents in Three Easy Steps

by Lindsay Lamb

We have talked a lot recently about tools for your evaluator’s toolbox. Today I have another essential tool to for you: consistent branding. Clear and consistent branding can help you more quickly and efficiently create short and long reports. Doing so will ensure that when people look at a report, they will instantly know that it came from you. I have found that consistent branding takes some of the guess work out of report writing. I no longer have to think through color choices, layout, font, or headers. That leg work has already been completed. I’m guessing you have a little free time now, so you might as well spend some time thinking through your branding. Trust me, you will thank me later.

Enough of my pep talk.

I’m sure you are wondering how to create a clean, consistent look for your reports, one-pagers, and blogs. Follow these three simple steps:

Logo

Make sure you have a unique logo that lets people know what your business/firm/program is about.

You can use a logo generator to help you out! We used logogenie.net.

Colors

If you have a logo, use colors from your logo in your reports.

Remember the color wheel? Use it! Make sure you incorporate complimentary colors (colors on opposite side of the arrows), and colors in a shade that is easy to see on screen, in print, and for those with color blindness. A good test for any report is to print it out in black and white. If your colors are all grey, try using gradations within the same color to convey meaning.

A color wheel, complimentary colors are located diagonally across from each other.

Use colors consistently such that all headers are the same color, etc.

Caveat – if you are generating a report for a client with clear branding, you might want to use their branding and simply insert your logo. Check with program staff, your supervisor, a colleague, a friend, or even reach out to us if you have questions and want some feedback!

Here is our style guide, and a link to the logo generator we used.

Font/layout

Choose a font that is unique and easy to read

Stephanie Evergreen goes into great detail on the importance of choosing fonts in her book, Presenting Data Effectively.

You can select two fonts, or fonts that are bold and light to use to signal to the reader level of importance. There are a plethora of fonts out there, download something unique online.

The layout of your reports should also be consistent. Check out our recent blog post on one-pagers. The idea is that people should pick up a report and instantly know that it came from your organization/consulting firm.

One final tip – try out your branding! If you have a few ideas, try them out and see what your clients, friends, colleagues think. Send your ideas to us and we will look them over. Your brand is the first thing that people will see when experiencing a report you have created, so you want to make sure it is right.

The Secret Reason Leaders Aren’t Reading Your Evaluation Reports

by Andrea Hutson

Evaluators: be honest: have you ever worked hard on an evaluation, found some really exciting findings, wrote an amazing report, and turned it in expecting….

praise from program leadership!
noticeable, positive program changes!
maybe a promotion? (or extended contract?)

And instead, got… crickets?

Let me raise my hand the highest. It’s definitely happened to me, and one time in particular is burned into my mind.

I was in a final report meeting with a large agency. The evaluation we were reporting on was a multi-year, multi-site evaluation with mixed methods, requiring multiple staff, travel, a ton of time with SPSS and R, and a very long time writing and editing the report.

There were about 10 people in the that final meeting, including the relatively new program director (sidenote: one interesting thing about long evaluations – in some organizations, change is so constant that often the only people who are with the program from start to finish are the evaluators).

Our final report was over 100 pages, with appendices, and lots of technical detail.

After I presented all of the results from the study, the program director looked at me and smiled. “Wow! These are really interesting results!” I smiled back.

“But…”, she started… and I felt my heart sink.

Then she dropped a bomb.

“But, you know no one will ever read this.”

Oh my goodness, so painful to hear – but I knew she was right.

It’s not that the content was bad (of course it wasn’t! I wrote it!), it’s just that it was too dang long.

Decision makers don’t have time to read.

The average 30-page evaluation report takes about an hour to carefully read and process (and at over 100 pages, my report would have taken 3+ hours)

The problem is, leaders – especially the higher you get up the food chain – don’t have an hour.

There was a fantastic study of CEOs conducted in 2018 by the Harvard Business Review (https://hbr.org/2018/07/the-leaders-calendar) that explains why. (Although the report only looked at CEOs from the business world, non-profit and governmental leadership likely have very similar constraints.)

According to the study, CEOs spend the vast majority of their time (about 72%) in meetings, either with their own staff, leadership, or clients. They have very little time alone, and most of that time is in small chunks (i.e., not conducive to report reading).

Check out this time graph of one CEO’s alone time, arranged in 15 minute green blocks.

chart of ceo's time arranged in blocks — This CEO has very little time to himself. Source: https://hbr.org/2018/07/the-leaders-calendar

Even when we include Saturday and Sunday, this CEO only had 8 hours of one-hour or more alone time for the entire week, even though he was working a TON of hours (note that he starts working before 6 AM most days).

The study noted that the CEO spent the majority of that alone time (55%) reading and responding to emails. After we remove that time, he had only had about 3 or so hour-long chunks for all other unscheduled tasks. FOR THE WEEK.

And that’s the reason that long reports don’t get widely read.

There’s just no time!

What can you do?

Now beyond going past the existential crisis about why we write reports at all, there are a few solutions. Here are three ways you can maximize your impact.

Don’t write a report; make a presentation instead. This method is getting more popular over time; UX Evaluators typically present results in one final PowerPoint. If you noticed above, we added on a presentation, which was the product that got us traction with the program leadership. That’s also a possibility, if a report is absolutely required (although the product we had worked for hundreds of hours on became essentially a doorstop).
Write a brief report instead of a long one. We’ve had tremendous success with 4 or 5 page evaluation briefs. When carefully written, these briefs can include the vast majority of findings, details, and recommendations that people need, in a format that’s digestible and a relatively quick read.
Add on a powerful one-pager. One pagers delete all but the most essential content, giving just the highest level findings from a study. The best thing about one-pagers is that they can be added on to a brief report, or even a very long one. Stephanie Evergreen has a great post on the 1-3-25 model, (https://stephanieevergreen.com/the-1-3-25-reporting-model/) in which a one-pager is tacked on to an evaluation brief which is tacked on to a longer report (but notice: the ‘longer report’ is only 25 pages).

Want to learn more about option 3?

Lindsay and I are currently working on a course on one-pagers that’s designed for anyone who wants to make more of an impact with their reports. This course will provide easy steps to turn a long report into a one page document that engages readers and packs a punch. And you’ll do it without needing graphic design skills or any fancy software. We’ll be sharing some of it on the blog, but if you’d like to be notified when the course goes live. subscribe below.

Teaching your kids about data

1 Comment

by Lindsay Lamb

Recently, I read a great article in the New York Times, Turn Your Demanding Child Into a Productive Co-Worker. As mentioned in an earlier blog post, I have taken on a majority of childcare responsibilities for my two children during the pandemic. Needless to say, the first few sentences of Michaeleen Doucleff’s article resonated with me: “At 10 a.m. on a Tuesday morning, I lock myself in our bedroom, slide down the back of the door and begin to cry. “But Mama, but Mama,” my 4-year-old daughter screams from the hallway.”

I found a kindred spirit. I had to read on.

After venting about her current situation (which we all need to do, regardless of whether or not we have young children to look over right now), Doucleff went on to describe how in her work she has observed families from cooperative cultures sharing work with children. She suggested that we stop endlessly entertaining our children (which is exhausting), and instead show them our work and share in the process with them.

Easier said than done.

How could I share my work with my kids?

I could teach Hannah (who is 5) to sit calmly next to me and “work,” but I seriously doubt I could get my destructive Godzilla-esque 20-moth old to sit calmly while I write without him a) destroying my computer, b) destroying whatever Hannah was working on, or c) injuring himself.

After pondering a bit, I came up with an idea that would both use data and get the kids outside.

Me: “Hey Hannah, do you want to do a data gathering nature walk? At the end we can make a graph and show Daddy what we found!”

Hannah: “YES!!!”

Hayes: “Mama!!! Outside, outside!!!”

Hannah and I discussed what we would search for on our nature walk. I explained that we would count how many of each thing we saw during our walk and use that information, or data, to make a picture.

Me: “What things do you think we might see on our nature walk?”

Hannah: “Rabbits!”

Me: “That would be really cool if we saw a rabbit! What is something that you know we will see just in case we don’t see a rabbit?”

Hannah: “Birds, maybe a cardinal, and squirrels! Oh, and a blue jay.”

Me: “What would Hayes like to see?”

Hannah and I: “TRAINS!”

Me: “Yes, let’s add trains.”

Hannah: “And butterflies.”

I loaded up the kids in the double-wide stroller, and gathered crayons, a notebook, an envelope in which to carry the crayons and notebook, and snacks (you always need snacks). Ten hours later (well, maybe 20 minutes), we hit the trail for our fun-filled adventure. Since the weather was nice, I decided to run. Because I felt like a pack mule, I was more than happy to stop whenever Hannah excitedly screamed that she saw something from her list.

Like any good scientist conducting an observational field study, Hannah added some animals to her list based on what we saw during our run (and what she wanted me to run by). For example, we saw a parrot fly by, so we added parrot to the list. She also realized that she forgot to add cats, so we added cats. Then she wanted to go by the house with goats in our neighborhood, so we added goats to the list.

To keep a record of what she saw, she created a list of each animal (or train or butterfly) and added a check (also in a box) next to each animal. This way she could not only keep track of what she saw, but also how many of each animal (or train or butterfly) she saw during our walk. Here is her completed check list:

Hannah’s completed nature walk check list.

In case you can’t read her handwriting (though I did help her a bit), we searched for rabbits, cardinals, squirrels, butterflies, trains, blue jays, cats, parrots, and goats.

Hannah had so much fun seeing each item and adding a check box to her list. It turned our daily run on the same streets into a new adventure. We saw things we had never seen before and took the time to really explore our neighborhood.

When we got home, she couldn’t wait to make a graph. She ran upstairs and grabbed some construction paper, crayons, markers, and tape.

We sat down at the dining table and got to work. I got out some paper and showed her what a graph of the data we collected could look like. I explained that you can show data in lots of different ways. In the end, she wanted me to make two different bar graphs. She used one graph that I created and gave the other one to Hayes.

I said that a lot of times I use different colors to indicate that it represents a different category of something. With our data, a different color would represent a different animal (or train or butterfly). She started coloring the bars using a color that represented each animal we saw. We added numbers to the bars so you could easily see how many of each animal we saw. Then she decided to draw a picture of each animal next to each bar. This was music to my ears! We have a data viz pro on our hands!

Me: “What did we see the most of?”

Hayes: “Pink! Pink marker!”

Hannah: “Squirrels… and butterflies! I wish we would have seen a rabbit.”

Me: “I know, and maybe next time we will! Hannah, I love your graph. Did you know that graphs have a title, like the title of a book? What do you think the title of your graph should be?”

Hannah: “Today I saw a lot of things in nature, but I didn’t see any rabbits.”

In the end, she made a pretty great one-pager…

Hayes’ creation could use a little help, but he had fun creating it (and seeing TWO TRAINS!) and that is all that matters.

For me, the best part about our data nature walk was talking – albeit abstractly – about what I do in my job and sharing in that experience with my daughter. Maybe it wasn’t exactly what Michaeleen Doucleff had in mind when she wrote her article, but it got us through another day and we all had fun.

The 24-Hour Evaluation

When visiting with organizations about the importance of evaluating their programs, we generally see a lot of enthusiasm. Nonprofits, school districts, and others realize that evaluations:

Let organizations know what’s working and what’s not
Give stakeholders a way to provide valuable feedback
Can give a sense of the impact the program is making
Save time and money in the long run

Why then, aren’t organizations doing evaluations for EVERY program, EVERY time?

Two things are missing – time & money. Evaluations are often perceived as (and can be) massive behemoths that take hours and hours to conduct and cost quite a bit of money.

It doesn’t have to be this way.

We have recently been working on a method to produce a comprehensive, focused evaluation with a polished final report or professional presentation in just 24 hours.

Of course, you could spend the longest day ever on the evaluation and get (some types) done in one day. But you could also spread those hours out and get a high quality evaluation completed in a week, with 4-6 hours of work a day.

The 24 hour evaluation has 3 phases.

Phase One: Planning (4 h, Day 1). In this phase, you’ll brainstorm ideas for your evaluation, create a mini-theory of change, select your evaluation questions, and decide on your strategy for data collection.
Phase Two: Data Collection & Analysis (12 h, Days 2 & 3). In the next phase, you’ll collect data (whether new or existing), clean it up, and analyze it. This is the longest phase, for obvious reasons, and if you choose to collect new data, it will necessarily take longer than one 24-hour day to complete the process.
Phase Three: Reporting (8 h, Days 4 & 5). In the final phase, you’ll pick a report type (either an evaluation brief or a presentation), write your content, edit, and publish or present.

A few caveats:

The number of evaluation questions asked will be small and targeted. This may look like one deep question or several small questions.
If you are planning on writing a report, the report will be a brief report of 5 pages with an additional one-pager summary. There’s not time to write a 30 page report in 24 hours! An evaluation brief, when written carefully, can actually be more effective than a long report because people are more likely to actually pick it up and read it.
If you are planning to do interviews and focus groups, you will use a rapid analysis method – no time to transcribe tapes and upload themes into nVivo or some other analysis software. Instead, you’ll be pulling out the big and consistent messages that you’ve heard.

With this method, 90% of what your organization needs to do to make real, meaningful change could be done in one work week spread over 24 hours.

Ready to get started?

Fill out the form below to get the 24-hour evaluation schedule for FREE.

We absolutely 100% pinky swear never ever to send you spam.

Making Over the One Pager

By Lindsay Lamb and Andrea Hutson

We are going to be talking a lot in the next few weeks about the wonderful tool in your evaluator’s toolbox called the One Pager.

One Pagers have all sorts of uses, and can help you get funding, pitch an idea, or share high level results with board members or executives.

To be effective, they need to grab your readers’ attention and give them just the right amount of information to make them want to know more.

This is more difficult than it sounds.

TMI

One of the biggest traps you can fall into with a one pager is putting too much information on the page.

Check out this example. Sure it is a one-pager, but can you easily see and read what is going on?

A one-pager on Covid-19 created by a Seattle internist.

To be fair, I’m sure the doctor who created this was in a fit of anxiety, and desperately wanted to get all of this information out to others to help save the world… but it is safe to say that this is a bit much.

How about this one? It’s actually pretty good. We’re definitely going in the right direction.

A one-pager designed to promote a business and win over investors.

Things I like about this one-pager:

Three columns that break up the text,
Consistent color, headers, and branding, and
Cool infographics

Things that could be improved:

It is too busy
There is redundant and unnecessary information
The story gets lost; you really have to jump around the narrative to determine the key takeaways

Makeover Time!

This one-pager contains the same information as the one above, but:

There’s an image that gives you an idea of what the product is right in the center.
There is more white space, drawing the reader in so that they can more easily make sense of the data in front of them.
It states the key facts from the beginning and moves some of the background information to a sidebar.
It tells a story. Note that the flow goes from Problem to Solution.
It keeps the branding of the organization but adds a bit of color.

It’s still not perfect. This is probably something we’d submit to a client as a draft, noodle over for awhile, and then revise.

Tips for Effective One Pagers

How do you create effective one-pagers? Follow these simple guidelines:

Keep it simple! Ensuring you have a lot of white space will highlight what you actually want your audience to read.
Break up your one-pager into sections. I try to use three sections per page (a one-pager can be two pages with a front and back). Use a sidebar, pull quotes, and icons to help break things up.
Use visualizations. Create simple figures or charts, use icons, use bold font and color to highlight key findings.

Ready to build your own on one-pager? We will be launching a mini-course that will walk you through the steps to create a simple, and effective one-pager soon. Stay tuned!