Recently, I was working with one of our clients in a board meeting and we discussed how to share results of our client’s work over the past year. I respectfully listened and eventually piped up, “How about a one-pager for donors?”
You all know Andrea have touted the importance of creating a one-pager... there are many ways one-pagers can work for you. They can help promote your findings, highlight opportunities for growth, showcase participants’ experiences with your program, and so much more.
In a separate meeting, I discussed that a one-pager for donors would not just be any one pager, it would be a one pager that would promote our client’s work and showcase the opportunities for growth. It had to be short, it had to be catchy, it had to show what our clients accomplished and also where they needed additional support without coming off as needy.
The way we tell the story to donors matters, and this was something I had not done before. I asked in my evaluation community if anyone had worked on a one-pager for donors before. I was surprised that most of my colleagues had not worked on something like this before. I decided to search some online resources. I found a few examples that some of my former colleagues created as well as from some organizations we work with, and the materials were too long, or not specific enough. I was hitting a dead end… there had to be some resources out there, right?
Finally, I found a great resource from spitfire strategies that made perfect sense.
The way to design an effective one-pager can focus on either on persuading the audience to donate or informing the audience about the great work the organization has engaged in over the past year. From there, the outline is pretty similar:
Some approaches on how to organize your one-pager with donors in mind based on recommendations from https://spitfirestrategies.com/
I shared this outline with our client and they were super exited. Their one-pager will actually be a combination of the two approaches with the idea to create a longer report for a few key donors and share the shorter version with all donors that highlights their program’s major successes over the past year as well as target opportunities for growth. Hopefully it will be a win win for all!
We all know the story. One year ago, life as we knew it changed forever. I will never forget it. Andrea and I were in the office on Thursday and were getting ready to leave for the day. We checked in about what projects we had coming up, what meetings we had on Friday to see which ones we both needed to attend, and where we could divide and conquer. We left and said, “See you tomorrow!”
That night, Andrea sent me an email saying there was a case of COVID-19 in Austin.
The next day, everything shut down. I sat outside and listened to the birds while holding my then 18-month-old in my arms and crying.
The next week was Spring Break and Andrea and I decided to work anyway. We had our first conference calls and Zoom meetings. Andrea and I checked in with each other almost daily to see how the other was doing mentally, physically, and emotionally. Some days were hard, and some days were harder. My husband and I set up a schedule to juggle work and childcare. I took the early work shift, getting up at 5 AM so I could work for a few hours before the kids got up. Then my husband would work while I took the kids outside and did some home-schooling activities with them.
In the midst of all of this, Andrea suggested I start writing a blog for our website. Um… what??? What on earth would I talk about???
Andrea is very business savvy, so I trusted her. She figured if we added some trainings and stories on our website someone might read them and seek out our help. At first, we both wrote mostly about what was going on with COVID-19, both in the world and in our homes. Eventually, we branched out and I started writing more and more. Potential clients reached out to us because of our blog posts. My dad, who started his career as a journalist, started a blog of his own!
As someone who takes the time to build relationships with others and collaborate in my work, this year has been hard… but the blog helped. Sure, I couldn’t physically reach out and talk to potential clients, share my experiences with peers or connect with other researchers in the field, but it helped me feel like I had a purpose in a very challenging time.
One year later, I am still writing this blog. I volunteered at a mass vaccination site and helped some of our most vulnerable population get their vaccine. I took my kids to visit my fully vaccinated parents who we had not seen since Christmas 2019. I am feeling hopeful.
Andrea and I weathered the storm (literally and figuratively!) and I honestly feel like our business is stronger than ever. Who knows what one year from now will look like, but I know after going through everything we have gone through this year, I can handle it… and I’ll probably tell you all about it too 🙂
Recently, Andrea wrote a post encouraging folks to break away from Excel and recode variables in R. I used this post (along with a lot of tutoring and hand-holding from Andrea) to begin to learn R. At the beginning of the year, I wrote about finally achieving my goal of learning R. To be sure, I still have a LONG way to go, but I am comfortable enough to stumble along, look up code, and find and fix problems on my own without bugging Andrea all the time. As one of my former colleagues told me, “You can’t break R, R is already broken. Our job is to fix it.”
In my previous role as an evaluation coordinator for the Austin Independent School District, we often talked about how much of what we did was figure out problems with data. Why did a student have multiple records? What did a code of F mean? Why did some students have missing data? How should we recode the data in order to get it into the right format for our analyses? We reveled in figuring out these logic puzzles.
The work Andrea and I engage in with our clients at Agile Analytics is pretty much the same… except it is a little more complicated. We are often pulling together disparate files from various data sources and often without a data dictionary. We have to do a lot of sleuthing to figure out if we received the data necessary for our analyses, what our data represent, where the data came from, and which students were included (or not included).
One thing I always do to help me answer these questions is to compute the mean, range of values, and standard deviation of values. When I trained new hires and graduate students in data analysis, I always emphasized the importance of computing measures of central tendency before running any analyses. It is even more important to conduct this first step in data analysis when you do not know the range of your data, how variables were coded, if values are character or numeric, if there is systematic missing data, how missing data are coded (or not), and if any variables are out of range.
Recently, I needed to review the data from one of our client’s previous evaluations that was not conducted by Agile Analytics. The program manager did not know much about the data, but knew it was messy and couldn’t tell me anything more about it. The data were coded in two ways that I typically don’t use: scores ranged from -1 to +1 or -2 to +2 on a 5 point Likert-scale. Our purpose in examining the data was to compute the mean and standard deviation for each variable and compute a factor analysis to help our client think through which items to trim on their survey. Because of this, I thought it was appropriate to recode the data to a standard 1 to 5 scale. This would also make it easier for our clients to understand the measures of central tendency I presented to them as we talked through which items to keep on their survey.
As I was recoding the data, I discovered that several variables included errors in their coding. These errors were super easy for me to identify because in my process of inspecting the data using R. First, I looked at the minimum and maximum of each variable in a table. I could easily see the codes assigned to each variable, and used this to help recode to a 1 to 5 scale. As a reminder, variables were either coded on a -2 to +2 scale or a -1 to +1. As you can see in my code below, I added a .5 even though this variable ranged from -2 to +2. That is because I noticed an error after I computed the table (see below):
Snapshot of code I used to examine data and recode data. I first made a table of the variable to examine the score range, then recoded the data, then made a table again.
When I computed the ables, I noticed some values were out of range (e.g., there was a score of 0.5) that had to be recoded. I confirmed my recoding when I computed the second table and could easily see what the original values were and how they corresponded with the new values. Note: some of the data names have been redacted to keep our client anonymous.
Had I been recoding in Excel, I would have completely missed these errors. Sure, I would have found them eventually when I brought the data back in to SPSS, SAS or whatever stats package I used, but then I would have to go back to Excel and recode the data… with no record of what I had done. In R, you save your script, so it is easy to show your clients where the errors were and how you fixed them. In Excel, you have to save different iterations of your recoding because there is no script to save. You can also save projects and some code in SAS and SPSS, but R has everything all in one place, or printout. You can see the tables and code you used to recode the data in a single glance rather than having to walk someone through your code first and the output second. Someone else can also open your project (if data are in a shared location) and run it or tweak it to see if you get the same result. Notes can easily be added to the script to ensure everyone is on the same page.
Like I said, I am still an R neophyte, but I’m getting better at fixing it and sharing what I have learned along the way with the rest of you all 😉
We have all been there: working long hours on tight deadlines on a grant. Establishing measures, outputs and outcomes that are informative, meaningful and tied to a logic model. And inevitably, you come to one that should be easy, but often is not: “number of students served by a program.”
Recently, Andrea and I were working on a grant proposal with one of our clients and we were reminded of just how tricky this metric is – particularly during the time of COVID. Oftentimes, programs count a student if they attended one of their sessions. Some count attendance using a threshold for students meet in order to count (say 50%), while others count a student as being served by a program if a student attended a specific session (e.g., FAFSA training), and others count a student if they participated in a set number of programming hours integral to the program.
In the grand scheme of things, these all “count” (pun intended) – but are they measuring the same thing? How were these thresholds set? What does it mean if a student attends one 3-hour session and another student participating in the same program attends a 1-hour session? Where is the tipping point? It might be that attending 3 one-hour sessions spread out over a few weeks is more meaningful to students compared to one 3-hour session. It might be that some students lose focus in a 3-hour session and cannot retain all of the information they learned whereas other students might forget what they learned in-between sessions if they are spread out.
How do you know where that sweet spot is, and in terms of grant compliance does it really matter? Let’s leave the existential dilemma out of this, and all agree that these counts matter.
Now that we have that out of the way, how do you count students served by your program?
If you have any data to draw on – attendance records from each session, session duration, session type – you are way ahead! First, figure out the total number of classes/meetings/sessions that are offered each semester or year. Then determine the length of time students spend in each session. What does 100% participation look like? Now, walk back from 100% participation to set realistic targets. For example, use these data to determine the average number of sessions your students attend, the average number of hours of programming students engage in, the most common type of programming students participate in, and more.
Once you have averages you can use this information to set a threshold for participation and include expected gains in students served by your program that is more robust than simply counting bodies. Perhaps your threshold is one standard deviation above the average number of hours of programming or number of sessions, or a percent increase in number of hours of participation.
Some of you may have heard of this referred to as dosage. Having this information is extremely powerful when you tie it to your program outcomes.
For example, if you know the average number of hours in which students engage in your programming, and the type of programming they engage in, you can say things like, “Students who engaged in 8 hours of programming related to growth mindset experienced greater improvements in their academics than did their peers who engaged in fewer hours of programming.” Below is a graph from an analysis Andrea created a while back using this type of data:
Students who participated in more hours of programming experienced a greater percent change in behavioral improvement than did their peers who participated in fewer hours of programming.
This information will also help you identify the types of programming most related to positive outcomes. For example, if your program focuses on helping students achieve post-secondary success, maybe students who attended a session helping them complete the FAFSA is most directly tied to enrolling in college. Or perhaps it is attending a session with a guest speaker who attended community college that was immensely helpful.
Having the right data can help you – and your program staff – gain more insight into your program and focus your efforts on what matters most to students.
How should you track this information? Some organizations use Google Sheets, some use various survey platforms, and some use Excel. The point is to get the data and start building your database. From there, you can build dashboards in Excel – yes Excel! – and start tracking this information and make more informed decisions when it comes to including this type of information in grant proposals and grant reporting.
This week brought us some very cold, very wintery weather here in Austin. We all spent a lot of time refreshing our various weather apps, checking the news, and listening to meteorologists’ weather reports on the radio. The reports shifted by the day, and sometimes even by the hour. I get it, this was an unprecedented weather storm which hasn’t been experienced in Austin in nearly 30 years. Many of us went without power, heat, water and food. Other than completely redoing the state of Texas’ power grid, offering some more detailed insights into the weather, power and water issues may have helped.
Let’s start with the weather. One thing that would have made knowing when the cold weather was coming, and what type of cold weather was likely to hit our area easier were error bars. Okay, okay, I know this wouldn’t work in a radio spot, but I think it could work in data visualizations on the news, and on apps or in a newspaper. Now, I’m not trying to upstage meteorologists (they have a hard job!) but perhaps some of their visualizations could use a little freshening up.
Here are some examples of what we typically see:
Source. The Weather Channel app on my iPhone
This is pretty a pretty typical experience that people have with navigating the weather. It is definitely informative, and you can click on the day and look at the forecast by hour, but there are still several unknowns. For example, it is always difficult interpreting (let alone explaining to an excited 2.5 and 6 year old) that a 35% chance of snow actually means there is a 35% chance that there will be snow in the greater Austin area, not that there is a 35% chance of snow at our house.
Here are some additional examples from the Weather Channel website and the Weather Underground website:
Source. The Weather Channel and The Weather Underground websites
The Weather Underground’s website contains much richer information such as time of day when the highs and lows will peak (which is helpful in Texas when sometimes the daily high occurs first thing in the morning), and what the temperature actually feels like versus what the reported temperature is.
Now, wouldn’t it be nice to also see something that includes a range of values, and the likelihood of wintry conditions happening? Here are two simple ideas. One is a traditional box and whisker’s plot, the other is a simple line graph. Obviously, having lots of these wouldn’t be helpful, but for one day it would be super helpful to see that the low temperature could drop below freezing, so you should probably take steps to protect your pipes.
Note. These are some simple ways to show what the projected temperature is (gray circle, or midline), and what the high or low ends might be based on the standard deviation of potential values.
After I created these charts, I watched the news. I was happy to see that one weather reporter showed an image with sliding bars indicating the likelihood of freezing rain, snow or sleet. This was incredibly helpful! Sure, we were going to get some snow, but the thing we were most likely to get was freezing rain. Had he not showed these sliders, we just would have heard 90% chance of a wintry mix. Not helpful.
So, speaking of freezing rain, that is what caused power outages in my neighborhood. Several trees were downed by the heavy ice causing additional power outages. Now there is no way a data visualization could have helped us on the front end here. The only thing that could have helped would have been the city trimming trees near power lines and Austin Energy winterizing equipment. Where data visualizations did come in and help is on the back end. Here is The New York Time’s take on explaining what happened:
This visualization shows us when the winter storm hit, so we can clearly see what normal power generation was, and how all sources of energy plummeted due to the storm. This made a bad situation worse as this storm affected nearly every Texan.
Another great data visualization came from Austin water:
These data visualizations are helpful because they show how much water people normally use, and how the reservoir was nearly depleted. The other image is a heat map by zip code indicating where there were water issues as of 2/19/21.
The good news is that as of this writing (which took a big pause in the middle due to power outages and spending all of my energy keeping my family safe), most people have power, water and food. My daughter’s school survived the storm, and we will get back to pandemic life soon. I am thankful to all of our neighbors who came together to help out during this challenging time – in a pandemic no less! We came together to make sure people had warmth, water and food. In a time where we have been learning about resiliency, wants vs. needs, I am constantly amazed by how resilient my children – and all children – have been this year.
About a year ago, Andrea and I met to plan out the best year ever! We went through our goals as a business, our goals working with clients, our professional goals and our personal goals. It was a really fun process to talk about where we were, both personally and professionally, and where we wanted to go both in our business and in our personal lives. We were ambitious, but we had each other for support and accountability.
And then the pandemic happened.
Although it wasn’t one of our goals, Andrea and I adapted. We spent nearly a year working apart. Checking in over zoom, a quick phone call, and sometimes when the stars aligned even a walk. We used Trello to keep all of our projects and activities on track, we were making it work (all while juggling watching kids, zoom kindergarten, and our spouse’s work schedule). Had someone come along during our year-in-review planning meeting and told us about the pandemic and how we would have to shift our work and personal lives, I probably would have said, “You are crazy. There is no way I can do all of that.” And yet, here we are, nearly one year later.
Andrea and I embraced our company name – Agile Analytics – and truly became agile. We met with clients virtually, used innovative tools like Miro to make planning meetings more intimate and engaging, sought out new clients and opportunities that didn’t seem possible a year ago, we thought of new resources our clients and colleagues in the field might benefit from having and started creating those. I started writing a blog!
While all of these activities are great, they were not my personal or professional goals for 2020.
My big goal? Learning R.
Learning R has been a personal and professional goal for a while now and I always end up getting out of it. I spent about 10 years using SAS and nearly 20 years using SPSS. In many ways, learning R was more like learning a new language. I didn’t need to learn R since I could use SAS, SPSS, and Excel for many things, but I wanted to learn R. I decided 2020 would be different. Andrea, an R guru, was teaching an R course to one of our clients and asked me to join. I jumped at the opportunity. We had our first class the week before everything shut down las Spring. I figured that was it for me and R… but we persisted and continued having classes throughout the Spring and Summer, virtually of course.
During this time, Andrea sent me bite-sized projects and I challenged myself to recreate projects Andrea worked on to make sure I understood what she did and to see if I could make them work on my end.
Without even recognizing it happening, I spent the past week uploading, cleaning, and merging data for one of our clients. I ran simple statistics and created tables, saved out subsets of data for Andrea to review. I was doing it! Sure, I have a LONG way to go, but I am no longer scared of using R. I can read the code and understand what is going on behind the scenes. I actually enjoy writing code and using online resources to figure out how to solve problems.
So here I am about a year later, and I feel like I am a totally different person (in more ways than one) than I was a year ago. I have adapted in ways I never thought possible, and I still found opportunities to push myself personally and professionally. Learning R was just one little piece of that, but it was something from my pre-pandemic life that I wanted to accomplish, and honestly helped make me feel a little normal. It gave me something outside of all this madness on which to focus. I made it past that first psychological hurdle in learning R – in the midst of a pandemic, no less – and I feel sure that you can learn R too!
This past summer, we had the opportunity to collaborate with Ignite MindShift as they partnered with San Antonio Works to provide high school students with an amazing virtual internship experience. We saw this as an opportunity to learn from these students, mentors, and non-profits who embarked on a new learning experience in the time of Covid. We are pleased to share with you the culmination of our learning in our Best Practices report.
We urge educators and non-profits to review the work and share with us your best practices as we continue to navigate online learning in the semesters – and years – to come. Here is a link to Ignite MindShift and their post, which includes a .pdf of our report:
As a reminder, we shared several posts related to this report (see below) and are excited to share the full report with you now. Our relevant posts are linked below:
As we near the end of what was a year like no other, I wanted to reflect on what I have learned and shared with you all while writing this blog.
Front and center for me are all of the innovative, telling, and nuanced data visualizations that have come from modeling and documenting Covid-19 cases. Initially, my go-to source was the John Hopkins Coivd-19 tracker. This interactive website was critical when the virus was beginning to spread worldwide and across the US. It painted a realistic picture and helped us mentally prepare for what was to come. We could drill down to countries, states, and counties with alarming precision. As the pandemic progressed, I found myself turning more and more to the visualizations presented in the New York Times. To this day, a visualization is included front and center and it can be filtered based on all sorts of criteria. Additionally, there are usually several other visualizations showing trends based on different populations, trends intersecting with state policies, and more. I have really enjoyed digging into these visualizations (okay, I guess enjoyed is the wrong word as I am constantly looking up Travis County (where I live), Dona Ana County where my folks live, and Orange County in California where my brother and his family live). In a weird way, I find comfort in looking at the trends in the data. It helps me mentally prepare for what is to come and to take comfort in knowing that we are all in this together. I also think they are really thought-provoking and have made folks who are not usually comfortable with data become more so.
I have also enjoyed the infographics the CDC have created. Some are pretty alarming, such as this one:
Source: CDC.gov Note: I would argue that all of dots should be in some variation of red as no one should swallow hand sanitizer, no matter who says it is a good idea, but other than that it is an effective infographic.
To be fair, I have not spent a lot of time reviewing the CDC’s infographics prior to the pandemic, so maybe they have always produced high quality work. Regardless, I have enjoyed their infographics and have gotten a lot of great ideas from them.
On a more personal level, one lesson I learned this year, okay re-learned, is to let your data tell the story. It is okay to have complicated data, let it tell its own story. I wrote a post about this issue earlier in the year when Andrea and I took data from a line graph and changed it into a lollipop graph. I also wrote about it in reworking some Covid data displays, which can get confusing. I think we all remember my extreme Covid makeover post 🙂 As a rule of thumb, if data cannot be explained on their own, consider breaking it up, making an infographic, or creating a one-pager.
Finally, as I recently discussed, simplify your presentations/reports/data visualizations and then simplify again. PowerPoint is a great tool to use in creating reports, and obviously presentations, and sometimes can get too busy with all the bells and whistles. As I wrote back in March, simpler is better.
This year was a challenge, to say the least. I spent a lot of time looking inward and forcing myself to work even when the world around me felt like it was falling apart. Having a blog was a good outlet for me and allowed me to share my experiences, some personal and some less so, in a way that also taught me important lessons. My hope is that at least one of you out there learned something or enjoyed my musings this year. I honestly believe that you are never too old/young/experienced/unexperienced/exhausted/stressed to learn something new.
So, my friends, go forth and visualize your data and remember my biggest lesson of this year: have grace and patience.
During the times of Covid-19, we have seen a boom for data visualizations. In my opinion, this is great for everyone (one of the few benefits of this awful pandemic)! Sure, there have been some misses (well, maybe some were more than misses) but people who never talked about or heard the term data visualization are now talking about data. Data visualizations are making it into mainstream media – TV news channels, newspapers, school district dashboards… everywhere.
We have written blogs about Covid-related data visualizations in the past, and as the end of the year is approaching — and with it, hopes of a vaccine — I thought it would be a nice time to share how folks are depicting vaccine timelines.
I came across this one on the Orange County Register that depicts how long it took other vaccines to get approved and stop the spread of a particular virus.
When I first saw it, I thought the syringe was a cannon. (Perhaps that says something about my current mental state.) I like the idea of using an icon (you know how I love icons!), but perhaps the shot icon is just too big. I do like the overall concept of the visualization — how you can see the timeline of when the disease emerged and when the vaccine was created. Simplifying could make this graphic so much more clear.
Here is a simpler version from A Learning a Day that averages the number of years it has taken to produce a vaccine and compares it to the rapid timeline for the Covid-19 vaccine. You can easily see that most vaccines take about 5 years to develop with several years devoted to the trial process. I like this graphic much better, but I also think it would be good to also include number of years across the top so you could see that the Covid-19 vaccine will be distributed in about a year and a half compared to 5 years. That information can be inferred, but it takes some previous knowledge.
The grey-ed out blocks indicate the normal timeline for producing a vaccine, which is also helpful in showcasing how truncated the timeline is for the Covid-19 vaccine. I also like this visualization because it shows the projected year when each component of the vaccine timeline is typically completed, which really hammers home how quickly and unprecedented the Covid-19 vaccination timeline has been.
As you can see, there are several ways to depict timeline data. When making a timeline visualization, the important rule is to choose what works best for the data. Let the data tell their story and build a visualization around that story. You might start with an idea in mind, but after messing around with your visualization realize you need to change plans. That is totally ok! Unsure what to do? Ask a colleague, friend, or family member (or all three)! Ideally, all of these folks should be able to read your visualization and understand it without having you explain it to them.
I hope you have found this era of data visualizations as inspiring as I have. Now get out there and share some data stories!
More and more of our clients are being asked to do power analyses for grant applications. This is good in a sense, but it’s also a barrier to those out there who are not statistics nerds like me. Don’t worry though – it’s not hard, and you can even DIY!
What is power? Power represents the percent chance that you will find a difference between groups if it is really there. If you have low power (anything below 80%), there’s a chance that you’ll go to all the trouble of conducting a study only to find…nothing. And, even worse, you won’t know if it was because there truly was no effect of the program you’re studying, or if it was just due to the study being ‘underpowered.’
How many participants do I need to have appropriate power?
Luckily, there’s an easy way to calculate this for a simple study that does not require MPLUS or STATA or any software. In fact, it’s available for free online (and it has the exact same results as the above programs).
What to enter? When you open up the calculator, the bottom section will look like this.
Assuming that you are trying to see if two groups are different (the program group vs. the control group), the only thing you will need to change is the effect size. The effect size statistic describes the magnitude of the difference between groups. Luckily there is standard guidance for this: a small effect is a size of about 0.2, a medium effect is about 0.5, and a large effect is about 0.8. In education research, I would say that these numbers change slightly: small is still 0.2, medium is 0.4. and large is 0.6. You might be able to find the effect size from other studies in your field. You can also calculate it from your own data if you have the means from each group and the standard deviation.
So let’s say you want to detect a medium effect in educational research. Simply change the “Effect Size” field to 0.4 and click “Submit.” At the top of the calculator, the result appears:
That is – we need 100 students in the experimental group and 100 in the control group to have a reliable study where we can detect a medium effect of 0.4 or above.
Here’s how you might write about it on a grant application:
Our sample size calculation is based on examining the effect sizes of similar studies in the extant literature. [Insert citation here if you can.]Â With an expected 200 participants (N = 100 in each group), we will be able to detect an effect size of 0.40 with 80% power.
What effect size can I detect?
If, on the other hand, you already know how many participants you have, you can play around with the calculator to find out the effect size you can detect. Let’s say we know we have 254 participants in the experimental group and 236 in the control group. Just to be safe, let’s go with the lower number – 236 participants.
Here’s how to write about it:
Given our sample size of 490 participants (N = 236 control; N=254 experiemental), we will be able to detect an effect size of 0.26 or greater with 80% power.
See? Not so bad!
What about groups?
if your data are GROUPED – let’s say that you have 450 participants participating from 10 different schools, the calculation gets a little more complicated. You actually need slightly MORE participants to account for the fact that students from similar schools are going to share many characteristics — teachers, school environment, neighborhood – that make their data more similar to each other than students from nearby schools. Because these data are correlated, our power to detect differences goes down slightly.
You’ll need to, therefore, generate an effective sample size that is affected by these things:
the number of participants per group
the number of clusters/groups
the Intraclass Correlation Coefficient, or ICC. This is probably the most challenging to obtain, but might be available from the literature. See this fantastic article for more on ICC (Killip, et. al, 2004): https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1466680/
Once you have all three pieces of data, simply use this formula:
numerator = (number of participants per group) * (number of clusters/groups)
denominator = 1 + (ICC * (number of participants per group – 1))
effective sample size = numerator / denominator
Example
Let’s say we have 10 schools with exactly 45 participants per school. Let’s say we know our ICC is 0.015.
numerator = (number of participants per group) * (number of clusters/groups) = 45 * 10 = 450
denominator = (number of participants per group) * (number of clusters/groups) = 1 + (0.015 * (45-1)) = 1 + (0.015 * 44) = 1.66
numerator / denominator = 450/1.66 = 271. That’s right, even though we have 450 total participants, because the data are clustered, we effectively only have 271 participants.
Using the calculator, we find that instead of being able to detect an effect of 0.19, we really can only detect an effect of about 0.25. This isn’t a huge difference, practically, but it could sink your grant application.
