By Lindsay Lamb
Recently, Andrea wrote a post encouraging folks to break away from Excel and recode variables in R. I used this post (along with a lot of tutoring and hand-holding from Andrea) to begin to learn R. At the beginning of the year, I wrote about finally achieving my goal of learning R. To be sure, I still have a LONG way to go, but I am comfortable enough to stumble along, look up code, and find and fix problems on my own without bugging Andrea all the time. As one of my former colleagues told me, “You can’t break R, R is already broken. Our job is to fix it.”
In my previous role as an evaluation coordinator for the Austin Independent School District, we often talked about how much of what we did was figure out problems with data. Why did a student have multiple records? What did a code of F mean? Why did some students have missing data? How should we recode the data in order to get it into the right format for our analyses? We reveled in figuring out these logic puzzles.
The work Andrea and I engage in with our clients at Agile Analytics is pretty much the same… except it is a little more complicated. We are often pulling together disparate files from various data sources and often without a data dictionary. We have to do a lot of sleuthing to figure out if we received the data necessary for our analyses, what our data represent, where the data came from, and which students were included (or not included).
One thing I always do to help me answer these questions is to compute the mean, range of values, and standard deviation of values. When I trained new hires and graduate students in data analysis, I always emphasized the importance of computing measures of central tendency before running any analyses. It is even more important to conduct this first step in data analysis when you do not know the range of your data, how variables were coded, if values are character or numeric, if there is systematic missing data, how missing data are coded (or not), and if any variables are out of range.
Recently, I needed to review the data from one of our client’s previous evaluations that was not conducted by Agile Analytics. The program manager did not know much about the data, but knew it was messy and couldn’t tell me anything more about it. The data were coded in two ways that I typically don’t use: scores ranged from -1 to +1 or -2 to +2 on a 5 point Likert-scale. Our purpose in examining the data was to compute the mean and standard deviation for each variable and compute a factor analysis to help our client think through which items to trim on their survey. Because of this, I thought it was appropriate to recode the data to a standard 1 to 5 scale. This would also make it easier for our clients to understand the measures of central tendency I presented to them as we talked through which items to keep on their survey.
As I was recoding the data, I discovered that several variables included errors in their coding. These errors were super easy for me to identify because in my process of inspecting the data using R. First, I looked at the minimum and maximum of each variable in a table. I could easily see the codes assigned to each variable, and used this to help recode to a 1 to 5 scale. As a reminder, variables were either coded on a -2 to +2 scale or a -1 to +1. As you can see in my code below, I added a .5 even though this variable ranged from -2 to +2. That is because I noticed an error after I computed the table (see below):
Note: some of the data names have been redacted to keep our client anonymous.
Had I been recoding in Excel, I would have completely missed these errors. Sure, I would have found them eventually when I brought the data back in to SPSS, SAS or whatever stats package I used, but then I would have to go back to Excel and recode the data… with no record of what I had done. In R, you save your script, so it is easy to show your clients where the errors were and how you fixed them. In Excel, you have to save different iterations of your recoding because there is no script to save. You can also save projects and some code in SAS and SPSS, but R has everything all in one place, or printout. You can see the tables and code you used to recode the data in a single glance rather than having to walk someone through your code first and the output second. Someone else can also open your project (if data are in a shared location) and run it or tweak it to see if you get the same result. Notes can easily be added to the script to ensure everyone is on the same page.
Like I said, I am still an R neophyte, but I’m getting better at fixing it and sharing what I have learned along the way with the rest of you all 😉