Friday, January 11, 2013

Known unknowns

When I was working on my dissertation, one question I tried to find an answer to was, "What proportion of board members at private colleges donate in year [x]?" This seems like a straightforward question - it's inherently quantitative, after all. Moreover, it's something frequently discussed in the nonprofit and education sectors. The phrase "give, get, or get off" is used to describe what board members ought to do when it comes to making donations. So, you might imagine someone has researched it - but you, and I, would be wrong.

The question breaks down into two parts. One, how many board members did you have on the board of Your College last year? Two, how many of them donated during that fiscal year?

IPEDS, the ultimate data source for researchers in this field, doesn't have the answer to either one. (What makes IPEDS so great? It's a survey that is conducted every year, so it's up to date, and every institution that gets federal funding must participate. That includes indirect funding, such as federal grants/loans to students, so it means nearly every institution in the country.) Various surveys do ask how many board members a college has, but none of them are conducted every year, nor are any of them comprehensive. Many college websites provide enough trustee data to at least find out how many there are, but not all.

The second question, about how many gave, isn't asked at all. I won't list every data source that can't help us, but I will mention one: The Voluntary Support of Education Survey, one of my personal go-to sources, doesn't tell us, either. It asks for the number of board gifts made in a year, which isn't the same thing: It includes gifts from emeritus and honorary members, who are non-voting. Besides, if a single donor gave twice during the year, that would show up as two gifts. We want to count donors, not donations.

But say you don't care about board giving - I use this example as an illustration. There are lots of data problems like this in education research. No one with serious funding has decided they are important enough to throw major resources at answering them, but they are important enough to practitioners that anecdotal evidence abounds in white papers and consulting firms.

Sunday, January 6, 2013

Mapping U.S. colleges and universities

I teach a course at CUNY Baruch on the structure of U.S. higher education, and a large part of what I try to do in that class is to encourage students to be able to segment higher education in various ways: How are research universities different from small liberal arts colleges? How are public institutions different from private, not-for-profit institutions?

One of the tools I use to help them do this is a Google Fusion map:

In the screen shot above, all institutions in the continental states and D.C. are shown (with the exception of Hillsdale College). Red dots are two-year institutions; blue dots are four-year institutions. Even in this view we can notice at least one pattern: Heavily populated areas have more institutions than more sparsely populated areas.

By clicking on "filter," users can view a subset of institutions, such as HBCUs, colleges with enrollment of 1,000-1,500 full-time students, or private colleges in California.

The map was created in Google Fusion using IPEDS data. I chose Google Fusion because I have no experience with GIS and it is free. However, it has some serious drawbacks that make it much less user-friendly than I would like; ultimately, this means my students don't take full advantage of it.

Fusion is often non-intuitive (for example, figuring out how to upload a CSV file and convince Google to use it as a Fusion table rather than placing it in your Drive) and has surprising limitations (a column can't be deleted if the table is the result of merged tables). That is less problematic than the difficulties it presents to the end users.

One challenge is that that what users see is exactly what the table editors see. I can't hide the spreadsheet from them, for example. Students have a fair amount of difficulty learning to operate the filters; there are too many irrelevant menus (e.g., File, Edit, Tools). They don't need to know how the sausage is made - they just need a sausage and a fork.

The larger problem is that the presentation of data is surprisingly limited. My biggest wish is for students to show different types of institutions in different colors. It would be very instructive, for example, to show all private and public institutions in different colors, and then turn that option off when you are examining something else. However, Fusion doesn't allow icons to change on the fly, so I settled on two-versus-four years as the default. But it makes it much harder to visually see the proportion of privates versus publics if one has to toggle back and forth between the two, or (worse) mouse over every individual institution.

Ultimately, the map is not entirely successful as a pedagogical tool. For a savvy user, it could be useful for the creation of graphics. Fusion seems to be a case study in forgetting who the most audience is for a software product - it should not be for the developers, but the users of what the developers produce.