WordPress as a Data Collection Tool

A few weeks ago (actually, I think it’s more like months at this point), I blogged about publishing my first plugin on the WordPress Repository. I thought I’d take a moment to write about what that plugin is — and the projects I’ve been working on that inspired it.

Making data collection as easy as cheese

Last fall, historic preservation professor Andi Livi Smith contacted me about working on digitizing a project that she’s been doing for several years in her Survey and Preservation Planning Course. Students in the course are required (working in groups) to survey a swath of properties around Fredericksburg. Each semester the class focuses on a particular neighborhood, and each group gets a selection of that neighborhood to work on.

In the past, students have filled out a paper form and then (as a group) transferred it into an Excel spreadsheet.  Then Andi would aggregate all of those spreadsheets into a master spreadsheet. Along the way, as you can imagine, there were plenty of opportunities for errors to be introduced. In addition, since they were filling out what is essentially a paper form, there was no way to normalize their entries so that they were using the same terminology, codes, etc.

Andi asked me if there was anyway we could facilitate this project with some kind of digital data collection system. I immediately thought of the work I’ve been doing over the last several years with WordPress custom post types — work that has increasingly been made easier by the arrival of several great plugins on the scene.

In the end, I built a new site for Andi’s class that allows students to use a front-end form to enter data about their sites. That data is collected as custom fields associated with a custom post type. We can then build special views of site listings as well as for individual sites – and the student who creates the entry can come back later and edit his/her data.

Andi also needed her students to be able to upload neighborhood maps which illustrate the location of each site that a group is working on — and I was able to create this as another custom post type with a parent/child relationship to the individual sites. Right now, if you click on an individual site, you can see the associated “parent” map. Eventually, it would be nice to build a way to use the maps to find sites, too.

The forms are fairly complex and allow Andi to set particular values for certain types of information — so that we can get that data normalization that she so desperately wanted. I also worked with a theme that’s responsive, so the site *should* work well on student’s smart phones, allowing them to easily upload data from the field,

(The forms require students to be logged in in order to fill them out, so I can’t link to them specifically, but hopefully you get the idea.)

The plugin I developed was to solve the final piece of the puzzle for Andi — after her students do all this work creating the collection of site data, they need to pull a subset of the data into SPSS for some analysis. Andi needed to be able to export the data in a format that could easily be imported into SPSS, and there wasn’t a plugin or technique I could find that quite fit the bill. So I wrote Custom CSV Exporter which lets you choose a post type, a set of custom fields, and then get a CSV file to use elsewhere.

Whew.

BUT WAIT! THERE’S MORE!

A few months after Andi approached me, another professor, Theresa Grana approached me about a project she does with her intro level BIO121 course. Theresa has her students go out and sample locations in the area (or in their hometown) for microscopic worms, which the students then study for the duration of the semester. I LOVE this project because it takes a standard, intro-level biology course and has the students doing authentic research. Thersa (like Andi) wanted a way to collect, manage, and share the data her students were collecting, so I used almost exactly the same technique to set up a site for her.

There are a few differences: Theresa’s students will only be adding their data once, so they didn’t need a way to edit existing entries. She also had some fields that her upper-level research students would be adding later. These aren’t available through the front-end forms. They’re just custom fields that can be added/edited through the post editor for the custom post type I created.

One thing I did get to experiment with for Theresa’s was mapping the data in Google Maps. I found a plugin that allows me to easily map according to longitude/latitude, so each individual sample page has a handy map showing where it came from.

At face value these projects seem really different, but really they’re very much alike — each professor wanted to empower her students to collect and share data using a Web site. They each had some specific requirements and controls that needed to be addressed, but really the basic requirements were identical.

To be honest, I’m amazed that in 2013 there aren’t more tools out there that make it possible to do this easily (maybe there are and I’ve just missed them?). I loved working on these projects, but they were both fairly complex and few faculty would have the time/expertise to do them on their own. With all the tools out there for easy publication and sharing on the Web, where are the tools that facilitate the creation of collaborative, well-designed, and well-presented datasets?

WHAT NEXT? 

I have at least one other faculty member who is interested in incorporating a site like this into an upper-level biology course. I’m very excited about this because one aspect of the project will involve the students having to determine what type of data needs to be collected. This introduces a whole new meta level to the project that I think is insanely important — it’s wonderful to have students collecting and sharing data, but what happens when they also have to spend time thinking about what kind of data/information needs to be collected, how it should be structured, and how it should be presented? This gets at technology AND information fluency in really rich and wonderful ways, I think.

A COUPLE OF NOTES

I used several plugins to make these sites happen, and while I haven’t written out specific instructions, perhaps knowing what I used will help others think about how to get started?

  • wp-types is a free plugin that makes it dead simple to create custom post types and custom fields.
  • wp-views is a premium plugin that allows you to very easily create custom queries/presentations of posts
  • cred is a premium plugin for creating front-end forms for creating/editing content
  • gravity forms is a well-known (premium) form plugin, but some may not know it also has an add-on for using it to create content from the front-end
  • pronamic google maps allows you to embed Google Maps in posts, and I was able to rig it to work with our custom fields
  • csv importer is an older plugin (hasn’t been updated in 2+ years) that allows you to import post data from a CSV file. Theresa had about 150 existing worm samples that we were able to import into the site using this.

Andi has written her own post about this project that you should read. Really you should just add her blog to your RSS reader of choice, because she’s awesome.

I’m hoping to do an episode of DTLT Today with both Andi and Theresa to talk about their projects. If and when that happens, I’ll post it here as an update.