Now that Tim and I have successfully built a site at community.umwdomains.com that aggregates the activity of the project, I’ve been focussing my efforts recently on what we can do to visualize and expose that activity. Every site that is created (as long as it uses Installatron to install a Web application) on the server as well as much of the content on those sites (as long as the content is available via RSS feed) is being pulled into the WordPress install that runs Community. That means currently we have information about 800 sites and almost 3000 pieces of content from those sites. For sites, we ask users to self-report the course they’re building it for as well as their “status” (student, faculty, staff). From the course data, we’re able to glean instructor and department. We’re also tagging sites with semester information. Content from these sites is similarly tagged with course, instructor, department, and semester information.
That’s a lot of content to play with, and it’s been fun to develop tools to allow users to explore all of the information.
The Site Directory has been in the works since last semester, but with the data all being pulled cleanly now, it’s finally become a pretty powerful tool for browsing the sites created on Domain of One’s Own. Basically, it allows you to use a form to filter the sites by status (student, faculty, staff), department, course, instructor, semester, and software. You can also search for all/part of a particular domain name.
With directory, we can easily provide faculty who are using Domain of One’s Own with a place to find all the sites that have been installed by students in their course.
There are a couple of “weak links” in this approach: First, users must self-report information at the time of installing software to create a site. If they forget to indicate the course they’re building the site for or their status, that data doesn’t get recorded. Tim and I have been talking about ways to let users come back to fix this data, so that’s one solution that might be developed.
Second, since the data is gathered entirely through a trigger in Installatron, we can only collect information about sites that are built using that script installer. Again, a way to let users login to Community to add additional sites/information may be one way to tackle this problem.
In the meantime, it’s not that a big deal for us to go in and fix site data.
The final “problem” is the screenshots that we’re using to visualize sites. We’re using a plugin called WP-Thumbnail to create these, and the plugin depends on WordPress.com’s screenshot generator. For some reason, the approach seems to be getting more and more wonky. We used to occasionally get shots that wouldn’t load. Recently, I’ve been seeing screenshots load and then quickly turn into 404 errors. I have no idea what’s causing that, but I think a different screenshot generator solution is in order. Over on his blog, Tom suggested a few alternatives in a comment thread. I’m hoping to start digging into a few of those in the next week or so.
In the meantime, I encourage you to explore DoOO using the Directory (and my apologies for any broken screenshot images).
For anyone who cares, this interface is built using WP-Views, a premium plugin for creating custom post views and filters.
Last week, I decided to explore WordPress plugins that I could use to build other interfaces for the data on Community. I stumbled across Calendar Archives which builds a visual calendar of your posts. The plugin displays daily posts on the calendar day in a scrollable list, but since there are days when we have upwards of 40+ posts published, this isn’t a really usable interface design. So I hacked the plugin to build a more sensible display for our purposes. Basically, on each day of the calendar there is a link that shows how many posts where published that day and how many sites were created. You can click those links to see the actual posts/sites.
Another features of this plugin I love is that it grabs the first image it encounters in the day’s posts and makes it the background for that day in the calendar. The overall effect is pretty interesting — and pretty.
I knew that I wanted a way to do some quick visualizations of activity on Domain of One’s Own based on the data we have in Community, but I wasn’t sure the best way to approach getting the data into a format that I could make of using something like Google Charts. I was hoping someone would have built a plugin for visualizing WordPress data, but the pickings were slim (to non-existent).
Then today I stumbled across a plugin called Display SQL Stats. It’s a pretty simple idea. Basically, you can write custom SQL queries against your WP database and then the plugin uses Google Charts to visualize those. The catch was that the charts were designed to show up in your WP dashboard not on the front-end.
As it turns out, though, it was pretty simple to hack this plugin as well. I wrote a quick shortcode to display the charts, calling the same functions that the plugin was using to show them in the dashboard, and then I dropped that into a page. That did the trick.
So, from there I wrote a couple of SQL queries to show things like daily post/site numbers and break down of sites by department or user status, and now we have our Stats page.
I’m hoping this is just the beginning of exploring how we can mine the data that we’re collecting in Community!
7 thoughts on “Visualizing & Exposing Domain of One’s Own Activity”
I like it. I’ve been trying to think through some of this so this helps me quite a bit. I was originally thinking D3 because it’s prettier than Google Charts because it’s prettier (and maybe doing some but simple sounds very appealing.
I’m not sure who wrote that comment but clearly he was distracted.
This is great! I’m a fan of stats as a way to help tell the story of a big project like this. I’m curious about the “by department” stat, by the way. I may have missed it, but how do you determine if a site is for, say, ELC? For faculty, it would be easy enough to pick out. But for students, is it a situation where you can tell that a student has made a site as part of an English class?
Partly, I’m surprised that ELC has such a small slice, but on the other hand, I’m impressed by how many sites are apparently not department-affiliated. If I read that correctly as students joining the project just because they want to, that’s a good indicator that people are recognizing and taking advantage of this valuable opportunity.
Anyway, great work here!
Hey Zach, I wrote this post in a hurry as I was getting ready to go out of town for spring break. Your comment reminded me that there are some nuances to the data that need to be explained. We’re now capturing data about new installs and new content in real time. But when built/launched this site a few weeks ago, we had to import data about past installations (from Installatron). Course data then had to be added by us “by hand,” and I’m sure our attempt ended in an incomplete (or not entirely accurate) set of data for the Fall. In addition, the RSS feeds we added at that point would have been limited to the last 10 items in the feed, so we don’t have complete post data for the fall, either.
Moving forward, I expect our data to become more and more complete and accurate, but I think the anomalies you’re seeing are a product of us ramping this site up.