For the last six months or so, I’ve been working with a committee here at UMW that is charged with researching ePortfolio systems. The push for this system comes on a number of fronts:
- An interest in a system for tracking institutional assessment learning outcomes (on program-, departmental-, and the University-level) and reporting on these otucomes.
- A space for meta-cognitive reflection by students on their learning, perhaps as part of a larger look at how we advise students.
- A desire to provide students with a “leg-up” by giving them a robust platform for showcasing their intellectual and professional work and development.
- A need (specifically in the College of Education) to track student outcomes BEYOND graduation. New laws are requiring us to demonstrate the effectiveness of our students as teachers after they’ve graduated (for a indeterminate amount of time)
It’s worth mentioning that another possibility with a system like this would be to have the ePortfolio become a transcendent presence throughout a student’s time at UMW — a place where they store work, track and share their learning progress, reflect upon their ideas and experiences, and, ultimately, build an online identity for themselves that represents their intellectual self. However, I believe that to do this really effectively we need some kind of programmatic commitment. The ePortfolio would have to be “baked in” to the way we teach from first year seminars to senior thesis. That kind of programming would, potentially, require a much bigger conversation about our curriculum, requirements, and teaching methodologies. I’m not sure that’s going to happen anytime soon here, so, for now, the four needs outlined above are guiding our process more specifically.
I will admit that the first goal (institutional assessment) is incredibly murky for me. I’m lucky in my position to be fairly removed from conversations about academic assessment. I say “lucky” because, frankly, whenever I am in conversations about this topic it feels like the cart is driving the horse. Rather that starting with a conversation about students and what we want them to experience and how we want them to change during their time here, we start at the top — what are our reporting needs and how to we build down from there? The entire institutional assessment process, for me, is sort of a black hole and I can never seem to have a conversation with anyone that sheds any light on it.
The second goal (a space for student reflection) is, by far, the piece of this that intrigues me most. It’s really the starting place for a conversation that could result in programmatic change, and, to me, that’s interesting.
The third goal (a professional online space for post-graduation) is sort of the practical flip side to the second. Theoretically if we do #2 right, I think #3 should happen naturally.
The fourth goal (a space for post-graduation tracking) is the newest for me, and pretty interesting. I didn’t know until a few weeks ago that the College of Education had this requirement. I’m not entirely sure how they’re supposed to realistically achieve it, and I worry if we can build or buy a system that can really answer this need.
Complicating the landscape for this project is our current existing systems which, in some cases, are already meeting some of these needs:
- UMW Blogs, our open source WordPress-based publishing platform, allows any student to create a site, upload content, and, generally, do whatever they want. Students have already started to use it as place to host online portfolios on their own, and some faculty have had students doing ePortfolio-like things in this space.
- Canvas, our new CMS (which also went open source earlier this year), is the space where students and faculty do your standard online course managementy types of activity: submitting assignments (or “artifacts as the ePortfolio crowd seems to like to say”) and grading said assignments. Canvas allows us to create institutional learning outcomes and rubrics which can be used at the course level, but it’s not clear that we can use it to actually do the kind of data gathering and assessment we need.
- TracDat is our institutional data tool into which faculty can submit assessment data and results.
Last spring, three faculty members at UMW (Steve Greenlaw, Anand Rao, and Krystyn Moon) all piloted WordPress as a space for doing different kinds of ePortfolios. Steve used them as a space for freshman advising. Anand had his students build personal portfolios based on their work in a visual rhetoric class. Krystyn and I worked on integrating a Google spreadsheet into a WordPress template to do online assessment of student work.
The projects were as much to understand the questions that we needed to answer as they were to provide the final solution.
This year, we’re hoping to pilot a few systems (both open source and commercial) to see if we can find one that meets our needs (or most of them?). However, we’re running into some challenges:
- Among the systems that are out there, many seems to do one piece of what we’re trying to do well. On the other requirements, they either don’t do it as well or don’t do it at all.
- Many of the systems seem to duplicate the functionality we already have in existing platforms.
- It’s difficult to imagine how yet another system would fit into this landscape. If faculty and students are already happy using UMW Blogs and Canvas to teach and learn, why are we going to thrust yet another system on them (that duplicates some of what UMW Blogs and Canvas already do)? I can imagine many of them just balking at the notion and not using the new system at all.
We’re beginning to wonder if there is another way forward. Is it worth trying to develop something on top of what we already have? Or, is it possible to build something on top of a new system we buy that would gracefully integrate them into what we already have?
I’m interested in talking to people at other institutions about this project. In my mind, we’re not the only ones facing these questions or looking for these solutions. We should be working together to share information about what systems work well (and work well together) and/or what we could build together.
I’d love to hear from people about their ideas. Specifically, here are a few questions I have:
- Do the four goals I outlined above align with any of your institution’s goals? What have I left off that is driving your own interest or research?
- Do you think it’s even possible for one system to meet all of these goals?
- If you’re already implementing a system for ePortfolios how are you handling integration with your existing systems (on both a technical and cultural level)?
- Would you be interested in talking with a group of others about collaborating on this research (and potentially, development)?
- What have I not mentioned here that I should be thinking about?
Hi Martha –
We’ve dabbled with using WP (& BP) for e-portfolio at York College. Myself (communications technology) and a colleague Xin Bai (teacher education) have used it quite a bit with students in our respective disciplines. I haven’t focused on the institutional assessment piece but Xin has a bit with the teacher education program’s need to showcase program and student assessment as part of their reaccreditation process.
I’ve focused more on your questions 2 and 3, with students creating blogs for various courses in our major, emphasizing that they should be describing their process as much as submitting assignments. This was capped with a senior requirement portfolio course, that asked students to prepare a professional portfolio (resume, statement, samples) using their various class blogs as a rich resource.
A lot of this sadly has recently stalled a bit with a lack of development resources for the platform (were still pre 3.xx on WordPress). Our IT department has little interest in WP preferring to show their affection for the college’s website/CMS they’ve built on Plone. We’re required to use Plone for faculty admin matters and it’s brutal with user interface/functionality so I can’t even imagine asking students to use it.
Let me know if there’s an opportunity to get like minds together to talk. I’m sure I could get Xin involved as well.
Also Matt Gold, mentioned in a Tweet to try and interest Joe Ugoretz from the Macaulay Honors College. He’s really been leading the way here at CUNY with eportfolios@Macaulay inWP. He describes some of his ideas for eportfolio assessment and documentation about the platform in his blog Prestidigitation.
Thanks for the reply, Michael. I’d love to see/hear more about what you and Xin are doing with WP. Truly, the part of this project that interests me most is the possibility of using WP as the tool for forward-facing portfolios. However, the project, as it’s defined, deals with more than that. The assessment piece of this is, for me, the most difficulty to wrap my head around. I’d be interested in hearing more about Xin’s experiences on this front. I will definitely follow-up on Joe’s work, too.
I was part of similar conversations at Queens College when I was working there. I argued (unsurprisingly) for an open source solution that we were already using: the blogging platform (we were in the process at the time of moving from Movable Type to WordPress). I lost, and they went with a separate, proprietary eportfolio platform. C’est la vie.
Anyway, my own take on the balance of your requirements is that such systems can only succeed in their “institutional” goals (like assessment and alumni connections) if they succeed in their “curricular” goals (genuine student engagement with the platform). The latter is an absolute prerequisite to the former, practically as well as ethically. As such, platform decisions should be made on the basis of student use above all; as long as you can make a reasonable case that the non-curricular goals can be met (somehow, if not out-of-the-box) with the option that best supports curricular goals, then that’s the option you should go with.
It seems pretty clear that at UMW you’ve had great success getting authentic student engagement using a WordPress-based platform. And since WordPress is nothing if not extensible, it’s quite reasonable to expect that tools for assessment could be found, adapted, or built. Makes WP a pretty easy sell, at least in my view, when compared with systems that are designed with non-curricular goals in mind. (Then again, as I say above, I lost that argument when I was an instructional tech.)
I’m not sure that I’m in a position to offer any genuine development resources to your project (I’m way over my head in open source commitments already), but I’m happy to lend an ear and a word of advice (technical or otherwise) when possible.
Boone,
I absolutely agree. I’ve argued strenuously in our project meetings that whatever path we take MUST be driven by the teaching and learning activities of faculty and students, not by the assessment and data collection goals of the administration. Picking a system because it addresses the latter to the detriment of the former would be a travesty. Luckily, there are lots of faculty on the project who completely agree. However, we still have this assessment piece looming out there, and, now, as I face the task of writing a requirements overview that would illustrate how the various pieces we already have fit together, I’m finding that the assessment piece is what I’m having to spend the most time describing and understanding.
If you’d be up for letting us pick your brain about the possibilities in WP, that would be awesome. I can probably carve out some consulting money for a few hours of your time. 🙂
Stay with what works. The tool is the internet. Here is what I saw:
1. A space for meta-cognitive reflection by teachers on their learning, perhaps as part of a larger look at how we advise teachers.
2. A desire to provide teachers with a “leg-up” by giving them a robust platform for showcasing their intellectual and professional work and development.
3. A need (specifically in the College of Education) to track teacher outcomes BEYOND reaching tenure. New laws are requiring us to demonstrate the effectiveness of our teachers as learners after they have reached tenure (for an indeterminate amount of time)
Sorry, I just replaced students with teachers.
I agree that “baked in” to courses is the best way to go. If the teachers are not role modeling the use of a learning/reflective/demonstration/portfolio then it will be hard to have them sell it honestly to the students.
Great, great point. We have had some interest in using whatever system or platform, we choose as a space for faculty to demonstrate their own professional activities. Unfortunately, it’s driven mostly by an interest in making the promotion and tenure process more palatable. But your point reminds me that this is ALSO about getting faculty to “walk the walk” when it comes to reflection and sharing.
Thanks for visiting my blog, Martha, and I would love to discuss this further. Some very good points from my colleagues above, and it’s great to see that we’re all struggling with the same kinds of issues. I really think your four goals, Martha, are pretty common in a wide range of institutions, and I think (as you hint) that they’re really not suited for a single solution.
Lately I’ve been thinking a lot about the different motivations for assessment and about when and why it becomes extra meaningful. One thing that these thoughts have led me to question is the need for a “baked in” or universalized program. To a certain extent I’m making a virtue out of necessity, because our structure is such that it’s very difficult for us to bake anything in any kind of general or common way. So what we end up with is customized pockets of self-directed or self-motivated work. It makes it impossible to say “x% of our students are doing y, and z% are doing w.” But what we can say is “student Q is doing H, I, L, Fed, 3445, seahorse, and abalone. And student H is trying to do HJ, but only making it as far as ~12, while she didn’t realize she was going to create a bunch of MLish.”
I know, that sounds like gibberish, but what I’m suggesting I think is an assessment that includes individuality, that celebrates it, and that allows us to understand what students themselves value about their accomplishments and their progress.
Of course, none of this really works for your goals #1 and #4. But maybe it’s better to keep things that fit better for those goals, and keep eportfolios for #2 and #3.
I’m still in the early stages of groping for all of this….
Hi again, Martha. You just wrote another very interesting piece. This time on **two** very interesting myths: e-Portfolios and Assessment.
You say that
“whenever I am in conversations about this topic [assessment] it feels like the cart is driving the horse. Rather that starting with a conversation about students and what we want them to experience and how we want them to change during their time here, we start at the top — what are our reporting needs and how to we build down from there?”
I feel more or less like you. Amidst the big waves of the current embodiment of some myth. In my institutions we are also in the assessment boat, and I can certainly appreciate why federal funding agencies and accreditation bodies want us to document our progress in teaching and our students’ learning achievements. The problem is that we have always done “assessment” when teaching, because when you *change* your methods, styles, technologies as a result of some reflection based upon observing how students do in your class, you are doing assessment. The issue is that “they” want us to **document it**. So, as you say, the risk is that the reflective part begins with exploring the “how” to do it (document, track, etc.) , instead of focusing on the process. Perhaps that will come as a result, eventually…?
Have you read Kathleen Blake Yancey’s piece in the Winter, 2009 issue of Peer Review? The entire issue is available here: http://www.aacu.org/peerreview/pr-wi09/documents/Peer_Review_Winter_2009.pdf. She presents compelling evidence that students must construct their own e-portfolios, not simply plug their “artifacts” into pre-existing data buckets. Her argument was a big part of the inspiration for my personal cyberinfrastructure piece.
Lately I’ve been thinking that part of the problem is the metaphor of “portfolio,” which suggests something large, standard, and external to me. The metaphor is all about an inert and meaningless container. What’s inside is meaningful. But if the “container” is the Web, then the container is not inert and meaningless. It’s not even a container. It’s like a brain that way.
The idea of “portfolio” worked when writing classes began to assess the entire scope of a student’s work instead of just marking up individual papers with a red pen. It linked the papers to the idea of artworks, or even (barely) to the idea of work-in-progress. There were traces of the old contract-teaching paradigm in there too. Not necessarily bad things, mind you. Yet as often happens, the metaphor gets an “e-” in front of it and begins to do damage to our ability to imagine the nature and potential of the new medium. It also invites data vampires to come in and do their little “assessment” dance.
What if we didn’t talk about “e-portfolios” but talked about “web presence”? Shifts the entire field around, I think.
Portfolio entry triad:
1. Artifact
2. Commentary
3. Criteria (rubric, etc)
Each of these triads (which can be separate files or integrated artifact+comment depending on software) become part of an overlay list organized according to need.
For example, student wants a list of work by year or theme, teacher wants list of all students with work, department wants list of all teachers with representative samples, institution wants list of representative graduates.
So, overlays (matrices in eportfolio-ese) can be independent of artifact repository as long as links are permanent and files do not disappear.
Hence need to distinguish between working repository (formative evaluation no overlay) and portfolio (summative evaluation and target of multiple overlays).
Technically I reckon you can mix ‘n match systems to do pieces of this but overhead for users is prohibitive. Systems like Mahara or Sakai or commercial solutions seem necessary to me, at least for the working repository and commenting with required notification to teachers and students that response is required as workflows progress.
There will always be tension between evaluative overlay requirements and functional usability of course…..until you have a solid instructional fire going you can put the flame out every time you open the stove door to prod the logs. So I would recommend establishing what “roaring fire” stage should look like with the most user friendly tools you can afford first.
Your evaluators should agree to wait until that milestone is reached before building overlays and overhead. Otherwise everyone will be frustrated during the pilot phase and adoption will be too unpleasant a prospect to sustain the kind of teaching and learning by all stakeholders that eportfolio-philes dream of.
Many two cents there…..I love the depth of this thread
Bram
My caution is that we shouldn’t just create a repository for artifacts made in response to conventional schooling. Those artifacts–papers, lab reports, whatever–should probably be preserved, but we have defined learning in terms of these “signature pedagogies” long enough. The e-portfolio should also be a platform for emergent understandings/instantiations/genres of knowledge construction, not just school-on-steroids.