Malapert

Started as the #CFHE12 blog; now trying to maintain it as the real blog. Comments are available by clicking on the post title
Mar 19, 2013

The Communicy College Research Center recently released a new study, "Adaptability to Online Learning: Differences Across Types of Students and Academic Subject Areas". It's another cut at a pool of data from the Washington State Community and Technical College system.

Not like I'm the only one doing this, but the headline conclusion could use some qualification.

About that data

First, some thoughts about the raw data. It has some challenges. While I can support the compromises made by the authors, as they would not otherwise have been able to push through an analysis, it would be wise for readers and policy makers to bear these issues in mind.

  • It's limited to transfer classes, which are taken by students completing the first two years of college. While it's possible the applicability to professional/technical (occupational) programs and students is similar, this should be done with caution, and ideally some statistical analysis.

  • Data were drawn from 2004-2008. We probably don't know a tremendous amount more than we did then in terms of good teaching practice. (We actually knew quite a bit!) That knowledge has, to some degree, been backed up by research. More importantly, to borrow from William Gibson, that knowledge is better distributed now than it was. Five years is a very, very long time in the online world.

  • Definitions of online and hybrid courses were in flux within the system during that time. Colleges used different definitions to describe online classes, and shall we say, enthusiasm for rigorous coding was limited.

Of results and known issues

A common, (and in my opinion valid) criticism of many "online learning yields poor results" studies is that they rarely consider that our face-to-face courses are often nothing to crow about. As the authors note:

Overall, the research on the impact of student characteristics on online success indicates that patterns of performance in online courses mirror those seen in postsecondary education overall...(page 4)

The comprehensive nature of the analysis (and the broad-based 2011 study) mitigates these limits, to some extent. However, the researchers occasionally fall back into the trap of comparing modes without referencing the larger context.

For instance, the study notes that students in English and Mass Communication attracted a higher proportion of less adaptable students (page 24). It would be fascinating to compare the concept of adaptability in face-to-face courses as well. English is anomalous in the college in that every single degree-seeking student takes that class. So does the result seen in this study stem from mode or from a much different population? And how does that population do across modes?

The concept of adaptability is a great approach into an important area of study (and practice) in the elearning world. At the time this data was gathered, few instructors and fewer students had deep experience with online learning. Particularly in the early years of this study, only a small demographic subset of the country was comfortable online for anything more robust than buying books and finding dates.

So it's not one bit surprising to see populations that struggle in face-to-face classes struggling in online classes. Old-school media richness theory would certainly support that conclusion: as students struggle, the reduced richness of the online environment prevents a) students from finding help and b) instructors from being aware of a problem. Few practictioners would claim otherwise.

But we're left with two critical questions:

  1. What is it like now?
  2. How does adaptability impact face-to-face courses?

What about the rest of the college

The results from English courses might provide some insight into applying this research across the community college. These courses showed similar impacts to the broader study in terms of reduced persistence and grades (page 20). They stood out for the negative peer effects: students were less likely to find adaptability skills from their peers.

Because English is taught to both professional/technical (occupational) students and university transfer students, it's possible that similar results might be found in professional classes. It might, however, be countered by the older student base often found in prof-tech degree programs. As the CCRC study found, older students are often better able to adapt.

It's not about mode

As those of us in higher education are regularly reminded by policy-makers, it's about completion. It's about a student successfully obtaining a degree or certificate. (Or as some more sophisticated analyses would point out, the students' educational goal. If they want two courses on Javascript, and get it, they've 'completed,' in their mind.)

We know that many students struggle with adjusting to college, regardless of mode. We know that adaptability is often predicated on past family experience with college, on income, on race. We suspected (and now have some dated, but statistically significant results) that these problems are worse in online courses.

What we also know is that if it takes longer for a student to complete a developmental sequence, or get enrolled in a course they need to graduate, then they are less likely to complete. Momentum matters. Perhaps the most fertile ground for discussion is encapsulated in this quote:

... older students may be willing to trade the ability to take an additional course for slightly poorer performance in that course (page 23).

Given what we know about the relationship between time and retention, we have to include this balancing act in discussions about the appropriate use of online courses in our institutional strategy.

Absolutely, we need to provide better support for students and faculty as they learn to operate in online courses. We need to do that for all students. And in many community colleges, there are activities toward that end.

But it's time to move past the headline grabbing arguments about better or worse. It's time to focus on what gets the student to their goals.

Dec 17, 2012

Once you're on board with captioning, how do you do it?

Plenty of schools have chosen the full-outsourcing route. (3Play Media, a captioning company, has profiled Penn State and University of Florida.) Locally, several of our peer institutions are doing it in-house. For instance, one school has an office assistant do the work in slow times. We've taken a mixed outsource/insource approach.

There are three key variables to consider when developing strategy.

Turnaround time

The way a video will be used often dictates how quickly it needs to move from recording to available. When creating an instructional video, post-production often takes longer than recording, so captioning can simply be added into that production timeline.

Expectations for lecture capture, on the other hand, might require a faster time-to-release. While something like lecture capture can be available almost immediately in un-captioned form, a captioned lecture might take longer to create. If there's a letter of accommodation requiring captioning, the capture shouldn't be released to the rest of the class while the hearing-impaired students are waiting for the captioned version.

Outsourcers can turn a video in 24-48 hours. In-house might take longer, depending on workload and scheduling (and how big your captioning team is).

Cost

Captioning and transcription are priced by the hour, no matter how you do it. Outsourcing runs $125-175 per captioned hour of video. Hiring someone with good transcription skills can be done for around $15/hour of work time, but manual captioning can take three to five hours per hour of captioned video.

Additionally, there's some time cost associated with properly linking the captions to the video. Some oursourcers will actually lay the captions on the video for you; others will send a formatted caption file (e.g. .srt), and either the browser/player or the production software will integrate them. Unformatted transcripts are not terribly useful, since adding the time code requires considerable time.

Volume

Economies of scale exist with both in-house and outsourced approaches. Most outsourcers we studied offer reduced rates for increased volumes. As volume goes up, staff get faster, investing in more specialized (better) software and hardware makes sense, more staff can cover more hours, etc.

And, of course, hours x cost/hour = budget.

Do Your Homework

Whether captioning is done in-house or outsourced is largely a product of how your captioning needs (and resources) play out across these variables.Each is related, so movement in one will impact the others. To get to a strategy:

  1. Start with categorizing what needs to be captioned, and how many hours are produced each year.
  2. Determine turnaround time for each category of video.
  3. Assign a cost, both in-house and outsourced, to each category of video.
  4. Play with the numbers: see where accommodations for disability and production requirements would be better met in-house or outsourced.
  5. Determine whether all-in-house or all-outsourced makes sense. (For instance, if you would only have 10 hours a year of in-house, maybe it makes sense to just outsource everything).

In the next post, I'll look at our categories and how that led to the mixed in-house and outsourced strategy.

Dec 14, 2012

I've been spending a bunch of time researching ways to caption video, in all its forms, that we produce and/or use as a campus. My goal is that no uncaptioned video is shown here. The captions may be off, but they're available.

Why? Well, captioning is a good accommodation for hearing loss. It has benefits as well for some with cognitive disabilies as well. Accommodations for disability have been law in the US since 1990. Clearly that's not enough pressure, since we're still having the conversation 20 years later. That said, it's a tough to make headway on disability accommodations. Even as an elearning pro, I've run into only a handful of cases where captioning mattered.

What's changed?

The completion agenda and the definition of access

Policy-makers and pundits have been focusing on college completion. Several states have already shifted to a model in which colleges receive funding based on how many students earn degrees or meet intermediate goals (completing a college-level math course, for example).

As a community college, open access is a core part of our mission. We take all students who can successfully fill out the form and pony up the $20 processing fee. Students with disabilities are, fundamentally, students. They are here, they are a part of our mission, and -- if we are to be honest to the mission -- it is a moral and legal obligation to provide accommodations.

Combine that with funding based on completion, and discussions about access and appropriate supports for learning become intertwined. It's now a financial necessity as well.

Hidden and un-diagnosed disabilities

There is a growing recognition of hidden disabilities, those which are not obvious to others. My colleagues in the Access/Disability support world have long been concerned about undiagnosed disabilities, especially cognitive disabilities.

The upshot of both issues is that there are likely more students who can benefit from what we think of as accommodations than we realize. The Universal Design for Learning (UDL) community has addressed this problem directly. Some of their work is overly reliant on debunked "learning styles" research, but the fundamental philosophy is sound:

  • design learning from the ground up to be usable to all; retrofitting accommodation is costly and time consuming
  • learning designed to reach students in multiple ways benefits all.

College students have grown up with ADA

ADA was passed in 1990. A person born in 1990 would be 22 today. It's not in the Beloit Mindset List, but students today do not know a world without it. That means three things: 1 Students have always seen accommodations in place for their peers 2 Students with disabilities have had accommodations throughout their K12 education 3 Students with disabilities are now able to succeed in K12, and are able to see college success as a possiblity.

They are also now at our door. (Ironically, with some hearing loss, according to #21 on the 2016 Mindset List.)

Increasing legal action

No doubt an outgrowth of the above, students and their families are much more aware of their rights, of the possibilities for accommodations, and of the options for recourse. With the passage of the ADA Amendments Act of 2008, and the Office of Civil Rights' Dear Colleague letter on eBooks, it's clear there's a stronger emphasis on compliance, especially in regard to electronic media.

Returning vets

Recent reports indicate that 45% of the 1.6 million veterans of the Iraq and Afghanistan conflicts have filed claims for some form of disability. (Article, VA Stats). From 2008-2012, 745,000 of those same veterans took advantage of the GI Bill to pursue higher education (article).

We expect to see more on all fronts: more vets, more with disabilities, and a good number of those will be hidden and undiagnosed disabilities. In other words, some folks who really deserve it are going to need support.

The technology

What surprised me, while researching captioning, was how much voice recognition software has improved. I created a screencast using Camtasia, then tested two strategies:

  • Camtasia's built-in speech-to-text feature, which uses Windows' built-in voice recognition.

  • YouTube's captioning feature, which automatically captions uploaded videos.

Neither got it perfect. Camtasia/Windows did better, which is to be expected, since the software requires a user to conduct training. But it was close enough to require only minimal editing.

Better yet, the captioning interface in Camtasia made it very easy to make those edits. The tab key plays the video for a caption and selects the caption for editing, enter key moves to the next caption. My fingers never left the keyboard. Total time to edit a 7 minute video was just over 15 minutes, the first time I ever did it.

In other technologies, HTML 5 has brought the web to the point where you really can completely separate content and presentation. I haven't tried OCR lately, but really... isn't the source digital already?

It's Time

The reason accommodations like captioning haven't happened, despite 20+ years of requirement, boil down to two things: it's not a big group, and it's hard. Well, the documented group is much bigger than it used to be. And the group with hidden disabilities? Bigger still. The relative difficulty has changed, too. It's not as hard as defending a lawsuit or a complaint. And advances in technology have helped lower the bar dramatically.

Nov 16, 2012

Eleni Boursinou and team are studying "learning through networks... to surface strategies that adult learners use for their learning in the context of the CFHE12." The survey asks:

What impact has your participation in the CFHE12 Massive Open Online Course had on you or your work? We'd like to hear a story (a short narrative) of something that changed for you through participation in the CFHE12 MOOC. It might be a new set of collaborators, a changed approach, understanding of new concepts, access to new resources, or something else. Please tell us the story of what changed, and how.

Below is my response.

The course suggested a viable strategy for professional development for people with somewhat unstructured, but busy jobs. The course:

  • Provided a loose structure that enabled time-shifting, at least within a week.
  • Metered the fire-hose. Instead of a massive stream of information in multiple channels, it enabled me to focus on one set of readings around a theme for a few weeks.
  • Used fairly small chunks for knowledge: 30 minute limits on speakers resulted in both a distilled presentation and one that could be squeezed into an available time slot.
  • Didn't over-structure response methods. I had been trying to get back up to speed with blogging, and the option to use that for participation allowed me to leverage the work.
  • Forced me to develop a workflow for digital readings, something I had been needing to do. Again, that structure was loose enough to support experimentation, and the types of 'readings' broad enough to require some flexibility to the approach.

Just yesterday I was grilled by our campus trustees about the future of online learning. My ability to deliver coherent advice would not have been possible had I not spent the time to construct a coherent understanding of the future of higher education. (Ironically, one of the reasons I haven't been very active in the course for the last two weeks was the time I spent preparing other materials for that presentation).

Now, to figure out how to adapt that approach to create professional development initiatives...

Nov 16, 2012

Reading the GSV Company Profiles (2.8MB PDF), I was struck by a notable dividing line between the companies listed. Companies either sold into the existing K12 or higher education markets, or sold direct-to-consumer. What I didn't see was a line that divided those that were truly transformative and those that tinkered on the margins. (Or for you Disrupting College fans, there were lots of sustaining innovations, but nothing disruptive.

Tweaks to existing markets

Business models that sell tools to help navigate the existing system, sell curriculum or instructional packages, or sell adjuncts (additional services, not the people) are selling into the existing market. For instance:

I don't deny that there's money to be made in these markets. I just don't see transformative change. Injecting game-based curriculum into a setting driven by standardized (and lowest common denominator?) classroom might reach a few students who would otherwise be lost, but doesn't yield the creative problem-solvers society needs and the business world claims to need. If you buy the "end of education as we know it rhetoric", maybe they're rearranging the deck chairs.

Would any of them survive in a world without publicly- funded education built on the industrial model of today? Not if they're selling textbooks to college students, or if they're selling to K12 districts. It's possible these companies could pivot to a new world learning order, but more likely that they'd be a case study of the perils, if you will, of sustaining innovations.

What about direct-to-consumer?

GSV's Fall of the Wall, Capital Flows to Education Innovation (6.1MB PDF) highlights a market opportunity among parents taking charge of their kids education. Again, this doesn't represent a revolution. Companies have been selling to home-schooling parents for generations. From the Profiles:

  • Mis-spelled name aside, Grockit sells test prep, with a social angle.

  • Motion Math sells math-focused games for elementary students.

Is there money to be made here? Absolutley. Are we revolutionizing anything? Absolutely not. I used low-res/8-bit teaching games as class examples while a student... in 1990. My parents bought me old-fashioned dead-tree test prep years before that.

A simple test

Deborah Quazzo of GSV Advisors recording here noted that the lack of funding for education is part of the reason there's a low level of investment in educational firms. Is the problem really that no one is doing anything truly innovative?

Certainly, there's an argument that we just haven't reached that point where someone creates a truly breakthrough approach to learning, and figures out how to bank it. Here are a few simple questions I think should be asked of any allegedly revolutionary business (or education) model:

  • From above: would any of them survive in a world without publicly- funded education built on the industrial model of today? In other words, are these companies really generating new money or significant efficiencies, or just sucking money out of an already underfunded system?
  • Does the model break the iron triangle in any significant way? Since we're talking about a trillion dollar system, lets define 'significant' as 'one less comma in the total system price tag.'
  • Would large numbers of students from all educational and family backgrounds benefit? Will it change the game for someone whose family has no experience, and doesn't value learning? (Rags-to-riches anecdotes don't count. The ticket price to claim systemic change is 100,000 students.)

Answer yes to all three, and I promise to read past the preview of your marketing message in my e-mail box.

Nov 2, 2012

I was talking two different languages this morning with our Institutional Researcher. She's currently in the market for tools that make sense of SIS data in ways that look a lot like predictive analytics for at-risk student populations. I was talking about using Angel (our current LMS/VLE) for real-time-ish early warning. Specifically, Angel can be set so a report of performance on an assessment is delivered to an instructor's email box.

Simon Buckingham Shun's presentation to #CFHE12 provided the universal translator. Our IR was looking for meso-level, or institutional data, with an eye toward institution-level interventions. I'm looking at micro-level, or course/user data for the same purpose.

Will it blend?

One of our institutional values is that student success must involve faculty activity beyond the usual content-driven transaction between instructor and student. This is not simply posturing by the faculty. They are most in contact with the students, more likely (simply through frequency and type of contact with students) to form meaningful relationships with large groups of students.

Buckignham Shum posited that big data will enable blending between micro-, meso- and macro-level analyses. The same is true for the resulting interventions. Targeting specific students based on demographic and historical profiles (e.g. Purdue's Signals project) and based on immediate assessment results both address early intervention. And both involve actions in the classroom, virtual or otherwise.

Chicken, Egg, etc.

There's a fairly good chance that we'll be switching LMS/VLE in the next year, and I've been mulling how much to evangelize micro-level analytics in our current system. Our IR noted that it's worth doing, simply because it opens up the conversation about early intervention and using analytics to address student retention and success. Broaching the subject of applying simple, street-level assessment activities is valuable.

As with every learning technology, discussions about pedagogy and values need to proceed in parallel with tool development. For many end users, it's difficult to discuss methods and goals in the abstract. For technologists, it's difficult to predict the affordances that a tool we create (or demonstrate) might have in all disciplines and all cases. Sometimes I think that should be a required bullet in every instructional design job description: the ability to maintain perspective when implementing a shiny new tool.

Can you un-ring the bell?

In which we wax cynical about workload and motivation.

If faculty are part of the solution, then the work of implementing it falls to them. They must set up the automatic reporting associated with assessments, they contact the students, they have the difficult conversations with the students. While the first two can be automated, the last is the most critical, the most time consuming, and the least automatable.

As an instructor, I would argue that the difficult conversation that occurs in week 2 of the quarter takes less time, and is less emotionally and mentally taxing, than the difficult conversation that occurs in week 7, when it's far too late to right the ship. And I would argue that teaching an engaged student who is on a path to success is far more efficient than teaching a student who is not. That view is not universally held.

With any classroom intervention, the instructor needs to see the value of the intervention. To date, research on early intervention is clearly positive. Making performance data easily accessible and actionable arguably demands action. That said, what are an instructor's values around retention? Does the instructor view their job as weeding out students presumed to be not cut out for a particular discipline? Is the correct course of action to simply acknowledge and ignore an under-performing student?

The normative assumption in every analytics presentation and paper I've seen is that our professional duty, our values, would require us to act. Certainly, the increased focus on completion argues for this.

Build the plane while flying it

For many of our instructors, a new technique to improve student performance will be applied with frenzied energy. Others will react as described above, or simply not engage. (Easy thesis idea: compare the performance of students based on instructor values and implementation of analytics, no?)

I'm skeptical of some big-data initiatives. EdX will have a trove of data about student interaction in their MOOCs; whether their conclusions will generalize to students in different learning environments is very questionable. Early warning/early intervention systems seem very promising. When the fuss has died down, it'll be critical to look at whether the interventions borne of analytics really do improve success.

We can start with discussions of pedagogy or start with a tool, implement a change and measure the impacts. And that is as it should be. Analytics show early promise. As teachers, and in partnership with our colleagues, it's time to act.

Oct 26, 2012

If there's a fault in the recent narrative about xMOOCs revolutionizing education, it's oversimplification. Come to think of it, that's a fault in almost any discussion about changing education. In particular, each new "innovation" is presented as a one-size-fits-all, winner-take-all proposition. Yesterday's post about competency- vs. creativity-based learning is a good example.

Although that sounds like a committee solution -- "why don't we do both!" -- there's space for multiple approaches to learning. The question is how to match learning needs, students, and approach/model.

I can think of several reasons for the simultaneous existance of different approaches:

  • Different learning needs - The things you need to learn when you haven't yet learned/been exposed to basic math - say, multiplication tables - are different than when you need to quickly re-tool for a job change. It stands to reason that professional development (like #cfhe12) and learning how to write would require different approaches.

  • Different time of life - Closely related to the above. Generally speaking, more 7-year olds need time working with foundational skills than 40-year olds.

  • Different learning backgrounds - Access and college readiness is strongly inter-twined with a person's background in learning/educational experiences. In short, past experiences with learning, formal and informal, equip students with tools to succeed in different learning environments. My history (3rd generation college student, raised two miles from an Ivy League institution) prepared me for college success in ways that a first-generation college student from a family with no history of success in schools does not.

  • Available resources - which applies to both students and institutions. OER, for instance, are much more valuable in countries with limited access to higher ed than in countries with greater access; students with the money (or credit rating) to attend 4 year residential schools simply have different opportunities than those who don't.

The critical issue with accepting multiple approaches to learning is matching the student, the learning needs, and the model. If the goal truly is efficient education, "letting the market (and students) figure it out," also known as "trial and error," is hardly the solution.

Education is a classic case of market failure due to imperfect information. Students are rarely sure what they need or want. Many "consumers" in this space lack information necessary to make a proper choice. As a student, how do you know which approach to learning is right for you? What market opportunities will exist in four years? In two years? How many students are accurately self-aware about what works for them?

To borrow from yesterday's metaphor of non-parallel lines that don't intersect,

  • Which line are you on? Which should you be on?
  • How do you move between lines?
Oct 25, 2012

It's strange how I remember a particular example from 10th grade geometry, now several years after my 20th reunion. The question was whether two lines that were not parallel would absolutely have to intersect at some point. The answer, of course, (of course!) was no. If they're in different planes, they don't have to. The diagram involved one line going up and down the page, and a second line, oblique to that one but not intersecting.

In various settings, I've been a party to conversations about the future of learning that illustrate this very geometrical concept. (Take that, you "math isn't important" people!). The two lines?

  1. An emphasis on competency-based learning, reduced time-to-degree, credential-focused and job-skills training.

  2. An emphasis on creativity, constructivist and connectivist pedagogy, a focus on habits of the mind and "transformative" learning.

Fundamentally, competency-based models and the like are goal-focused. To build them, you start at the job to be done (or the skills to be demonstrated) and build back until you have a checklist of needed skills. Students then move through that list until the demonstrate mastery. Crossed with discussions of new cost controlled models of learning, time-to-degree, and decoupling learning from seat time, and the model is about checking off the list as quickly as possible.

Critics note three core problems with this model:

  • The process of creating the list of competencies has to be very thorough in order to ensure the language and the needs are understandable to the student, and that skills being taught match labor market needs.

  • Racing to complete concepts isn't compatible with skills that require dwell time, or thinking time, to be developed to a higher level. In writing, for example, your first idea or first draft is rarely (never?) the best, or even good enough.

  • Chief among these skills are things like creativity, synthesis across different knowledge domains, pattern-recognition, learning to fail, and so forth. (Failure is a particularly interesting one: sometimes it takes a while to fall on your nose).

Creativity-focused models of the future of education, at their most basic level, emphasize process. Students are exposed to concepts or problems, and then work through learning the content and skills necessary to understand the concept or solve the problems.

This model predominates in business and pop press discussions about what is required in a 21st Century economy. It overlaps with access, disadvantaged populations, and economic development considerations, especially when higher ed is seen as an engine of that development. I've yet to see it crossed with narratives around cost containment/reduction.

Critics do well by pointing out:

  • There is little linkage between creativity and specific learning outcomes. The process focus and lack of hard skills make it difficult to link returns with investment. In an age of increased scrutiny of the cost of education anything that can't be measured is a tough sell. Selling it to employers, in particular, is a challenge, even though many employers are begging for exactly those skills.

  • There are elitist overtones to the conversation; much of this line of reasoning comes from the independent school (read: elite private high school) world, small high-cost liberal arts colleges, and high-cost, small scale, low-access experiments in education reform.

Certainly one can build competencies around creativity, problem solving, and the like. But as Claudia Scholz notes, ways to assess mastery of these characteristics are not very advanced. And while it can be argued that those competencies will just take students longer to master, that doesn't seem to enter into discussions about how competency-based learning saves time.

So are these two lines that don't intersect? How do these two seemingly opposed approaches to the future of education interact?

Oct 25, 2012

Michael Feldstein at e-Literate

I don’t understand the logic of, “Well, we don’t do a good job of teaching students in classes of 150, but what the heck, let’s go for classes of 15,000. It’ll be fine.” I haven’t yet seen evidence that any of the xMOOCs have come up with better ways to teach effectively at scale...

Oct 16, 2012

One of my side goals for taking an online class (MOOC or otherwise) is to force myself to find an effective workflow for online reading. I've never really found a sweet spot of comfortable reading, note-taking/highlighting, quick and easy reference back to the notes/highlights to review (or write a blog post), and preservation of the original reading.

So far, this is what I've come up with:

  1. Click on the reading.
  2. Save a PDF of the web page (or the actual PDF, depending on how it’s done) to my Readings folder in Dropbox.
  3. Bookmark the reading in Diigo, tagged #cfhe12 and based on content.
  4. Sync the Reading folder on my iPad in Goodreader, so that I can read on the bus/train/etc.
  5. Read it. If I'm at the office, I'll read on the big screen and take notes in a text file (using BBEdit, FWIW). If I’m in Goodreader (on the iPad), I’ll highlight and make notes with that. I'm also playing with Highlighter.com.

Theoretically, the text files are searchable with Spotlight (I'm on a Mac, if you can't tell) and should be available for recall later. I'm also writing them in Markdown, so a MD previewer would make reading them nicer.

Both Goodreader and Highlighter.com have a way to show a summary of notes and highlights. I've found display of notes and highlights loses a lot of useful context, so it may require a shift in highlighting strategy.

Of course, all of this assumes I don't lose traction and stop keeping up with the course.