Online learning 101: Costs vs. efficacy

Reda SadkiLearning strategy

Having presented three online learning approaches, here are three aspects to consider together:

  1. What is the cost of developing an online course based on each approach?
  2. What is the cost of delivering the course, per learner or per hour?
  3. What is the learning efficacy (outcome) that can be expected?
Costs vs. efficacy

Costs vs. efficacy

Development costs for modules are comparatively expensive, as they are media-intensive and require complex production and technical skills.

Often this leads to under-spending on the instructional design.

The main attraction of this approach is its low delivery cost.

It scales really well.

Once you have a self-guided module online, the delivery cost is marginal.

All of a sudden, you can abandon the elaborate schemes in place mostly to restrict access to limited numbers of seats.

Unfortunately, the death knell for this approach is its limited efficacy.

It doesn’t work very well and, probably, only marginally better than giving a motivated learner the raw content to prepare.

Developing an online, scenario-based simulation does not necessarily require intensive media production.

We are not talking about building a humanitarian ‘Second Life’.

Combine compelling, well-written prose with a few images.

What matters is the complexity of the scenario’s decision points, the diversity of its cast of characters, and the design of a scorecard that provides rich formative feedback.

Add a time element to put the pressure on.

The development costs can be high, but the investment is in the design of a powerful experience for the learner, not in the bells and whistles.

Best of all, such simulations can be self-guided (while modeling collaboration and team work by interaction with the fictional cast of characters) and therefore have the same marginal delivery cost as the ineffective information modules do.

And, of course, there is a powerful case (and accumulated evidence) that allowing people to make choices and experience their consequences (as success, failure or somewhere in between) generates a virtuous cycle of engagement, retrieval, and retention.

The knowledge community concept starts with an online course in which peer-to-peer relationships are the basis for the co-construction of knowledge.

A learning environment like Scholar, grounded in Bill Cope’s and Mary Kalantzis’s New Learning, can fully align the way we learn to the way we work and collaborate.

Because learners are the ones producing knowledge, the roles of teachers and experts are transformed.

A seemingly absurd parallel can be made between this approach and reality television.

When learners are teachers and teachers are learners, the development costs are low.

The fact that peer review and other forms of learner dialogue require a synchronous cohort implies more-than-marginal delivery costs, but does not prevent scale.

A large cohort can be split into smaller communities, reinforcing bonds of knowledge and collaboration.

Together, I believe that scenario-based simulations and knowledge communities can sustain an agenda for new forms of humanitarian learning and assessment. And, yes, lower costs along the way.

This infographic is excerpted from a comprehensive (65 minutes) talk originally presented to the Assessment Capacities Project (ACAPS) on 22 September 2014. Its content is largely based on my experience in managing a 1.7 million CHF pipeline of online course development.

Online learning 101: Approaches

Reda SadkiLearning strategy

There are myriad approaches to online learning. I’ve selected three. One of them should no longer be recommended. This is the production of information modules that test information recall. In some cases, aware of the limited outcomes using this approach, attempts have been made to encourage reflection or analysis, but then the limitation of the approach leaves the learner with limited or no formative feedback and reductive forms of assessment. We need to stop producing these “click-click” modules, as they are teaching all of the wrong things, even if the subject matter content is spot on. They are purely transmissive, leaving the learner to passively consume information. They substitute multimedia bells and whistle for substance. Their only real usefulness, in the past, was to introduce people in the sector to “e-learning” as a digital version of transmissive trainings in which the slide deck is the pedagogy.

The other two approaches, fortunately, are grounded in more constructive (and constructivist) pedagogies. They have been shown to scaffold, support and promote realistic outcomes that matter for developing competencies around analysis, team work and leadership. Once we realize that how we teach is at least as important as what we teach, these two distinct approaches open up new possibilities for humanitarian learning. They are the topic of much of my presenting, which reviews the evidence, case studies, and practical aspects of each.

Three online learning approaches relevant for humanitarians

Three online learning approaches relevant for humanitarians

This infographic is excerpted from a comprehensive (65 minutes) talk originally presented to the Assessment Capacities Project (ACAPS) on 22 September 2014. Its content is largely based on my experience in managing a 1.7 million CHF pipeline of online course development. The full set and recording are available for LSi.io members via this link. LSi.io is a non-profit talent network for learning leaders from corporate, academic, and humanitarian/development sectors interested in solving wicked problems. (Note: there are some display problems on lsi.io which should be fixed soon. Thank you for your patience.)

 

Online learning 101 mindmap excerpt

Online learning 101: learning objectives and mind map

Reda SadkiLearning strategy

My LSi.io presentation on the foundational knowledge about online learning in the humanitarian context could provide fodder for… an online course. And here are some of the learning objectives that would be included in such a course, together with a mind map showing some of the items addressed by the presentation.

  1. Summarize the challenges of adapting to constant technological change in learning design
  2. List humanitarian learning problems that can be addressed by online and distance learning
  3. Distinguish between three evidence-based learning approaches relevant to the humanitarian context
  4. Explain the main criteria used to distinguish these approaches
  5. Distinguish development costs from delivery costs
  6. Compare cost vs. efficacy for three learning approaches
  7. Explain the relevance of blended learning for humanitarian learning
  8. Distinguish between self-guided and cohort-based learning
  9. Scope the complexity of an online learning project
  10. Identify possible sources of funding for online humanitarian education
  11. Identify factors to consider when developing a learning system
  12. Evaluate when scaling up is relevant to developing online learning
  13. Relate humanitarian training needs to the affordances of New Learning
  14. Summarize the benefits of scenario-based simulation for humanitarian training
  15. Compare the learning outcomes between distance learning and face-to-face
  16. Summarize changes in the nature of knowledge
  17. Reflect on the significance of changes in the nature of knowledge for humanitarian learning
Online learning 101 mind map

Online learning 101 mind map

This post is based on a comprehensive (65 minutes) talk originally presented to the Assessment Capacities Project (ACAPS) on 22 September 2014. Its content is largely based on my experience in managing a 1.7 million CHF pipeline of online course development. The full set and recording are available for LSi.io members via this link. LSi.io is a non-profit talent network for learning leaders from corporate, academic, and humanitarian/development sectors interested in solving wicked problems. (Note: there are some display problems on lsi.io which should be fixed soon. Thank you for your patience.)

Learning Strategies International

Online learning 101: Criteria to distinguish approaches

Reda SadkiLearning strategy

The table below summarizes criteria that you should consider to identify the appropriate approach for your online learning needs. At the top is the pedagogy and specific learning architecture. The key question is to ask: What does the learner get to do? Key decisions include the choice between self-guided learning (which scales up easily as it does not require synchronous interaction with other learners) and cohorts (which enable synchronous peer-to-peer relationships between learners).

Criteria to distinguish approaches
Criteria to distinguish approaches

For a long time, a ferocious debate was waged between advocates of face-to-face learning who fetichized the value of IRL (“in the real world” interaction and advocates of online or distance learning. The evidence fairly definitively demonstrates that distance learning delivers slightly better learning outcomes, and that there is no learning efficacy benefit when you blend. However, your professional network is how you find your next job. It is also how you learn from others. Face-to-face contact is necessary for cultural reasons, at least for the current generation of humanitarians over 30. The bottom line is that in the humanitarian context, social relationships are so important that they provide the sole justification for a blended approach. Distance technology (read: Skype) can help scaffold, grow, and sustain these relationships and their value for learning, as can a well-designed online knowledge community.

Next in the table are outcomes. The industry standard is Kirkpatrick. It really is that simple – and comprehensive. What is staggering is the dearth of learning evaluation in the sector. Training is assumed to be inherently good. This is no longer good enough, hence the necessity of not only reactive evaluation (the “happy sheet”) but the impact of learning on performance at both individual and organizational levels.

When considering costs, one needs to distinguish development costs from the expenses associated with delivery and evaluation, as well as ongoing quality development. Often, an organization will budget for development without considering what training will cost to deliver.

Last but not least, and I’ve written and presented on this extensively elsewhere, is scaling up. Self-guided learning scales up at low cost and cohorts do not (very easily). This intersects with the question of pedagogy.

This post is excerpted from a comprehensive (65 minutes) talk originally presented to the Assessment Capacities Project (ACAPS) on 22 September 2014. Its content is largely based on my experience in managing a 1.7 million CHF pipeline of online course development.

Complexity in humanitarian and development

Online learning 101 for humanitarian managers and decision makers

Reda SadkiLearning strategy, Presentations

I’ve just posted on LSi.io a comprehensive (65-minute) presentation intended for humanitarian managers and decision makers working in organizations without prior experience in online or distance learning. It includes numerous practical examples and case studies, as well as a description of the best available learning theory and best practice approaches most appropriate for the humanitarian learning context.

Here are the 10 questions addressed:

  1. It’s not about technology. Really?
  2. What learning problems do you want to solve?
  3. What kind of online learning can prepare humanitarians?
  4. What do you need to know about costs, time, and complexity?
  5. Where’s the money?
  6. Do you need scale?
  7. Can you do more than transmit information with e-learning?
  8. If experience is the best teacher, how can e-learning help?
  9. Does e-learning work at all?
  10. How does all this fit together?

This slide set was originally presented to the Assessment Capacities Project (ACAPS) on 22 September 2014. It is available for LSi.io members via this link. LSi.io is a non-profit talent network for learning leaders from corporate, academic, and humanitarian/development sectors interested in solving wicked problems. (Note: there are some display problems on lsi.io which should be fixed soon. Thank you for your patience.)

 

Bookshelves

Thick knowledge

Reda SadkiContent strategy, Learning strategy, Quotes

Toby Mundy on books as thick knowledge:

“[…] Books have a unique place in our civilisation […] because they are the only medium for thick descriptions of the world that human beings possess. By ‘thick’ description, I mean an extended, detailed, evidence-based, written interpretation of a subject. If you want to write a feature or blog or wikipedia entry, be it about the origins of the first world war; the authoritarian turn in Russia; or the causes and effects of the 2008 financial crisis, in the end you will have to refer to a book. Or at least refer to other people who have referred to books. Even the best magazine pieces and TV documentaries — and the best of these are very good indeed — are only puddle-deep compared with the thick descriptions laid out in books. They are ‘thin’ descriptions and the creators and authors of them will have referred extensively to books to produce their work.”

I’ve found myself going back to searching for well-written, comprehensive, in-depth books for sourcing both foundational and most-current knowledge. This notion of ‘thick knowledge’ makes a lot of sense.

Photo: Rainbow (Katey/flickr).

Lenses rainbow

Unified Knowledge Universe

Reda SadkiContent strategy, Learning design, Learning strategy, Writing

“Knowledge is the economy. What used to be the means has today become the end. Knowledge is a river, not a reservoir. A process, not a product. It’s the pipes that matter, because learning is in the network.” – George Siemens  in Knowing Knowledge (2006)

Harnessing the proliferation of knowledge systems and the rapid pace of technological change is a key problem for 21st century organizations. When knowledge is more of a deluge than a trickle, old command-control methods of creating, controlling, and distributing knowledge encased in a container view do little to crack how we can tame this flood. How do you scaffold continual improvement in learning and knowledge production to maximize depth, dissemination and impact? A new approach is needed to apply multiple lenses to a specific organizational context.

What the organization wants to enable, improve and accelerate:

  1. Give decision makers instant, ubiquitous and predictive access to all the knowledge in its universe – and connect it to everywhere.
  2. Rapidly curate, collate and circulate most-current content as a publication (print on demand, ebooks, etc.) when it is thick knowledge, and for everything else as a set of web pages (micro-site or blog), or individual, granular bits of content suitable for embed anywhere.
  3. Accelerate co-construction of new, most-current knowledge using peer review to deliver high-quality case studies, strategies, implementation plans, etc.

How do you crack this? Here are some of the steps:

  1. Benchmark existing knowledge production workflows and identify bottlenecks, using multiple lenses and mixed methods.
  2. In the short term, fix publishing bottlenecks by improving existing systems (software) and performance support (people).
  3. In the longer view, adopt a total quality management (TQM) approach to build ‘scaffolding’ and ‘pipes’ that maximize production, capture, flow, and impact of high-quality, most-current knowledge production, with everything replicated in a centralized, unstructured repository.

Multiple lenses are needed as no single way of seeing can unravel the complexity of knowledge flows:

  • The lens of complexity: Systems thinking recognizes that we do not need a full understanding of the constituent objects in order to benchmark, analyze, or make decisions to improve processes, outcomes, and quality.
  • The lens of learning: Learning theory provides the framework to map knowledge flows beyond production to dissemination to impact. The co-construction of knowledge provides a ‘deeper’, less fleeting perspective than conventional social media approaches. More pragmatically, a number of tools from learning and development and education research can be used to benchmark.
  • The lens of talent: Staff lose precious time and experience frustration due to duplication of effort, repetitive tasks, and anxiety due to the risk of errors. They may feel overwhelmed by the complexity and intricacies of multiple systems, as well as by the requirement to learn and adapt to each one. Informal learning communities can bring together in the workflow to identify potential, develop competencies, and drive performance. Hiring, on boarding and handover can be used to identify gaps and improve fitness for purpose.
  • The lens of culture: Determinants of quality through print-centric publishing processes are grounded in a rich cultural legacy, for example. Other specialists (IT, comms, etc.) also have their own, overlapping universes. Correct analysis of these and how they interact is indispensable.
  • The lens of total quality management (TQM): This lens includes quality development, business process improvement (BPI), and risk management. It can help both in the initial diagnosis (process maps) and in designing systems and procedures for continual improvement.
  • The lens of IT: Information technology management includes both agile methods as well as traditional requirements-and-specifications. Although such approaches on their own are unlikely to achieve the desired outcomes, their familiarity may facilitate acceptance and usage of the other lenses.

The remaining pieces of the puzzle involve standards, mixed methods, and deliverables.

Unified Knowledge Universe

Unified Knowledge Universe

Photo: Lenses rainbow (csaveanu/flickr).

Baby at seven months

Seven months

Reda SadkiQuotes

Alan Todd, Founder and CEO, Corp U:

Today’s workplace culture is evolving at a break-neck pace as technological advances, shifting demographics, and new economic realities force corporations to reorganize, on average, every seven months. The challenge is real. Fortunately, so are the solutions.

Source: The Science of Learning: How to Develop Mindsets for Success in the Workplace

Photo: My baby at seven months, on a beach in Soulac, France. August 2012. Personal collection.

Learning dashboard

Elements of a learning dashboard

Reda SadkiLearning strategy, Theory

“What is clear is that a learning rich culture will emphasize informal learning and more open learning designs rather than relying only on formal training approaches. The learning infrastructure consists of all of the formal, informal, and incidental activities, systems, and policies that promote individual, team, and organizational learning and knowledge creation.”

Elements of a learning dashboard-Watkins

Source: Watkins, K., 2013. Building a Learning Dashboard. The HR Review 16–21.

Belle Nuit 1920x1080

Webcasts, then and now

Reda SadkiEvents, Video

(No, this is not a post about the Apple keynote meltdown.)

When I started organizing live webcast events for the first time in 2006, they required extensive technical preparation, specialized software and hardware, and – most important – a group of really smart people gifted with more than a little bit of luck to pull off each event. Even as recently as 2011, I remember a time in Budapest when my young cameraman (one of a team of four) announced to me that his fancy P2 broadcast-quality camera could not connect to his equally-fancy webcasting software. I ended up hacking our MacBook Pro’s webcam, piloted remotely from another laptop using VNC… It was exciting to transform what had been a local, 19th Century-style lecture series into a series of global participatory learning events, but so much energy had to be expended on the technical issues that many people missed the point about the amazing affordances of technology to fundamentally transform how we teach and learn.

Participants in today’s blended learning event (a “15-Minute Global Health Practicum”) still experienced the technology mediation as interference. However, the blurry video and mediocre sound came from the crappy hotel wifi of our presenter, and nothing else. We were nevertheless able to focus on the substance (games for health) and the learning process  (a 5-minute Ignite presentation). I spent little more time testing and checking the Google Hangouts than I did visiting the meeting room where we gathered for the event in Geneva. Hence my effort went into reimagining how to solve some of the learning problems, not the technical ones.

For future events, the lesson for me is that figuring out how much of a burden  technology is likely to be (as it remains across the Digital Divide, for example) should really be a first diagnostic step when planning such events. Then you can determine where to focus your efforts.

Having said that, note to self: Crappy hotel wifi may be a rich people problem, but it really sucks. Sorry, Ben.