What is a rubric and why you should use it in global health education-small

What is a “rubric” and why use rubrics in global health education?

Reda Sadki Global health, Learning, Learning design, Theory

Rubrics are well-established, evidence-based tools in education, but largely unknown in global health.

At the Geneva Learning Foundation (TGLF), the rubric is a key tool that we use – as part of a comprehensive package of interventions – to transform high-cost, low-volume training dependent on the limited availability of global experts into scalable peer learning to improve access, quality, and outcomes.

The more prosaic definition of the rubric – reduced from any pedagogical questioning – is “a type of scoring guide that assesses and articulates specific components and expectations for an assignment” (Source).

The rubric is a practical solution to a number of complex issues that prevent effective teaching and learning in global health.

Developing a rubric provides a practical method for turning complex content and expertise into a learning process in which learners will learn primarily from each other.

Hence, making sense of a rubric requires recognizing and appreciating the value of peer learning.

This may be difficult to understand for those working in global health, due to a legacy of scientifically and morally wrong norms for learning and teaching primarily through face-to-face training. The first norm is that global experts teach staff in countries who are presumed to not know. The second is that the expert who knows (their subject) also necessarily knows how to teach, discounting or dismissing the science of pedagogy. Experts consistently believe that they can just “wing it” because they have the requisite technical knowledge. This ingrained belief also rests on the third mistaken assumption: that teaching is the job of transmitting information to those who lack it. (Paradoxically, the proliferation of online information modules and webinars has strengthened this norm, rather than weakened it).

Indeed, although almost everyone agrees in principle that peer learning is “great”, there remains deep skepticism about its value. Unfortunately, learner preferences do not correlate with outcomes. Given the choice, learners prefer sitting passively to listen to a great lecture from a globally-renowned figure, rather than the drudgery of working in a group of peers whose level of expertise is unknown and who may or may not be engaged in the activities. (Yet, when assessed formally, the group that works together will out-perform the group that was lectured.) For subject matter experts, there can even be an existential question: if peers can learn without me, the expert, then am I still needed? What is my value to learners? What is my role?

Developing a rubric provides a way to resolve such tensions and augment rather than diminish the significance of expertise. This requires, for the subject matter expert, a willingness to rethink and reframe their role from sage on the stage to guide on the side.

Rubric development requires:

  1. expert input and review to think through what set of instructions and considerations will guide learners in developing useful knowledge they can use; and
  2. expertise to select the specific resources (such as guidance documents, case studies, etc.) that will help the learner as they develop this new knowledge.

In this approach, an information module, a webinar, a guidance document, or any other piece of knowledge becomes a potential resource for learning that can be referenced into a rubric, with specific indications to when and how it may be used to support learning.

In a peer learning context, a rubric is also a tool for reflection, stirring metacognition (thinking about thinking) that helps build critical thinking “muscles”.

Our rubrics combine didactic instructions (“do this, do that”), reflective and exploratory questions, and as many considerations as necessary to guide the development of high-quality knowledge. These instructions are organized into versatile, specific criterion that can be as simple as “Calculate sample size” (where there will be only one correct answer), focus on practicalities (“Formulate your three top recommendations to your national manager”), or allow for exploration (“Reflect on the strategic value of your vaccination coverage survey for your country’s national immunization programme”).

Yes, we use a scoring guide on a 0-4 scale, where the 4 out of 4 for each criterion summarizes what excellent work looks like. This often initially confuses both learners and subject matter experts, who assume that peers (whose prior expertise has not been evaluated) are being asked to grade each other. It turns out that, with a well-designed rubric, a neophyte can provide useful, constructive feedback to a seasoned expert – and vice versa. Both are using the same quality standard, so they are not sharing their personal opinion but applying that standard by using their critical thinking capabilities to do so.

Before using the rubric to review the work of peers, each learner has had to use it to develop their own work. This ensures a kind of parity between peers: whatever the differences in experience and expertise, countries, or specializations, everyone has first practiced using the rubric for their own needs.

In such a context, the key is not the rating, but the explanation that the peer reviewer will provide to explain it, with the requirements that she provides constructive, practical suggestions for how the author can improve their work.

In some cases, learners are surprised to receive contradictory feedback: two reviewers give opposite ratings – one very high, and the other very low – together with conflicting explanations for these ratings. In such cases, it is an opportunity for learners to review the rubric, again, while critically examining the feedbacks received, in order to adjudicate between them. Ultimately, rubric-based feedback allows for significantly more learner agency in making the determination of what to do with the feedback received – as the next task is to translate this feedback into practical revisions to improve their work. This is, in and of itself, conducive to significant learning.

Learn more about rubrics as part of effective teaching and learning from Bill Cope and Mary Kalantzis, two education pioneers who taught me to use them.

Image: Mondrian’s classroom. The Geneva Learning Foundation Collection.

Epidemic preparedness through connected transnational digital networks of local actors-small

Pandemic preparedness through connected transnational digital networks of local actors

Reda Sadki Global health, Learning strategy

In the Geneva Learning Foundation’s approach to effective humanitarian learning, knowledge acquisition and competency development are both necessary but insufficient. This is why, in July 2019, we built the first Impact Accelerator, to support local practitioners beyond learning outcomes all the way to achieving actual health outcomes.

What we now call the Full Learning Cycle has become a mature package of interventions that covers the full spectrum from knowledge acquisition to implementation and continuous improvement. This package has produced the same effects in every area of work where we have been able to test it: self-motivated groups manifesting remarkable, emergent leadership, connected laterally to each other in each country and between countries, with a remarkable ability to quickly learn and adapt in the face of the unknown.

In 2020, we got to test this package during the COVID-19 pandemic, co-creating the COVID-19 Peer Hub with over 6,000 frontline health professionals, and building together the Ideas Engine to rapidly share ideas and practices to problem-solve and take action quickly in the face of dramatic consequences of the new virus on immunization services (largely due to fear, risk, and misinformation). By January 2021, over a third of Peer Hub members had successfully implemented their immunization service recovery project, far faster than colleagues who faced the same problems but worked alone, without a global support network. Once connected to each other, these country teams then organized inter-country peer learning to help them figure out “what works” for COVID-19 vaccine introduction and scale-up.

Such a holistic approach is about mobilizing and connecting country-based impact networks that reach and involve practitioners at the local levels, as well as national MoH leaders and planners – quite different from conventional approaches (whether online or face-to-face) to building capacity and preparedness.

TGLF’s global health network and platform reach significant numbers of practitioners at all levels of the health system. It is not only the number of people who participate (47,000 as I write this) but also the depth of engagement and diversity of contexts that they work in. Globally, 21.2% face armed conflict; 24.5% work with refugees or internally-displaced populations; 61.6% work in remote rural; 47% with the urban poor; 35.7% support the needs of nomadic/migrant populations. This is across 110 countries, with over 70 percent in “high burden” countries. Many have deep experience in responding to epidemic outbreaks of all kinds. Health professionals who join come from all levels of the health system, but most are (logically) from health facilities and districts, the bottom of the health pyramid.

Through the network and platform, they build lateral connections, forging bonds not only of knowledge but also of trust. They do this not because they are from the same profession, but primarily (we believe) because they face similar challenges and see the benefit of sharing their experience in support of each other. Engagement is voluntary (ie people opt in and contribute because they want to), with no per diem or other extrinsic incentives offered. 

Individuals develop and implement corrective actions to tackle the root causes of the challenges they are taking on, drawing on both peer learning and the best available global guidelines. For the IA2030 Movement, our largest initiative so far, participants are simultaneously implementing 1,024 projects in 99 countries, learning from each other what works, sharing successes, lessons learned, and challenges. Here are four examples of what collective action through digital networks looks like :

  • In Ghana, TGLF’s alumni (including national and regional MoH EPI directors) decided to organize online sessions country-wide to share the latest information about COVID-19 with local staff, starting in April 2020. They had learned how to use digital tools to find the best available global knowledge and to combine it with their local expertise and experience to inform collective action.
  • In Burkina Faso, the national EPI manager entrusted the first “masked” vaccination campaign to the TGLF alumni team, which has organized itself country-wide, with over half of alumni working in conflict-affected areas. He told me no one else had the network and the capacity for change to figure out quickly how to get this right.
  • In the Democratic Republic of Congo, the TGLF alumni team is increasingly being asked by national EPI to contribute to various activities, due to their effectiveness in connecting and coordinating. The alumni network is country-wide and includes many from very remote areas. When Monkeypox was reported in Europe and North America, we were already seeing a steady stream of information through the DRC and other country networks.

We believe that this continuous learning and action is actually the definition of preparedness. Trying to imagine preparedness and response to new pandemics using old, failed methods of training and capacity building – whether face-to-face or online – is both dangerous and irrational.

Image: Remote villages illuminated by rays of light, with mountains beyond mountains in the background. The Geneva Learning Foundation Collection.

Reinventing the path from knowledge to action in global health

Reda Sadki Global health, Learning strategy, The Geneva Learning Foundation

At the Geneva Learning Foundation (TGLF), we have just begun to share a publication like no other. It is titled Overcoming barriers to vaccine acceptance in the community: Key learning from the experiences of 734 frontline health workers.

You can access the full report here in French and in English. Short summaries are also available in three special issues of The Double Loop, the Foundation’s free Insights newsletter, now available in both English and French. The report, prefaced by Heidi Larson who leads the Vaccine Confidence Project, includes DOI to facilitate citation in academic research. (The Foundation uses a repository established and maintained by the Geneva-based CERN for this purpose.)

However, knowing that academic papers have (arguably) an average of three readers, we have a different aspiration for dissemination.

As a global community, we recognize the significance of local action to achieve the global goals.

The report documents vaccine confidence practices just weeks before the introduction of COVID-19 vaccines. It is grounded in the experience of 734 practitioners from local communities, districts, regions, and national teams, who developed case studies documenting a situation in which they were able to successfully lead individuals and groups toward better understanding and acceptance of the benefits of vaccines and vaccination.

Immunization staff from all levels of the health system became citizen scientists, active knowledge-makers drawing on their personal experience of a situation in which they successfully overcame the barriers to vaccine acceptance in the community.

Experiential learning offers a unique opportunity to discover unfiltered experiences and insights from thousands of people whose daily lives revolve around delivering immunization services. But what happens once experience has been shared? What is to be done with what we learn?

Sharing this report, we have found, has triggered remarkable dialogue and led to the co-creation of a steadily growing collection of new practices actually used to build vaccine confidence (as opposed to the many theoretical frameworks on the topic), submitted through our new Insights system. New stories and their analysis are being shared back with local practitioners and with TGLF’s Insights partners, fostering continuous learning that is an action imperative of a strong learning culture. (For Insights, we work with Bridges to Development, the Centre for Change and Complexity in Learning (C3L), and the International Vaccine Access Center at Johns Hopkins.)

In the coming weeks, we will be inviting 10,000 leaders of the Movement for Immunization Agenda 2030 to share this report to their colleagues, teams, and organizations (in both ministries of health and civil society organizations). They will be sharing back their own insights on how the findings can be used to improve demand for vaccines – and colleagues who listen to their presentation of the report will also be able to share back what they learn, connecting with each other through our Insights system.

Then, the Foundation’s Impact Accelerator will track if and how insights from this report are linked to reported positive outcomes, and we should be able to document this, at least in some cases. This will not only foster double-loop learning but also explicitly link learning to implementation and results.

In this way, local practitioners will be putting to use global knowledge grounded in their local experiences, for their own needs. We believe that this provides a complementary, more organic mechanism than current top-down processes for developing normative guidance driven by global assumptions and priorities.

As Kate O’Brien, WHO’s Director of Immunization, said during a recent Insights Live session: “The global role on immunization is actually to bring together everything that is known by people at the grassroots level. That’s where the action is. Global guidance is basically one means to share knowledge and expertise that’s coming from the grassroots level around the world with others who may not have had that experience yet.”

What we are doing with this report is part of a larger initiative to build the IA2030 Movement Knowledge to Action Hub. New knowledge produced by local practitioners will be available as both static and living documents that local and global practitioners can add their inputs to, at any time. This Hub will be launched at Teach to Reach 7 on 14 October 2022, with over 13,000 local practitioners registered for this event.

Image: Many paths to moving mountains. The Geneva Learning Foundation Collection.

Mindjourney-online learning network-abstract-colorful

Which is better for global health: online, blended, or face-to-face learning?

Reda Sadki Learning, Research, Theory

Question 1. Does supplementing face-to-face instruction with online instruction enhance learning?

No. Positive effects associated with blended learning should not be attributed to the media, per se. (It is more likely that positive effects are due to people doing more work in blended learning, once online and then again in a physical space.)

This is the conclusion of the U.S. Department of Education’s “Evaluation of evidence-based practices in online learning: a meta-analysis and review of online learning studies” in September 2010. You can find the full document here.

Question 2. Is the final academic performance of students in distance learning programs better than that of those enrolled in traditional FTF programs, in the last twenty-year period?

Yes. Distance learning results in increasingly better learning outcomes since 1991 – when learning technologies to support distance learning were far more rudimentary than they are now.

This is the meta-analysis done by Mickey Shachar and Yoram Nuemann reviewing twenty years of research on the academic performance differences between traditional and distance learning: summative meta-analysis and trend examination in the Merlot Journal of Online Learning and Teaching. Vol 6, No. 2, June 2010.

A long time ago, I asked Bill Cope what the evidence says about the superiority of online learning over blended and face-to-face. My experience had already consistently been that you could achieve so much more with the confines and constraints of physical space removed.

Of course, it is complicated. But Bill pointed me to the two meta-analyses published in 2010 that provided fair and definitive evidence to answer two questions. Yet, in the field of global health, the underlying assumption of funders and technical partners remains that there is no better way to learn than by flying bodies and materials at high cost. This is scientifically and morally wrong, does not scale, and has created a per diem economy of perverse incentives. It is wrong even if it is easy to understand why international trainers and trainees both express a preference for the least effective, low volume, high cost approach to learning.

Image: Online learning networks. Personal collection generated by Mindjourney.

How we make sense of complexity, together, at the Geneva Learning Foundation

Reda Sadki The Geneva Learning Foundation

Unique learning experiences generate not just data points but complex stories about what it takes to make change actually happenBy connecting the dots between ideas and implementation, we can zero in on the highest-value insights. 

Our Insights Unit uses the latest advances in learning analytics to map how ideas and practices shared between countries and system levels make a difference. 

The Unit facilitates international partners to work hand-in-hand with local practitioners. 

In addition to thousands of local practitioners contributing and using insights to drive shared learning and action, our Insights Unit’s work is being used by leading global agencies. Examples include: 

  • Effective strategies to overcome vaccine hesitancy in districts and health facilities (BMGF) 
  • Motivation of local health professionals for COVID-19 vaccination (Gavi, the Vaccine Alliance) 
  • Learning culture as a key driver of frontline health worker performance (Wellcome Trust) 
  • Gender barriers, vaccine confidence, and other immunization challenges (WellcomeTrust) 
  • Analysis of implementation of recovery plans in TGLF’s COVID-19 peer support programme (WHO and USAID Momentum) 

We are exploring affordable, practical ways to extract meaning from large data sets 

To learn more about the Geneva Learning Foundation (TGLF), download our brochure, listen to our podcast, view our latest livestreams, subscribe to our insights, and follow us on Instagram, LinkedIn, Facebook, Twitter, and YouTube. Or introduce yourself to our Partnerships team.

Teach to Reach: peer learning at scale

Reda Sadki The Geneva Learning Foundation

Teach to Reach are fast-paced, dynamic digital events connecting local and global practitioners to each other in a new, potentially transformative shared dialogue. 

Teach to Reach and other TGLF special events rally thousands, serving as powerful moments of inspiration, providing the amazing sensation of being connected with thousands of fellow, like-minded people and the impetus to transform this feeling into shared purpose and action. 

Meet, network, and learn with colleagues from all over the world 

Successive editions of TGLF’s flagship event series, “Teach to Reach: Connect”, enabled a cumulative total of 27,000 health professionals to share experiences, test approaches, and identify solutions with international experts listening and learning with them. 

To learn more about the Geneva Learning Foundation (TGLF), download our brochure, listen to our podcast, view our latest livestreams, subscribe to our insights, and follow us on Instagram, LinkedIn, Facebook, Twitter, and YouTube. Or introduce yourself to our Partnerships team.

How does the Geneva Learning Foundation’s approach break the norm?

Reda Sadki The Geneva Learning Foundation

100% digital 100% human: using the latest learning technologies and interfaces, we adapt our digital networking interfaces to learner needs and habits to augment their digital and networking capabilities. 

Grounded in experience: we foster problem-solving that values both participants’ lived experience and the world’s best available global knowledge. 

We open access: participation can be opened for all, across geographic, sectoral or institutional barriers. 

New knowledge is created through peer learning: national and international practitioners sharing experience, giving and receiving feedback, and using this new knowledge to solve problems together. 

We build trust and mutual respect: safe spaces encourage authentic sharing of experiences to learn what actually works, how, and why. 

Driven by intrinsic motivation: proven high engagement rates with no per diem or other extrinsic incentives. 

Sustainability built-in: 78% of TGLF programme participants feel “capable” of using TGLF’s methodology for their own needs, and 82% want to organize their own activities using it with their colleagues. 

To learn more about the Geneva Learning Foundation (TGLF), download our brochure, listen to our podcast, view our latest livestreams, subscribe to our insights, and follow us on Instagram, LinkedIn, Facebook, Twitter, and YouTube. Or introduce yourself to our Partnerships team.

The Geneva Learning Foundation: Localizing programming and grounding policy

Reda Sadki The Geneva Learning Foundation

By defying distance to connect with each other, practitioners expand the realm of what they are able to know beyond their local boundaries. 

Listening to these diverse voices and experiences is critical to inform programming, policy and decision-making and build bridges across sectoral silos and other boundaries, by providing: 

  • A direct, unmediated connection to the priorities and challenges of frontline staff, as well as their perceptions of key obstacles and enablers of progress. 
  • Impactful learning and knowledge building by and for frontline responders and practitioners. 
  • A “reality check” to assess whether current global assumptions match those of frontline workers. 
  • A “test bed” to co-design, develop and pilot tools or resources. 

Thousands of ideas are turned into action, results, and impact 

In every TGLF programme, practitioners develop action plans and then report to each other as they implement, documenting results, outcomes, and impact to help each other. 

Such peer accountability has proven more reliable, in some cases, than conventional monitoring and evaluation mechanisms. 

For individuals, TGLF enables: 

  • Increased knowledge of low-cost digital tools for learning and networking at scale. 
  • Opportunities to blossom as a leader, no matter who you are or where you are. 
  • Sense of community across system level, sectoral, geographic and institutional boundaries. 

Measurable impact in countries: Examples of outcomes tracked in immunization since July 2019 

  • Following up on finding and vaccinating zero dose and defaulting children 
  • Tracking and vaccinating migrant populations 
  • Setting up a Missed Opportunities in Vaccination (MOV) system to ensure eligible children present at outpatient/other PHC “stations” in a facility receive vaccinations 
  • Improving geographic equity by increasing outreach sites in hard- to-reach areas 
  • Increasing frequency of services in higher volume urban facilities 
  • Using community engagement approaches to bring on board leaders to support immunization, who were previously opposed. 

To learn more about the Geneva Learning Foundation (TGLF), download our brochure, listen to our podcast, view our latest livestreams, subscribe to our insights, and follow us on Instagram, LinkedIn, Facebook, Twitter, and YouTube. Or introduce yourself to our Partnerships team.

The Geneva Learning Foundation: Scale, reach, and sustainability

Reda Sadki The Geneva Learning Foundation

In its first years of operation, the Geneva Learning Foundation (TGLF) built digital infrastructure to foster and support several global networks and platforms connecting practitioners.

Communities supported included:
•  immunization and primary health care professionals,
•  humanitarian workers advocating gender equality during disasters and other emergency operations,
•  doctors, other health workers, and communities addressing neglected needs in women’s health, and
•  health workers tackling neglected tropical diseases.

This digital infrastructure enabled TGLF to rapidly respond to the challenges of the COVID-19 pandemic.

In the first two years of the pandemic, a team of three people developed and implemented… 

To learn more about the Geneva Learning Foundation (TGLF), download our brochure, listen to our podcast, view our latest livestreams, subscribe to our insights, and follow us on Instagram, LinkedIn, Facebook, Twitter, and YouTube. Or introduce yourself to our Partnerships team.

The Geneva Learning Foundation: Spanning the full spectrum of learning

Reda Sadki The Geneva Learning Foundation

We empower practitioners to tailor learning experiences that fit their own needs to drive change: Participants do not  stop work to learn, every step of the process is embedded in and focused on their daily work.

Typical learning events include:  

“Hackathons”: 2 to 4 days fast-paced context and challenge analysis and idea generation

“Peer learning exercises” : 2 to 4 weeks, on and offline facilitated learning among and between practitioners and international experts, including knowledge sharing, situational analysis and action planning.  

 “Full Learning Cycles”, a nurturing space for learners and leaders over several months to explore and take action together, identifying common challenges, generating and sharing ideas, testing innovative solutions, and implementing action plans.

To learn more about the Geneva Learning Foundation (TGLF), download our brochure, listen to our podcast, view our latest livestreams, subscribe to our insights, and follow us on Instagram, LinkedIn, Facebook, Twitter, and YouTube. Or introduce yourself to our Partnerships team.