GLOBAL CAPACITY BUILDING COURSES

– Written by Karen Milton and Paul Kelly, United Kingdom and Justin Richards, Australia

Physical activity is considered a “best buy” for public health¹. Being physically active is associated with reduced risk of a range of health conditions including cardiovascular disease, stroke, diabetes, and certain types of cancer¹. Physical activity can improve sleep, lead to better physical and executive function, and can reduce the risk of mental health problems such as anxiety and depression¹. Although research evidence on the physical and mental health benefits of being active is well established and growing, data show that worldwide activity levels remain low². Furthermore, although many countries have developed policies and actions to promote physical activity, levels of insufficient physical activity have remained stable over the past 15 years².

The Global Action Plan on Physical Activity (GAPPA), adopted at the World Health Assembly in May 2018, emphasised political support for the promotion of physical activity at the highest level³. GAPPA provides a framework for action, which includes 20 policy actions across four areas – active societies, active environments, active people, and active systems. Several actions state the importance of monitoring and evaluation to ensure accountability as well as informing policy and practice. Ensuring researchers, practitioners and policymakers have competency in evaluation is critical for demonstrating implementation and effectiveness of national and sub-national actions. 

The need for greater capacity in physical activity and health has been recognised previously and courses have been delivered in various formats in different regions of the world. For example, a postgraduate course on research directions and strategies was developed by the Centres for Disease Control and Prevention (CDC) in the USA. The 8-day course aims to build competencies in topics such as study design, physical activity measurement, dose-response issues, interventions, and grant writing⁴. Alongside this course, the CDC delivers a parallel course for practitioners involved in community-based initiatives. The content of the practitioner course covers topics such as models for health promotion, best practice intervention strategies, partnership development, and policy and environmental supports for physical activity. The practitioner course also includes a programme evaluation component⁵. These courses, which take place in North America (but include international delegates), were delivered annually from 1996 to 2013, and in 2016 and 2018. 

Building on the CDC North American courses, the International Society for Physical Activity and Health (ISPAH) developed a similar course for the Latin American region. The rationale for taking this type of course to other regions was to reduce travel costs for delegates and therefore broaden reach. To date ISPAH has delivered its physical activity and health course in Brazil (2015), Mexico (2015, 2016) and Chile (2016)⁶. Similar efforts have also been attempted in other regions on a more ad hoc basis. In addition, the Prevention Research Collaboration at the University of Sydney has developed an online introductory course on the topic of physical activity and health. Again, the rationale for using an online delivery platform was to broaden the reach of these types of courses globally, and particularly among low- and middle-income countries⁷. 

While some of the aforementioned courses have contained an evaluation component, this has typically been a small part of the course content. No previous course had focused solely on building capacity on evaluation in physical activity and health. Evaluation is critical to realising the GAPPA, with Action 4.3 specifically highlighting the need for national and institutional research and evaluation capacity. Without evaluation we are unable to understand what works to get more people active and why. Importantly, programme evaluation in a real world context is needed to understand what actions can be implemented at scale. It is only by identifying feasible at-scale interventions that we can begin to change population levels of physical activity. Therefore, the Pragmatic Evaluation in Physical Activity and Health course was established to fill this important gap in training provision. It was also targeted at directly bridging the gap between researchers, practitioners and policy makers through concurrent and collaborative capacity building (i.e. co-attendance at one course, bringing together representatives from each of these stakeholder groups).  

The concept of ‘Pragmatic Evaluation’ is based on seven underlying principles which de-lineate it from (academic) research: 

1. Pragmatic Evaluation has similarities to Realist Research and methods⁸, with a core focus on context and complexity. Experimental research designs (randomised controlled trials for example) are usually not effective at handling these issues in behavioural science and health promotion; 
2. Pragmatic Evaluation recognises that the most appropriate design and methods for an evaluation may not be the scientific ‘gold-standard’ (i.e. the evaluation may necessarily adopt a non-RCT design, the tools available may not be validated, and the sample may not be representative). Pragmatic alternatives (and compromises) may be appropriate and provide information that is more useful. In many cases, adopting a Pragmatic approach allows us to obtain more contextually relevant results, by optimising the available resources. The availability of Pragmatic Evaluation designs also make evaluation more feasible, increasing the likelihood of obtaining useful evidence to inform policy and practice; 
3. Pragmatic Evaluation often occurs with limited time, budget and resources;
4. Pragmatic Evaluation relies on the use of theory based assumptions to build logic models of varying complexity;
5. Testing the logic model is a core tenet of Pragmatic Evaluation; this places a focus on process evaluation, but does not exclude assessment of programme effectiveness; 
6. The Pragmatic Evaluation approach may allow an understand-ing of why an intervention/ action did or did not have the expected effects, and can inform judgements of replicability, scalability, and generalisability; 
7. Pragmatic Evaluation allows us to work with practitioners and policy makers in a way that is meaningful to them, to positively influence health promotion.

The Pragmatic Evaluation course was founded by the co-authors in 2015. The course has two delivery models. The first model involves an open application process, and operates on a cost recovery basis.  Attendees at these courses are selected based on the quality of their application, with a large focus placed on interest and potential to apply pragmatic and creative solutions to complex problems. These courses are delivered in association with an international physical activity and health conference and this has several advantages. Firstly, it means that the course is delivered in different parts of the world, helping to ensure global reach. Secondly, both delegates and faculty can combine participation in the course with attending the conference, minimising additional travel costs and adding value to the travel expenses incurred by individuals and their organisations. We strive to ensure that the faculty for each course consists of approximately 50% local experts and 50% international experts. Faculty members contribute to the course ‘in kind’ (with travel covered but for no fee). Any profit made from the course is used to provide scholarships and cover running costs for future courses. Scholarships are particularly designed for people from low- and middle-income countries. This was the original delivery model for the course and courses using this 'standard' format have been delivered in Edinburgh, Scotland (2015), Cape Town, South Africa (2016), Vancouver, Canada (2017), London, England (2018) and Olomouc, Czech Republic (2019). 

The success of the Pragmatic Evaluation course has led to requests from organisations to deliver specialised courses, tailored to the organisation’s needs. This second model involves consultation with organisations to understand evaluation needs and to tailor the resources and teaching accordingly. No application process is involved, rather the organisation identifies and nominates staff members, partners, or other participants who they feel would benefit from attendance. There is a cost to the organisation for the development and delivery of the course, but no cost to individual participants. The first pilot of a specialised course was conducted in Doha, Qatar, on behalf of Aspetar, in February 2016. A total of 15 people working within Aspire Zone in Qatar attended the course. The course evaluation showed that the organisers/hosts and the participants felt the course met its objectives and built local capacity to evaluate community-based programmes. A further specialised course was delivered in Wellington, New Zealand, in May 2019.

The Pragmatic Evaluation course has been delivered to 139 participants from 35 countries. Alumni of the Pragmatic Evaluation course is growing and we are in the process of establishing an online platform to support continued networking among delegates and faculty. Furthermore, we have been working with course alumni to co-create a Level 2 (advanced) course on Pragmatic Evaluation. We delivered a pilot Level 2 course in Bangkok, Thailand, in 2016, which was attended by alumni from previous courses that had been delivered, including the two standard Pragmatic Evaluation courses in Edinburgh and Cape Town and the specialised course in Doha. 

The Pragmatic Evaluation course is addressing an important gap in capacity building opportunities in the field of physical activity and health. Each of the standard courses has been over-subscribed and we have received numerous requests to deliver specialised courses, demonstrating the demand for this type of training. We continue to revise the scope of the course in response to delegate feedback and the growing evidence base on how to conduct robust evaluation of physical activity interventions in real-world settings. We believe that continued delivery and expansion of the suite of Pragmatic Evaluation courses and activities will lead to a stronger evidence base on real-world physical activity promotion and an increased ability of all countries globally to demonstrate implementation and effectiveness of national and sub-national actions.

Karen Milton Ph.D.

Lecturer in Public Health 

University of East Anglia

Norwich, United Kingdom

Paul Kelly Ph.D.

Lecturer in Physical Activity for Health

University of Edinburgh

Edinburgh, United Kingdom

Justin Richards Ph.D.

Senior Research Fellow

University of Sydney

Sydney, Australia 

Adjunct Research Fellow

Victoria University of Wellington

Wellington, New Zealand

Contact: pragmatic.evaluation@gmail.com

References

1. 2018 Physical Activity Guidelines Advisory Committee. 2018 Physical Activity Guidelines Advisory Committee Scientific Report. Washington, DC, USA; 2018. 
2. Guthold R, Stevens G, Riley L, Bull F. Worldwide trends in insufficient physical activity from 2001 to 2016: a pooled analysis of 358 population-based surveys with 1·9 million participants. Lancet Glob Heal. 2018;6(10):1077–86. 
3. World Health Organization. Global Action Plan on Physical Activity 2018 - 2030 - More active people for a healthier world. Geneva, Switzerland; 2018. 
4. University of South Carolina. Postgraduate Course on Research Directions & Strategies [Internet]. 2018 [cited 2018 Dec 4]. Available from: http://www.asph.sc.edu/paph/research.html
5. University of South Carolina. Practitioner’s Course on Community Interventions [Internet]. [cited 2018 Dec 4]. Available from: http://www.asph.sc.edu/paph/practitioner.html
6. International Society for Physical Activity and Health. Training development opportunities [Internet]. 2018 [cited 2018 Dec 4]. Available from: http://www.ispah.org/training-development-opportunities/
7. Engelen L, Bauman A, the Physical Activity Network group. Capacity building in physical activity and non-communicable disease prevention: a low-cost online training course can reach isolated practitioners. Glob Health Promot. 2015;24(1):27–33. 
8. Pawson R, Tilley N. Realistic Evaluation. London: Sage Publications.