The value of engagement

Engaging with different stakeholders is regarded as very important, but there is little knowledge about what works. Sharing experiences about engagement was the central theme of the plenary workshop presented by John Young of the Overseas Development Institute on Friday morning. Increasingly, donors want to invest as much effort as getting research into use, as investing in research itself.

To illustrate the complexity of public engagement, the group discussed “what do you think would be convincing evidence of the value of public engagement?” which showed the diverse ways of looking at this type of evidence and what is needed for engaging with the public. Among the many suggestions indicating evidence of the value of public engagement were: participation of the community; trust of the community; two-way communication and greater debate in the community; positive health/behaviour outcomes flowing from public engagement; visible evidence in the form of buildings, programmes and services; positive policy change; legislative reform; new links and networks.

Resuming the session after the break-away, John discussed one of the Overseas Development Initiative programmes, RAPID, which promotes greater use of research based evidence in development policy and practice (www.odi.org.uk/rapid). The group engages with a wide group of actors through a range of tools including the media, web, publications, meetings and collaborative work. Policy process is “fantastically complex” – everyone tries to influence everyone else. Research-based evidence generally plays a small role in policy development, (see presentation for more detail), although it can have significant impact, as seen for example in health service reform as seen in Tanzania which led to a 43-46% reduction in infant mortality.

“In terms of engagement, it is important to know what is need to be achieved. Indicators should be SMART in order to achieve this. Change takes a long time and many projects fail when the input ceases. If you want to see lasting benefits of research, we need to see changes of behaviour of stakeholders, whether it be communities, researchers, and policy makers,” said John.

There are several tools in terms of measuring and assessing change (see IDRC’s website: www.idrc.ca). Learning through stories (what was the situation, what was the challenge, what was done, what was the result, what lessons can be drawn) is one way of learning and documenting stories of change. (www.odi.org.uk/rapid/toolkits). Horizontal evaluation (www.chsrf.ca) is done very well by the Canadian health research foundation to evaluate their programmes.

ODI also does “after-action” reviews to evaluate how projects could have been done better (what was supposed to happen, what actually happened, why was there a difference, and what can we learn from it) which is simple and quick to do. Case studies (how did evidence shape policy decisions) and episode studies which retrospectively tracks back from policy change can be useful as well. ODI combines the classical case study approach and retrospective approach, with “RAPID outcome mapping” to look at behavioural change. A social network analysis (www.rcuk.ac.uk/cmsweb/downloads/rcuk/pulications/evaluationsguide.pdf) is very helpful too to assess who can influence who needs to be influenced.

The group then split into four to discuss what sort of evidence was needed in individual projects, what sort of evidence was needed about other public engagement projects, how should that evidence be obtain, and who should do what. The groups focused on policy, community, media and creative/other. Group feedback will be posted to the blog at a later stage.

posted by Greer van Zyl

Advertisements

Engaging the media to influence policy

By Greer van Zyl, Healthwrite

Media can play a critical role in influencing policy positively, according to Jo Carpenter from PANOS (UK). In the final media engagement session on Thursday afternoon, she outlined research which PANOS had done in Uganda and Jamaica, selected because of their high proportion of media independence and diversity, where coverage had influenced policy. Four case studies (two in each country) were examined, focusing on labour practices, developmental status of children (Jamaica), the Mabira forest, and use of polythene bags (Uganda) which had all had good media coverage. The research involved 12 in-depth interviews with key informants and content analysis methodology. The study found that researchers were equivocal on their roles in promoting the research in the media; some actively engaged the media; others were more reticent.

The key findings were that media were critical to influencing policy, but didn’t work in isolation. Debates were not initiated by journalists, but sustained by them. Civil society was most successful in catalysing media debates because of their skills and resources. Professional links were seen as critical between researchers, policy-makers, civil society and the media. In all cases, policy-makers saw the media as a source of information and a window on public opinion. The influence of a media debate was not dependent on its quality.

These findings show that communication strategies need to be systematically integrated into all research initiatives. Researchers need to be trained; professional relationships between media, civil society, policy and researchers need to be strengthened; and capacity building of the media is important, specifically for talk show hosts. Editors need to be engaged because they don’t have the time, resources, power to transfer skills to reporters through standard journalism processes.

Jo also discussed how the RELAY programme of PANOS works with developing countries to connect journalists and editors with researchers to enhance their capacity and communicate research. PANOS supports research institutes in their media strategies, and hosts events which bring academics and journalists together. Before working in a new country, PANOS scopes the media and research environments to spot gaps and identify lead journalists and researchers who are currently working with media. (See www.panos.org.uk/relay for more details)

Luisa Massarani from Brazil discussed the importance of radio to engage people in science as it has high penetration in the country. There were only a few science radio programmes in Brazil, and consequently people were not using radio as a source of information on science and technology.

A project was launched to engage 10-year-old children in science and health issues through a radio programme which trains the children in radio and science communication by deciding the format, issues to be covered etc. Another project deals with a radio science programme called “Electron” which integrates science and culture once a week in short snippets which features music and poetry related to science. Programmes have dealt with AIDS (data, vaccine development and Brazilian research on AIDS) and genetics (the use of genetic information on health insurance and jobs; research in genetics and tropical diseases) which is packaged with appropriate music (including heavy metal “Pity and the bloody chromosomes”). (www.museudavida.fiocruz.br/electron)

Communication does not happen in a vacuum or in a linear fashion, explained Monica Bonaccorso, a lecturer and senior research associate at Goldsmiths, University of London. “The question is how to really reach people through the media and how effective is it; what are we communicating?”

She explained that many communication models and expectations of communicators is that once a message has been conveyed, it will be accurately received, which is problematic as it does not account for feedback and cultural contexts within which messages are received. “Much of the effort we put into communicating science may be wasted because we are not communicating in the right way, which recognises that communication is an evolving process and is the product of learned events defined by social frameworks.” She said it was incredibly difficult to predict the effect of communicative efforts by studying audiences and doing evaluation because communication is so complex.

Julie Clayton, UK-based freelancer, trainer and organiser of the World Conference of Science Journalists taking place in London in 2009 (www.wcsj2009.org), spoke candidly about SciDev.net’s involvement in building science journalism capacity through training and workshops to promote the standards of science journalism globally.

She also discussed the World Federation of Science Journalists (WFSJ) which is an umbrella organisation promoting peer-to-peer mentoring in Africa and the Middle-East which aims to improve the quality of science reporting and improve networking. “It’s not only about linking mentors (who are practicing journalists) and mentees, it’s also about new opportunities such as setting up journalistic associations which can twin with others,” said Julie. The federation has created an online science journalism course: (see www.wfsj.org/course). Outcomes of the programme include better stories, more responsibility, more space for coverage, and recognition by editors.

Strategies in policy engagement

In the second of two sessions on policy engagement, three speakers highlighted various strategies in engaging policy-makers during the Thursday parallel session before lunch. Geoffrey Lairumbi from Kemri-Wellcome in Kenya presented a snapshot of the different players involved in research and their expectations on benefit sharing. The “what, who, and when” of the social value of research was explored in 50 interviews of stakeholder groups in Kenya including researchers, policy makers and industry.

The key question was “what is the value of the research?” The community voices appeared to be interested in short-term benefits such as regular monitoring, general assistance, and strengthening of services. “From what they have been telling me, as we get into that research, there are those small benefits that they will get free treatment”. In terms of researchers, they want training, career development, patents and knowledge. “When you are looking at a research project, the primary thing is that you to think in terms of yourself and then others. The question is “what is in this project for me.” Policy makers were interested in community mobilisation and capacity building among other things.

Due to challenges each group has to circumvent, they each appear to do their own thing based on what they think is important. There are also practical barriers including the complexity and longevity of the research process and actor interests. Another issue for public engagement was whether there should be continual and long-term engagement vs once-off study specific encounters. Should the existing institutions be strengthened, or are we looking at new “knowledge brokers”?

Rose Oronje of the African Population and Health Research Centre (APHRC) in Kenya discussed the bold and difficult task of targeting parliamentarians in a presentation on “Sharing research with parliamentarians and journalists – lessons learnt”.

The APHRC is a Pan-African organisation conducting research on population, health and education to inform policy and practice. Policy makers are regarded as parliamentarians, technocrats and civil servants who write policies, as well as district level civil servants who prepare plans. Members of Parliament (MPs) have an oversight role to solve societal problems, while journalists expand development information, thereby promoting its uptake.

The programme actively involves MPs in research dissemination meetings, and forms partnerships around issues. Champion MPs are identified by multiple organisations agreeing on a common issue framed to highlight benefits to MPs by using research in their work. Prior to the elections in Kenya, the Centre highlighted to MPs their future benefits by investing in the youth resulting in the establishment of Youth Empowerment Centres using constituent development funds. Clerk assistants (civil servants) responsible for parliamentary committees are key stakeholders in getting MPs’ attention. The Centre uses research to demonstrate the extent of the problems facing MPs’ constituencies, and how it can help development.

“We clearly state the role MPs can play in addressing challenges, and we try to understand the problems they have in accessing and using research evidence,” said Rose. Some challenges included MPs not knowing which research institutions were credible; lack of internet access; interest in amassing wealth rather than evidence-based policy.

In working with journalists, the APHRC works through journalistic associations, targets individual journalists, makes awards of excellence, writes media releases, conducts training and sensitisation sessions, field visits and writes media features/letters to the editor/piggy-backing on other issues. The Centre’s achievements include the establishment of the Kenya Parliamentary Network on Population and Development, and for the first time in Kenya, a budget line for Family Planning services was included in Kenya’s national budget. Lessons learnt include that sustained engagement is critical; research needs to be framed in specific ways; and working in partnerships has more impact than working individually.

Maurice Yaogo, GREFSaD, Burkina Faso showed a French video on “Promoting evidence-based decision making in health, science and new technologies”. Through community-theatre, it featured the problems women face in Burkina Faso through community-theatre, after which the actors engaged with the audience. About 80% of the women are poorly educated and this method is a good way of conveying key health messages. The video was shown to policy-makers prior to being shown to the community, which illustrated how various media can be used for disseminating information to policy-makers.

John Young of the Overseas Development Institute ventured that lessons can be learnt from other disciplines in terms of policy engagements.

1) Partnership brokering through a partnership broking accreditation scheme (PBAS);
2) Evidence policy-making in the UK to encourage bureaucrats to be more evidence-based: the policy hub;
3) The innovation systems approach recognises that there’s a need for a rich network of actors and connections between them for research based evidence to flow into policy in practice;
4) The World Bank Knowledge Economy Approach is another way of looking at innovative systems, to assess the ability of countries to capitalise on the knowledge they produce. It is focused on emerging countries in Europe; and
5) How coalitions can affect change is covered by DFID in Nigeria called “Coalitions for Change” which include researchers, policy makers and the private sector to promote better development policy in practice.

(web links of these policy engagement sites will be added in due course)

Posted by Greer van Zyl, Healthwrite

Is engagement ethically neutral?

Even when we try to get the rules of engagement right, we can get it wrong. What are the guiding principles around the ethics of engagement? Whose responsibility is this? In the first parallel session on Thursday morning, these issues were explored in an experimental role play chaired by Wellcome Trust’s Bella Starling.

Seven members of the audience were selected to pose as a research ethics committee considering public engagement activities into a previously approved proposal on a new medical intervention to induce abortion. The proposed public engagement activity was in two phases: firstly, a discussion through a national radio debate about sexual health research, particularly with regard to abortion and when it might need to be carried out with voting on the issue, followed by phase two, where the research results would be disseminated through intermediaries in the local community where the research took place, and through radio.

The audience was divided into teams with specific roles which included a research scientist, local clinician, social scientist, community chief, policy voice, community representative, public engagement practitioner. A representative from each team then participated in a role-play debate. It emerged from the debate that different stakeholders come from particular paradigms in terms of ethics and public engagement. Public engagement is not ethically neutral, the group found.

Questions which arose included: whose responsibility is it to do public engagement? When should public engagement be reviewed? Who sits on the ethics committee, and is the chair neutral? When do public engagement plans become explicit? What happens when there are several studies? Should communities develop shared principles? Who is going to monitor the ethics during the public engagement?

It emerged during discussion that when public engagement occurs, it should be based on an informed public engagement strategy. There are a range of public engagement issues, some of which are more controversial than others, like abortion which has a scientific component, but is not integrally involved in science. Perhaps one way to deal with deciding on controversy in public engagement is to decide on the role of science in the controversy, and then assign roles.

Some words kept being repeated: one was time – when should public engagement take place? – and the time available to the people doing the research. Protection of people was another, as well as review and competence to do public engagement.

The group was asked whether it thought having guidelines underlying public engagement would be useful – the show of hands was nearly unanimous. A guiding framework might be helpful which is not too prescriptive, and is flexible enough for country-specific contexts. It was suggested that the blog would be a valuable platform to continue this debate. Please visit https://scienceincommunity.wordpress.com
after the workshop to comment or see more about this issue!

Posted by Greer van Zyl, Healthwrite

Intercontinental Community Engagement

Muza Gondwe, TropIKA Reviews & Malawi Medical Journal

Whether you are doing public engagement in Colombia, Ghana or Sri Lanka the methods and challenges are similar. This second session on community engagement highlighted three different contexts of public engagement.

Kwaku Poku Asante, a Wellcome Trust International Engagement recipient, discussed Ghana’s Kintampo Health Research Centre’s (KHRC) communication strategy which was developed based on gaps they identified in a study in 2007. The activities in their communication strategy will create awareness about KHRC, inform communities about KHRC research agenda, strengthen the communications unit, and share their lessons learnt. When Kwaku said during his presentation “get the media before they get you” he alluding to providing media with factual information before they construct misleading stories from unreliable sources. He also mentioned that KHRC uses the media as a tool to dispel rumours associated with KHRC research and intervention programs.

Paulina Tindana, Mclaughlin – Rotman Centre and Navrongo Health Research Centre, was unable to attend the conference but her presentation on ethical, social, and cultural issues of community engagement was ably given by Kwaku. She recommended start ing community engagement early, knowing the community you are working in, establishing relationships, and very importantly, feeding back to the community.

Community meetings are valuable opportunities to connect with people, but in some instances, such as in Ghana, at the durbars held by KHRC, a majority of the participants are men. This the delegates attributed to meetings being called by traditional leaders that invite household decision makers who are mostly men. Multiple mechanisms should be used to reach target groups e.g. women can be reached in other arenas such as womens clubs or church groups. However, in a majority of low income countries, the patriarchal nature of society, deems men as heads of households and thus the importance of their approval is vital for studies that will involve women and children.

Lisbeth Fog presented a research study in Colombia that developed a communication strategy that is being implemented by the Columbian Ministry of Health with the assistance of the Colombian Institute for the Development of Science and Technology, Colciencias. The study looked at three critical questions surrounding research findings: Who knows about them? Who cares? Who uses the information? The study explored three stakeholders – knowledge produces e.g researchers, intermediaries e.g journalist, and audiences. The results of the study have led to the establishment of a website and provision of training for researchers, amongst other activities.

Even the most obscure of research such as twin studies can engage the community. Sisira Siribaddana’s Sri Lankan Twin Registry capitalized on the change in the A level school curriculum to engage students in conducting research projects on twins.

An interesting question was raised by one of the delegates, “Are there research agendas which are community driven?” Simply put no. However, the Africa Centre has in the course of its activities picked up research ideas. An example was given of a schistosomiasis study that was initiated from reports during engagement activities of children peeing blood.

The burning question that still remains unanswered is evaluating the impact of community engagement. Examples where given by PANOS where they examine case studies where media debates have influenced policy. It was also suggested analyzing enrollment figures and asking during enrollment where people heard of the study. The impact of engagement certainly remains a very topical issue for science communicators as the development of parameters to assess this, can certainly provide evidence that will support communicators promotion and improvement of public engagement.

The Power of the Press

Muza Gondwe, TropIKA Reviews & Malawi Medical Journal

The importance of the media in achieving wider coverage of research was captured by Thulane Cele, public relations officer for the Africa Centre, when he said “Media is a very powerful tool for disseminating information, influencing public opinions and educating the public”. His was one of four presentations on media engagement that took place on Wednesday, 3rd December. Thulani described strategies that he has used to engage media that include media conferences, media partnerships, live broadcasts, and podcasts.

Luisa Massarani, Latin America and the Caribbean Regional Coordinator of Science and Development Network (SciDev.Net), spoke of the journey of SciDev.Net and David Dickson, Director of SciDev.Net, being the driving force for its current international status. The SciDev.Net free access website is a highly recognized platform for disseminating science and technology information. 70% of its 40,000 registered users come from developing countries. The site gets 250,000 page views per month, and has 100 collaborators around the world. SciDev.Net’s work extends beyond the web platform to capacity building. They have conducted science communication workshops around the world for journalists, scientists, and parliamentarians.

A little can go along way with support and collaboration, this was implied from Dan Kaye’s (Makerere University) and William Odinga’s (Ugandan Science Journalist Association) presentations. Dan with funds from the Wellcome Trust’s International Engagement program, is soon to embark on a project that will build a critical mass of science communication trained journalists (90) and young researchers (150) over a 24 month period. William Odinga, Chair, of Uganda Science Journalist Association is just coming off the first ever Ugandan Conference on Science Communication. The idea started in February with a team of science journalists that had no funds but gained support from several local and international stakeholders including government organizations, scientists, media associations, and research institutions. Thanks to collaborative efforts the conference was able to take place on November 24th to 26th, 2008.

The challenges of engaging with the media are numerous but are similar in both developed and developing countries. These range from translation of scientific language, getting the interest of the media, poor funding for communication activities, and research institutions not recognizing the role of media. Two key questions on media engagement challenges were posed in this session. How do you can make science appealing to the media and how do you judge the success of your engagement? Suggestions were made on the former: giving science stories a human face, advocating for daily science and health sections in the newspaper, scoping the media and identifying missed opportunities for editors, and forming media partnerships. Judging success and evaluating impact still remains unanswered. One starting point could be defining the outcome or the end point of the engagement. Do you use metrics e.g. the numbers of articles published on research? Or the quality of the article? Should the engagement outcome even go beyond that, to behaviour change and influencing of policy. Hopefully these questions can be answered in the second session on media engagement on Thursday.

Focus on Research and Policy

Wendy Graham, University of Aberdeen, UK

“Researchers are from Venus, policy-makers are from Mars”

Using the issue of maternal mortality, (Immpact study*), Prof Graham showed that the worlds of policy-makers and researchers are very different and they keep changing. Maternal deaths are those during pregnancy and six weeks after birth; 98% of deaths are preventable, and 99% take place in the developing world. Maternal mortality is one of the UN millennium goals and is significantly off-track. Because of the urgency for change and progress around this area, it is becoming increasingly volatile.

The Immpact study worked in 8 countries to reach policy makers for national change, asking which of their major strategies they would like to be evaluated for impact. The research-policy continuum moves from research priority setting to knowledge generation and dissemination (and back) to evidence filtering and amplification, leading to policy-making processes. The evidence to policy environment is now occupied by more than researchers and policy-makers. The demand for evidence is more diverse, beyond the Ministry of Health, including the media who play a critical role in holding people to account.

Prof. Graham posed the question: What is the appropriate evaluation framework; how do we know what works? The Immpact study found that enabling environments were necessary and included working from the perspective of decision-makers; timing findings appropriately; capitalising on personal contact; and effective communication. The Immpact study didn’t reduce maternal mortality in the countries evaluated – which brings up the issue of only telling the good news.

“We need a journal of negative findings! Just because something didn’t work, doesn’t mean it should be binned. This is a very serious issue. There is no one thing that will reduce maternal mortality – you need a revised health system, there is no magic bullet. In terms of translating research into policy, knowledge brokers can be very useful,” said Prof. Graham.

*Impact: ‘Initiative for Maternal Mortality Programme Assessment’ which aims to improve the evidence base for policy makers through comprehensive evaluations of safe motherhood intervention strategies at a national and regional level.

Questions arose on whether knowledge brokers were feasible. The response is that they do exist, but there’s no one size fits all – sometimes they’re in research institutions, sometimes in NGOs. Prof. Graham commented that an evaluation platform needed to be established to see whether it will work or not. The issue of translating research to evidence in terms of maternal mortality was how to translate the findings because the answers are clear. Who to translate it and how to package it was another issue. The Canadian Health Services Research Foundation does this very well; for more information, see their website: www.chsrf.ca

John Young, Overseas Development Institute

“Key findings from research on reaching policy-makers”

SciDev.net undertook a literature review coupled with expert interviews and 7 country case studies to determine how policy makers in developing countries use evidence for policy. The review found several tensions:

  1. There is politicisation of science vs scientisation of policy
  2. Should scientists be involved in policy-making?
  3. Demand for certainty by policy-makers vs scientific uncertainty by scientists
  4. Challenge of providing specialised expertise
  5. Divergent motivations and time-frames
  6. Western vs indigenous knowledge.

The review found general dissatisfaction about the lack of using of research-based science in policy in developing countries. The major obstacles were insufficient information and information was outdated; in developed countries, information was too lengthy and general, while in developing countries, information was too minimal. Major uptake issues were perceived more in the south than the north. The more technical ministries valued scientific information, compared to the more general ministries.

Worldwide, 13% of people thought scientists should only provide research findings, 39% thought they should provide opinions and advocate policy positions, whereas 41% thought they should provide both. In terms of the most effective mediators of scientific information, the most effective were thought to be scientific organisations, while the least effective were thought to be the media (19%) and corporate business. The media were regarded as useful for getting issues on the table, but not in terms of the detail. “What is key to remember is that everything is highly context-specific; no one size fits all,” said John Young.

The study recommended that the role of intermediaries should be promoted; these should be identified and disseminate information and represent scientists or policy makers. “If there is to be more investment in these outcomes, there is a need to measure the impact for direct policy change as well as conceptual influence,” he said.

For more information: see http://www.odi.org.uk/publications/working_papers/wp294.pdf (or contact Nicola Jones at n.jones@odi.org.uk)

Questions included whether knowledge intermediaries and brokers were the same thing (they are), and who funds them (A mix of public/private and subscriptions if it’s a networking organisation). There is a big debate on whether they should spend more on this issue, which will impact negatively on research budgets. Another question dealt with the role of professional associations, who were generally found to impede policy implementation. In terms of translating policy into action and why some resulted in change and other did not, Young responded that this was often due to politics and drivers of change. “Policy processes are political and we delude ourselves if we think that evidence plays a significant role. There are lots of examples of appalling policies which were not evidence-based, and the north is not much better at this than the south,” he concluded.

Mammusa Rantsoti-Lekoa – Africa Centre

“Bridging the gap between research and policy-making”

There is a clear gap between policy and research, and the Africa Centre wanted to bridge that gap. The position of policy liaison officer was created and is responsible for familiarising policy makers with data from the centre relevant to their policy and programme needs, and to work with Centre scientists to effectively present data to service providers and policy-makers. It is very challenging to get a platform for this. The work also involves relationship-building with key stakeholders (government, NGOs, civil society) and proactively promoting the value of the Centre’s work for policy-makers. This is achieved by means of scientific meetings, policy forums, etc.

New initiatives include developing a prevention research network. Successes include the vertical transmission study; challenges include stimulating researchers’ interest, time constraints and working alone. The question is whether to be independent or to work within an institution. Who would fund this intermediary body?

Sheila Ochugboju – Global Women Inventors and Innovators Network, UK

“From idea to market: brokering knowledge to promote invention and innovation.”

In the late 1990s, the British Government declared a moratorium on GMO research due to volatile public pressure. This led the establishment of a programme on promoting the public’s understanding of science, where scientists received training in science communication. This initiative effectively caused a career change for many people including Dr Ochugboju who had been conducting post-doctoral studies on plant viruses.

“This wave of public feeling made us look at our science in a different perspective,” she said, explaining that the British Council had invited her to come to South Africa to speak to school children about being a science role model which led to her working with international development work. In the knowledge brokering role, roles are clearly defined and science education focused on meeting the needs of economies. “Women need to be empowered, and in the UK, the model of brokering knowledge is quite developed,” she told the session.

One of the roles the network is playing is facilitating funding. For example, IBM is putting money into South Africa, and in August the South African Women Inventors and Innovators Symposium, funded by the Minerals and Energy was launched. In October 2008, a partnership was signed with the Department of Trade and Industry to work on a Pan-African Women Inventors and Innovators initiative which will result in an international conference in June 2009 in South Africa.

“The public backlash in 1999 caused quite a few career changes, and public pressure can sometimes have more impact on policy decisions than research,” said Sheila. “Now, I work in three areas: creative science, innovation and leadership development which is all influenced by my scientific background and ability to interpret systems. I feel that every aspect of my work expresses who I am, and I am really fulfilled!” she said.

policy engagementA

Wendy Graham, Sheila Ochugboju, John Young, Bella Starling (chair) and Mmamusa Rantsoti-Lekoa