Engagement should bring value. It is important to monitor and evaluate engaged research to ensure it's potential is maximised.
There are three levels of evaluation to consider:
- Process evaluation: how you are engaging.
- Performance evaluation: does your team have the skills/capacity to do it?
- Outcome evaluation: is it achieving what you projected?
.jpg)
Horizon Europe
In Horizon Europe, the European Research Council (ERC) emphasises the importance of citizen and societal engagement in research projects. While not mandatory for all ERC projects, engaging citizens can significantly enhance the societal relevance and impact of your research.
Upon project close, ERC projects may ask for reporting as to whether citizens or end-users were involved in the below aspects of the project. Therefore, put a strategy in place to collect this information throughout the project.
| Co-creating R&I visions, agendas, policies or frameworks |
Monitoring and/or evaluating R&I results |
| Co-creating R&I action plans or technology roadmaps | Testing & experimenting with innovative R&I solutions |
| Collecting data for the project | Contributing to scientific publications or patent applications |
| Analysing data for the project | Debating R&I findings and implications for them |
| Providing resources, e.g., computational, space/locations, practical support | Not applicable |
Source: A presentation by George Papanagnou (European Commission).
Knowledge valorisation
Knowledge valorisation is the process of creating social and economic value from knowledge by linking different areas and sectors and by transforming data, know-how and research results into sustainable products, services, solutions and knowledge-based policies that benefit society (European Commission definition). Citizen engagement was identified as a key enabler for knowledge valorisation. The (opens in a new window)(opens in a new window)European Commission's Code of practice on (opens in a new window)(opens in a new window)citizen engagement for knowledge valorisation provides guidance on creating an enabling environment, participatory processes, and practices of sustainable citizen engagement for improved knowledge valorisation.
Knowledge Valorisation Evaluation framework
An evaluation framework is one of the main elements of The Code. It gives recommendations for assessing the efficacy of citizen engagement and participatory processes that lead to knowledge valorisation and including appropriate indicators and metrics. The associated (opens in a new window)independent report outlines a wide array of potential indicators (annex 3) that can be used at project level.
There is a mix of quantitative and qualitative indicators available for use throughout projects and for a variety of purposes. We highly recommend you review and implement relevant KPIs into your projects. Each KPI includes the aim and objective, collection interval, calculation formula, unit of measurement and data requirements.
.jpg)
UNICEF Minimum Quality Community Engagement Indicators
UNICEF developed (opens in a new window)(opens in a new window)minimum standards for community engagement. Although these are not directed at research, there is significant cross over in their intent and they may be useful in a research context. In addition to the UNICEF standards, community engagement indicators were also developed to address the consistent and rigorous measurement of community engagement in order to demonstrate its value to key stakeholders.
UNICEF provide two sets of indicators: one for adaptation by National and local governments; and one for adaptation by Non Governmental Organisations, Civil Society Organisations and implementing agencies. Neither set is designed for universal adaptation, but rather to be adapted and selected based on local need and where possible selected via co‑evaluation with affected communities or populations.
(opens in a new window)(opens in a new window)Review table 4 (chapter 4)to review and select the indicators appropriate for use on your project.