Evaluating the performance of your communication
When to evaluate your communication
- Conduct evaluations of your communication plan both during its implementation, and after its completion. Conducting evaluations during implementation allows you to make necessary adjustments and respond to new information and needs mid-project cycle. Conducting evaluations after implementation is finished can provide valuable lessons for future projects.
- During implementation, evaluations can take two forms:
- Evaluation can occur informally throughout the process of implementation, such as by holding regular team meetings to get updates on progress and identify needs. Changes can be made accordingly. Another way of evaluating your communication is through interactions with your target group that occur as part of your project.
- If possible, formal interim evaluations of your communication should be held at pre-agreed stages. These should occur as often as appropriate for the length of your project (eg: multiple interim evaluations for longer projects).
- After implementation has been completed, a rigorous evaluation of your communication should also be conducted, comparing output (eg: the various output of your communication activities), outtake (eg: what your target group has taken away from your communication activities) and outcomes (eg: the impact and changes that have resulted from your communication activities). These should be compared to the communication goals established in your communication strategy.
- Additionally, use meetings in the field with your target group as another method of continually assessing the suitability of your communication strategy. This will emerge as you explain the strategy to your target group and better learn the local conditions.
What to evaluate about your communication
- Evaluations should be as robust as possible, but the broader the scope of evaluation, the more time and financial resources will be necessary. If constraints exist, a more targeted evaluation should focus on:
- Evaluation of your communication should be both ‘internally’ and ‘externally’ focused.
- Internally focused evaluation examines the performance of your team and the execution of your communication plan. This focuses on whether implementation is occurring as planned, and what strengths and weaknesses can be identified in the plan. Key questions and elements to be considered include:
- Is the plan being followed (or was the plan followed)?
- What remains to be done (or what remained unfinished)?
- Who has responsibility for the remaining elements?
- What are the remaining deadlines, and will they be met (or did you achieve your deadlines)?
- Did you budget correctly for your communication activities? Which measures are/were the most cost effective?
- Have your communication activities all worked towards the objectives defined in the communication strategy?
- In terms of outputs, how many people in your target group were reached, how many times and over what period of time? What were your ‘push statistics’ (eg, how many newsletters were sent out, how many articles published).
- Externally focused evaluation examines the impact that your communication activities had on your target group. If this is the first evaluation in your project, these impacts can measured against the baselines identified in your target group analysis.
- Is your intended target group being reached? Are the communication channels working as perceived?
- How is your target group receiving your messages and tools, and are their perceptions changing? Do they understand your message?
- In terms of outtakes, what messages did target groups take away from your communication, and were these accurate and positive? What are your ‘pull statistics’ (eg: how many flyers were taken by individuals, how many times has your website been viewed, and which sections received the most traffic)?
- In terms of outcomes, have your measurable communication goals been reached? Do you observe any behavioural change in your target group? Did you contribute to this behavioural shift? How did you contribute to this behavioural shift? Which of your messages and tools had an impact upon your target group, and which did not?
- If the project is still ongoing, how can the communications strategy be altered to better integrate the views and experiences of your target group into your communication?
Determining which form of evaluation to use
- A variety of methods of evaluation exist to help you answer the questions, including:
Baeyaert P. 2005. “Developing an External Communications Strategy". Presentation at Communicating European Research, November 14, 2005.
Research Matters: The RM Knowledge Translation Toolkit: A Resource for Researchers. Chapter 6: Designing a Communications Strategy.
United Nations Development Fund for Women (UNIFEM). Guidance Note on Developing an Evaluation Dissemination Strategy. Evaluation Guidance Note Series No.10 UNIFEM Evaluation Unit. December 2009