Skip to main content

Zoom Logo

A start-up guide for designing a data management plan & selecting a software for monitoring and evaluation for non-profits - Shared screen with speaker view
Noble Kelly
31:21
Noble Kelly - Education Beyond Borders
Elaine - Salanga/HCA
31:44
Hello everyone, welcome. Please share greetings, your organization and where you're calling in from.
Elaine - Salanga/HCA
31:52
Please fill in the poll!
Elaine - Salanga/HCA
32:23
Elaine Stavnitzky, Salanga / Harry Cummings & Associates. I'll be helping facilitate the chats and questions.
Maria Wong
32:41
Hello! I’m Maria, ED of Shanti Uganda (Vancouver). A special hello to Edith!
Kris Schmitz
32:49
Kris from HOPE International Development Agency - hi everyone!
Laura Solberg
33:06
Hi, I’m Laura, ED of the Canadian Christian Relief and Development Association.
Marvin Lubega
33:17
Marvin from Clinton Health Access Initiative - Uganda
Anja Kessler
38:14
Hello from Anja at Oxfam Canada
Bruce Petch
38:24
Hello, Bruce Petch here from World Neighbours Canada
Sarah Coates
38:25
Hello! Sarah from a high school in Vancouver.
Samuel Sunday
38:36
Hello, I am Samuel From ADRA Nigeria
Ian Chow
38:41
Hello from Ian from the United Nations Association in Canada
Todd Biderman
38:49
Hello, Todd from TAPOL (UK), joining from Victoria
Elaine - Salanga/HCA
38:56
Yes, this is being recorded
Elaine - Salanga/HCA
39:08
Edith's sound is garbled
Noble Kelly
39:27
It is a connectivity lag
Chelsey Andrews
39:41
that is better Edith
Adetoun Ilumoka
39:47
Hi everyone, Toun Ilumoka - from AOIA/EMPARC.
Noble Kelly
39:57
Yes, maybe Edith use audio only.
Adetoun Ilumoka
39:59
Edith is a bit slurred
Elaine - Salanga/HCA
40:23
If you would like to share a message in the comment with everyone, select "all panelists and attendees". If you would like to share just with the presenters, you can send a message to "all panelists"
Elaine - Salanga/HCA
42:07
You can also put your "hands down" now. Unless you have a question.
Ian Chow
47:38
tracking indicators that fit with the LM
Bruce Petch
48:09
measurement of progress towards indicators
Jennifer
48:16
monitoring your project progress using indicators
Anja Kessler
48:30
also why we collect data
Elaine - Salanga/HCA
51:57
Hi Rebecca, could you type your question into the Q&A or into the chat for now?
Elaine - Salanga/HCA
52:15
Hi Srikanthi, did you have a question? Or was your hand still raised from before?
Elaine - Salanga/HCA
52:53
I have seen some people call the PMF a Monitoring and Evaluation (M&E) framework. I have seen them include a separate column for indicator definitions
Sarah Coates
53:20
Thanks for the overview of Logic Model- I appreciate not the focus of today’s session… any “go to” resources you could share to help build that knowledge would be so helpful.
Noble Kelly
55:23
I’m not seeing those responses
Noble Kelly
56:42
It’s also important to have indicators of “failure”
Elaine - Salanga/HCA
01:00:11
Noble, do you have examples of indicators of "failure"? I find that interesting.
Dan - BCCIC
01:00:12
OK, where do I grant permission?
Karen Koster
01:05:31
what is the difference between a monitoring indicator and an evaluation indicator?
Tracy Bain
01:06:54
Need to make sure team has correct expertise to collect indicators
Diana Gresku
01:07:04
I find it really difficult to balance real-life context and academic theory (in creating indicators).
Noble Kelly
01:08:04
@Elaine: In general, just looking at the main things that restrict success. For example in education, knowing the things that will interfere/impede with learning and measuring that as part of the baseline or at least to help identify indicators of success.
Elaine - Salanga/HCA
01:08:15
@Tracy, that can be true. I find that if possible, having project implementers understand their Theory of Change or logic model and how the indicators measure those steps of change is important
Elaine - Salanga/HCA
01:08:32
Thanks Noble
Tracy Bain
01:08:46
Agreed!
Noble Kelly
01:08:51
ditto
Elaine - Salanga/HCA
01:09:04
@Diana, I would also add that the link to the Theory of Change/logic model is key
Anja Kessler
01:16:28
targets need to aligned with project budget, timeline and how you assume the causal pathways/assuptions to pan out - especially important for outcome level indicators.
Elaine - Salanga/HCA
01:17:17
I agree with you Anja. You could also set ideal targets initially, e.g. you would like 80% of people to meet that indicator standard and then change it once you have more information.
Anja Kessler
01:17:25
It is also important to take into consideration risks/shocks a project might face that may impede progress made by project or even lead to reversals - so those too need to be taken into account in considering targets
Paula - Salanga
01:18:31
Yes Agreed Anja! And targets can also be updated throughout the project if needed
Paula - Salanga
01:20:23
I would be curious to know what kind of qualitative methods folks use as well
Noble Kelly
01:20:30
video interviews, surveys, observations
Adetoun Ilumoka
01:20:55
With certain kinds of programs (educational) unexpected outcomes also occur (positive ones) so your framework has to be flexible enough to capture them.
Bruce Petch
01:21:17
simple counts of attendees at training sessions is nice and easy
Elaine - Salanga/HCA
01:22:18
@Adetoun, that's a great point! I would say that if this is happen often, you might want to look again at your Theory of Change/logic model and build it into your logframe/PMF. You can add these results to narrative reports otherwise
Anja Kessler
01:23:52
stories of change (first person accounts)
Anja Kessler
01:25:53
it is important to have check-ins beyond what is required by the donors to assess the assumptions and change theories noted as part of the project's theory of change hold true on thepath to reach the ultimate outcome - and adjust and adpt project interventions, strategies accordingly. To that end, we shouldn
Anja Kessler
01:25:58
shouldn
Anja Kessler
01:26:17
shouldn't leave looking at ultimate outcome close to the end of the project
Elaine - Salanga/HCA
01:27:00
I agree Anja, I am an advocate for doing outcome monitoring throughout the project lifespan and to try to empower the clients or beneficairies to do it as much as possible
Karen Koster
01:28:55
how long till the TOC component will be available?
Noble Kelly
01:28:55
yep
Elaine - Salanga/HCA
01:29:26
Karen, I'm not clear on your question. It is available as long as you have an account, but with the free account you are limited to one project
Elaine - Salanga/HCA
01:31:17
Hi Jessica, do you have a question or were you raising your hand to show that you could see the screen?
Karen Koster
01:31:27
ok. Thx. I thought Paula said it was in the process of being added to the application.
Elaine - Salanga/HCA
01:32:38
@Karen, what she is showing is already available. I think they are thinking how they can make it more flexible and have it be not just a flowchart
Elaine - Salanga/HCA
01:33:03
@Srikanthi, it usually starts with a blank template
Anja Kessler
01:33:23
with multiple people working on the same project. is there a way to track changes, 'lock cells'?
Elaine - Salanga/HCA
01:34:03
@ Anja, I think you can share a visual without giving options to edit, but if they have access to edit, I don't believe there is that functionality right now. I'll double check.
Anja Kessler
01:35:33
Is it possible to do work planning here too? like paln activities for the year and add timelines as well and have those activities be autogenerated for reporting against planned activities?
Anja Kessler
01:35:52
plan*
Elaine - Salanga/HCA
01:36:16
@Srikanthi, it's one of the best I have seen (I have been working as an independent consultant in the past year)
Elaine - Salanga/HCA
01:36:52
@Anja, in as much as you create them into an indicator, this is possible, but it doesn't have a workplan like a gantt chart right now
Noble Kelly
01:36:59
I can see linking to the indicators of the SDG’s as well
Elaine - Salanga/HCA
01:37:23
@Srikanthi, yes it can. I will need to have Paula respond to the details more.
Elaine - Salanga/HCA
01:37:36
@Noble, yes!
Anja Kessler
01:38:08
@elaine is customizations possible with Kinaki to allow for work plan functionality?
Anja Kessler
01:38:18
@elaine - thanks!
Elaine - Salanga/HCA
01:39:03
There are software out there that do integrate workplan, but the one I know is more expensive.
Anja Kessler
01:39:10
not task management - but work planning for project - similar to what GAC requires.
Anja Kessler
01:39:18
but thanks for the Asana tip though
Anja Kessler
01:39:46
yes
Anja Kessler
01:40:15
so customization is possible?
Sarah Coates
01:42:34
Not used because it’s yet another platform to navigate (regardless of ease of use)
Noble Kelly
01:43:08
not supported by platforms of all users
Jennifer
01:48:43
the connection is freezing for me
Elaine - Salanga/HCA
01:48:46
Edith's sound is hanging...
Elaine - Salanga/HCA
01:49:23
I think Edith was mostly reading from the slide, but if there is anything you would like to inquire more about, please let us know
Jennifer
01:49:37
Okay thanks
Elaine - Salanga/HCA
01:58:21
DevResults does include workplanning. It is designed more for USAID projects and has a higher price tag
Elaine - Salanga/HCA
01:58:38
DevResults also integrates budget
Sarah Coates
02:06:45
Can you share some experiences with using a formal measurement and evaluation approach to convince skeptics that evidence-based decisions matter or that the evidence they have been basing their actions on is incomplete/out of date?
Noble Kelly
02:07:54
Also, result vs. process measurement
Bruce Petch
02:08:08
One of the challenges we have faced with our very simple data management is that progress is seldom linear so we find the indicators aren't quite right and also we need to change direction so targets are no longer relevant. How can we prepare for these kind of things?
Elaine - Salanga/HCA
02:09:39
Hey Noble, was your comment in response to what Paula was saying or is it a question?
Noble Kelly
02:09:50
a question
Noble Kelly
02:10:44
So much emphasis is placed on results while process is, I feel, as important
Anja Kessler
02:11:08
does your qualitative data analyis tool work similar to that of Nvivo or Impact mapper?
Noble Kelly
02:11:28
yep
Elaine - Salanga/HCA
02:14:03
@Noble, on process indicators, sometimes these can be captured through management reports or Key Performance Indicators... it sometimes is captured outside of PMFs
Noble Kelly
02:16:28
Yea, it is more important to educate stakeholders that sometimes the process is more important because the ultimate goal cannot be achieved in our life time
Anja Kessler
02:17:03
@bruce it is also about acknowledging and influencing your donor to realize that the PMF is actually a very limiting tool. it is one way of knowing things are working - like someone said - it is a signpost. but to trully capture the complex change you described, as Paula mentioned, there needs to be flexibility to adjust. Another way is to do a really rigourous risk analysis of project outcomes at the beginning
Elaine - Salanga/HCA
02:17:15
@Anja, qualitative part of Kinaki not yet developed
Noble Kelly
02:17:58
Thank you to Paula, Edith and Elaine! Very good information and examples!
Elaine - Salanga/HCA
02:17:59
@Noble, @ Bruce,I think some of the issues identified can be captured by revising ToC and ensuring key steps/preconditions are included
Samuel Sunday
02:18:39
kindly share the presentation as well
Jennifer
02:18:51
Thank you, this was great!
Elaine - Salanga/HCA
02:18:51
Great questions and discussions and contributions! Thanks
Bruce Petch
02:18:54
Thanks very much. I learned a lot.
Laura Solberg
02:19:04
Thank you! This was very valuable.
Kris Schmitz
02:19:33
Thank you all! really great
Sarah Coates
02:20:02
Thanks for sharing.