Welcome, everyone! Please feel free to introduce yourselves in the chat as you join, and make sure to keep yourselves on mute to reduce any background noise. Thanks for joining us today!

Hi, everyone!Here's speaking Wiki from Indonesia, currently working as a government employee in tax authority. Cheers!

Hi, I am Plangsat Dayil, from University of Jos, Nigeria. It is good to be a part of this noble initiative.

Hello everyone! Mariana here, I'm an immigration officer working in the Ministry of Home Affairs in Namibia

Hello all, I am Ahmed Khaver from the Sustainable Development Policy Institute, Pakistan.

Hallo everybody!

Hi! I’m Francisca Pinto from Espacio Público, Chile

welcome all of you!

Greetings to everyone. I'm Stevan Stanisic from Montenegro.

Hi! I’m from Dominican Republic

Hi I am from UK

Hi, am Vania Ivanova in Brussels

I am from Canada

Hi all, I'm from the UK working for a Local Authority

Hello from Wisconsin, USA

My camera is playing hide and seek this morning.Good morning from Edmonton, Alberta, Canada.

Good morning, Tim Douglas from City of Vancouver here

Christine, GOPACKGO!

Okay. I am Plangsat Dayil from University of Jos, Nigeria. I am the Director of the Centre for Gender and Women Studies.

Stephen from Newport, South Wales

Hi all, I am from Albania

I am very happy

Thank you for organising this webinar!

@Tanya. 😍

hi every body Im Mahmoud najjar from united arab emirates - Dubai

Good morning folks, I'm participating from Toronto. Looking forward to the discussion.

Mohamed Aboubakari from Benin in West Africa, Social Policy Specialist

It's a pleasure to be here. Thank you!

Hello to everybody! I'm from Albania.

Thank you for organising this!

Hi there Bardha, Jessica in Canada here. Yes, thanks to the organizers.

Hello, I am from Ottawa, Canada

Hi

Hi from Toronto

Hi everyone - and hey Jessica :) I'm here from Winnipeg, Canada.

Hi Anne!

hi everyone, Onora from Nigeria

Hi from Ottawa Canada :-)

Hi from Edmonton Canada

Hello, Marine in London here!

Hi there Sarah from London

Hello Tanya from Swansea, UK

Hi everyone! Jenna from Ottawa, Canada

Hi- Karen from Washington, DC

Hi everyone from Milton Keynes, UK

Hi Beverley from Victoria, BC Canada

Hi everyone - Vanessa from Calgary, AB Canada

Hi, from Vienna, Austria :)

Hi everyone, Alexander from Indonesia~

Hi Dianne from Winnipeg, Mb. Canada

Hello- Ritikaa from Sault Ste Marie, Ontario, Canada

Hi- Eleanor from Birmingham, UK :)

Hi, Jo here from UK

Hello everyone, Edna from Montreal, Canada

Hi from Noumea, New Caledonia

Hi all, Umida from Berlin, Germany

Hello, Ammaarah from Cape Town, SA

Hi Everyone... this is Sahar from Toronto

Hi, Kanksshi- From India :)

Hi, from the UK- just outside of London

Hello! Nicole from Ottawa

Hello from Alberta Canada

Hi! This is Katie from the Measuring the Mountain project in Wales

Hi everyone, Emin from Toronto! :)

Hello, Gwen from Cape Town.

Hi everyone, Leanne in London :)

Hi everyone, I'm Julie from Wales.

Hi!, Cyan from London

sonny from Cardiff

Hello from Toronto!

Hello from Vancouver Island, BC, Canada

Hi, Marina, from Brazil

Jeff from Institute on Governance in Ottawa

Mariam from Dundee

Hi, Alex from London

Hi, Shadeequa from Los Angeles, CA

Hi! This is Melieha from Lincolnshire in the UK

Ismail a masters student in research and public policy in Nigeria

Joana from Victoria, British Columbia, Canada

Hi, Sole from Chile

Katherine, Climate Change Policy, Ontario Canada

Hi there, Paula from Ontario, Canada!!

Hi from my chilly home office (garage) in North Vancouver!

Hi - from Deborah from Belfast

Jenna, In Swindon UK :)

Hi, I'm Chris based in London! :)

Alan from Vancouver!

Hi all, another Paula here from Victoria, BC

Hi this is Sapna from India

hello. here from Victoria, BC, Canada

Hi I'm Muireann joining from London

Jen from Nova Scotia, Canada

Hi, Abbie from London

Hi, Liliana from El Sal Salvador, Central America.

Hi, everyone, Lin from New York

Hi from Vancouver, Canada

Louann from Manitoba, Canada

Hi from Serbia

Hello Everyone, I,m Shabira Nupur from World vision Bangladesh

Hi - Emily from London.

Hi, Cindy from Hertfordshire (UK)

Martha from Washington, DC

Hi from Bristol, UK

Hello - Debbie from London here

Hi everyone, joining from Calgary, Canada

hi from St Andrews, Scotland!

hi from CPI at London!

Hi! Pumi from Norwich, UK

hi ,,im hala from Amman- Jordan

Kabura Zakama, joining from DFID in Nigeria

Hi all, Christine from Washington, DC

HI all - Grace from London

Hi Julia, from Rio de Janeiro, Brazil

Hi all, Lucy from London

Hi Pierre Seguin from Ottawa Canada

Hi all. Suze here from Kuala Lumpur, Malaysia

Hello from John in Vancouver...

Hi there is Hollman from Bogotá Colombia

Hello--Lauren in Baltimore, MD here!

Hi all, Lisa from Victoria, BC, Canada

Fiona from Brandon Canada

Faisal Islam from Winnipeg, Manitoba. Also a former UK civil servant.

Hi all, Orla joining from Ireland

Hi, Shawn McGuirk joining from Ottawa, Canada

Hi everyone, I am Simone from Milan, Italy

Hello - Kevin here from sunny Preston, Lancashire

Hi all, I’m Inés from Buenos Aires, Argentina

Hi, Gladys from Paris, France

Hi all, I'm Roan joining from Halifax (Canada)

Brittany here, joining you from my make shift home office in Arnprior, Ontario, Canada.

Lindsay from Manitoba, Canada

Lauren from Atlanta, Georgia USA

Susie Abson from Welsh Government - Food Policy team..

Hi, from Ministry of Science, Chile

What is your experience of evidence-based policy, if any?A. I’m confident that I practice good evidence-based policyB. I’ve tried but found it’s really challengingC. I’m interested but I don’t understand itD. I’m sceptical about itE. Something else (please share on the chat!)

DC, Kuala Lumpur

I'm new to policy so want to know all about evidence based policy

Hi, Isaac from Oakland, CA USA

I'm also new to policy

Hi! Ben Tate, Head of Service Design and Standards for the UK Government

I'm new to policy so am learning about evidence based policy making.

New to policy, experienced in project delivery

B

At initial stage

Inexperienced but interested!

Im a scientist, so an expert at evidence, but new to bringing it to policy.

new to policy

C. I hope that we in the UK govt use it but it's difficult to know what to believe right now, especially in respect of COVID-19

Sujoy Dey from Ministry of Labour, Training and Skills Development, Ontario, Canada

I have provided the evidence when requiered for a research

A/B Evidence based policy making should be challenging- whether you are confident or not ;)

B

I work in the implementation space so work with policy designer

I'm new to policy so inexperienced - but have used research backed info for decision making

What do you find most challenging about using evidence in policymaking?A. I don’t know where to look to get the best evidenceB. I can’t get buy-in from colleagues or stakeholdersC. It’s very costly (in time or money) to get good evidenceD. I find that evidence-based policy ignores a human narrativeE. Something else (please share on the chat!)

new to policy but used data heavily when working in strategy consultant

many of those are applicable

A

Costly to get good evidence

D

A

DATA!!!

A where to get credible evidence

Time is the issue with Ministers demanding work at pace

something else - political barriers

Politics

all of the above

I am new to policy

there's never one 'right' answer

Conflicts with political priorities

Al

Researchers aren't always looking into the questions we need answers to

Often timelines are so short

also challenging to "groundtruth" evidence in llocal context

Evidences from many disciplines can contradict each other, how to reconcile? Also evidence is dynamic, how to use over time?

Don't always believe that evidence is best used ir weighted appropriately

I don't know where to look the get the best evidence... quickly!

Evidence rarely gives a direct answer. World is always slightly different to past precedence. And different sources rarely agree.

It can be challenging as there will be times when some evidence base is not so well received by colleagues because it challenges their current practice

@DeySu1 --- Hi!

Policy making the wrong way round - policy first then scrabble to find the evidence

Something else: as a user researcher, how get the user voice heard alongside other sources of evidence

All of the above - particularly human-centred methodologies take time and money

does the data really show the behaviour

What effect do you think COVID-19 will have on the way we use evidence in policymaking?' and then offer some choices?A. It will make us more likely to use evidence in policymakingB. It will make us less likely to use evidence in policymakingC. It will have no effect on how we use evidence in policymakingD. Something else (please share on the chat!)

Hi all, Sharmistha from Kolkata, India

Depends for who

Huge effect

slightly more but not much

We strive for evidence based policy making but too often end up with policy based evidence making

A

i think that Covid-19 gives us fear, much more fear

Short term effect (positively) and then back to business as usual?

D. depends on the will of each government and how they want to spin it...

something else - could make political leaders more receptive to advice

increased interest, but time pressure makes it less likely

Some of us are very cynical!

But there is also a need to act quickly - some tactical elements may need to proceed without evidence

"action while planning"

so yeah we might end up using less

It will make us more careful. Data viz can be misleading!

I think we are starting to see COVID responce become politized

We will be need to deal with each situation

I would hope more likely but I am worried of how resources will be used

B - I worry B because if we respond to COVID really well, it might make the case for not spending time on evidence

Gathering evidence may become more difficult due to resource constraints

I think it will depend on who is using - US feels like it's going backwards in using evidence in policymaking

There is need to move quickly - sometimes evidence is not available.

we will move towards a new definition of "evidence"

we're all here obviously interested in data

anyone from US government?

Not sure it will increase the use, but more people will be wary of misinformation, will value more trustyworthy sources of evidence

evidence based can be timely. much harder during COVID 19

Decisions are being made on the fly and this may introduce a new approach to policy development.

Huge amounts of data available, but skills to interpret it are not evenly distributed.

Lots of responses are crisis responses, not necessarily based on evidence but “best evidence” available

We desperately need to gathering more evidence now, and using disciplines such as health psychology

I think we'll use evidence more as we will be required to have ironclad business cases for any decisions

Rapid sourcing and confidence

the effect of COVID-19 will be limited to policy in the public health realm

agile approach to gathering evidence maybe? could be quicker

Agile definitely

Data literacy is key. Our health official explanations of flattening the curve are grounded in real world experiences. Data + holistic experience

Yeah Agile

HAving worked on policy advocacy in a non-profit, but only in frontline in the government, I have a question ... is evidence ever NOT used in government policy making? Certainly in advocacy work on policy, it was crucial to back up any policy recommendation we made with evidence. Perhaps for us newbies, you can explain what you mean by EVIDENCE BASED POLICY MAKING?

thanks!

Also be highly aware of "Goodheart's Law" - measurements that become targets become bad measurements.

Loving the comments. You folks rock!

How we can address the issue of reliability of evidences in our daily activity and its impact on policymaking

Be cautious what indicators are chosen. If people are incentivized to under-report or torque figures, it's likely some will do so.

Sorry, unfamiliar with “clearing houses":

Me too

From Gary: Just picking an evidence based approach is not enough you need a disciplined approach to delivery the policy well. This takes care, monitoring the situation on the ground and continuing to look at what the new evidence is.

Sounds like the comms/ marketing people are doing a good job, though. :)

Timmy stories... ever all seen them!

The evidentiary basis to inform policy is most important at the problem definition stage. before a policy is proposed, you need to establish a problem.

can you repeat gary's title and expertise?

"good" evidence takes time and government officials often are under intense pressure to act quickly

What about evidence for "BLACK SWAN EVENTS"?

Gary Vanlandingham is a professor and nationally recognised expert in evidence-based policymaking, evaluation, and public sector performance management and is currently the Reubin Askew Senior Practitioner in Residence at Florida State University. Prior to this Gary was director of the Pew-MacArthur Results First Initiative, a nationwide initiative to promote evidence-based policymaking in state and local governments. Gary has over 30 years of successful leadership experience in a variety of public and nonprofit organisations.

One of the challenges also comes when team members change and the evidence is no longer taken into account

There are so many ways we can use as an evidence to influence policy makers

question to Kimberly - do you do research regarding evidence base policy and policing?

Could you repeat the name of Kimberly’s organization?

Hi Shannon :-)

Evidence for Democracy

Kimberly Girling is the interim executive director of Evidence for Democracy, a Canadian organisation promoting the transparent use of evidence in government decision-making. Prior to her time here Kimberly worked in government as a Science Policy Analyst with Defence Research and Development Canada, and Innovation, Science and Economic Development Canada. Kimberly holds a PhD and worked on a number of initiatives relating to global and public health, drug policy and harm reduction during her work as a scientist.

Thanks! 😊

what's worked wer

could you share that report?

Yes, we can share resources mentioned today in the follow up email tomorrow

Time is important in terms of policy development. So sometimes political principals want to use evidence but short timeframes doesn't always allow policy analysts to do everything they would have done if they had more time. How do you counter this challenge?

The report mentioned by Kim: https://evidencefordemocracy.ca/en/research/reports/evidence-action

what are ways to determine the strength of evidence?

qualitative v quantitative data?

both data can be used as an evidence

Should evidence based practices that when vetted through an equity lens, have been found to create racial disparities, be used in policymaking? E.g. algorithims being used to assist in pretrial release decisions

we used both data in Bangladesh

The data we have access to is often transactional/operationally focused and often not very useful for more strategic policy purposes. Or very expensive/time consuming to make the linkages between the dat elements in order to make it useful for policy.

From Kimberly: Covid has the potential to lead to a boost in the public’s trust in evidence and science which has been flagging in recent years. It also presents an opportunity to gather more evidence and data that can be used in the future.

what do you do when you have a Government that doesn't believe in the evidence presented by scientists, because it doesn't fit their political manifesto?

Jonathan Shepherd is Professor of Oral and Maxillofacial Surgery at Cardiff University, he is also Professor at the University's Crime and Security Research Institute. Jonathan’s research on clinical decisions, community violence and the evidence ecosystem has made many contributions to public policy and to legislation including the 1998 UK Crime and Disorder Act. Jonathan is currently a member of the Cabinet Office What Works Council, the Home Office Science Advisory Council and the Welsh Government Public Services Leadership Group.

If the "Timmy" example is a Black Swan Event, then are we going to say no to that evidence as the first speaker mentioned?

Re: clearinghouses. They are similar to the UK What Works Centres in that they review and summarize rigorous evaluations of different programs. Many clearinghouses then rate the programs based on that evidence. A particularly useful tool for exploring clearinghouses and understanding the evidence base behind a variety of programs is the Results First Clearinghouse Database: https://www.pewtrusts.org/en/research-and-analysis/data-visualizations/2015/results-first-clearinghouse-database. Check it out!

Evidence based approach to shape policymaking process which is mostly used from NGO, think tank and international organizations and evidence based approach to make policymaking are not always the same

Policy makers don't have time to read the academic journals and even if they did, they wouldn't understand most of what's there!

Response to Jeff - govt departments need knowledge broker / knowledge translator positions who can guide policy analysts in accessing and synthesizing the best evidence for the issue at hand.

But surely good civil servants are recruited to rapidly understand compex evidence and present it to their poitical masters in digestible formats.

boom, evidence alone is not enough. so how we do we convince others?

A lot of the success of current measures is based on behaviours. Will citizens tolerate this or that measure, how do we communicate certain messages? Are behavioural scientists using evidence to inform decision makers, and collecting 'behavioural evidence' from the current crisis? I have not seen very much on this.

From Jonathan: There are many factors which can achieve policy and practise change: how evidence was used to achieve policy and practice change in these two examples. First, a bedrock of published, quantitative evidence. Second, a campaign for change built on this evidence. Third, as part of this campaign, wide media reporting of this new evidence. Fourth, development and refinement of local, prototype arrangements necessary to implement the new policy. Fifth, advocacy with selected parliamentarians to achieve legislative change. Sixth, detailed work through government departments and arm’s length bodies to achieve scale up.

Thanks

in Bangladesh we used base line survey data and findings, evaluation report, law policy analysis fundings, published investigative report by media, organised field visits by policy makers, organised dialogue Between survivors(with permission) and policy makers

Will we have the recording of this meeting

In an interview with a former chief scientific adviser on a UK news program last night, he raised concerns around Covid-19 government policy formulation, and in particular, control of messages to the public.He said that during his tenure the opinions of chief scientific advisers were made public before final policy formulation, allowing external scrutiny, and that now discussions take place behind closed doors so that a single consistent message is it put out to the public.Do the speakers have any idea how as policy developers we might be able to strike the correct balance?

My

we also conducted issue based research to use as evidence to influence policy makers

How the speakers would use or apply evidence in the development of policy in complex adaptive systems where you have evidence that is conflicting or suggests many different policy options?

How much time do we spend on evidence mining in the face of emergency.

That's such an important point!!!

Totally agree on need for knowledge brokers, also plain-language policy briefs, co-production of knowledge, etc.

timing in a political is key for sure! Also any advice on good methods of cost benefit analysis

cost benefit analysis doesn't benefit all

CBA: don't put too much emphasis on it!

what about Covid-19 communication, when we work remote

ist the information reliable

I agree. As a public servant I see far too much focus on the bottom line and lining up a budget on a dine

di

dime!

Consider the "integration of science and policy" as a more two-way process than a unidirectional "evidence-informed decision-making". Kim's comment on understanding the decision maker's thinking, needs, timing, etc. is VERY important.

if scientists have divergent opinions

Kimberly: One useful way to get buy-in is to identify stakeholders & identify how ignoring evidence might impact the department. Be clear about what the implications will be if you don't use evidence. Frame science & evidence in light of what a government's mandate is. Relationships are also really important!

what if there is pressure to create a policy before sufficient evidence is actually available

sahar

sam cowan- thats the struggle for sure

https://www.pewtrusts.org/en/projects/results-first-initiative

good qs

link to results first initiative here -https://www.pewtrusts.org/en/projects/results-first-initiative

Sam Cowan- that is a common scenario when translating manifestos into programmes for government

If anyone is interested, here’s a piece in Apolitical I wrote summarizing our E4D study in light of how the findings could be applied in the public service https://apolitical.co/en/solution_article/public-servants-can-be-the-fact-finders-politicians-need

Gary: The potential is there for policymakers to get on board - but it requires us all to make it easy for them by engaging in relationship-building, so people understand what is possible.

I have to leave the meeting now, thank you to all the speakers, and to you all, so helpful!

thanks Maggie! pleasure having you with us

In 20 years in various policy administrations I have never seen "true" decisions/solutions/programs built on "research evidence". It's a back and forth, at best - from policy idea to research evidence/input - to policy idea. Top-down largely rules the day

How do you think about the role of context in using evidence?For example, many "green-rated" programs in the pre-Covid world are likely not possible / effective in an during / post-Covid worldI fear overreliance on the evidence can lead us to overlook unique, local contexts

Jonathan: Certain strategies work in some countries but not others. There is flexibility needed, and country-specific and city-specific factors to consider.

Here is the link to the document with evidence resources Gary mentioned. This also has some of the resources Jonathan just mentioned: https://www.pewtrusts.org/en/research-and-analysis/fact-sheets/2020/04/where-to-search-for-evidence-of-effective-programs

Do the best you can? And commit to review the decision as needed...

question please: public sector innovation is very important and how could the seemingly conflict between evidence based policy and an innovation based approach which often sees failures

Covid-19 is creating new futures for people and countries- what approach should we take to evidence in policy making that is futures focused?

What areas of evidence should policy makers be focusing on now to help inform the drive for recovery post COVID19? What evidence could be used to encourage policy makers to think beyond the immediate at such an intense time?

Jonathan: There are situations where there aren't any randomised trials or experiments - you have to make a reasoned decision based on the information available. Sometimes, that means doing nothing, as politically difficult as that may be.

What if you did human-centered policy design, and used that to inform your policy design — so you knew you were solving the right problem to begin with?

Also really important to carefully and specifically define the research questions and who gets invited to provide input.It is so important to include users in ways that are appropriate to their experience. Justice systems have traditionally relied on judges and senior lawyers as the definers of truth.

there often is much appetite to failure in policy decisions.

Kimberly: People often expect science to have the obvious answer - but scientists need to be more clear about science being an iterative process. Like innovation, science is a series of failures and continuous development.

yes - if industry/business use human centred approaches to designing products and services for the public to use and consume - why not use human centred policy design ??

How do we communicate how science is iterative, and sometimes and exercise in failure, when there is already such low public understanding and public trust in science? Not to mention a lack a trust in public policy?

Yes and communicating the process and managing expectations for our leaders.

The results of scientific research are often based on a set of assumptions - these assumptions can have a massive effect on the results. And can be challenged - weakening the percieved 'correctness' of the science.

Jonathan: Systematic reviews & meta-analyses can be useful for figuring out if an approach works or if it doesn't.

any experience with Systematic Reviews in the social sciences? I.e., using policy research articles for carrying out systematic reviews ...

Good question Meghan

There is a role for bodies such as the Council of Canadian Academies or other national academies to do the synthesis of the state of knowledge behind policy questions...

Gary: A lot of policy questions fundamentally get down to value choices, not just scientific evidence choices. We can say 'if you want to go down x path, here is what you should think about'. It might not be a yes/no answer, but it ensures that policymakers are informed. (Evidence-informed policymaking rather than evidence-based policymaking)

Where do ethics come into the discussion of possibly competing evidence?

'Experts should be on tap, rather than on top'

YES!!!

Kimberly: One thing that might come out of COVID is the need for more interdisciplinary scientific research. Science will need to come out of the lab and be used by policymakers. Crossing barriers can be really challenging, from communication to funding, but this can provide a great opportunity.

Interdisciplinary approaches - yes!

well said

policy making doesn't have the same iterative luxury necessarily

Absolutely! Issues cross ministry/topic boundaries and it can be difficult for policy initiatives to follow across those boundaries.

Agreed, need horizontal collaboration/integration across federal departments and across the academic science/government science divide under a broader umbrella of "public science"

Kimberly: Scientists need to publish more iterative steps to show the process more transparently. They should also be putting work into communicating in more accessible places (like Apolitical!)

totally agree we do a lot to work with evidence "folk" to speak in a way they helps more people access their insight...

A helpful resource from the Urban Institute in the US: https://www.urban.org/urban-wire/introducing-evidence-based-policymaking-collaborative

Awesome!

Illuminating!

inspiring

Fantastic

collaborative!!

informative

Hopeful

collaborate

collaboration

encouraging

Collaborative

iterative

insightful and practical

thought provoking

Tricky

informative

enlightening

collaboration

empowering

curious

communication

stimulating

Evidence-informed

informative thanks

Great!

Science!

Relationships!

Thought Provoking (two words I know!)

valuable

interdisciplinary

brilliant, would love to participate in more

encouraging

The Big Picture

collaboration

Collaboration

Insightful

transparency

Collaboration

hopeful

very interesting, thank you!

collaboration

community

Communicate

All of the above!

Stimulating!

Thanks to our speakers for the presentation and to Ema for moderating.

Learning!

invigorating 😊

collaborative perfect word

Strategic

Lit

Integrated

interdisciplinary

Many thanks for sharing and the time you give us to learn from you!!!

being comfortable with the uncomfortable

Communirt

thank you very much! This was great.

The need for more insight

motivational

Community*

Thank you!

collaboration

Thank you

Thank you very much!

crossing barriers

thank you!

interdisciplinary team work

Advicacy

STAY SAFE AND HAPPY

complexity

Advocacy

Learnings were immense!

Thank you!

willing to innovate and make everything better

I agree with CAMPAIGN

thank you, really great session

Thanks to the presenters and Apolitical

Thank you everyone!

Thank you!

Thanks!!

Thank you!

Thank you

thank you!

building bridges

thank you!

Thanks a lot!

thanks so much!

thank you

Thanks so much

Thank you all, stay safe

Thanks! Great experience.

Thank you!

Have a great day everyone, and thanks again

Thankyou!

diolch!!

Thanks!

thank you!

Thank you so much

Thank you!

Merci!

Thanks

thank you!

Brilliant

Will we get a recording from today?+

thank you

Thank you very much!

Thanks To YOU

thnks everyone!

Thank you!

Thank you all.

Thank you!

Many thanks