Practical Evaluation Tips in a Time of Crisis

Hi everyone-

Today I am joined by Jenn Ballentine of Highland Nonprofit Consulting to talk about, what else, evaluation in the time of Covid-19. Granted, my last blog was a bit of a rant, so today, I would like to strike a  more positive and helpful tone.  

To tell you the truth, some of the conversation around data collection during the pandemic has me a little squirmy because it has felt kind of opportunistic. I don’t think rushing out to survey people when they are really worried and anxious feels helpful, or frankly ethical.

But we are evaluators, so we do believe evaluation is important and we just can’t stop doing what we do. I am a community psychologist and Jenn, a public health professional. We believe in a public health approach to prevention and in systems-level change. And if anything, this pandemic should teach us is that we are all connected. Systems-level change is needed now more than ever to correct the inequities in our society so evident in disproportionate impact of COVID-19 in communities of color.

Adaptions in the Time of Crisis

Sanjeev Sridharan recently wrote a thoughtful and poignant piece called Adaptions and Nimbleness in the Time of Crisis: Some Questions for Evaluators. In it, he observes that both program implementers and evaluators must now think about how to adapt. He raises a set of questions for evaluators to consider, and I urge you to read the article for yourself.

Today we would like to address the nonprofit and program implementers and provide some practical and feasible tips, inspired by some of the issues he raises.

Jenn and I are evaluating a federally-funded teen pregnancy prevention program that for the last year, has been implemented at 5 community-based centers for teenage boys and girls. I also am the evaluator for several Drug-free Coalitions and Alcohol substance Abuse Prevention Programs, all of whom have a school component. Jenn serves as the evaluator for school-based sexual health education programs facilitated by a statewide training and advocacy organization.

As to be expected, nearly all programming and thus, data collection stopped mid-March. This left Jenn and I wondering, what the heck we were going to evaluate beyond the data we already collected this year?

The technical assistance from funders for the most part, included four specific questions:

  1. What were your intended enrollment numbers and what are your actual numbers, and what is are the reasons for these differences?
  2. What is the status of your programming and how has that changed?
  3. How has data collection changed (number pretests/number posttests) and how were participants affected- e.g. missed content, sessions provided out of order, etc.).
  4. How will the program use Continuous Quality Improvement (CQI) strategies to document and learn from the events?

What is missing is the story here, as Sridharan points out in his first question: “What are exemplars of good evaluation stories related to the adaptiveness/nimbleness of specific interventions.” Yes, we need to understand changes regarding what was planned versus what was done, but we need the why, in order to tell the entire story. When did they have to close their doors and why did they make that decision? What happened to staff and why? And as a result of the situation, were program staff able to pivot and if so, in what way?  For example, did program staff decide to shift from in-person to online delivery?

For one of my clients, they have shifted, rather nimbly I might say, to online meetings with their youth advisory committee. They are taking notes about their discussions and developing interventions that they can do online via social media. Similarly, another client that trains health and physical education teachers to implement comprehensive sex education offered to facilitate virtual lessons for one new district in an effort to ensure that students received this valuable information.                                                                                                                                      

Another of Sridharan’s questions is “Are there examples of evaluations that have taken a developmental approach to enhance the coordination at this time of the crisis?”  He observes that the “pandemic has highlighted the need to better understand the connections between the intervention and its underlying systemic contexts/supportive structures.” During a time of crisis, coordination can be improved, perhaps accelerated, or could also break down altogether.

Some school systems for example, have enlisted bus drivers, community volunteers and even local law enforcement to deliver food to students eligible through the National School Lunch Program. Some have expanded food distribution beyond those eligible through these federal programs. Other school systems have maintained the status quo, requiring families and guardians to drive to school with the eligible children present to collect the food distribution. Those without transportation, or without a large enough car to transport the whole family, were out of luck.                                                                                                                                 

There are a lot more gems to unpack (like the dynamics of vulnerability), but we will end with this question posed by Sridharan: “Can a focus on a minimal set of components needed to produce change help enhance a focus on meeting the needs of the disadvantaged given limited resources?”

For our teen pregnancy prevention program, we can’t even imagine where to start on a minimal set of components for this implementation fidelity evaluation. How do you deliver an evidenced-based, comprehensive teen pregnancy prevention program virtually? Is it even ethical to do so with parents, siblings in the next or same room? What about students without internet access, laptops or other devices or when access to devices must be shared by multiple youth?                                                                                             

Looking Forward

We are pretty sure that six months from now, funders will be asking nonprofits what happened? What is the program implementer to do? We think it’s critically important to document the changes programs made and the various ways in which the disruption impacted their organization and the people they serve. But program staff are busy people, especially during times of crisis. Evaluators can help the nonprofits they serve by helping staff document the changes they made and why they made them. Evaluators need to stress how this information will be useful when reporting to funders, partners, board members and others. The learning that comes from this process can help the organization plan for future disruptions.

We developed a guide to help in this process. Just let me know you want the guide and I will send it to you. Depending the needs of your client and their situation, these questions can be adapted in a variety of ways. You might want to change the order, eliminate some questions and add others. Do let us know what you think and if you find it useful. Stay safe and well!

Comments

  1. I have recently started a web site, the information you provide on this site has helped me greatly. Thank you for all of your time & work.

  2. How to play online slots for money? How to Win Real Money Playing online casino Slots? The Best Place To Play Online Casino Games! Picked By Experts. Best Reviewed.

  3. Спасибо за пост
    _________________
    pin up bet partner

  4. Как каждый догодался я предлагаю заработать с помощью рынка форекс, установив торгового робота.
    Многие задают вопрос безопастно ли моя схема 50/50 как повезет
    Подробности тут:https://wsb-robot.ru/

  5. Game LIFE 遊戲情報 http://gamelife.tw/

  6. Sexy photo galleries, daily updated collections
    http://beardfetish.bloglag.com/?fabiola

    venus porn wikipedia teen shower porn tube riley brooks porn star free ameatur porn videos hairy secretaries porn

  7. Aged Twitter 2009 to 2015 Accounts For Sale – check new thread for new prices

    The accounts are empty or with less than 50 followers.

    2008 – 10$ Per Account
    2009 – 9$ Per Account
    2010 – 8$ Per Account
    2011 – 7$ Per Account
    2012 – 6$ Per Account
    2013 – 5$ Per Account
    2014 – 4$ Per Account
    2015 – 3$ Per Account

    with Format
    Username:Password Twitter:Live Email:Password Email,
    joindate,tweets,following,followers and likes.

    Buy for testing accept! pls check with me before you buy

    https://sellaccs.net

    You Can Contact me via
    Email – congmmo@gmail . com

    Discord : CongMMO#9766
    Skype & Telegram : congmmo
    ICQ : @652720497

    Msg me on discord or skype for faster response.

    Payments accepted PayPal,Payoneer or ETH,ETC,LTC or Another Crypto.
    (Extra 5% transaction fee)

    I will provide Username:Password Twitter:Live Email:Password Email of the account.

    Accounts are delivered instant after paying.

    Thank you!

  8. r2 online видео – фогейм р2, r2 online forum

  9. Пару минут назад смотрел данные инет, неожиданно к своему удивлению увидел красивый ресурс. Смотрите: купить insta ник . Для моих близких этот вебсайт явился очень неплохим. До встречи!

  10. Позавчера мониторил содержание сети интернет, неожиданно к своему восторгу заметил лучший ресурс. Посмотрите: инди рок новинки 2020 . Для моих близких вышеуказанный сайт оказал хорошее впечатление. До свидания!

  11. Добрый день
    Доктор посоветовала
    Высочайшее качество лекарств

    Sidovis (Цидофовир) – Сидовис (Cidofovir) отзывы

  12. хайп инвестиции 2019 мониторинг exclusive-profit – перспективные хайпы для инвестиций exclusive-profit, легкие деньги мобильный

  13. Sexy teen photo galleries
    http://hdpornpregnant.xblognetwork.com/?yvette

    free home teen porn movies free pee porns free teen porn big tits pics porn hitz real european classic porn video free

  14. взломанная яндекс музыка – раздача ключей стим, аккаунты ps3

  15. Browse over 500 000 of the best porn galleries, daily updated collections
    http://lexixxx.com/?juliet
    hardcore screaming porn and muff diving lesbian mean and rough porn persian porn ladies aylar caramel moore free teen porn trailer o porn

  16. скачать последнюю версию world of tanks – инвайт код на черчилль и голду, купить аккаунт world of tanks

  17. Серверы с повышенной частотой процессора 4.5 GHz. Для задач, требовательных к скорости CPU, отлично подходят для сайтов на Bitrix и игровых проектов.
    От 16 рублей / день
    От 480 руб. / месяц
    Официальный сайт: https://t.co/YpYNB0ng8U

  18. New super hot photo galleries, daily updated collections
    http://shemalepenang.hotblognetwork.com/?avery

    video gay teen porn iphone porn genuine amateurs free toon porn gallery chinese porn movie galleries free best 2008 homemade free porn

  19. Dirty Porn Photos, daily updated galleries
    http://sissybrapanties.allproblog.com/?naomi

    80s french porn stars porn dorm tour hungarian hungary porn stars porn fee pictures search panty boi slave porn

  20. Сегодня вечером анализировал данные интернет, неожиданно к своему восторгу открыл лучший сайт. Посмотрите: красивый адрес страницы . Для моих близких вышеуказанный веб-сайт оказался весьма полезным. Пока!

  21. памятники в осиповичах – купить памятник в могилеве, эксклюзивные памятники в могилеве

  22. купить adrenaline bot – adrenalin bot, бот для ла2

  23. Hot teen pics
    http://alypics.com/?talia
    amature hand job porn movies frree young porn movies swapping wives porn tube videos porn on rss channel free video porn charmane and cheyne

  24. Mailades says

    What outfit could you put together from clothes you own to get the most laughs?

  25. Scandal porn galleries, daily updated lists
    http://bigpussyporn.instakink.com/?delilah

    hentai hot porn sex teen youn porn catwoman porn claws dani rich porn star tits biggest bra busters free porn

  26. ethereum – initial coin offering, how to

  27. Pretty nice post. I just stumbled upon your article and wished to say that I’ve really enjoyed browsing your article. After all I’ll be subscribing to your rss feed and I hope you write again soon!

  28. Viagra no script overnight
    Viagra no script overnight
    Viagra overnight delivery no rx
    Viagra no script overnight
    Viagra overnight delivery no rx
    Viagra cod saturday delivery fedex

  29. Буст часов – Буст кс го, Накрутка часов

  30. Xrumer ist ein schwarzes SEO-Tool, ein einzigartiges Programm zum Spammen von Kommentaren, Foren oder Gastebuchern.
    Khrumer wird aufgerufen, um fur die Website oder ihre einzelnen Dienste zu werben, und platziert automatisch Texte, Links oder Ankundigungen auf Ressourcen von Drittanbietern.

    Fruher wurden Programme wie dieses mit Captcha bekampft, aber die aktualisierte Version von Khrumer handelt sorgfaltig, sodass Nachrichten in alle Bereiche der Site eingefugt werden konnen.

    Sie konnen Khrumer nur fur Geld kaufen, es funktioniert mit Windows OC. Unter seinen Merkmalen kann Folgendes unterschieden werden:

    Wenn Sie mit mehreren Ressourcen gleichzeitig arbeiten, konnen Sie mehrere Nachrichten gleichzeitig einfugen.
    Khrumer kann sich unabhangig auf einer Website oder einem Forum registrieren, ein Profil ausfullen und Kommentare hinterlassen.
    Das Programm kann mit einer Vielzahl von CMS arbeiten.
    Der veroffentlichte Beitrag kann in verschiedenen Versionen platziert werden. Aus diesem Grund nehmen Suchmaschinen solche Texte nicht fur einen Beitrag und bestrafen sie nicht mit Filtern oder einem Verbot.
    Die Werkzeuge von Hrumer werden von Jahr zu Jahr mehr.

    Die Meinungen der Optimierer zu Khrumer waren geteilt. Einerseits hilft das Programm, die Website zu bewerben und Besucher anzulocken, andererseits ist es immer noch Betrug in Bezug auf Suchmaschinen.
    http://t.co/aYIw5s04LF

  31. https://man-r20.com/

    XYZ 壯陽藥春藥專賣店 大量購買有優惠

  32. books-top (Cookbooks, Food & Wine) Free Ebook – books-top (Mathematics) Free Ebook, books-top (Travel) Free Ebook

  33. 0445120236 форсунка – 0445110594, дизельная форсунка

  34. emex ru – купить форсунки ленд крузер 200, Форсунки Delphi

  35. Спасибо за пост
    _________________
    Предупреждение незаконных азартных игр

  36. Hot photo galleries blogs and pictures
    http://freeteenpornwmv.lostinporn.relayblog.com/?kamryn

    girl russia porn free you pornh best blow job ever aidan lane porn dirty british porn stars i m watching gay porn

  37. смотреть фильмы и сериалы онлайн

    https://lutikhd2.ru/film/20664-posle-smerti.html

  38. Фильмы и сериалы 2017-2018-2019 годов

    https://hdclaps.me/3167-doktor-son-2019-smotret-onlajn-22-11-05-2020.html

  39. На прошлой неделе просматривал материалы сети интернет, неожиданно к своему удивлению обнаружил актуальный веб-сайт. Вот смотрите: Купить топливные форсунки . Для меня вышеуказанный веб-сайт оказал незабываемое впечатление. До свидания!

  40. 歐客佬精品咖啡 https://www.oklaocoffee.tw/

  41. DoloresUtesy says

    How would your country change if everyone, regardless of age, could vote?

  42. Минуту назад разглядывал содержание сети интернет, вдруг к своему удивлению заметил познавательный вебсайт. Ссылка на него: Контрактные автозапчасти Красноярск . Для нас вышеуказанный сайт оказал яркое впечатление. Всех благ!

  43. Enjoy daily galleries
    http://androidshemale.instasexyblog.com/?brandi

    mass effect porn galleries free no sign up porn farmers daughter porn dvd alex dane porn european swingers porn

  44. IT’S TODAY!!! Free shipping on orders over AU$45.00
    Swimwear, Dresses, Gothic Style,, Tops, Intimates & Lingerie, Bottoms, Sweaters & Cardigans, Jumpsuits & Rompers, Outerwear, Activewear.
    http://ali.pub/4rzldy

  45. My new hot project|enjoy new website
    http://pinupgirltattoo.allproblog.com/?aspen

    porn medieval blue gay porn wife fucks porn star at home porn couple and babysitter romania free gay porn

  46. Hurensohnror says

    christoph
    christoph
    christoph
    christoph
    christoph
    christoph
    christoph
    christoph
    christoph
    christoph

  47. Heya! I just wanted to ask if you ever have any problems with hackers? My last blog (wordpress) was hacked and I ended up losing several weeks of hard work due to no backup. Do you have any solutions to protect against hackers?

  48. Stephenanamn says

    Urgent loans in the USA. 5 minutes approval in https://maybeloan.com. Fast, easy, safe and secure. Online. – Need $1,000-$5,000 and more fast? – Yes. Bad credit OK. Instant approval. Online 24/7 Customer Service. Get Started now!

  49. Pretty amazing post. I just stumbled upon your article and wished to say that I’ve really enjoyed browsing your blog posts. After all I’ll be subscribing to your rss feed and I hope you write again soon!

  50. ColinGuife says

    схемы онлайн казино https://t.me/casinoxslots

  51. Новости распространения коронавируса https://karta-coronavirusa.info/statistika-na-kaymanovykh-ostrovakh.html

  52. CarlosStync says
  53. Pretty cool post. I stumbled upon your article and wished to say that I’ve really enjoyed reading your blog posts. After all I’ll be subscribing to your rss feed and I hope you write again soon!

  54. As a Newbie, I am continuously browsing online for articles that can benefit me. Thank you!

  55. My acquaintance with VTOPE

    I learned about the site a month ago, watching another video on YouTube. The site is in Russian, but the browser translator does a good job.

    In this video, the guy tells how he makes money on the #vtope bot by connecting his social network accounts, which automatically earned him points, and they could already be exchanged for real money.

    It all seemed very interesting to me and I began to try this way of earning … https://t.co/x4YZQolkXF

  56. Dirty Porn Photos, daily updated galleries
    http://bigboy36.hot7thgrade.bloglag.com/?kelsie

    web site porn list of porn stars penis watch porn movie taboo online real couple porn stream yiff drawn porn thumbs

Speak Your Mind

*