And welcome to Tony Martignetti Nonprofit Radio. Big nonprofit ideas for the other 95%. I’m your aptly named host, and I’m the pod father of your favorite hebdominal podcast. Oh, I’m glad you’re with us. I’d suffer the effects of aspergillosis. If you inflamed me with the idea that you missed this week’s show. Here’s our associate producer, Kate, to give you the highlights. Hey Tony, here’s what’s up. Bring fun to evaluation and impact data. Kayla Meyers reveals the fun in what may sound dull, through braids, data galleries, and data escape rooms. She distinguishes between outputs, outcomes and impact, and explains the disconnect between traditional evaluation and the impact measurement we need in today’s environment. She helps you recognize common evaluation challenges and sets you up to overcome them. Kayla is the founder of Bridgepoint Evaluation. On Tony’s take 2. They need your opinion. Here is bring fun to evaluation and impact data. It’s my pleasure to welcome Kayla Meyers to nonprofit Radio. Kayla is the founder of Bridgepoint Evaluation. Through training, coaching, and hands-on evaluation support, she guides organizations in designing practical, action-oriented evaluations. She’s an adjunct professor teaching program evaluation and survey design. The company is at bridgepointevaluation.com. And you’ll find Kayla on LinkedIn. Welcome to Nonprofit Radio, Kayla. Hi, Tony, thank you so much for having me. It’s so fun to be here. Uh, it is fun. Thank you. I’m glad you, I’m glad you’re having fun already and we haven’t even started. That’s outstanding. All right, I love, thank you for bringing that. Thank you for bringing that energy. OK, evaluation, like impact measurement, um, these things get like nonprofits, I think, I fear get intimidated, uh, plus they sound boring, although essential. Help us, help allay like fears of intimidation and boredom. Yeah, absolutely. So I think sometimes, uh, when folks think about evaluation, they think about auditing, they think about compliance, they think about proving their impact in a way that’s going to be deemed enough. Um, whereas when we talk about evaluation over at Bridgepoint evaluation, we focus on improving and learning and growing. Um, and then also having a, a strong basis for making decisions confidently, for having a shared understanding amongst your entire organization, and so really just trying to flip the script a little bit on why we do evaluation, acknowledging that demonstrating our impact is only one piece of why we go about evaluation and learning versus thinking about evaluation as an auditing and compliance mechanism. OK, and for decision making, as you mentioned. So, you know, can we, can we humanize this process? Yeah, yeah, absolutely, I think, I think that’s, that’s doable, that’s achievable. Absolutely. And, and I think sometimes when we think about evaluation, we think about numbers, we think about how many people accomplished what and how do I make sure those numbers are as big as possible. I see all the times nonprofits. Saying we’re going to reach 1000 people, 1 million people, and what gets lost is that human moment, exactly. And so I think a grounding ourselves and getting back there is so important. OK, and we’re gonna talk about, you know, going from 1000 to a million, you know, that’s not gonna happen in 6 months or even in a year, but you want to celebrate the steps reaching, reaching. We’ll, we’ll we’ll get to the, the, you know, the, the incremental nature that you ought to celebrate. Um, I feel like we should distinguish at, uh, near the beginning here between outcomes and impact. Can you, you know, make that clear for us? Yeah, absolutely. And so you can think about this. The easiest framework to think about this, in my opinion, is to think about the logic model, which I know I just said logic model, and then people just like turn off their ears, and they’re like, never mind. No, no, we’re giving you a chance. No, no, not yet. It hasn’t happened yet. Maybe at the 5 minute mark, but not, not the, not the 3:40 mark. No, we’re, you’re OK. All right, no, so just think about it instead of like an outcomes chain. Um, so. We host this training to what? To, to achieve what outcome. Um, so you can think about outputs, which is how many people attended the training, but then what happened to those people in the training? What do those people do immediately after the training? Those are your short term outcomes, then you start building out that train, right? So people learned um how to be financially healthy, they learned how to budget. And now the next outcomes on that train are things like, they tried budgeting out. Um, once they tried budgeting out, they were able to better align their income with their expenditures. Once they got that, you know, kind of just working through that chain and really thinking about that overarching chain, what it results in, so we’re talking about a financial health program, what it Results in is the impact, the result there being that folks have the financial stability and the financial wellness to be able to have increased decision making, increased resiliency, and increased ability for, um, social movement, for financial movement within their lives. It’s an impact. All the things that come before that are outcomes. OK, thank you. And outputs too. You added outputs too. Thank you. All right, cool, cool. Um, so you believe there’s a disconnect between traditional evaluation and the impact measurement that you want to be talking about, that you want, you want, you’re encouraging folks to do, you encourage clients to do. So help us see how the old model is, uh, not, uh, not so good. I don’t know. Not, the old model is not new. The old model is not current. That’s why it’s old. Share the, share the, the, the, the differences here, where, where we, where we are versus where we used to be. Yeah, absolutely. And I think this is, you know, this is truly why Bridgepoint evaluation exists. Um, I have been an evaluation consultant for almost 15 years now, and much of my early career was spent on these really big 3-year, 5-year projects, um, huge budgets coming in from the federal government or research institutions, and then at the same time, I’d be working with nonprofits, medium size, um, large size, but even some smaller, smaller size nonprofits. Who are being held to very similar standards as those, you know, large, huge consulting opportunities. And so what I, what I wanna bring together is the, the two, bridge across those words, right, across those worlds, so that’s look how fancy you’re bridge, Bridgepoint, you’re bridging. Yeah, bridging those worlds. Oh, absolutely brilliant, brilliant bridge point. OK. Right. And so, and so you’re often seeing, um, these large studies of demonstrating impact, proving impact, and, and instead, I want to work with organizations to reduce uncertainty and make better decisions with their data. So it’s about improving your impact, not necessarily. Improving it, but improving it, um, and just generally kind of trying to take some of the tools from that impact measurement world and translate them into your in-house nonprofit of staff who wear 3 to 5 hats, of perhaps folks doing impact measurement who maybe didn’t have, uh, necessarily, um, formal training on that, but they grew into that role and they’re fantastic at that role. So how do we support them even more? Let’s talk more about the decision-making that you mentioned earlier. You know, maybe, I don’t know, maybe you have a story about how a, a nonprofit used data in decision making, but I, I’ll put you on the spot. If you don’t, that’s OK, you know, but, but help us see the, the value in, in clearer decisions. Yeah, absolutely, um, and so I think it can be, it can come from a few different reasons, but I, I do have one example in particular working with a client, like many organizations, they had all in-person programming. COVID happened. They had all virtual programming, and now they’re moving forward, and they want to know the best direction. What is the, what, how can they bring together all of those resources they just put into virtual programming, a high, high demand for their services. And their love for in-person programming. And so what we did was we used program evaluation to answer that exact question. What is working well about virtual programming? What’s maybe not so working? What was their work? What, what, what kind of programming were they doing? Yeah, they do, um, financial coaching, um, as well as, um, sorry, financial coaching and Uh, education, financial education, coaching, um, and so, so they have kind of, you know, this, this whole course that they typically do in person, then they shifted it all virtual. Now they’re kind of in a blended approach, and they’re saying, well, if we continue virtual, we could rapidly expand. And geographically, and that’s not just a self-serving thing, but folks all through the world could use financial education and coaching, especially when we’re talking about closing the homeownership gap, um, for people of color compared to, to white people. And so, um, they brought me in, they brought in our team, and they said, how do we go about this? How do we, how do we kind of design an evaluation for this purpose? So we brought everything together, we looked at the quantitative, we looked at um the organizational data, we looked at operational data, and then we also looked at things like test scores, surveys, things of that nature, attendance scores, and also brought in the qualitative. So trying to kind of get the how, what, um, output side of things with the why, and so what qualitative side of things. Bringing all this together to highlight where my virtual programming maybe have some shortfalls, and how can you kind of close those gaps based on what your stakeholders are telling you, and where is there incredible opportunity for virtual programming? How can that, um, how can that be coupled perhaps with satellite offices or something like that, something of that nature, to be able to expand your reach and really change the way that you’re able to do your work? It sounds like a lot of that work was surveying. It was, honestly, what we, what we really leaned into was operational data, existing operational data, which this is gonna be something you hear me repeat. Let’s use what you have, you probably have some great stuff. Um, and then, and then focus groups and uh collaborative evaluation layered on top of it. So I worked with their entire staff. To have consistent learning workshops. I think, again, trying to get, trying to get to that cultural change of evaluation isn’t an audit, it’s not a test, it is a learning opportunity, it’s a, it’s a cultural change, um, and, and kind of coming together to look at the numbers and then match them with interviews and focus groups to add more context and depth. Focus groups. So you, you, uh, facilitate focus groups. Is that facilitate, moderate? I don’t know what it’s called. What, what is it, facilitate, facilitate focus groups. Yeah, OK. Um, how do you, uh, how do you get people at ease? This is something I always wondered, uh, in a focus group now, so that was a program around financial literacy. Um, how do you get people to open up about, you know, maybe where they started before the, before. Working with the, with this nonprofit, you know, that, that they were, I mean, They were essentially saying I was financially illiterate because I’m in a financial literacy program. Like how do you get people to open up, not just about that, but, uh, about, you know, the, not talking about money. Like people would rather talk about death or go to the dentist, I think, than, than talk about money. How do you get people to open up in that kind of, in a, in a group setting? Oh yeah, you are like nail on the head there because it is so hard to even recruit people to come to that setting, let alone open up the door $50 gift cards or I don’t know what, but yes, uh, yes, exactly, um, yeah, so definitely compensating people for their time and expertise, sharing food, breaking bread. I mean trying to understand like what are things that generally are going to Provide comfort and then I think really flipping the script a little bit, right? So, talking to folks about we are navigating a broken system, an intentionally broken system. There’s a system in place that seeks to hold this information back from you, that it tries to hold communities down, especially when it comes to financial wealth and power. And you are here as experts within your community, right? You are here as people with lived expertise, community expertise, network expertise, and that’s what we’re gonna talk about. So what we’re actually gonna talk about in this focus group is how you as experts in the room, see this program lifting up those around you. What are the ways in which you See it lifting people up in which you see it perhaps letting people fall through the cracks, which is something the organization really wants to reconcile with. How do we, how do we build some safety nets in there? And how can we all that work together to kind of overcome this systemic effort, um, so really, you know, kind of de-individualizing every single component of it and saying we’re not here to necessarily talk about you. But you as an expert, right, in all of these different areas and how you can help us, um, really have a reciprocal exchange of information about this. That’s excellent. So you, you, it’s an appeal to community, that you can, you can help us help the community if you’re, if, if, if you share, if you’re, if you’re honest and plus you mean that that’s, uh, I think that, I think most people do want to help the community. Uh, Especially when they were helped by the program. Like they wanna, they want others, it’s almost empowering. I can help, I can help, I can help this group help others if I, you know, if I help them. Um, plus you mentioned meals. I happen to, I’m, I’m writing a book, uh, it’s on planned giving fundraising, but I have something, I, I love shared meals. I mean, I, I do, so I do plan giving, fundraising consulting, so I do a lot of individual like donor and prospect meetings. I love over a meal. You said breaking bread, you know, sharing a meal. I even, I even created a, an acronym like meal. Meals expertly allow learning. Oh, I love that. I know, I just, you know, it’s just something like cute for the book, but, but, but it does, you know, the shared space, the shared, you, you know, you, you like you’re at a communal table with the focus group, right before, I assume this is before or maybe even during their eating. People are literally sharing that physical space of the table, and they’re sharing the food. Yeah, that’s a metaphor for what you want them to do. You want them to share. It is, it’s perfect. So there’s all of those layers, right? And then there’s just this really basic layer, Tony, which I’m sure you’ve seen, where it lets people kind of crack their voice a little bit, right? So they get in the room and you’re nervous and you’re like, kind of, why am I here? But then, we’re not gonna talk about that first. We’re gonna talk about like, isn’t, aren’t these ribs amazing? Isn’t this burrito bowl incredible? And just start talking and just warm up our voices. Warm up our, our relationship with one another, much more smooth sailing than just getting in the room and saying, here’s the consent language, let’s go. Yeah, yeah, right, right, right. Were, were they beef ribs or pork ribs? Pork. OK, um, I live in North Carolina. It’s gotta be pork ribs, but I think Texas, Texas, I think is the beef, well, Midwest generally beef, more beef ribs, but, uh, yeah, pork ribs. Did you have pulled pork too or just the ribs? We did all sorts of good meals. Um, we did, we did see some pulled pork at the corn, and that was the thing too. We revealed, we told the folks the menu in advance, and then we had all these women making cornbread and so then we’re talking about cornbread recipes. I mean, it’s just a, it, and I’ll note here, the focus group’s long. If you haven’t caught on, the focus groups not a short. Endeavor because we got to show up, we got to make our plates, we got to talk, we got to share this cornbread recipe, and then we’re gonna get into it. I love that. I love people came with their homemade cornbread. Outstanding and then they’re sharing the recipes. Oh no, I use canned corn. No, no, you got to use fresh. No, canned corn is better. No, OK, no, I love that. All right, sorry, sorry. Um. So, all right, how do we overcome the, these, these challenges? I mean, how do we like sort of get nuts and bolts, you know, start to, like, what should we be thinking about or, you know, if we, even if we don’t have a, suppose we’re not even a grant recipient, but we do want better data. We wanna, like you’re saying, we have the data, but we need to, we need to coalesce our data so that it can help us. Maybe it’s not even for a third party. Maybe it is just for our own purposes. Uh, what do we need to like think about? I don’t know, culturally, leadership-wise. You know, I like, I like steps, uh, if we can, and you know, I’m not that you have to say 12345, but, you know, I like steps that people, takeaways. I like takeaways that people can take action on or at least start thinking about. Absolutely, and I think about it as a framework, right? Because we try it out. It’s just a general idea, um, and I do a lot of this, so like I said, I do a lot of coaching, uh, which is more about one on one coaching, as well as fractional management. So if you just need someone to come in and kind of help your team, you know, navigate this real sea change in evaluation culture, we can come in and kind of manage that for you, um, and, and, you know, leave, leave you kind of with that. Uh, learning system in place. And so what that tends to look like is we start off with an inventory of what you have, what’s already going well, what do you already, um, answer in terms of what data, data points you have, where you have interest, where you have staff interest, and really bringing people together in that identification, because on one hand, yes, your team’s, you know, filling out those spreadsheets. On the other hand, What you might not have a finger on the pulse of, is that your country manager is always having lunch with um their main volunteer, and is kind of always doing a bit of a pulse check in terms of how things are going with them. That’s data. Formalizing that is the challenge. So we start off with just this inventory, what do you already have in your pocket? Then we start to think about what are the strategic questions that are going to guide learning. So, they might ask, you know, to what extent are we achieving our outcomes, with whom, for whom, under what circumstances. And they’re gonna examine the process by which you do that, the appropriateness of that process, the relevance of that process. So we’re just gonna put together those questions, we’re gonna then put everything on a grid, right? So we’re gonna take those questions, we’re gonna put those across the top, we’re gonna take all of your data sources, all of the work that you’re doing, put that in rows, and identify where you have overlap, where we’re answering those questions with the data we’re already collecting. And examine it through the lens of breadth and depth. So breadth is participation counts. It’s the number of people you reach, the number of activities you you were saying before, the outputs. Exactly. Bread is going to tell you how far your work is going. And then depth. So then we’re gonna look for depth. How are we going to hear about stories of what that actually looks like on the ground, um, as well as perhaps case studies of how this one person, like, really went all in on the 4 programs we offer and changed her life in 9 months, like, really. Really changed her whole, her whole way of operating. Those are our depth stories. That is usually where we need to turn things up a little bit. So, we go through this table, we look at this, those are all depth measures, these are breadth measures, and here’s where we need to kind of add in some new tools, or maybe just add in questions on the tools we already have. Get all of that in place, we talk about a reasonable implementation timeline, reasonable being key, if it’s not reasonable, it will break and it will fall apart. Um, But then the most important part that I think tends to get forgotten is when are we going to have our learning moments? When are we gonna come back together? Is it gonna be quarterly? Is it gonna be monthly dashboards and then quarterly conversations? But really kind of reorienting ourselves again, how are we going to learn? When are we going to learn? In what ways are we’re going to learn? And that learning tends to look the same as that. Focus group I just talked about. We’re gonna get together, we’re gonna set aside enough time to have a meal, to talk, and to get out of our daily routines so that we can really come and think strategically about our work. And what can we incrementally look for to celebrate little, little moments, whether, you know, it’s a monthly dashboard or a monthly meeting, a quarterly meeting, whatever it is, you know, like what, what movement. In our, in our, in this data formalizing process, can we, can we see incrementally? Yeah, and one of my, uh, one of my fractional management clients was such a great exemplar of this, because within the first two months, we got together, we identified their data systems, and we found key quick changes. One of those changes was we moved from one spreadsheet. Um, that wasn’t linked to anything, it wasn’t talking to anything else, right? They were having to work really hard on their data. We replaced it with a form that spoke directly to a dashboard. And so now their data is working for them. And within 3 months, we changed the way their team meetings were structured. So instead of each of their program managers coming together and trying to remember, oh, in the last month, we held this training, I think there were 30 people there, um, yeah, it felt like a success. People weren’t that engaged to doing these kind of like report outs to instead opening up their team meeting and Having their dashboard and having different program managers be like, Whoa, you had 100 people come to that workshop? What on earth happened? And they were like, Oh, right. I found this nurse who works in churches, and she told everyone about our workshop, and all of them came. And just having like these really amazing learning moments that changed when we stopped making people work so hard on their data and started making our data work so hard for them. It’s time for Tony’s take 2. Thank you, Kate. Someone needs your opinion. Uh, the, it’s a, it’s a former guest. She’s been on a few times, uh, Karen Graham. Very smart uh data scientist, a tech person in the nonprofit community, and she is leading a donor management software study. So she wants to get information about how you use your CRM or even if you don’t have a CRM what you, what you do in place of that, just everything around your, your donor management software. So I’m helping her to encourage you to participate in the study. Uh, N10 is also involved. They’re also, I think Karen’s leading it, but N10 is helping fund it. So, you know, how closely we partner with N10. Amy Sample Ward, of course, being our tech contributor. She’s the CEO of N10, and of course, I bring the podcast to N10’s conference each year. This year we’ll be going to Detroit in March. So, helping Karen, helping N10. I would ask you to participate in the study. If you go to my LinkedIn this week, you will see, uh, just check the posts, and I will have posted, uh, I have posted about it this week. Uh, and the link to participate in the study will be, will be there. It’ll be, uh, in my comment. So that’s it. So I hope you can help out Karen, help out N10. By participating in the, in their donor management software study. I thank you very much. And that is Tony’s take too. Kate, Kate, like my voice just cracked. Thank you. Like I’m 14. I was gonna say something and now I forgot, cause now I just wanted to make, I wanna make fun of you for your, your voice crack, but. Whatever I had, it was good, but I’m sure you were going to encourage listeners to participate in the study in the study, yes, please go participate in the study, um. Kate We’ve got booco butt loads more time. Here’s the rest of Bring fun to evaluation and Impact data with Kayla Meyers. You teach, um, program evaluation, survey design. Where, where is, where is that you teach? Yeah, the Humphrey School of Public Affairs at the University of Minnesota. Oh, Hubert named, they gotta be named for Hubert Humphrey, the senator from Minnesota who ran for president in, I don’t know what election, presidential election, Hubert Humphrey, right? Yeah, yeah, you got it. Um, so what’s that like? What, uh, do students need a, a, a, a, well, let me ask you, uh, uh, jump all around. You got a lackluster host. I’m sorry. Do you, do you have a background in, I don’t know, mathematics or data design, data analysis, data design, computer science, or you like Spanish, Spanish and music major? What, uh, I, I, so this is, uh, and this speaks to how I ended up in my career. I am a generalist with a generalist training, so I got a degree, an interdisciplinary degree in development. Um, so, economic development, community development, etc. where I took a different, uh, every semester I was taking different classes in College of Health, College of Natural Resource Management, um, because I was like, well, I love all of this. I love the intersection of how all of these systems work together. And then, uh, got a mixed methods, had a mixed methods focus, so I was taking qualitative analysis classes alongside my statistics classes and policy analysis classes, um, and I, I am a graduate of the Humphrey School of Public Affairs, so I got to come back, uh, almost a decade later to then teach the exact classes that were the inspiration for my career. That’s outstanding. You’re teaching the classes that were the inspiration for your career. I love that. And, uh, do you find, uh, a lot of interdisciplinary or like self-defined majors in, in your classes? I think, you know, uh, the biggest degree program at the Humphrey School is the Masters of Public Policy, which is inherently, you know, allows itself to really be customized quite nicely. Um, but then we also do have students from, you know, the business school, from the education school, we’re kind of converging into this one space. We, I really teach the class, um, because it’s at the, at the public affairs school with a lens to policy, um, and a lens to public affairs. Work. Um, so, you know, if you take an evaluation class at an education institution, they’re really gonna focus on evaluation and education, or the public health school, it’s really gonna focus on uh like health evaluation. But mine is really looking, um, a little bit more both broadly and focused on um public face. programs, so how the uh programs that are intended to really help the public, how do we evaluate those and really measure that, especially because most of the time the public is interfacing with much more than just one program. They’re, they’re interfacing with all sorts of different nonprofits. Uh, do you teach at all, uh, in person, or is it, or is it all virtual? You’re doing it in person, all in person, which is fantastic. That’s excellent. Yeah. So now what is that like? Are you looking at a sea of laptop screen, you know, the back of laptop screens? Is that how, is that how students in college take notes? Um, you know, I, like when you look up, you see a bunch of like Apple and Dell and Lenovo logos and, and glowing eyeballs because their screens are lit up. Is that what you see or what? I’ll give them credit. Their laptops usually have some great funny stickers on them, so I do get that. Um, the class is 3 hours long, and so we only do lecture for about 45 minutes, and then we, we switch it up. We do a lot of applied learning, a lot of, um, a lot of group work, a lot of, you know, facilitated discussion. I’m, I’m trained in technology of participation and art of hosting, and I bring a lot of that into the classroom because I don’t just want them to learn the concepts of evaluation, I want them to learn how to facilitate an evaluation. And so, I will use a lot of times facilitation methods, but I just want them to learn by having them go through it. So we host world cafe conversations where they then get to learn, like, this is a world cafe conversation, this is why you use it, this is when you use it, great. Now we’re gonna use it to talk about how to apply this learning. You said you’re trained in technology and participation. Is that participation, the technology of participation. What you, I think you just gave an example of it. What, what does that mean generally, the tech the technology of participation. Yeah, and I, this was the, I think it was designed by a group. It’s ICCA, um, International Culture, I’m gonna forget their acronym. It’s basically putting together neuroscience and facilitation. So how do we, you know, overlay what neuroscience tells us about how people think and process information with how we how we help them navigate a conversation. One specific example that, and then your listeners can look this up and use it for their next conversation. Is the OID framework. So the arid framework O O R ID. Yes, thank you, O R ID observation, reflection, interpretation, decision making. OK, so we’re gonna start off when we bring a group together. Perhaps we have a data dashboard, right? This is something I, I will often use whenever we’re, um, coming together over some data. We’ll start off with just what do you see? What stands out to you? Or you have questions. What’s surprising? What’s not surprising? We don’t have to ask all these questions, but these are all observational questions. Interpretation. What, um, what seems, wait, what was the R? Wait, the O was observation. What was the R? Skip the R, yeah, you went from O to I. OK, oid, it’s no good. You can’t, you can’t redo the oid into oid. OK. Reflection. What, what, what are we reflecting on? Reflecting, um, what, what stands out to you, what feels critical, what feels uh desired, what feels undesired, right? So just reflecting, like, how are we, how are we feeling? So what do we see, how do we feel about it? Interpret, what does it mean for us? Why, what feels of most importance? What feels of most importance for external communications? What feels of most importance for decision making? Where do we need to dive deeper? So, interpreting that information. So, here’s what I see, here’s how I’m feeling about it, here’s what I think I need to do about it, and then decision-making. So, what’s, what’s most important for us to carry forward the next steps? What next steps are we seeing here? Who needs to be involved in the next conversation? So where do we go from here? All right, cool. Thank you. Taking a little digression into uh technology and, and. Wait, what do you, techno, I just lost the word participation, participation, technology of participation. Thank you. Um. Well, so what about, you know, we can, we can talk about some of the tools. I don’t know, are there, are there tools that folks can use, or, or, or is everything, I don’t know, proprietary to Bridgepoint or like are there, are there tools that we, that we can offer resources? Yeah, absolutely, and, and I truly, I’m in this work because I want folks to take on evaluation. So I can tell, like you’re, you’re passionate about, about evaluation and, and program management, measurement, and no, it’s, it’s, it’s moving, like you’re moving. Your, your passion is, uh, is convinced, I’m, I’m, I’m feeling it. What’s the word I’m looking, it’s uh, yes, it’s palpable. Um, our tagline is we turn data into stories and so can you. That’s the first phrase you’ll see when you open up our website, cause that’s what we believe. And so, I often work with folks to think about a data braid. So when you bring everyone together, and sometimes when I like to bring a lot of tactical work, um, into, into my conversations and so, and so if we show up together in person, we might pull together some yarn and, and start actually braiding together pieces. But think about your data braid that your organization is pulling together. The strands are going to be qualitative. They’re gonna be stories, they’re gonna be um those mission moments that really bring your team together, that they say, I was just talking to this one participant and gosh, I just, I, I was re-energized, I’m ready to go. They’re going to be your quantitative measures. Maybe you have some pre-post survey or some pre-post test scores. Maybe you have um some confidence measures. Those different quantitative measures. Um, as well as your operational data. So, you’re looking at number of people coming to your trainings, number of trainings worked. These are all different strands within your data braid. And what you, first thing that you wanna do is make sure that those strands are equal, right? So if you have a data braid, and one strand is a string and another is a thick piece of yarn, How do we kind of change that up to make it a little bit more balanced, to make sure that we’re properly pairing things like pre-post test scores with qualitative stories that help us understand why the change in test score is important. Um, I often work in food systems. So, working, you know, whether that be with the regional food supplier or aggregator, right? So thinking about those agricultural producers who bring together all these different products and, and provide them in a CSA or provide them to local schools. Um, so, I work on that side of things as well as the food shelf, food distribution, food bank side. And a lot of times what we hear is the number of grocery bags sent home with students on a Friday, right? So, the idea here being that, that all of us can assume is that now this family has a meal for Saturday, which is really vital when we’re talking about students and making sure families are fed cause those 5 days a week is so important for families. What we don’t hear is why that is so important. And so when we start talking to participants and we hear them say, well, actually, I matched it with some of the food I already had in my cupboard, and we went to my mom’s house, and my kids were able to have time with their grandparents, interacting. I was able to check on my parents and see how they’re doing, and Groceries were taken care of, I wouldn’t have been able to feed all of these people had it not been for some of the supplemental food. And that resonates with people. People get that. That is why people want to do this work. Right now it’s grandchildren with their grandparents, and, and there can be a bountiful meal. Yeah, right. And so that’s the grade, right? Like we just put these pieces together to really tell a holistic story of what your organization is doing and how it’s, and how it’s going about doing that. Cause what we also heard in this is that she didn’t necessarily have enough to feed all of, are they all her dependents, right? Is she, is she a caretaker for her, for her parents there? And so we can start digging into that too, and saying like, hmm. I’m hearing this amazing thing, and maybe we can expand our definition of dependence in our next intake survey and hear from folks, like, is it more than just the people in your household that you’re, that you’re providing meals with, that way we can adjust the, the amount of food that we’re sending home with you. All right, Kayla, what else, what else can you share in the second part of your mission that, and, and you can too. So the braid is excellent. The braid is excellent. I love, I love the metaphor of the braid equal, you know, because if you have a ball and a strand, your braid is gonna be, you know, well fecacta. It’s gonna be messed up. Uh, it’s not, it’s gonna be weak. It’s not, it’s a, it’s not gonna be symmetric. So that’s a, that’s, that’s a terrific metaphor. What else for the, uh, and, and you can too part. Yeah, I think, you know, really thinking about how to bring your team together for collaborative learning. Um, and so, and so I kind of brought some of this up, but a lot of what we do is grounded in fun. Um, so I, I actually took a lot of trainings in youth-based evaluation, and so we’re learning about theater exercises and play-based exercises. And my main takeaway was like, grown-ups love having fun. Why aren’t we trying this out? Um, and so, So, some of the things that we do is we do data gallery walks where we, um, we bring everyone together and set it up almost as an art gallery of charts and quotes and, um, and different, you know, tables and just invite people to walk through it like a gallery, right? Where some folks are gonna be silent observers, they’re gonna be reflecting, other folks are gonna be having quiet conversations to help say, like, what are you seeing here? I’m seeing this. What do you think? Um, and then facilitating a large group conversation. So, all to say, how are you learning? How are you collaboratively using the information to bring it to life, to tell the story of what’s happening, and help you learn the story of where to go next. The braid, the data gallery. You got one more. I love threes. Is there one more that you can, one more. OK, great. Here’s a, here’s another fun one, and I learned this from a colleague, um, a data escape room. This one is, is so fun if you, if you just want to get people together to be silly. Um, but take your data dashboard. So instead of that gallery where you put it all over the walls, put it on a dashboard, break into teams. I recommend shaking people up, right? So if you have a programs team, like shake them up, mix them all up, get them mixed up, right, right, right, yeah. Um, and now you’re going to start a timer and you’re gonna tell people, these are all clues that come together to tell a story, maybe 3 stories. You escape the data escape room when your team can cohesively tell a story. So you need to make sure there’s no gaps. You need to make sure all these data points are kind of coming together to help you. Understand the story that’s, that’s coming out here. And my students know this one because I do it in class and I, we always have a blast with it. Um, but it really helps people examine their information, examine their program data, examine their organizational data, and put it together with different quotes, with different case studies, um, with different test scores, things of that nature, and be able to then. And say, like, here’s a comprehensive interpretation of what I’m seeing, and here’s some things that are still a little bit of gaps for me. Like, here’s, here’s some things that I, I want to learn more, so I can really test and prove this. Um, and so we come back together and have a, have a large group discussion, and that’s where a lot of the learning and revealing new insights comes out. I love the data escape room. It’s so fun. Yeah, that’s, it does sound like fun. All right, all right. Um. What about, um, some, some challenges, like, you know, obstacles we, we’re likely, oh, you know what, before we get to the obstacles, and see, I told you, lackluster host, um, this sounds like we, this is, this is, this is a, uh, An endeavor that is gonna require, you know, leadership buy-in, right? I mean, you’ve got to get, because this is, this is gonna take some time. Um, folks are gonna come together, you know, even, even the quarterly meetings can be fun, it sounds like, but still, it’s taking time away. We, we need leadership buy-in for this initiative, right? Mhm, yeah, we do. We do, and I, and I think just keeping in mind we’re already spending that time, we’re already, we’re already doing these things, right? So when we are having staff all fill out their own unique spreadsheets and then we’re having a maybe a grants manager or someone take all of these spreadsheets and input them, like all this manual data entry, all of this, you know, kind of rote work that’s going in, that’s, it’s time already spent. I’m saying like, let’s take that, instead make it into a system where you’re not working so hard on your data, but your data is working much harder for you. And it, and revealing what it can do for you. So, you know, think back to that team meeting too, right, where we’re starting off our team meeting the 1st 20 minutes are folks trying to remember what on earth they did last week and think about things they can share with their colleagues. We’re gonna change that. We’re gonna instead make sure that our data is working harder. It’s in a better spreadsheet, it’s in a better dashboard, and we’re gonna have a much, our conversation’s gonna go much further. I love systems. I mean, I think they’re essential systems, processes, you know, the every, we have this, we have this cadence, but each time is not the first time we’ve done whatever it is we’re talking about. So let’s have a, let’s have a process so that it, it just becomes easy, it becomes natural, becomes routine instead of it being an event every time. Right, right, and that’s, and that’s, you know, something that we love to bring together, right? It’s like we are, we are the fun folks. We’re gonna come in and have a data escape room, and we’re gonna be able to convert your spreadsheets into Microsoft forms that go through Power Automate and show up in this really nice Power BI dashboard. Like, we are, we are nerds who can dig into those different things and still share a great meal and have fun, uh, at the, at the AO meeting. Can we have a glass of wine with the meeting, or? Absolutely, we’re always in. OK, OK. And I love you, you, you’re coming in like we’re, we’re the fun. We’re gonna have fun with data. Like people are like rolling their eyes. Yeah, I don’t think so, but I’m sure within the 1st 10 minutes or so, you, you change minds. Um, all right, so I was starting to ask you about, you know, obstacles we might see in our Data, our measurement and data analysis. Uh, flow, our, our, our process, what, you know, what are some common way, things we might see and how do we circumvent them? Yeah, I think one of the first challenges I often encounter journey, that was the word I was like, journey, journey. I tell you, lackluster host, I told you this, this journey that we’re on together, that’s what I meant to say. Yeah, all right, sorry. Yeah. No, you’re good. So the journey, so I think one of the biggest, um, obstacles on that journey is folks are gonna struggle with how much do I leave behind and how much do I, do I kind of retroactively fit. Um, so, for example, when we’re talking about a new way to enter in your data, right? That is going to give us a bit of a bump in the road in terms of, well, do we want to program all of our past data into this new dashboard, or are we just starting on day one and moving forward? That’s a really important conversation to have, to talk about the implications of either decision making, um, but we can, we can think about the ways that we can utilize your community, your volunteers to help us out with that backlog if we need to do that. Um, and, and that shows up in a lot of different ways, right? It’s kind of like, if we, if we refine and change our systems right now, how does that impact us in terms of having this longitudinal story, this longitudinal measure? It, it might, um, but at the end of the day, if you have a longitudinal story that doesn’t really represent your work, doesn’t go that deep, can’t tell you that much, how valuable is it, right? So, let, so let’s talk about setting up the framework for a longitudinal, uh, work moving forward. The other The other challenge that’s gonna come up is that um you’re gonna have funders who want different measures, and that is a pain point that a lot of organizations go through. Um, so, you know, any funder or any uh development team will tell you that getting in those diverse types of funding is really key. The challenge is that when you bring in all these different funders, they might have all their own reporting requirements. And so how do you build out a system, an evaluation system in which you can meet all of those requirements while still being proactive about your learning, your strategy, your decision making. Um, that’s where I think it can be really helpful to bring in, to bring in a team, because it’s a heavy load to lift at the front end. Once you have it figured out, you’re good to go. OK. Only 2? Only 2 obstacles, uh, to our journey? Those are the two that pretty much always come up. Um, yeah, those, those are the two that it’s just they’re, they’re just real challenges. They’re just in the world and, and I think, you know, staff. Staff feelings and staff sentiments kind of like you brought up when we started this, right? Like getting people to come to an evaluation workshop is like getting them to show up to algebra. Like no one wants to be there. Um, I actually had a participant come up to me after a recent workshop and she said, I’m, I want you to know that they made me come here kicking and screaming. I I brought out the entire book. I’m too busy, I’m sick, I’m this, and leadership said, please go. And now, I actually have tangible tools, right? Let’s, like, once we get out of this, um, oh, can we do, you know, a cross tap here? Can we do a T test there? People just gloss over it, right? They’re like, oh my gosh, let’s move that. Like, that, that doesn’t need to be. Something you have to worry yourself with is like, what is a T test? What is she talking about? But instead saying, here’s the results, here’s what this actually says. What we know is that the more times people come to our workshop, the better their savings account grows. That’s what the main takeaway is, that’s what I’m trying to tell you. And kind of get yourself away from some of that jargon, some of that. You know, just that middle school math feeling of like, oh my gosh, why is there a letter in this math equation? I don’t understand what’s going on. Um, and I say that as someone who, who did really struggle with, uh, with some of the, some of those math lessons, and so I, I understand where people are coming from. That’s a great place to wrap because it’s, it’s uplifting. Uh, T, you talk about T test. I took statistics for poets at, uh, Carnegie Mellon University. So, you know, what, uh, that’s, that, I recognize the phrase T test. All right, Kayla, professor, Professor Kayla Meyers, outstanding. Thank you. Founder of Bridgepoint Evaluation, the company is at bridgepointevaluation.com. And Kayla is on LinkedIn. Kayla, thank you so much. Thank you so much. This was fantastic. I’m glad. I think that it’s fun. You make data fun. I try. You’re succeeding. Now, a lot of people are trying. You’ve, you’ve, you’re, you, you’ve succeeded. Next week, grow your personal brand and your nonprofit. If you missed any part of this week’s show, I beseech you, find it at Tony Martignetti.com. Our creative producer is Claire Meyerhoff. I’m your associate producer, Kate Martinetti. The show’s social media is by Susan Chavez. Mark Silverman is our web guy, and this music is by Scott Stein. Thank you for that affirmation, Scotty. Be with us next week for nonprofit radio. Big nonprofit ideas for the other 95%. Go out and be great.