 My name is Kiskeya Addison. I am Director of Consulting at Just Tech. And we are here to talk about RFPs and bender selection. And we have our esteemed panelists, Susan Cho and Simon Raub, who will discuss with us their own process, the trials and tribulations of creating RFPs and also selecting the right bender. As we were talking about earlier, selecting the right bender can almost seem like a marriage. It's a relationship that's trying to figure things out together and work together. So I think we should just get started because we have a full party here. OK, so I put this one on. We're really excited, right? We just found out we got funding. We have the money to do that project. We did some kind of needs assessment potentially and we know that this is what our community is asking for and we want to get it done. And so we're in that super excited. We're ready. We're going to get this done. And we're just like, let's do this. And then we're just confused, right? Because we're trying to figure out, OK, how do I do this? The work that needs to be done can't be completed internally. And maybe I knew that, right? So I already thought I will have to look for a contractor or I may have to look for a bender. But maybe I haven't actually created the documentation to help me do that work. So we're here today to start helping you just think about, OK, one of the things that think about as you are developing your first RFP or your second or your third. And then what is the selection process that you can use that is, we would say, equitable, right? That's fair and that we can ensure that we have also a trail that we can show people. This is how we made this determination, this is the vendor we work with. These are the requirements that we set forth. And it just helps with our relationship and future. So today we're just going to talk about RFPs, vendor selection. So how do you develop an RFP? I remember the first time I had to develop an RFP. And honestly, what I did was I scoured LSNTAP and I looked for other sample RFPs. And then I talked to people in my office and I asked them for some samples. So I just looked for everything I possibly could to see what was out there. And they were all different. Some of them were one page. Some of them were 10 pages. And so I tried to figure out what was that happy medium where I'd be able to determine what do I need to communicate and what do I need from a vendor. And so we're going to talk about what to include in the RFP, how to create the criteria to select. And part of that criteria is so we can ensure equity and fairness. And then what are just some DIB and GI considerations? So diversity, equity, inclusion, belonging, or justice, equity, and diversity, and inclusion considerations? So I'm going to pass it over to Susan, who will present. OK, so we're talking prior to and we're talking about whether Simon and I are RFP gurus, I will tell you I am not. I think I struggle with this as much as anyone. And in fact, I'm going to talk about a process where we did our own kind of version of an RFP process because the task of what we were trying to accomplish was so daunting in a way that we had to re-think that RFP process for a project. But I'm going to talk about here are some key components for an RFP. I'm sure everyone has worked on an RFP, everyone has seen this, everyone has said, this is the general rubric. So project overview, basically you want to give one or three sentences about what the project is and then a description of your organization or company. A lot of companies will say, this is where we are in our market positions. This is what we do. But I'm going to assume that for the most part you folks are nonprofits. So talk about who you are, what you do, what's important to you, and also, who's your target audience, low income clients maybe in terms of, or maybe you're part of a particular niche within a nonprofit. But make sure that because this really is a marriage or like dating, we talked about that as well. And so they have to know kind of where you're coming from in order to get a sense of whether this is a relationship that they want to try to engage in. So and then I want to talk a little bit then. I'll talk a little bit more about some of these things. And then I'm going to break out scope of work. And then in terms of, I'm going to talk about it within a case study of the Ohio Legal Help, I'm going to say redesign is not a strong enough word. I would say complete build in terms of how we approach that. Timelines are important because this really is for the vendor. They need to understand both the scope of the work, what it entails, what your functional requirements, technical requirements, what infrastructure are they working with, what's your goals. But at the same time, they need to make sure that your timeline is going to work in with their current, like staffing structure. The software developers are who they have available. And so that's really critical. Now, in terms of the budget, I put optional here. I'm sure every time I've done an RFP, the vendor would have loved to know what's my kind of top line budget. But sometimes I don't want to know that. I really literally just want to know what they're proposing to meet the requirements that I've set forth and what they're going to cost it out there. Because they'll give me an opportunity then to look at different proposals. They'll give me an opportunity then to think through each of the cost lines. And sometimes it's not the cheapest proposal, in a sense, the lowest proposal that you'll want to go with. Because you'll want to think through that relationship. You'll want to think through all the plus factors that you want to think through when you finally select the vendor for your project. And then Simon's going to touch a lot more on this, which is the selection criteria and submission instructions. But what I thought I would do for each of these components is to kind of go through an S of W that we recently did in terms of what some of this might sound like, if you will. And I'm also glad to share one of our S of W so that folks have a sense of the detail sometimes that you need to get a good bid. Because unless you really lay out what your minimum kind of functional requirements are, it's going to be really hard for the vendor to make an appropriate bid. So for instance, I'm going to talk a little bit about. So Ohio Legal Health is on Drupal 9. We use a module called Web Forms within the Drupal infrastructure to build out our document assembly. It's integrated with a bunch of third party products, which is important to know. But we would have talked about that in the RFP in terms of the infrastructure in which we're asking for developers to build. But in terms of the project overview, basically what we were getting a scope of work for or a project bid for was we wanted to be able to enhance our current Web Forms infrastructure so that we could scale up forms quickly. So that's our goal. And then in terms of our project overview, what we wanted to do was to have the ability to use our Web Forms infrastructure to create individual data sets that could then be collated into our document assembly. And those data sets needed to be reusable as well. And so that was our project overview. Description of who we are, we always talk about how legal health, our history, recommendation, our placement within the continuum of services and then our role within the court system and so forth. And then our project goals, this is important because while we're going to talk a lot about the technical aspects of what that build would entail, our project goal here at the end of the day is to make it easier for my staff to scale up forms for multiple jurisdictions. So by having data sets of information that are reusable across multiple collated data sets and forms, it makes it easier for my staff then to create something from Montgomery County, something for Cuyahoga County, something for Hamilton County. And we're not a unified court. And so even though there are uniform forms, each court likes to add, let's just say, its own special little nuance to that form. And so that's the reason why we're doing this. And so our goal here is really about scalability, about forms, about standing up product faster for both the constituents and our customers, which are our users, but then also to accommodate some special requests from courts. And so we want to make sure that folks understand that. Now, in terms of the scope of work and those deliverables, that's where it gets tricky. So go to the next slide. So functional requirements, how to develop requirements. This is tricky. So I will tell you, because of everything we kind of learned through our build, we were able to create kind of the critical functionality and lay that out much easier. So we had in terms of our overarching kind of, like high level, if you will, functional requirements was, we said essentially an administrator has to be able to use the same data set within our infrastructure for any form. And we kind of went through kind of each of the things. We made sure to explain that, what we wanted the user experience functional requirements to be. And so we kind of laid it out within kind of a user experience functional requirements. We laid it out within kind of some of the backend requirements. And then we also laid out things within the infrastructure within which the vendor would ultimately be working. So obviously we're Drupal 9, we talked about that, Webforms modules and things like that. But then we also laid out our preferred project management, we're Jarrah Shop. So we wanted to make sure that folks understood that, because that's our workflow. And so we wanted to make sure there was a good match on that. So we go to that level of detail in terms of the project management. And then in terms of the testing in QA, how that will go. And then we always ask and make sure that folks understand that we want documentation. I think that that's something that folks sometimes miss in an RFP. We always make sure that folks understand that we will want the documentation. So while we have our own Jarrah, it's not always what folks use. And so what we'll make sure is that, we make sure that they're a Jarrah Shop. And then you can actually ask for the Jarrah export afterwards, because what we don't want to do is to, we want to make sure that we work within a vendor's workflow as well. We don't want added costs just to adjust to us, if that makes sense. So then one of the things that folks asked was, so what if you just don't know where to begin? We know our product at this point to a point where it's much easier for us to build out an RFP. We could probably do an RFQ and you'll hear different terms, right? So an RFI is where, and I think Simon will touch on this as well. So some folks will do an RFI, which is more of a question for information. That's where you're, it's a little bit more lucid usey, so it's kind of in between, right? So an RFP is in between its request for proposals. And then when I was with the state of Ohio, sometimes we would do an RFQ, which is we knew exactly what we wanted built. We knew exactly the way we wanted it built. And we literally asked for a quote for a vendor to build exactly in that way, right? So that's an RFQ. So there's multiple kind of in between. Now for an RFP, the functional requirements is often the, I think, especially when, let's say you had an idea. I'm gonna build an app. I want to build an app for low-income client to be able to, let's say, maybe put together an eviction defense with an answer. Let's say that, right? But you're not exactly sure where to start, but you have this idea. This is what you got funded. You knew that you know what the need was. You knew all this and then, and you have this amount of dollars to spend, right? And so you need to put together the functional requirements. So what we did with Ohio Legal Health because it was a complete build of a website and we weren't sure what the stakeholders would need. We weren't sure what our clients would ultimately need. We actually hired consultants to help us put together a statewide portal, if you will, with the components. And folks have heard this from the Pew, right? About asking, refining, right? And then, so we put together a group of consultants to help us work with one. We identified the stakeholders that would be involved. We then did landscaping to identify the needs, right? Of our primary target user, low-income Ohioans, right? And then we identified, based upon that landscaping, what the gaps were and what the needs were. And what we learned when we did both the voice of stakeholder and voice of the customer was that there was really great alignment. Folks requested forms, so simple, easy, document assembly. And folks requested information about how to connect to lawyers and other local resources to help them, right? And then also folks said family law was their number one request, right? And so, and then when we further identified gaps, folks said it would be nice if instead of me kind of hunting and pecking through the site, if you guys helped me and guide me. So some guided interviews, if you will. And then we identified that at a high level and then we did some refinement of those requirements. And then this is what we did. We didn't issue an RFP. So I'm gonna make a confession here. We didn't issue an RFP because we, to be fair, I'm sure we've all done RFPs where we think it's, we've identified, we've gone through all the rubric, we've set forth, we put together the selection criteria. We sent it out to everyone we know, we sent it out to every listserv. And then you get one proposal, right? Or maybe you don't get any proposals and then you have to re-issue again. And so what we did was instead of doing, going through that RFP process, we had a requirement stock. And what we did was we made a lot of phone calls and we said, hey, we're looking for, we wanna do a Drupal build. Can you give us some recommendations? And we just made a ton of calls. I talked to other consultants, I talked to other folks. And we then basically called all of the recommendations and said, here's what we're thinking. Let me send you the requirements. We'd love to then have a conversation with you. And so that's what we did. And we ended up with three final interviews where we got proposals based upon those conversations and then went through that three kind of typical RFP interview and then made the selection that way via stakeholder input and the typical process. But we didn't do the typical process for the RFP. And I will tell you, it worked well for us because we had the opportunity to really, and it's funny, we keep on talking about the relationship so we had an opportunity to really talk with folks, have real conversations and have that back and forth. And even after that interview, we did a couple of follow-up interviews because this was a large six-figure project. We wanted to make sure that we had the right vendor for this project. And so I will say that if folks are really at a place where they're not quite sure, one, seek help from a consultant, potentially to help you put together the RFP, it might be the best investment you make. And two, unless you are, unless, let's say, I'm gonna call out LSE here because LSE does require you to go through this particular RFP process, if you don't need to do that, think outside the box a little bit because you might actually get a more equitable process in that by doing that outreach, by picking up those phones and going through that process. And then in our scope of work, we always kind of put our general project management philosophy and then again, I wanna emphasize documentation. Okay, I'm glad to take any questions. I see Simon's up, so I'm gonna turn it over to him. Thank you, Susan. Should we wait until the end for questions or kind of take each section at a time with questions? Okay, something on people's minds? I would say we can take questions as we go and people can also put it into the chat, I'll be monitoring the chat. I want it to be open so that people feel as if they can ask questions. I don't know if Bianca, if you had a question. Okay, go ahead, Simon. Okay. So my name is Simon Rao. I've been in the IT world doing development or project management or leading IT departments for the last 20 plus years. So my focus has always been on vendor selection in the IT world development, web application development, mobile app development, and so forth. So I'm gonna give you a bit of a bent on that coming from someone who's more technical. I wanna start off with just reminding people what the general or common set of steps are. So my focus is primarily on the mechanics, the logistics of actually putting out an RFP in a review, gathering feedback and then the interviews and so forth and selection. So I'm gonna talk about the mechanics, primarily. If you have any questions later on in terms of some of the selection criteria and what kind of companies and so forth and how to write up the RFPs and so forth, I could ask that separately after the presentation and I could attract those specific questions. So in terms of general process, you started with, of course, sending out the RFP or RFI, then you have a Q&A process. Typically this is remote, people send in their email questions and then you kind of consolidate all the questions to a single document that has all the answers. And because many of the questions are very common across different vendors, you wanna consolidate them into a single set of answers and then send it out to all the vendors to respond it. So in fact, this is actually like the first round of selection. People who ask questions are typically the ones that you will wanna focus on when you actually receive the proposal later on. Then once you receive the proposal, you do a first round of selection. So assuming you have a good selection to choose from, let's say 10 or 20, you usually wanna pair that down to the find list, two to five find list. And during that process, we have to put in some kind of objective selection process. And that's where I'm gonna talk a little bit more about the selection tool that we've been using. Once we have the find list, then we, at that point, we tend to do the interviews. So we actually talk to them in detail and ask about the proposal and ask for clarifications or ask them, I know you said that you want to do this step, but can you do it some other way or can you skip that step or and so forth? So you could like adjust the proposal during the interview and ask if they're willing to change the budget even. So that's a more, it's a pseudo negotiation during the interview. From the interview, then you could do the final selection. That's when you do another round of under selection. Typically, it's essentially reviewing the same set of criteria and scores you gave in the first round, but just adjusting it based on the interview and how the vendors can cross when you talk to them and their flexibility, flexibility is very important. So this URL at the bottom is a Excel file or sorry, it's a Google Sheet file, which anyone in this meeting have access to. So you could go to it and then make a copy of it into your own Google Doc or download it as an Excel file and work on it. Next slide, please. I could go through this tool in a little bit more detail later on if you're interested, but what it does is it essentially has a different tab for each of the vendors. And for each of the vendors, you have the criteria of scoring. So how they would up to summary what the company background is and so forth. Then you can weigh each of them depending on how important those criteria are. And then for each of the reviewers, and assuming you might have like three, four or five reviewers, each of the reviewers can put in their own scores. And then the tool would automatically average them and they give you like one score for the vendor. And then the screen on the right, the screenshot on the right takes those single scores and breaks down by the criteria and then kind of puts it on the one sheet. That way when the reviewers meet after this and discuss each of the vendors for the first round or the second round of selection, they can look at the scores and see what everyone has gave and then compare and contrast. And this is where the qualitative discussion comes in as well. So even if the score for vendor one was five out of 10, but you really liked them because you've been working with them for the last 10 years, that is a very strong reason to maybe choose vendor one even though their score is lower. So although it is supposed to be primarily objective, you need to provide rule for subjective arguments. So this is a tool that you guys could look at later on have access to and you could download if you need to. So let's move on to the next. I just mentioned one thing since I have worked with Simon and I worked with Simon using this tool in the past. And one of the things that he sort of stresses is that thinking about, so that other that you determine what the criteria are, right? You determine and how much you wanna weight each of the criteria. So it might be, for instance, locality might be very important. It might be important that the provider have relationships grounding in the community. So the fairness of it is that you're gonna be comparing based on the same criteria, but also remember that sort of that criteria is what you would determine and you will come together and make that determination what those criteria are as part of your stakeholder group. And I found it very helpful to work with Simon in that way. Exactly. And the weighting of course is important because even though you're scoring on, let's say, budgets, that may not be your most important criteria. So understanding the different weighting is important. So this slide kind of describes why we have this more structured, objective way of reviewing. Why not just have people read the proposals and discuss during a meeting. By having these scores and weightings and criteria, it does make the three things. One is that each of the vendors know exactly what to focus on when they put in their proposal, when they write the proposal. So they know they're going to be rated on the writing of the proposal. They're gonna be graded on the history of the company, graded on how they write up the project plan or project approach. So they know where to focus their energy if it's limited. So that's one. The other is that we also, of course, then have to be as fair as possible to all the reviewers. So all the reviewers need to be given some direction on what to focus on. Some may, there might be individual preferences in terms of like write up of how well the writing is done with the pros or how short the document is. They may have some preferences in those areas, but if that's not part of the evaluation, then it doesn't really matter because we want to have the same criteria across all the reviewers and all the vendors. So having that kind of a clear criteria for evaluation is important, both of the vendors and the reviewers. Secondly, by structuring it this way, and allowing people to or requiring people to put its scores at some number, you could objectively average those numbers up appropriately. So even though maybe the overall score is five out of 10, you are very impressed with the price. So it's much lower than everyone else. So you might think that because the price is so low, we should go with this vendor one, but it forces people to look at all the criteria together rather than one specific impressive attribute of the vendor. So that's the second reason for having a numeric score. So it's as objective as possible. And then lastly, as I mentioned in the previous slide, even with all this objective numbering and criteria, you still need to leave room for subjective discussion. So even with all the scoring and all the very structured criteria, when you do meet among the reviewers for selecting the vendors, there's going to be people who make an argument for certain criteria that no one's ever thought of. Maybe a minority owned business is very big thing that we didn't consider when we were writing the RFP or maybe location or the fact that a particular company worked on a very similar project, very close by, that might be a criteria that you can put into the RFP, but that's something that you might want to bring up during this interview or during this review meeting. So I think I have one more slide. The next place, yeah. So what are the different criteria? Susan touched upon this as well. And it's very similar to what you put in the RFP are more or less the two criteria that you're looking for when you're scoring them or reviewing them. Believe it or not, the proposal writeup is actually key important. It goes to whether the vendors are serious about the proposal because very often when you see a lot of proposals you're going to see they sound very boilerplate. So if you see a very pretty proposal that's like 30 pages that doesn't mean they actually put a lot of effort into it. So the fact that they're following your instructions for how to write the proposal is actually a very good indicator of how serious they are about the project. Secondly, of course, company background is important how they understand the scope of work. And this is something I saw also many vendors especially the larger ones, they'll literally copy and paste your scope of work paragraph into their proposal, which is for me a huge red flag. That means that again, they didn't really think through what the work is about and they didn't reiterate or rewrite the scope of work as they understood it, which is very important. And then of course project plan is important because that means that they actually thought about if they had the job, how would they approach the project? So that's a very important indicator of whether they're good project managers and they know how to organize projects. And then of course budget is very important. And then there's always other, as I said, always taking into consideration that there are other criteria that you're not thinking about that needs to be a serious consideration. So for me personally, I found that the company background which is the work they did in the past and examples of their work as well as how they write the project plan are the two most important indicators of a successful vendor. Yeah, I was just gonna add like, I'm always a little wary when I see really slick proposals on a template that are the beautiful graphics and like tells a lot about their executive team but tells me nothing about how they're gonna actually like build the technical functional requirements to get me to MVP. Those things are troubling to me. So everything Simon said, it sounds weird but the slicker that template, the more wary I am, the more they emphasize who their executive team and less about what the work they're going to do tells me that they cobbled together because they're always cobbling together tons of vendor proposals. And so I will say those are kind of my red flags as well. And one thing that's at least for me, I love it when a vendor contacts me and says, hey, I'm really looking at this, I have questions and that to me, you know, and then to make sure that the RFP process is fair, if we know, we'll make sure to post those questions but just, I like that when someone engages me and says, you know, I read this and I sure if I quite understand and then just to make sure the process is fair and equitable, we'll go ahead and post those questions to everyone that has expressed an interest. Yeah. My one last recommendation that I always tell everyone who's involved in the RFP process is that the proposal we receive is not the final proposal. Very often what they propose for the budget or how they're going to do the project is not the final way we're going to do it or find the budget. Even if, let's say you had a budget of $60,000 for this project and someone came up with a $100,000 proposal, that's fine. You could still select them and talk to them and say, you may be thinking of something much more sophisticated than what we're thinking, can you do this for more like $40,000 and then very often vendors will say, oh, okay, sure, no problem. So don't think that what's sent in for the proposal is a final version that it's a yes or no situation. More about a sense of their competence and their understanding of the project. That's more important in a proposal. I just had one caveat, which is have realistic budgets about what it costs to actually build something. That's the only thing I would ask. Yes, one of the things that I noticed is budgets can be all over the place. I've received proposals where I did wonder, did the company truly understand what were the requirements? And sometimes I've noticed that those are companies that are younger, newer, that are BIPOC companies that are, and I've spoken to them and asked them, what do you understand this project to be? Do you wanna talk about it a little bit further? Have you thought about budget range potentially? And maybe that's not the best interest of your organization, but I do wanna be fair and I do wanna be equitable and if I'm willing to have this combination, I think that can also help someone next time when they do their next proposal. Just to think about, okay, maybe I should look at what's out there in the market. So budgets, I think, are a hot topic. We can talk more about about whether to put one in, whether or not to put one in, and what does it mean when people go really off your budget? Because sometimes it might be that you actually didn't plan it accordingly and you don't really know what their requirements are and you are really under the market. So thank you. So we just have two more slides. I just wanted to bring into some considerations about diversity, equity, inclusion, belonging, and justice, equity, diversity, and inclusion. Just thinking about like, since we are most primarily, I assume, justice-oriented organizations, are we looking at vendors that are committed to justice and fairness? Are they committed to an aware of the cultures and identities in our communities? Like, do the vendors have connections to our communities? And what can that mean? Sometimes that can be incredibly beneficial for a project that could be a website rebuild, like you're gonna need community input. So are these vendors gonna be able to get that community input? Do they have trusted partners or partnerships within those communities that you're serving so that they can also help you with potentially user testing? But there's so many different ways. So just trying to think, as we're thinking about justice in the work that we do, also thinking about our engagements with partners and with consultants. And so here are just some things that I found online. I think Susan mentioned earlier about sort of government agencies giving a lot of information about equity inclusion. So just think about putting your mission vision and values in your RFP. So people that are clear about it. And also then you can see if when people are responding to your RFP, do they use some of that language? Do they understand that language? Is it clear to them or does it seem as if they're, I don't want them to copy and paste, but I do want to see some mirroring of the language to understand where we're coming from. And then think about requesting information about their own plans for diversity, equity, inclusion and belonging of how they're gonna staff the project, how are they gonna work on a project? And think about like, is your RFP, like, are you, is it excessive unnecessary? Are you asking for things that truly aren't necessary for you to make a determination as to whether this is the right vendor for you or for your organization? Because sometimes it can be unduly burdened and sometimes have a negative impact on communities that are just starting out in this process. And I would just really stress performing outreach. As Susan mentioned at the beginning, it's not enough to just post on LSNTAP, it's not enough to post on Progressive Exchange. Again, you need to really be reaching out to other networks, other communities. You need to reach out to the grassroots organizations to figure out who are the people that they're working with. And performing that outreach is a really important part to think of this process. And also then you're gonna learn some more about this project that you're trying to engage in and it will give you some new information. So that is what we have. And now we're open for questions. We want this to be more of a questions. We did want to lay sort of a groundwork for RFPs and vendor selections, but now it's open for you all. I mean, one first question that I just had is like what are those red flags? And I think Susan, you mentioned one and Simon, you mentioned one. Like what are red flags that you've seen in proposals or moments where you've had a poor experience and you probably could have noticed if you hadn't checked for those red flags in the beginning, not mentioning any names. For me, the biggest one is when they understaff is when the proposal says we have one consultant working on this project. And it's a project that I know is gonna need multiple consultants or if they're significantly under budget. Like they say 17,000 and you're like, it's a whole rebuild of a platform. Those are red flags for me. We're gonna figure out what to do. Yeah, I'll second that also with the budget. Early on in my career, we were so focused on the budget. We said, yeah, here's our 30,000 budget. So make sure our proposals come somewhere near that. And the ones that's closest is the one that we go with. The problem is that budget in proposals are quite variable, as we said, because it's a project itself is very undefined, very often when our fees go up. So when someone comes in with the half the budget that you expected or double the budget you expected, it's not because that's what the project requires is because of the understanding of the project based on the information that was given. So don't rely too much on the budget. At the same time, though, as Keith mentioned, do not go with the cheapest option because that is a huge red flag, more than two expensive budgets, in fact, because too cheap means that people are just trying to get their foot in the door and they're do anything to just say yes and you end up with a project that's a mess and gets over budget. The other one has to do with when the proposals seem overly accommodated. So when they essentially regurgitate everything you're saying, say, oh yes, we will make sure we'll reach out to the community. We'll do everything that you put in your R-speed that they just regurgitate just to make it as close to what you're saying. That is a red flag that just means that they haven't really thought through if they were doing the project, how they would do it, that they not agree with you. So it's better to have a critical thinking kind of write up rather than someone who's just too agreeable. I think Rochelle had something in the chat about where to... So since we use Drupal as our content management system, but then we built tools on Drupal, like my DrupalCon hasn't happened, I think maybe it'll happen this year, but he'll attend that. My product manager attends that. We'll sometimes send out things through membership so that we can think through outside of traditional, I guess, even commercial development shops in terms of Drupal. So I think folks can think outside of the nonprofit kind of channels and think through where like DrupalCon is one, they have memberships, you can get a member. There's different things. And then there's different opportunities there as well, but obviously LSNTAP and things like that. Sometimes folks can reach out to program officers at LSC just to get names, obviously they can't make recommendations, but things like that, right? And then just within, I know like obviously there's a group of statewide websites. And so maybe we will ask folks to send things out through those networks and things like that. So I think you just need to think through both on the nonprofit and on the commercial side. Yeah. I just think, do you have your hand up? Yeah, just in follow up to that. I mean, one thing that I would find helpful is if there was some way to have a list of places that people have worked with that they would recommend or even proposals that they got. Sometimes you get more than one good proposal but you go with someone else but the proposal seemed likely. I mean, it would just be helpful to have some kind of centralized list somewhere. I don't know if LSNTAP could keep something like that or rather than have to go reach out individually to every single person in the country who's done an RFP recently. Yeah, believe it or not, I actually use Google. I look for like, let's say a technology consulting firm in a New York area and of course get hundreds but just by doing Google use and there are of course lists of like best performing whatever. And I usually start with that. And of course, I supplement that with any recommended organizations that are vendors I've worked with or word of mouth kind of thing. But that works pretty well just doing a simple search for a certain type of a result, especially if you say technology consulting in nonprofit technology for this particular area. You could get a pretty good list that way because whether it's word of mouth whether it's people you've worked with in the past they're not necessarily the team of the crop that's out there already. So yes, they may be more trustworthy because you've worked with them before but there may be people out there that you've never talked to that no one around you knows could be very good. And you only know that when they send them calls. I also noticed that there are consultant networks and especially in Massachusetts there's some really great consultant networks and the social innovation forum. For instance, they have lists of diverse consultants that they work with. So tapping into those different networks in your state or if you want it within your metropolitan area can be really beneficial because I've met some really great consultants that way. And I think Mark you had a comment slash question. Mark Larson. Yeah, put it in the chat. I was just saying as a prospective vendor in some of these engagements sometimes I get the impression that the RFP has been written and it's really wired for a specific consultant and the company's just going through the motions because they have to have cover or they're required to and you spend a lot of time responding and you don't get selected. Is that an illusion? Does that actually happen from your experience? Is that a common phenomenon or is that just a paranoia on the part of responding parties? I think it depends on what it is. There are some things that are so particular and that are specifically within one group's area of focus like one consultant group area of focus where there aren't that many there's not that much competition for that type of work. So I've had times where I'm just like there is no one else that actually can do this work but when I've done website design projects it's open to everyone because there's a very large community. So I don't think that's not what my experience has been but I've been on both sides of the nonprofit and the vendor side but I don't know. So I will tell you that when we have a particular vendor in mind we don't do the RFP. And sometimes that requires us to do a lot of paperwork like for TIGs for instance, right? Where we know we're going to do a source contract but we don't think it's fair to other vendors when we know that we're gonna use a particular vendor for reasons obviously LSE wants us to go through all the reasons but we'll do that. And if I issue an RFP it's an open selection. But I'm not saying that that is what always happens. So Mark I think what you're suggesting potentially probably does happen because I agree is sometimes there's something that's so niche that you may have to because your funder is requesting that you go through these hoops and there's no other way to do it that basically you have to go through those hoops. And so, and I think someone else put in the chat that some vendors don't even respond to RFPs because of that process, yeah. Yeah, I would second that actually it is more prevalent. Mark you're not imagining things. There are times when RFPs do go out and definitely it looks like it's catered for one specific vendor and that does happen. And primarily it's because the funder requires that you do the selection and vendor selection. And the best way to identify those is if the RFP mentioned specific product and specific ways of doing the project that's requirements then you know they have someone in mind. If it's more open-ended and say, I wanna create a website that does this for the community more general like that that means that it's a fair selection process. I just ask the audience, are you having any challenges or anything, I didn't see any hands up but any things that you want to bring up, challenges that you've seen as you've tried to develop RFPs or as you've tried to engage in the selection process? Actually related to that, there was a question that amongst us, we talked about it's like, what if you are not technically minded and you were given the project to doing some technical IT project, how do you write an RFP that's informative? Has anyone had the issue with that or how do you deal with something like that when you don't have the technical knowledge to put together a good functional requirement or even know what the criteria should be to choosing an IT consulting firm? Don't say copy another RFP because those requirements may not be right for you. Yeah, because I know that sometimes the suggestion and I think I went through our scenario which is we actually hired a consultant to help us. They weren't a vendor that we would, they had the technical consulting know how to help us put together the words in a way that we get responses and folks would know what we were talking about. So we actually ended up hiring a consultant to help us. But I realized that everyone has the budget to do that but that's what we ended up doing. The other possibility is to get a skills-based volunteer. So we have LexisNexis on our board and obviously they built and they help us because they have skills-based pro bono volunteers. And so that's a possibility. If you have that kind of shop that's willing to do that and work with you folks, that is a possibility. Like, and that could be a way to get some no cost essentially assistance in building out requirements of folks who have to do this every single day so that you can say the right things so that folks actually understand what you're trying to build. Then it also improves the quality of the proposals because what I found is when I didn't quite understand the technical requirements, I got the wrong proposals and they did not meet the needs. And I had to start over, that's not a good use of time. I see that someone says, has anyone built that kind of technical consulting cost into TIG budget? I wonder has anyone framed it as like a business process improvement or analysis that might be a way? Has anyone built that kind of technical consulting into their budget? Yeah, the way that I've done it is through BPA and BPI and then through there I was able to get the information of requirements and recommendations that I could then build out the proposal. Jamie Levine? Yes, Susan, who did you hire and how did you hire that person and how far did that consultant take you? So we hired the Ripples Group out of Boston, Massachusetts and they helped us, they do MVP consulting for large commercial and they work with multinationals. They also work with nonprofits, it's part of Attila who's the founder, it's kind of his Ripples, right? So he is part of very large corporations and consulting, like he worked for Maine, he worked for a lot of different large Boston multinationals and then he and his wife now though also have a nonprofit practice and we connected with them and they helped us. They also helped us with the landscaping, they work with a lot of data providers like Ipsos and everything and so they helped us pull together that information as well and they did it at a reduced nonprofit price for us. And I'm guessing it worked out well for you? It worked out well for us, yes. Because I think one, we had to do the landscaping regardless because it was part of the community led design, right and we needed a consultant to do that. So we did issue an RFP for this, but we wanted someone with that technical expertise that had done IT like large software or product builds that could help us think through how to place those requirements in language that we could then build out an RFP. And so we just happened to be able to marry those two with one consultant. And then John Grayson didn't help us with the technical aspects of it, but he helped us in terms of that community connections. And so those were the two consultants that we actually had multiple proposals, proposals from both of those folks and I asked those folks to work together and they agreed to do it. So I also negotiate contracts, I'm a lawyer. So, and I basically asked them to do it and they agreed to do it and to bring their unique skill sets to the table. Oh, thank you. And I don't think you should underestimate the power of RFIs as well, especially if you don't have anyone to like reach out to help or you don't have the money for it. If you just find the vendors near you, as I said, so through a Google search and then send that email to each of them saying, you know, we're thinking about starting this project. We're not sure how it should be done or what costs should be. Could you just recommend what tools to use or what technology to use or what costs you think it will be. Many of the vendors will reply back because they're interested in projects like that. So they'll say, oh, this sounds like a $50,000 thing and you could do it on WordPress and you shouldn't take it more than three months. People respond very quickly like that now. That's not enough to like fully flesh out an RFP but at least you'll get a sense of what vendors are expecting for a project. So that would give you a very good head start especially with the terminology that they use. If they say this is a document management system or reporting system or content management system, you know to put those kinds of words or phrases in the RFP because that's what the vendors understand. Great, thanks. Mark? Yeah, I was just gonna say that my colleagues and I often find ourselves in this role of being a consultant to the process where we're helping an organization send out and deal with RFPs. And one of the biggest issues we run into is the use of the vocabulary. We talk about requirements and very often requirements are not true requirements. There are must-haves, there are must-not-haves and there are nice-to-haves, right? And especially if you've got a team of people who've never been through an RFP process before they get them very confused. Your tool Simon I think is a very good way to deal with that, to clarify what do we actually care about and how much when we wait it. But I find that just a common issue is just the vocabulary people use when they talk about vendor selection. The questions or recommendations. Thank you Danielle for your comment in there. I understand that she just said I just went through this and had to have many conversations with developers do a bunch of research and talk with folks within our community who have more tech expertise than I do. It was a learning process and I wish we could have hired a consultant to help determine what the best options are for our particular project given new technologies available. Yes, I hear that. That's what I was suggesting potentially business process analysis or business process improvement because you can sometimes get funding to do that and it could be a small project, it could be a larger project but it can really help you think through, okay here's the process that we're engaged in here's what it could look like your recommendations for the future and here's what you need. But I know it is hard to get funding for this. I have a quick question, I guess for everyone. What is the average time span of, well, first writing the RFP and then going to the selection process? Are we talking about four or five months worth or can this be done in one month? I don't think a month, who's done it in a month? Does anyone want to put their hands up? I feel like this is just me. Okay. But it is an involved process I guess people should be aware that you do have to commit to a lot of hours going through the process. Hopefully have staff internally who can work with you to do this. Maybe you could let them have a slightly lower case load or something or give them some kind of ability to be able to work on this project and engage in technical work within your organization. It's a plug I always make, sir. It took our team many months to get in the RFP app. It's hard. The problem is the longer it takes the more things shift. Sometimes you just have to get it out the door before everything changes again. Sometimes if you do it in a month and then you start to realize, oh, okay, actually I need to adjust this and I need to reissue it because this is not what I was looking for. I don't want to keep people just to keep people but if people have any other questions, concerns, ideas I think we went through red flags. We looked at some questions about differences. We can talk about, we look at common mistakes sometimes people make in the selection process but if people have any other like final observations or questions, please speak now, we will end. Can I ask one more question? Are there any resources that people know of like the list for RFP templates or list of good vendors or any kind of resource that people can follow up on? I found some good information on government websites RFP RFI samples. It's like plug and play and I haven't actually used them but when we were going to do this panel I started to notice them and just think, okay, this is really interesting and helpful. So, yeah, I was gonna, oh, go ahead, yeah. Just to follow up, I think it would be great if we could, Alice and Tap could help build out somebody. I think they're already starting to have a section on the website for RFPs but it would be really great to encourage people to send their RFPs I mean, many of the RFPs do end up on the Alice and Tap email list and I know I have like searched that list to try to get RFP samples before but also related to that, I guess, you know what I said before about companies that people would recommend and also costs. I would find all those things really helpful. Yeah, I was just gonna say I'm kind of following up a little bit on the government. When I was with the state of Ohio because the rubric is so specific and because of the sunshine nature of an RFP or an RFQ or an RFI process at the state level those rubrics and the templates are very like step one, step two, step three. I haven't looked at my state of Ohio ones in a while but I know like when I used to do RFQs for experts as part of our litigation unit within the AG's office like it was very specific like what we had to put in because those went out as part of sunshine. And so I do think those are good resources and I could try to see if I can find some and then kind of to Richelle's point I think everyone's asking for kind of a central repository of information. Yeah. Thank you everyone.