 Good afternoon Sue. Good afternoon. Thank you for joining us this afternoon. We appreciate your time Certainly welcome to my living room And and all of ours So We asked Sue to join us today. She's the chief chief performance officer and the agency administration and She is work. She works with the government accountability committee and I think we like to learn your role and your thoughts Welcome Thank you. So for the record, my name is Sue zellers Susan Zeller and I'm the state's first chief performance officer And so I've been here for 15 years and the first Half of that I was the deputy a commissioner of finance working for Jim marine for eight years and Subsequently, I've been the chief performance officer and I do I actually sit non-voting member on the government accountability committee as the governor's Representative so what I thought I'd do is I'd walk you through What? The government accountability committee has been up to and how that intersects with my role is chief performance officer So I have a presentation that I'd like to do and feel free at any time We don't have to wait till the end for questions. If you have any questions, just let me know Okay, let's see if I can get this thing up Wait I'm not as good on zoom as I am on teams The administration we use teams more than zoom. So let me know if you can see this We can all right good. Well, that's a good sign Come on Okay, yeah, do you mind putting it in the presenter mode? We had this with another presenter before is that possible? Presenter mode you want me to get out? Thank you, so Here we go. So how did we get here? So in the 1990s Con Hogan who was secretary of the agency of human services brought RBA Which is results-based accountability into AHS and they began using it and having a program and they were doing all of their county Trends and looking at programmatic outcomes at a vastly large high level But he introduced the concept and that program lasted until the 2009 2010 Recess great recession and it was that job was eliminated when secretary Racine started he brought back RBA and he dedicated two staff members to it, which was great and In 2013 Commissioner David Mears brought lean into DEC the Department of Environmental Conservation and then AOT picked up on it So in October of that year, I was appointed as chief performance officer, but I've been involved in a lot of this Efforts for efficiency and modernization. I think I come by that genetically my father was what you call a manufacturing efficiency expert So I can hear him always saying Susan. There's got to be a system so And the reason my position of CPO was created was actually in preparation for what we knew The legislature was going to pass the following Session 2014 session But then an effort led by then senator Diane Snelling to for the legislature to adopt RBA and as part of that GAC was established and had been established and They got to work on setting statewide population level outcomes and Indicators to be used To for me as the CPO to report So how that process Went along like that for a couple of years until governor Scott came in and governor Scott really wanted to Create a full-blown system that used both lean and RBA Originally he just wanted to use lean because as a construction guy he knows about that But I convinced him and I said look the legislature's adopted another methodology and lean and RBA and six sigma and Baldrige and every other Improvement methodology you can think of are all the same their goal is to improve Or to be able to evaluate results or outcomes So I saw a way that we could do both That we could use lean at sort of a lower level When you're talking about making a process more efficient But how many hands touch that same piece of paper that expense report or that timesheet or that Contract that has to get signed How do you make that efficient make processes efficient or communications with Vendors or clients or whatever so we could use lean because lean is more directed at transactional Processes and we use that at that level and then we could slow up into results-based accountability and the overall outcome And we called this the governor called it pivot and I won't go there, but we called it Continuous improvement because that is really what we're trying to do is continuously improve our processes our tasks our technology our Programs the results of our programs and the overall outcomes that we want for Vermonters We built with Governor Scott's support an entire course structure for continuous improvement and We have almost 2,000 state employees that have gone through our programs. That's at either the basic intermediate or advanced level We also tied in a strategic plan that the governor did for the Administration for the executive branch and we tied those his top level Strategic outcomes into the 10 outcomes set by the legislature We had a little we had a little bump in the road called oh, I think you guys know what this thing is COVID-19 around March of last year and We had to reinvent our whole core structure to be able to do it remotely and in a strange way we've actually benefited from that because more employees Can get approval to take the classes because they don't have to travel so their supervisors are more willing to let them have You know two hours is two hours. It's not four hours anymore It's they don't have to travel to Montpelier So we're very pleased with that So, where are we right now? Well, GAC right now is working on Selecting with stakeholder and public input Indicators that will be used to look at our 10 outcomes from the perception or from the demographic of for racial equity of the Black indigenous and people of color perspective that's being led by Susanna Davis our director of equity myself and Drew Ressley from AHS and Emily representative Emily Kornheiser and coach Christie So in addition in my shop, we have a planned expansion of what we call the programmatic performance measure budget Ultimately what we're trying to get to is where each program It probably of a certain size. I don't think we could I don't think we have the staffing the data or the wherewithal to produce Metrics on every single program we have and publish them all but we're going to try to take our largest or most impactful programs and Present them not only On a basis of what are they achieving and what are the results but also How much of an investment are we making and we're trying to do it beyond just the direct program costs We're trying to incorporate indirect and overhead costs So how does all of this stuff look so here's a holistic look at the whole continuous improvement project Lean Rba everything. This is how we look at it. You know, this is the peel the onion thing If you start at the outside the highest level are the outcomes and for that we use The methodology and the language of Rba So you have an outcome which you want for example, we have 10 outcomes set in statute and They are things like Vermont has a prosperous economy Which is the one that I think you'd be most interested in the second one is Vermonters are healthy. The third one is Vermont has a clean and sustainable environment So that's your ultimate outcome great. That's wonderful. How do we know if we have a prosperous economy? We use indicators and indicators are metrics That can somehow give you information about the economy In most of these outcomes, there is no one formula metric analysis that you can do So for Vermonters are healthy. There is no one percentage I can give you for how many Vermonters are healthy. But what I can do is I can tell you Things like smoking rate Premature baby birth how many people have health insurance. So those are proxy indicators that give you an idea as to how you're doing in the overall Outcome that you're trying to achieve and in the case of the economy we look at unemployment. We look at the median wage We look at house prices Things like that that tell us How we're doing now below that level is something that we have invented that We've created a level here called a service domain And a service domain are sort of a big bucket where you can put all of your alike kind Areas in so for example, you might have all nutrition programs and in that you would be able to cluster wick and three squares Vermont and In the case of COVID right now our emergency food distribution Or housing you could look at housing and you could look at all the different housing programs that we have currently Our Appropriation system and our accounting system are in silos Your analysis can go up and down the silo. You can go up and down human services and you can go up and down The ACCD but if you want to look at housing you can't go across You cannot look at housing and drill across the three or four Areas where housing is because housing is in human services housing is in a CCD Okay housing is in the housing and conservation board so we have no way except manually to go across so that's the purpose of service Domains is we're trying in a longer term to be able to do that For example a few years ago three years ago maybe four The governor proposed and the legislature approved a thirty five million dollar bond for improving affordable housing At the time someone asked, you know, what are the reporters or something what percentage of our overall housing investments did this 35 million You know, what's what's its ratio to you know, is this a 10 percent investment of well we can actually answer that question because housing programs are all over the place and Some departments in their programs they charge actual employee salaries there some charge salaries all in administration and don't distribute the cost So service domains is a coming feature Below that we have our strategic plan, which is where The administration gets down to brass tacks and the governor has Three a four primary strategic outcomes that he has And each governor is going to have strategic outcomes of their own depending upon their Their philosophy depending upon the times you know Whether you've got economic challenges or whether you're you want to do a lot of things in education or the environment But what I tried to enforce and what I think we did and I would hope that future administrations follow this We tied these outcomes directly to the 10 outcomes that the legislature already has established so the governor's number one outcome is Vermont has a prosperous economy Now he would tell you and I would tell you that that ties into vermont has um Oh, I'm sorry The legislative one is vermont has a prosperous economy The governor's is vermont has a growing economy because I think he'd say it has to grow before it's prosperous and he has His his second one is affordability and the affordability one falls in very nicely under the economy About median house prices It also falls in about health care, which you know, how much do you have to pay on health care or percentage of your income for taxes? His third one is to support The most vulnerable and so that falls into two two of um or three actually of the And outcomes of the legislature has and his fourth one is modernization and efficiency And that ties in to also to one of the 10 outcomes And I'll show you how that works later on So when you get below the strategic plan and the strategic plan should be driving what you're going to do In your programs what programs? What are you going to do in your programs or how are you going to change your program? to to drive toward your outcome And then uh below that is our performance measures, which are the same as indicators except at a lower level We judge and measure how well we're meeting our outcome with indicators for a program We do it with performance measures. So for example, you have a training program your Result that you're looking to get is you want 80 of the people that enroll in the program to finish it So that's a performance measure And of that 80 that finish it you want 75 of them Actually, oh, I don't know get a job. Let's say it's a it's a workforce development program So you want them to get a job The mission of the program and the result that you want and you want to be able to measure is not just That we have this training program and we push people through it And great, you know, if you give me more money, I can have more people come to the program Well, that's wonderful, but that's not what the program is really supposed to do The program is supposed to help people get the trend give them the skills so that they can get a job Or a better job or a higher paying job So you use performance measures to analyze whether your program Is actually achieving what the legislature or the federal government Gave you the money to do for to do It's not just to have a classroom and and put people through it It's so that they a graduate and b get a job out of what they've learned So how do you know? In history the state has been always great on giving Metrics of how many well we enroll 50 people in our training program, uh-huh And or our smoking cessation program. Okay at the end of the program how many of them had actually quit smoking And three months later how many of them were still quit It's not just to have You know people go through the class. I mean men people And smoking cessation is one of those areas where they may have to take the class repeatedly to finally conquer that And you can build that into the metrics to see whether you're actually being successful The people that you have teaching the training class or the smoking sensation class Are they qualified? I mean heck you could have me teach the smoking sensation class, but hey, I'd never smoke And b I don't really know much about it. So I'm not sure how successful I'd be My classes might not be very successful. So there are many ways to evaluate your program And how it's doing is it is it doing Well by the numbers is it doing well on a quality basis? Or is it actually doing what we really want which is You're getting results that were intended And once you get below performance measures, you're down to actual processes and this is The process of writing and and sending a contract around to get signed. How do we do that? How many hands does it have to go through? Does it really need seven signatures? I mean literally we had now the department will Remain nameless, but there was a big kerfuffle about how many people All went to some conference one year and the and the secretary went a little mental and said, okay, that's it Every expense report has to come to me. I have to prove everyone So of course all the directors and and the commissioners Didn't want expense reports going to the boss unless they've seen it And then all the supervisors so literally there were seven people that looked at every expense report when we were still using paper expense report Because of the fear factor Well, we stopped that First of all, we have an automatic system now And we found that some departments weren't using it. You guys use it for your your expenses And they weren't using it and why because it only allows for two signatures So they weren't using it Okay Why do we hit you? Can you add signatures? No, we cannot add signatures. How about you don't need seven signatures? So you work through these problems with departments and challenges and you come up with better ways to do it And that's the process improvement In the brackets next to each of those onion layers, you'll see A name of the tool or methodology that we use and these are all Improvement continuous improvement processes to trying not to be Wed to rba or lead or hoshin konri or a six sigma or plan do study at that We've developed a way to use the best part of each of those things at the place we need it And we also feel that not saying oh, you have to use rba. Oh, you have to use lean Also tells a better story Because people get sort of tired and Resistant if you're trying to shove a particular methodology um In particular language and all of that down their throats they get tired of here. Oh that rba stuff again. Are you back again? That doesn't work go away so We've tried to change that into what we call continuous improvement so GAC gets involved at well at the level of one and two And um We're going to go on and talk a little bit more about this so How do we change an outcome or an indicator? Well, the outcomes are set in statute and they're in three vsa 23 11 b And they have to be changed with legislative action And GAC Uh mostly does their meetings off session and uh and here, you know has hearings with Departments and takes testimony and If they want or find a reason to change An outcome or add one or split one and two which we have done in the past since 2013 They have to put it in a bill and generally GAC does a You know a maintenance Kitchen sink bill every year To fix little things and that would be where that would come out and so that outcome Change would have to go through both GOV ops committees and would also have to go through the committee of jurisdiction. So if it were You know Vermont has a prosperous economy. It would have to come through your committee And to see if you Like the way it's written. Do you think it's a valuable outcome? You know now on the indicator side The indicators are actually set by Departments or me or I've suggested by us And GAC approved them, but they're not in statute and the reason for that is Sometimes we have to use an indicator. That's not our preferred indicator Because we don't actually have the data for the preferred indicator And so we may use something else in the meanwhile. That's close enough approximation And it's one that we know Has good data comes out on a regular basis And it's from a trusted source Those three things are all very important For example, when we first started back in 2014 after After act 186 was passed We had one in there for agriculture. That was something about how much fruit and vegetables get sold every year Well, we took that out after two years because come to find out It's the feds that report that. Okay, so that's good a reliable source But it comes out every five years And by the time they issue it after the fifth year, it's really partway through the sixth year So what the heck does that tell you not much of anything? It's old It doesn't mean what are you going to do? How do you react to that to five year? You know, essentially five year old information So we took that out because it Didn't really tell us anything It didn't give us information to allow management or the legislature To decide we need to do something Where things like unemployment rate that come out every month Gives you the opportunity to see trends happening and to react so indicators are a Not a legislative action, but there are committee and administration action And gack will approve them We've we've had one swirling around for the last couple of years if you look at poverty rate there's They use 185 percent 200 percent 300 percent 400 percent. Well, what one are we supposed to use? The feds generally for federal reporting use 185 so AHS would like us to use that but lots of the of the stakeholder community use 200 Now in some ways it sort of doesn't matter whether you use 185 or 200 because The information comes from using the same metric over a period of time and seeing how the trends change You're not going to see a much different trend line if you're looking at the population that's 185 percent of the federal poverty level or a 200 percent But it is nice if you are using a metric that everybody understands and uses So that's some of the other work we do. So here's a chart. I developed quite a while ago to show how both The legislative outcomes which is to the left three vsa 2311 c population level outcomes Vermont has a prosperous economy tied directly into with the governors the state strategic outcome Vermont has a growing economy And one of the metrics that we use on both is median house price And median house price then feeds into oh missing an arrow feeds into service domains for housing services And at that level there's a budget There would be a budget that we could actually if we can go across strata we could give you a budget Below that you have the strategies that both the legislature and or the administration use For how you're going to affect median house prices affordability of houses or access to housing or housing stock And that is split by that dotted line And so above the dotted line, which is half of the strategy and up Is the legislature all your policies and statutes Below that line is the administration. And so we're working on how do we implement those policies and statutes In our programs and we have multiple programs in this case. These are just some of the programs we have That deal with housing And each one of those programs has performance measures that tell you Family supportive housing how many of those people have supportive housing for six months Transitional housing how many Transitional housing units do we have available for either people in economic transition or in corrections transition Things like that and all of those programs have budgets. That's the piece we'd like to get to So that the legislature and management can look at The family supportive housing program and see what it actually costs us What it actually costs the taxpayer And then look at the results and see if those mesh. Maybe it needs a bigger investment Maybe the investment is not getting the return that you want Data is neither good nor bad. It's not meant to Judge whether a program is doing well or not It just gives you the information for you to evaluate And we call those two levels above and below the dotted line Above the dotted line are the population accountability, which is Whole populations either all vermoners or all chitin county residents or all people with mental health issues or All people that are being trained in workforce development And below the dotted line is performance accountability So how are we delivering service? Who are we delivering service to and and what are these things costing us? So we have something online called the outcome scorecard and this essentially is the annual Um outcome report on the 10 indicators and so i'm just showing you Outcome one what it looks like we can actually look at this later And we have under vermont has a prosperous outcome. We have The percent or rate of 1 000 jobs of non-sector and that's essentially the unemployment rate the change in non-public sector employment the Unemployment rate per 1 000 in the labor force median household income net change in jobs net change in business establishments who i'm not looking forward to seeing that one post covid The gross state product And i'm going to skip the genuine Progress indicator because i'll talk about that and then percent of population living at or below 185 percent of federal poverty So obviously the larger population you have living in poverty the less A dynamic your economy is because it's pushing people into poverty The genuine progress indicator was supported by a number of legislators a few years ago including senator palina Because it doesn't just measure the traditional gross state product Things it also takes into account the effect on the economy above of sort of uh softer things like How much uh recreational activity do you have how much you know forest or or or Lakes or um You know things that are quality of life items and not just purely dollars and cents items And uh, we were using that it was done on a three-year rolling basis, but the uh professor at uvm who invented this and Um worked on it and he would do it each year with a graduate student died last year uh pre covid and No one has seen fit at uvm At the gun center. No one's really taken it over. So I'm not sure what the What the future for that metric holds Now you notice there are little plus signs next to Each of the yellow eyes and now that's eye for indicator and the green o is for outcome Now you you can click on those and underneath that you'll get a graph that plots the multiple gears that we have that data for And it also has some narrative sections that you can Click open and you can either read comments Some people write a lot a hs really does a lot with their outcomes and indicators um, and some people just write the bare minimum But um, you can keep drilling down to get more and more information about uh and below indicator then You can actually add programs So under um You know new change in jobs or you know new business establishment accd has a program for assisting new businesses in either expanding or coming to ramon So you could have underneath that indicator. You could have those programs And see the metrics for them So what am I trying to do here? Well When I first came first of all of 15 years ago, we had no strategic plan. We had a budget And we'd had very little performance management But rightfully they should all be part of the same circle And your strategic plan should drive your budget the things you're trying to do strategically you should be funding And then you determine How well you are using that money to get the outcomes you want And whether you're moving the needle on your strategic plan with performance management So it should be a cycle Uh, and it's slowly getting there. It was very disparate before Um, you know previous governors, some of them have written strategic plans without any connection to what Um what they were asking for uh from the legislature in budgets So how do you how do you uh drive a strategy forward if it's not funded? And then you expect that you're going to magically get some kind of result That's one of the reasons why strategic plans wind up being done and then they go sit on a shelf somewhere and nobody ever looks at it again Everybody goes oh great job. We did a strategic plan as a matter of fact I just was on the phone earlier today with a woman in Maine who's in University of Maine. She's doing her her doctoral thesis and she's doing it on strategic planning and so she interviewed me Because Maine has a strategic plan, but they've been having trouble Getting it off the ground and so So we we went through a talk like this and you'll notice the things that the governor comes to you about almost always have to do with growing the economy making for more affordable Helping the most vulnerable and something having to do with making state government more efficient and um and effective So he um You know, of course, there's other things that he'll come to you with that Right now. I mean have a ton to do with our current pandemic state But generally if you look back over the things that the governor's proposed over the last four years they They pretty well revolve around his strategic outcome And that's how he's trying to drive those strategic outcomes because you can't really implement a strategy if it isn't funded So just to give you an idea of how and why Um Indicators or performance measures are such challenges This is the the previously mentioned family supported housing So That you've got the purpose of the program is to support families to secure housing and transition into permanent housing over time Blah blah. Okay. So what do we do? And we have case coordinate coordinating services and case management Try to get them employment stability financial stability child safety and family health and wellness So here are 16 different performance measures That we could use and collect the data on and report on that all Work to tell you if this family supported housing program is working 16 is way too many It would be a full-time job just to track this data So one of our challenges with both performance measures and indicators is how do I decide which ones of those I should use Okay, I could cheat and use the cherry picking ones that will make me look the best I could I could use the ones from I could find ones From sources that I know will sort of lead the conclusions to the ideology I want But neither of those things are really what? As the chief performance officer I want I want to use measures that come from a valid source. So if I'm not collecting it myself I better be getting it from the federal government the annie kasey organization the pew charitable trust, you know, one of the real Stall war noted reliable Producers of this data If I can't produce it myself, then I have to get it from somewhere that I know are the census, you know, things like that There are many measures you can find on the internet you can go out and you can look and lots of groups support groups advocacy groups, whatever We'll use what we call composite Metrics, so they will take several metrics of family supported housing and they will You know use weighted averages to come up with this combined thing, which is fine if you If you understand and look at the pieces they're using Because for example, if I want to talk about how much it cost a family To live, you know their rent their taxes their Utilities, you know Do I put child care in or do I not put child care? Is child care relevant to The majority of my population or not Do I put health insurance in? Do I not? So there are ways That organizations who produce all these metrics of these things that you read and go online and you get newsletters And i'm not going to say they're jury rigging it, but what I am going to say is Is that it's not a it's not a straight on Mathematic exercise Because things are incorporated or left out To lead the reader to a certain conclusion And we try very hard not to do that We want the real data and if we can't collect it ourselves. That's why we go to some place that has Data that you can rely on that hasn't been manipulated that is Always reliable always current timely and available and accepted So that's part of our challenge You know people when when we meet we're going to have a public meeting and then a meeting with stakeholders about the racial equity Indicators and you know, so I know we're going to get somebody's going to want well. I want you know Native americans, I want one indicator for them by themselves You know, no, we're going to do this we're going to do this as a totality We're going to look at all of these groups together And then we're going to say, okay, what's the unemployment rate for example among this group of black indigenous And people of color in vermont compared to the aggregate statewide unemployment rate And I want to see those two right together. I want to present them together. I don't want to take all the racial equity Metrics and put them in their own section. I want to see The unemployment rate for bipoc right next to the statewide everybody in because I'm figuring it's going to be higher I'm I'm I'm assuming here and you know that on me, right? You know what they say about assuming But the value in showing that has got to be How it compares to the aggregate Not in and of itself Okay, so let's say so our unemployment rate right now is you know below three percent So so we're going to say well, maybe the BIPOC unemployment rate is five percent So if you just saw that five percent by itself, what does that tell you? Tells you it's five percent It's about all it tells you But if you see it next to the 2.8 percent that is the state aggregate It makes you ask questions. Why is it higher? What can we do if we want to bring it down and make it more in line with the with the state average? That's what all of this is supposed to do. It's supposed to help you Ask questions leave you down the path to where you can actually look at What are we going to do? What would you propose in economic development terms to bring down a rate of unemployment For a certain population of people that is higher than the aggregate How do you make how do you craft that legislation to hit that mark? And not everybody it doesn't hit everybody. I mean, it's kind of like all the coronavirus relief bunch Remember when it first came out and you have, you know, giant companies applying for the business Loans and the little companies that it's supposed to go to they didn't even get out of the door You know, they didn't even get to file the application before the money ran out because it wasn't crafted Specifically enough to get it to the exact place that you wanted to get it And I'm not blaming the federal government, but you know, that's what happens when you do things in a week and a half, you know Instead of take six months to investigate it and take all the testimony So it's a real challenge. What metrics you use It also does not help if you change metrics all the time because metrics are a point in time It's what is it at the end of the year? It will only help you if you look at it over three or four years and or Three or four quarters or something and you can see where the trend is going Again, if the trend is going up and that's a good thing. Yay If the trend is not going up and that's what we want. Okay. Why isn't it going up? Is it a specific area? Is it a specific population or it's are they specific count? You look at a lot of You look at a lot of Those indicators We actually break them out by chitin in county We do the state and then I do chitin in county and the rest of the counties lumped together Because you look at the state economic information with chitin in in it looks fine Going up There population going up jobs are going up wages are going up You take chitin in by itself and you take all the other counties In a lump and uh doesn't look so good Chitin and looks even better and the rest of it looks worse And then if you actually look at it by county you get two or three counties and you all know who they are Who are really hurting? So You've got your most populist Economic engine driver county overshadowing all the information from The other counties so if you just look at that You would think we're doing great. Yay more housing More jobs higher level income. Yay You know don't be going to Essex Because it's not happening there. It's a matter of fact. They're going in the opposite direction. So that's what this um, that's what makes this a challenge is finding the right metric understanding what it tells you And using it consistently over a period of time So you can see what's happening So that's the end of my lovely presentation. I have some contact information for you If you'll notice the state transparency site spotlight dot vermont dot gov Is um where you can find out a lot of information about my office and what we do And I've given you the direct link which is actually at spotlight it on spotlight vermont.gov for the scorecard And I have my email address there So if you have any questions after this or you would like more information about something in particular I'm happy to do that One thing I would like to do. Let's see if I can stop sharing As I want to show you the actual scorecard Now share screen Okay, where is it? I know it's here someplace Right, let's see Let's find out where it is Okay Oh, it's Here we are. Let's see if we can oh, I have an even better idea. Why don't I just go back? Okay share See let's go back to this share Okay, so what are you looking at? Anything Yeah, we can see here the scorecard. Yep Okay, good. So if I open You can when you go to the scorecard you can use this little blue thing here and open it all up And and you'll see that that will open as many layers as there are or you could have open just if you're interested in um Median house price you could just open that one And in addition to the chart which has all the You know data points on it. You could also open the story behind the curve. So here for example Is one of the ones that we do the state and chit and then we actually don't do non chit and then But I'll do another one. Let's see unemployment rate is a better one to look at so Here is the unemployment rate. Okay below three percent Looks like we're doing great before COVID But look at what the unemployment rate in chit and then really is it's 1.8 percent The whole state is 2.4, but chit and then is 1.8 and non chit and is 2.6 That's a big difference and if you were to drill down into that non chit and then even more You would find that Two of the counties are really in bad shape Okay, so You know, this is this is where you will find your um information That you can use then to say Okay, is 2.6 percent for the rest of the Counties acceptable to us If it's not I mean go back to 2015 and look where it was 3.9 percent for non chit and then 2.7 even back then chit and then was below 3 percent And the state was at 3.6 So 2.7 to 3.9 that's pretty pretty different So you can see that chit and then by the virtue of its low rate And by virtue of its weight within the total of the state how many people they have It overshadows the state rate To tell a different story than if you just look at the state rate So, you know and things that you look at I would I don't know If if joint fiscal, you know has the ability or you certainly can get this from things like this from the Department of Labor from Matt Barowitz But when you're looking at things and trying to think about how you want to help housing or how you want to help uh employment or or new businesses Take a look at it beyond just the state Because it may change How you decide to direct Your investment or how you decide to change regulations or whatever And it'll give you a more informed picture about The economy and how these individual things are affected Okay, so that's um You know and so outcome two is Um brahmanas are healthy and so this goes all the way down. It's got all 10 outcomes and each of the outcomes Have indicators and each of the indicators you can open By clicking that little Plus and you can read Uh information, you know, here's one of ahs as they do a great job They even give you other things to click on and go and go find out more information There are stars in this frankly and Part of the reason for that is because conhogan started it there in the 90s and doug were seen picked it up again And governor scott has pushed it forward. They have the most experience in using These kinds of metrics and have been doing it for many many years. So For them, this is just built into how they run their business For other agencies Um, not so much. It's for them. It's confusing and hard and extra work and whatever Transportation is another one that does a really good job and ag to its credit Has really adopted this and is really moving forward and so is education. They've even put a new chief Data officer into education who is helping to corral all of the data Education has masses of data because of all the federal programs and everything But it's all over the place and it's you can't use it. You can't compare it and so they actually have Wendy geller who's their director of of data and is you know, trying to corral all this stuff and put it into One place where you can use it and compare it and and make Informed decision So that's what I do Is anybody have any questions because I'd be happy to answer any Now that I've just talked babbled away for however long michael Thank you I really enjoyed this presentation. Uh, you know, I think one of the areas that The public sometimes have a dim view of is accountability in government and You know when when choosing kpi's outcome measures Around issues that can be kind of A little harder to get your hands around like well-being. Yes, um, you know, obviously that's that's a challenge I think that gpi measure you were talking about that might not be continuing right now That you know was kind of driving towards some of that I mean are there times have there been times where there's been major policies or initiatives that really You never felt like we were able to get a quality measure for right So That's where you have to use what we call proxy measures So our vermonters are healthy. There's no one metric I can give you That says what percentage of vermonters are healthy? Okay, because it's it's too complicated and it Too many variables. So you have to pick The metrics that you have that have good data and you can get too easily and are are updated all the time That are approximate. So hence the proxy So obesity Who has health care because we know people that have health care The insurance right who has health care insurance because people that have health care insurance are Healthier than the same person with the same health conditions who doesn't have health care So that's one now people might might say well, what are you putting health insurance in there for it? Well, because it's a big factor It's a big factor chronic disease Immunization oh that one's you know become an even more important one. So You try to find measures not too many You know Three to five maybe seven if you have 16 nobody's looking at them anymore after they get about down to the third one Their eyes glaze over and they don't know what they're looking at even mine do So it's that balance of trying to find um The right measures that broadcast what you're trying to say Without having that one definitive you know Measure because it doesn't exist You know, I mean it just doesn't exist. So in transportation, what are you looking at in transportation? You're looking at things like how many bridges have passed the uh, you know structural test How many um new miles of paving have we done? Right, um, you know different things like that in in uh, in uh, the environmental, you know Osphorus and lake champlain. Hello, you know how many miles of river do we test or watershed do we test every year to to From the water basins that are going into lake champlain because it's not just the phosphorus in the lake It's what's coming in every year from all these other places I have to I always have to tell this story When we were when I was just starting with this and david mirs had just brought Lean into uh, the department of environmental conservation I had a meeting with him and he was already in the little conference room and so I come walking in He's just finishing finishing an obviously difficult conversation and like sort of slam And looks up at me goes they want me to get rid of phosphorus in the lake fine I can get rid of phosphorus in the lake chuck has to get rid of all of the cows Chuck was the secretary of agriculture at the time right that's so so you also have this going on Okay, we don't want all the cows to go away But we want the lake cleaned up You know, that's sort of uh competing priority. So how do you deal with that? What's it worth to us? I mean is cleaning up the lake and this is a rhetorical question Is cleaning up the lake more important than having cows? I mean, are we willing to sacrifice all of our dairies so that we can clean up the lake? Probably not But on the other hand, we're also not willing to just have the cows You know doing their thing haphazard unmanaged To just keep continuing the problem of phosphorus in the lake. So what do you do? You know, so you have competing measures dairies doing great. Yeah more poop Water in the lake is getting dirtier not good, you know So it's a challenge. I hope did that answer your question? Okay Charlie Thank you, Mr. Chair. Thank you. So your presentation is always very, uh, wonderful. I really enjoy it. Uh I'm wondering a couple things uh, and I think when you're describing the governor's priorities of now being a four legged stool And I think he probably does a good job of Organizing the different programs. He's uh introducing to try to support his four different initiatives The question is that our our measurements are fixed in statute, as you said So how often does the administration compare compare our level outcomes with what he's trying to do? So it's the administration versus the legislature Your the measures are not fixed in statute. The indicators are can change From year to year based on the decisions that GAC makes but we try to keep them stable But we have a strategic plan and we have some of the same metrics And other if you just google Vermont strategic plan, you'll get it Let me go see what the actual link is for that So it is yeah strategic plan one word dot vermont.gov. So some of those are updated Um some are annual and we have um our own indicators if you will for his outcomes and we also have it by agency each agency has a strategic plan now We honestly we never We didn't promote this because It's at it's out there. It's public anybody can find it But it's it's our guidepost. It's the governor's guidepost to how his cabinet is doing Are we bending that curve because you know, he came in under six three one, you know, how many people Are you know, we're losing population We're the second in the nation Our schools have 20 capacity Gap between, you know, we've lost 20 percent of our students over the past however many years And um, you know, at least at the time when he first came in we were having almost one Baby born a day that was um addicted to drugs So those were um, you know, his areas of priority and so his He expects each of our agencies To be contributing to that in whatever way we can now you're going to say well human resources. How can human resources? You know connect to that. Well human resources can make sure that the training we're giving our state employees Is the best we can give them So that We are making them more valuable employees who are better able to do their jobs And do a better job of getting Assistance to beneficiaries So, you know, I mean, it's all it's all a big swirling pot, right? So it's big swirling state government and one of the major things that he's done is Is try to get the agencies and departments. We're really we've really gotten good at working together He he's been busting the solar the silos down Like i've never seen before and this is you know, he's my third governor. So He puts we have right now for covet recovery. We have cross Agency teams working on what are we you know, I'm on one headed by secretary young that has secretary flinn smith and Quinn and some other people What are we going to do about all the employees and the state buildings when it's time to come back? What are we going to do? We're just going to flip a switch Are we going to let some people continue to work remotely? What's our policy going to be? How are we going to deal with the fact that You know, if we're bringing people back They're still coveted, you know, I mean, are we going to wait until next year when it's all gone? And what are we going to do if it comes back in next november? So And then they have an education team that are working on trying to figure out. Are we going to get schools back? To quote normal whatever that is And how are we going to deal with this? issue that we've had Where it's so inconsistent about which schools can provide Remote learning and which schools haven't been able to and which students have access and which students don't have access So we have and then there's another group that's working on the economy. You know, how are we going to Help all these businesses that must just be hanging on, you know by their fingernails And how are we going to take advantage of the fact that we seem to actually have an influx of people? I mean one of the things the governor's been Trying to do is how do you get people to move to vermont? Well, I you know, I hate to say it but Just like Irene taught us or Irene settled the 20-year discussion on whether we should build a new state health bill or not in one night You know, this has Showed us that you can actually get people to move here You have to provide them with wi-fi. You know, and that doesn't necessarily mean we have to have IBM back That they can come here and work their job remotely And that's what we're getting. I mean, look at the housing boom, you know, that's how the houses are selling to out-of-state people The day they go on the listing site unseen You know, so Necessity the mother of invention I The other question I have for you and it's probably not a fair question. I apologize on on lean training There are some agencies that are known for being particularly difficult to navigate for vermonters Oh, yeah, yeah, and and and particularly in this covid crisis when many people filing Various things. Uh, so wondering if if your office gets involved with Helping those uh agencies or departments Yeah, we actually had a um a particular program That on we have a project going for the last Um, uh, a couple of years which is going to be uh ramped up. You'll hear more about that. I'm not taking the governor's thunder away um about, um One of the big frustrations of businesses is how to navigate the seven Agency department permitting system We are we just finished one successfully about, um Vendors who come into state buildings Have to get processed for security By each department. So we've just pulled all of that together Um, I believe there's there's probably a bill uh in about language that gives bgs the authority to um Like do all the all the work with the vendors to get them authorized to come into a state building Before like the judiciary did their own and bgs did it and but then corrections did each separate correction facility separately and then public safety and so All of this is now is going to be a combined system so if Joe the plumber gets approved to go into The springfield prison he's now approved to go into Any building for a period of time before he then has to re you know reapply and bgs is going to keep track of all that So that's um an example of you know, how we do this. It takes a while It's not instantaneous, but we do work through these things. Um, we did You know DMV's done several One of the DMV ones that I love which was early on which was these motorboat permits That have they're like this shiny foily thing, right? Well, they kept running out of them Because you because that special shiny foily thing It's like special papers So you have to order the special paper and then send it to the printer and then when you're getting low and yep Someone asked the question in the meeting. Why do we have that special shiny foily paper? Do we need that is that in statue? Well, of course it must be not Regular coated paper is just fine so it's It's things like that that seem and that's mostly where The governor's program has been working where I've been working and and uh, justin kennie who works for me my my little genius um Scratched a little my genius. Um Goes out and it's just how you challenge people You know why do we have that special tiny shiny paper because somebody thought it 30 years ago It was good stuff, you know, I mean so is it in statue? Oh statue says we have to publish in the So-and-so paper which doesn't even exist anymore Sure, right how many statutes say report Deliver a report to the legislature Almost all of my reports now are electronic And I send, you know, sometimes I send a pdf over but basically I send the link Here look at it yourself. No, I'm not printing out 25 copies three whole punch You know So um, so that's that's where we use that a lot, you know And they're just you know people well, where where are you saving money? Well, I you know, that's the really hard thing to calculate is how much I can tell you how much money we save going from Citrix to To teams because that was a contract. Okay, but I can't tell you How much I saved by Making the department that had seven signatures for time sheets or expense report Go away. What did it say? I mean, you know six minutes a week for this person and an hour for that person And so those are hard to get a hold of but what you do have to say What you're doing is you're building capacity in the department So if you've got a back load They're backlog that just keeps building and you figure out how to deal with that The department is not going to come to you and say legislature. I need another person because we're backlog No, you're backlog because you're inefficient You're backlog because you have seven freaking people signing this one piece of paper, you know That's why you're backlog You know, you don't have too much work. You're doing unnecessary so It's uh, yeah, those are the things where I'm terrible because I my mother always told me I have a face like a plate glass window So I have to work very hard to like keep that We go into these things and people go or like the comment when we were doing some work on permitting and We may we're making suggestions. Someone says, oh, yeah, we tried that in the 80s and it didn't work I have to go So it's really I want to go excuse me Okay, first of all, you've been here since the 80s. That's a problem and b Do you not think that with technology or information might have moved on a bit since 1988? But hey, you know, what do I know? Yeah, yeah, it gets hard sometimes just sit there and go I could hear my father going and there has to be a system Susan. Yes, okay I just want to say that this has been really fun to listen to and it has put a lot of things in perspective in the 12 years I've been here I sat on the committee which will not be named regarding a department or an agency or whatever was that won't be named but we brought in david mares and the young lawyer a couple of people from his office a couple of There was a young lawyer who was hired to deal with the Backlog of appealed water permits or something out of whatever it was This poor girl looked at this pile of stuff and said she won She thought she would retire and never get through the pile And she put together david had her put together a lean process And that each one of those permits must have gone through up and down these lines of um, you've seen the lean And there must have been 14 things that it had to go through before it finally got it to the decision point and It was we we actually did that with them and one of the biggest problems was 75 percent of every permit that was mailed in because they couldn't do them online Was not filled out completely and had to go back So this is what she sat there doing most of the day open the mail look at it Take an envelope write it out stuff it and send it back So you put them online they cannot submit them without The information in the little boxes now that's not to say That they may have the wrong information But you know it got it down to where only 30 percent of them had to be sent back and And the error rate of incorrect information You know what we found she don't tell anybody but it it became very clear that a lot of the engineering firms Under pressure from their clients. They just send them in And so they can save the client. Oh, yeah, we sent it in, you know, and it's that damn department Except for the thing game and it's only half filled out Yeah, no, but this had had to go through approval rating someone initial did and they sent it back to the original one And he moved over to another person and they got in it. It was it was amazing to see and it was um I always found that that david did a remarkable job I didn't know what the motivation. I mean what the circumstances were but he did a remarkable job with it And I'm beginning to put all the pieces together Yeah, and it's amazing some of the things that you that you find it's um, like I said I have to like bite the inside of my cheeks dig my nails and my palms because I just want to go Come on people, but it's um, you know and employees are just doing the job as they learned it that's all They're doing they're working very hard and they're They don't always know and maybe they shouldn't know or should Whether this step is actually critical, but it's their job and they do it Yeah, and until you can help them And the managers understand and of course that's where I have my biggest problem It's what I call the frozen middle The employees are great. They all get it. They come to class. They love it the cabinet's really super supportive But in the middle is the supervisor who's like, I don't have time to do things right just get your work done, you know Or this is the flavor of the month Which is you know, one of the things I battled for a long time Um, and as soon as the next administration comes in this will all go away. So I'm not going to do it Hence why I said to the governor you better be giving me six years here because otherwise I'm not going to do this again So, um, thank you. It was a really well done explanation of how the thing is supposed to work. It's one of the best explanations I've heard Thank you. And it's really interesting when you Realize that there are some really simple ways and that's something kovat has really done for us You can file things online. Oh my god. You can have meetings online. Oh my you know You can share documents in SharePoint. We have SharePoint. We have Microsoft SharePoint. So So five people can be in the same document at the same time editing it and it keeps track of all of that and it tells each of you Oh, you and bob are trying to change that same line. So you can send a little message electronically to bob gone bob What are you doing? So I mean all of this technology is there It takes money and it takes some planning to institutionalize it But it is amazing how Much more efficient and confident you can you can be About what you're what you're doing You know, look at all the doctors that now do telemedicine Because I had to And labor I mean, you know that damn System in labor it's been there for 30 years the feds built it 30 years ago And all it can do is pay on employment and the only thing you can do is change the rate So come in and say now I have to give a flat amount to everybody and not a calculation You know Can't do it can't do it And we have 200 people on the phone lines because there was no way to do online. I mean All of that, you know been worked on and being changed now because we had to so There's a great line that Sean Connery says at the end of the hunt for red october if anybody's ever watched that movie and he He says at the end to jack ryan You know a little revolution is good for the for the country or the world every once in a while And although this isn't a revolution. It's like these crises as painful as they are do lead to some really Innovative changes that you would never have gotten to you would never have done because you weren't being forced to so Thank you. It's been a pleasure. Please feel free to email me Certainly will sue. This has been excellent great Been a great conversation Because I was looking at the At the outcomes And looking at the economy and I'm just thinking, you know, what's missing there Right, and you know, there are things that are missing. It's you know, our workforce continues to contract That's correct. And so how how is that affecting our prosperity? Well, you if you go to the strategic plan, you'll find one of the metrics we have there is The workforce by county And that's a really Sad chart. Yeah, I think it's a bit that that's a big eye opener. Yeah, it's it's it's sad It makes me sad, you know, even though I try to be Agnostic about the numbers the numbers are what the numbers are, you know, they tell a story and It's it's sad sometimes to look at what the numbers say because you Know that these are people There am I, you know people in this town people in other towns? I may not know them, but they're all Vermont people and and you, you know, you It's hard to keep that Acknowledgement out, you know that rat go holy crow, you know, look at all the people in whatever county that How many fewer people there are and how many are on or out of work or Yep So, um, yeah, I would encourage you to go there too to get some more economic metrics Yeah, I'm happy to send you the link again if you would like by email or It's on the presentation. So feel free I think it's a plan dot vermont dot gov, right? Yeah, that's right. Okay. Okay This has been great very much so thank you. Thank you. Have a good day you too committee, thank you very much for another good week, I think I think today was very informative from everybody that we saw and talked to and I think a good week overall We'll try to finish up the agenda so Amy can get it out to everybody tonight