 Okay I see Quentin is joined. Okay so I think we can get started. Okay so should we first go through the questionnaire? That sounds good. Okay. Sure I just need to step away for two minutes but please get started without me so long. Okay I'll just share on a screen. So this is the survey we looked at this last time and addressed the comments. So I'll go over this again. Can you see the screen? Yes. Okay so the first question, how would you identify yourself? So here are those are the categories and then are you aware of the CNCF storage landscape white paper? So have a link there. Yes or no? Which CNCF storage project are you using or contributing to? So this is the more with the surveys more for the user side. So we have this you know whether you're using it or not here so we have we have Kubernetes, Duke, that is TKV, service brokers, CSI and other and how do you classify your experience? So any any comments so far? Nope no I'm trying to read as you go along but okay okay and what attributes of storage systems are the most important for you when making decisions on which storage system to choose? So this is the new I added this one I think this is a Quinten's feedback so I don't know maybe people will choose everything basically just saying there's some trade-off that you have to between your choices right so and then if you have any problems making those decisions. I can think of a few more you can add into that list yeah you could add snapshot replication. Oh so those are I think we have more about those like the some of the later questions we have those snapshots replication so we have those the later on so that those are in the data services section right I'm not sure that you couldn't put them up there in the other one though because I believe there are solutions that lack support for those things maybe I'm wrong but oh I think I think this is more based on the storage white paper basically we have the those five attributes in the beginning and then we talked about some of the other services what can provide those so we could add but maybe there will be overlapping right so so that's why it's not there so those are the basic attributes that we are adding here basically so is it okay we can we can we can go through the rest of them and then then you let me know if you think if this is okay so basically the in the white paper we first talked about okay so those are the the main five storage attributes and then when you're making decisions you know do you have to decide what is most important for you right so if you let's say I have some example here if you choose something that is a a system that is strongly consistent but maybe that cannot scale horizontally you know that maybe some system has has very high durability but then maybe performance is not good so things like that so some of the other like the data protection those those are on top of that there are those are the other services that you add on top so putting them together then impact your decision and impact to the all the basic attributes basically I think that's what it means so so going down to q6 yeah deployed the first one deployed as a hardware solution I think maybe amplification you mean I think a hardware storage solution right I mean right yeah hardware storage solution okay that hardware it's somewhat confusing with confusing okay okay so this yeah so this basically the first one meaning yeah it's like the hardware storage yeah like the traditional yeah like a enterprise story yeah and then versus the second one is you know the software component on commodity hardware that's the software yeah so that one's maybe if they're maybe getting long but software on okay on commodity server software might be a way to make it clear commodity hardware commodity server hardware might be a good way to okay or you could just say commodity servers that would work commodity servers better okay yeah okay is that better now maybe servers servers oh okay yeah I doubt that many people do it on a single pin but yeah so this they also say they can multi-choice right so we can have both yeah so everything's almost every question is like a multi-select because they probably have multiple storage not just one and so if you consume just three different public public cloud providers and then or maybe storage as a service so this is a yeah this is also just a Quintan's feedback maybe somebody will provide some database service on top of like aws or google cloud and then you know customer will just use that one directly so that will be the slayer on top so we have this one there anything else we could add or this is i think either it's a hardware it's purely the hardware storage solution or software on commodity servers or public cloud prior to storage it looks good to me okay okay and then this is basically just you know what it's a file using file block object key value stores databases and then I missed everything there but I think a category that I know is out there a lot that isn't in would be a screening solution like Kafka okay so yeah well so what should we there's well a lot of people would respond to Kafka but I hate to put the name of one specific solution so so they could have like e.g. Kafka but I'm open to other people suggestions trying to just e.g. Kafka where do you want to put that I mean maybe under databases but it is a different category is that the data there's a message yeah no no under databases it's a different category oh you mean like a streaming streaming it's not storage so what do you call that well you can use Kafka to persistently store things yeah we didn't we didn't include message buses as part of the scope of this working group I don't think so I think I mean if we you know we need to restrict the scope somewhere I don't think Kafka is what people understand to be a storage system okay so they could include that if they think about that maybe they could put you know we have other all right that sounds yeah so we have other and we have don't know yeah Kafka is I'm not sure yeah okay block okay so block then you specify what blocks three system then you give a name then actually if you give a name then if it's well known that we probably already know the or the other truth about that particular three system so we still have ideas two questions why is the data protection uh basically those are the you know we basically just listed rate and erasure coding those type of technologies and then the next question we have uh other advanced services like replications snapshots clone encryption or other so just some people may not know all those details about like the the first one I'm not sure so so but we have other or don't know if they don't know the details just they could just say don't know so is this okay uh Steve you know you you mentioned the snapshots so we have it here basically so do you have that okay yeah that's okay okay yes I think oh should we add one more question related to how the block storage is accessed I think I have that before but then after the last round of feedback I removed them because the same if you if they specify the name then you probably know that already so that's why I removed that we have we had a very long question long question in the last time the protocols okay doesn't have any examples for local or like you know on prem block storage options except for self I don't think we have anything that is just a what so there's just some names it can be anything right so just I this is just a it's just some examples you can be anything right maybe along with self should we just put fce is to see that kind of stuff uh we could we could do that yeah okay wait a minute those are not block storage systems those are yeah those are just a protocol yeah that's why it's a savvy 3d rbd but yeah we don't have the we removed the protocol do we need to put that back but if they specify storage then probably we know sometimes it's for both sometimes it's just one I'm not sure that we really care very much do we okay uh well but the protocols it's kind of important oh I can I can add like lbm that could be it's asca's yeah at least that's covered at least that's one one more protocol there isca see just to be clear the vast majority of people running on virtualized hardware they don't see that protocol at all they they just see a block device in the virtual machine um so for the vast majority of remember we're talking about cloud storage here so yeah a lot of people using cloud storage don't see that detail okay so probably yeah for yeah maybe for the enterprise storage then usually they probably know that okay it's fc or asca they probably will make some choice there so but no no but in that case I'm not sure in that case they're running open stack and system disks and open stack are backed by somewhere the enterprise storage system and they get plugged into the bottom of their virtual machine and they still don't see asca's or fiber channel or whatever um I think it's probably depending on which level that they are I mean if if they are actually if they're setting it up actually they have to know you're setting up fc is difficult so I mean if they are actually it's gonna have to set up those systems right and they would have to know so but you might have a specialized question that simply says if you manage your own storage hmm oh yeah maybe okay that was actually a general comment that I gave shing was uh-huh we've got users of the stuff and they have knowledge and a set of interests um we have administrators of this stuff or installers or builders of you know self maintainer or whatever uh and if they're answering this they have a different set of knowledge and a different set of requirements um so I think we need to be very careful that we don't confuse particularly end users who I think are the ones we kind of care the most about by asking them questions either they don't know or they don't care okay and I think if they asca's fiber channel would be an example of something where they don't know or care right I think I agree with that one I think it's more for a cluster administrator than the end user that question so we so we do in the beginning we do have this uh the category can we just take a look at this to make sure that we cover the other you know whoever will be responding to this we do have the cluster operator, flow provider, project distributor, the storage vendor, just storage user yeah I'm not sure from that if you could to Quintin's point if someone was on open stack some subset would actually manage the tier underneath the VMs and some might not yeah fell from their answer here which category they'd fall into they could they could they could use a multiple right they could pick multiple categories if they think they are uh so do we need to add something I'll just say manage um should admin or no well that would be like cluster operator type it could be yeah but even then it doesn't tell you whether they go below whether they actually manage below the hypervisor layer right so I don't know do we need to add a question for that but then um uh do we need to add a question do you manage your own storage I think it depends on what some customers do some customers don't do you need to say do you manage your own storage do we need to add a question somewhere on top or I don't know I'm whether their interpretation of a question like that might just even be that somebody who buys EBS volumes and pays for it thinks they manage it yeah that's true so maybe so we'll just leave it like this and then we we will be talking to the end user group and if any of you know they can still give us feedback if they think you know there's more questions they want to answer all right so um so one more uh suggestion around the lines of instead of asking you know protocols isca z fiber channel whatever I think the more interesting bit that I'd be interested in is does your storage uh system require the installation of the kernel module that I've noticed as a kind of uh a challenge for folks who are trying to deploy the kernel module are your storage Kubernetes you're thinking about a kernel module in the vm and without it uh the storage right or the host whether or even on bare metal but yeah the other one is for the for software defined storage that's containerized whether it requires running as root or privileged containers right on that side that seems strikes me as in the same category yeah I mean for for as far as I can tell anybody who's deploying a sds on top of kubernetes basically has to run privileged they have to be able to access uh the the block devices underneath to expose them um maybe if they expose just file system they can get away with not being privileged but the vast majority are privileged uh but doesn't hurt to ask also should we have a separate section for this cluster operators or cluster administrators where these questions are present and normal users can skip that section yeah I think you're right because um I think a lot of users would even be confused by this and wouldn't even understand where the kernel module is yeah those are the admin I guess the admin would be the one that would do those so um I had a lot of similar comments I think as soon as somebody answers you know EBS of the thing they use then we kind of know what the answers to a whole bunch of other questions are so we should not ask them and similarly if they choose sef or they didn't choose whatever something else then a whole bunch of questions are irrelevant yeah the only thing I'd say there is there's lots of people who are multi-cloud so just choosing EBS doesn't mean you're not also maintaining a physical on-prem data center I agree and and and that's why I'm saying this kind of is looking more and more like it needs to be a proper interactive questionnaire where depending on the answers to your questions you get asked other questions um because you're totally right I mean when you when you ask them which of these do you use and they say I use EBS and I use sef on-prem or whatever because I'm a hybrid cloud user then the answers to the EBS questions are different to the answers to the sef questions and you need to ask the same question twice once for EBS and once for sef except that for EBS we already know what the answers are for the most case because they're defined by the sef product or sorry the EBS product so yeah we end up making a very we either I see Chris is saying yeah we need a branching questionnaire and and the other overarching kind of comment I made was if we give people more than you know a very small number of questions to answer particularly if a percentage of them are like questions we shouldn't need to ask them they will just give up and not give us an answer so yeah so that's why yeah we try not to make it too long yeah when they don't understand so I was I don't know I was wondering if we should just have like a maybe a simple set of questions that get get some feedback and then maybe we need to um get a I don't know another round of survey or yeah sure um yeah well I guess a form of branching as we do an initial broad survey and depending on the answers you give them follow-up secondary surveys on specific subjects if that's what you mean that would work it's a form of branching that mm-hmm okay how is this going to be distributed by the way as an email with a link or I need to yeah I try to need to ask a acquittal by this one so you said this is the like the release team or you have the um a template of this one right from another survey how did they distribute the survey it's not not using this right has to be some kind of format that they distribute the survey uh well when you say I think Steve's question was different is is how we get it out to people are we going to email it to them or are we going to you know announce it at a community meeting or whatever was that your question Steve yes okay so so I don't know the precise answer to that I believe it was just mailing lists uh I can find out precisely how they did it um I think it's a different question is is how do we actually uh expose this to to users is it right that's my question yeah in there or is it a piece of web software that we develop or is it something else or is it a spreadsheet they fill in or whatever that's a different question and it's looking increasingly to me I think if we had a simple set of questions that we asked everybody and they just went and clicked through multiple choice questions you know survey monkey would be a reasonably decent answer it's not clear to me that we can actually formulate this branching questionnaire effectively using survey monkey but I'm not an expert on that so maybe we can figure that out later okay yeah yeah I think that if we have to branching the survey monkey probably doesn't give us that much capabilities to do that uh so okay yeah we can decide on that later okay so we we should you sorry sorry Shing I keep interrupting you I don't know it's possible that we can actually make links from one survey to another in survey monkey where you you know you answer two questions and then it says if you answer yes to either of these then click on this link to fill in this other survey monkey thing so it looks like a branching survey but it's actually multiple survey monkey don't check okay okay one other thing while I'm babbling on I just wanted to point out that I think the the idea that you had Steve of having multiple rounds of surveys where you ask you know one set of questions and then depending on who answers what you send out another set of surveys I think the problem with that is that you end up with a potential like survey fatigue problem where people say I've already done the bloody the storage working group survey I'm not doing another one or in the worst case they actually just think that it's one they've filled in already and they just have more the second one so I would caution us the other thing is to send them a follow-up requires exposure to privacy rules and things that vastly complicate it right because you assume you start taking IDs it becomes messy by the way while you were talking about survey monkey and it does do branching but it's only available in so okay maybe we'll just the first to get the questions first and then we decide what to do with it so if we do let's say if we do a branching um well maybe we I don't know after this long discussion I'm thinking maybe getting a simple one that has pretty generic questions is the better way to go here rather than to cover yeah every every last corner in every room so there are yeah so if we do branching we have so I think we also need to decide how many categories we want to go so here like we have several different identities right so that's another thing if we wanted to do branching for it could yeah could go differently for each one of those so can we just ask some general questions first and then we'll see if you know if it's too much or you know we can say if you if you are your row is something then for example like this question right and then we'll just say if you are an admin and this this one requires something we can add this and we'll see how that how that goes because if they are not they can just keep this question rather than design a completely different survey for a different row it's probably not necessary there are still a lot of common questions I think yeah yeah I agree one other idea that just came to mind is I think different people have different tolerances for questionnaires and you know I think there'll be a large number of people who are only gonna answer like somewhere around 10 questions and then they'll just get irritated and then there are other people who's prepared to spend you know much longer answering much more detailed questions and maybe if we structure the questionnaire in such a way that we can accommodate both essentially with mandatory questions you have to answer these and then once you've answered these based on your answers to those here's a bunch of additional optional questions then then at least we can you know get the basic answers from the people who not prepared to spend much time and get more information from people who are yeah it's a bit like those freeform questions at the end of some of these things like is there any other stuff you want to tell us and then some people will write tons of stuff there and other people will leave it blank but maybe we want to do it in a slightly more structured way okay yeah I was having the same thoughts that I started searching for data on recommended numbers of questions and it looks like there's there's people who say that suggest that you tell them not not just the number of questions but your request says this will take 10 to 15 minutes and a lot of people will enter into the path if you promise them that but beyond that you're not going to get much participation for an unpaid survey so maybe what we should do is rather than questions we should test this on ourselves just how long it actually takes to answer and come up with this you know come up with a survey that is legitimately going to take 10 to 15 minutes yeah I agree I think that's a very useful goal yeah okay 10 minutes okay yeah with that case and okay we'll just and I don't honestly know whether that's 10 questions or not but whatever it whatever you do in 10 minutes then we cut it off okay okay so we may need you also see if we want to rearrange some of the questions too if we think some questions are more important for them to see upfront before they quit okay so okay um should we okay so should we continue so basically I think those this two might be yeah some of those uh depending on the the row right some people may not know how to answer those um uh so do we let's say if we want to have some optional questions at the end um so the simple question if you are watch let me let go back to the we don't well I'm guessing just a guess here but I'm guessing what we have in this doc now is already at the 10-minute mark but it's already yeah it's already we can cut but um I was just thinking the cluster we don't really have a cluster admin do we need to add that one we only have you see the the vendor the contributor distributor user do we need to add a admin here probably should or is the operator will be you know operator will be the honestly I I think there's if you only describe it as admin there will be a lot of user confusion as to what what it means to be an admin I mean somebody who simply consumes as a service out of aws yeah but as admin credentials probably considers themselves to be an admin okay so basically we just remove this one just let them decide if they can if they want so let's say for example if we make this one optional question right optional this is a more detailed question if they really need this then they can answer this one right they can say don't know so we will put this one at the end is that okay or yeah that looks okay so sad wifey yes yeah that sounds good to me okay Steve you're trying to say something oh I was going to say that I thought sod might be only on a phone so much but oh camera showing on the thing so yeah looks like he is on the phone yeah hard to find a new button sorry it's okay I'll just yeah I'll just the desert storage system require instead of a kernel modus move that to the end as optional more deep question okay so um those type of questions should those also go to the end or we'll leave it here for now okay and then for file system is basically similar what type what file system are you using and then how you know that's the same right so we have how is data protected and whether more advanced so on that specifying on file system you chose not to make it multiple choice um oh I basically I I asked him to to specify I mean we unless if you want to give some examples because I actually this one basically just this is just well if I try to answer myself I'd be wondering do you want like a me to identify a specific vendor or are you asking for um nfs versus uh smb for example okay it's a protocol or okay you mean by the question all right so okay so maybe we should we have something similar just give some examples uh what is actually I'm I don't know what is the one on the what's the name of the the one on EWS is that the EFS something like that right something like that and then the last FSS something all right something like that uh we actually I do have the protocol questions earlier but I also removed that I think after the feedback so but they if they specify this one then we would know right um we could add the protocol question as a optional right at the end we can add some optional questions as for the protocol I think what we what we need to ask ourselves for every question we ask here why do we want to know this information and right so I thought the Alex was saying you know he he was actually wanted to keep those type of like the nine and ten those type of questions that he thought maybe with this we could have some ideas for the next round of we could have some use case you know look at what type of use case we want to look into next I think that's what he was saying so know a little bit more details rather than or just a very high level questions so I think that's what he was saying um yes and just to be clear I'm not saying we shouldn't ask them questions I'm just saying every question we ask we should be clear why we asking and what we're going to do with the information if we get it otherwise we're going to end up asking them all the questions we can think of asking them and they're not going to answer them right right so if they um because in the last you know white paper right and then we would presented it was said okay how the white paper now the next step we want to uh do some use case right so basically look at some specific use cases so go through the survey and then after they answer this we can look at survey and then decide what use cases we want to choose and then maybe do some more deep analysis of those use cases so all right so should I should move to next I think it's almost actually we almost there so object store just so you choose what object store you're using key value so we gave some examples and databases so didn't list any of the some of those actually it's up here in the key values but actually they are also databases and then oxidation yeah I wonder if this oxidation section should move up a little bit this is kind of towards the end we have this what container oxidation system are you using and what type of workloads are you currently running so this you know this is you know what we discussed last time so uh steep steep drop off I think he uh gave some suggestions um you know we just went to the the docker hop uh site and then just see what are the most used ones so picked a few just give give them as an example saying okay tell us what type of workloads are you running uh not on containers on containers and uh what are you planning to run in the next three two uh six months and are you using a voting plugin and then you know then tell us if you're using entry plugin uh docker uh plugin or flex voting csi that's that's the end of it so all of those questions you incorrect me if i'm wrong those are only things that a cluster administrator would know or care about right um right yeah those are probably uh cluster administrator mm-hmm the reason i'm asking is i'm wondering if one way of simplifying this dramatically would be just too limited to end users in the first round mm-hmm interested if you are a uh you know someone consuming cloud uh and so we remove essentially all but one of the options in the first question and then we can uh basically delete that question and then any question in the questionnaire that does not pertain to an end user um we remove and the benefit of that is one the questionnaire gets much simpler it becomes very easy to decide what to put in and what not to and then if we wanted to do another questionnaire later for uh like cncf project contributors we can do that and they won't they will be very minor overlap that the audiences will be completely different so we won't have the questionnaire fatigue problem right but i think for the i was just thinking i was looking at the people in the end user uh group um they are people who are actually managing their own data centers so it's not like you're just using it so they probably actually do have more knowledge i mean um if we are just completely saying okay you're going to talk to us but he's muted oh i was saying yes you're you're right the Venn diagram isn't two separate circles there's an overlap there which one i'm sorry which one is a little the concept of use some you some people are both users and admins at the same time yeah i i do know a lot of end users they actually have their own data center they actually manage it it's kind of hard to say that you they are in the cncf end user group and you're saying you can answer those questions or but i think quint might have a point that maybe we should get out an introductory short survey just to get the experience of doing a survey and simplify it and it would be an attribute of uh not pushing up against that 10 minute limit limit either and let's see what happens and we can do more okay so so how about we can we just choose those questions that we think are just for end users first and then we then we'll see so let's see right now yeah there are more there are 20 questions so we still want to know i think we want to know the workload yes definitely um excuse me i think i need to go get some water so we just keep like this three question they're actually uh pretty similar can we really count them as like three questions or seven questions i would suggest perhaps getting rid of the first one and uh combining the other two oh okay 19 and combine 20 and 21 and say which types of workloads are you currently or planning to run oh so you're saying like not not asking this one it was just a suggestion that that we uh just have a combined question saying which of the following are you either currently or planning to run in the next three to six months oh okay all right okay this questions don't have to be under the orchestration section right i think they're pretty yeah we can move this up so if we are going to say have only 10 single questions we maybe we don't need those titles anymore it's just we initially have those sections because there are many many questions for each section so if we just have 10 questions i would just get rid of those sections i think yeah we we can move this do we this one we still should have right where is it that one is good i think yeah i agree okay so so just move with us everything to the front without the without the section okay so and then we don't want the plug-in ones okay um and so we still have yeah we will have those so we have the this will keep 16 we'll keep we'll keep this one and we'll keep this so yeah we then we maybe we'll see if we still need this this question is a little hard to answer i think i'm gonna have to drop because i've got a prep for another meeting coming up okay thanks dude thank you okay then this one this one is like a more like intro kind of thing if they answer the below then we know which one they're using so then maybe this one's still but this one maybe if they want to answer other then it will be here they can say other so so what about what about this one we need this one right so they sorry shing and one one other suggestion we could potentially get rid of question seven and merge it into the sections you have below just just a thought okay if you have a block section the first question can be if you're using block complete the section and then you've got an answer to the questions of it automatically okay i was only thinking that if they are saying like like steven was mentioning um what kaka if someone was considering that one then they answer other that's why i thought maybe you know keep this but otherwise yeah we probably don't need this yeah or you can put it at the section at the end are you using any other stuff that we haven't you know asked you about this questionnaire right at the end you might have some free form free form uh you know other information okay so that's optional okay who's to have yeah we'll move everything else in the optional okay and this probably will be optional too i think it's maybe not that important and the attributes one we need this one right we need to know they what is important for them but they probably will answer everything i'm assuming um yeah maybe uh maybe a better way to ask that is to rank them yeah yeah i was just thinking i i need check if if that that is possible in sorry monkey or maybe that is possible sorry monkey ranking yeah yeah we're i'm sure there's a way you can do it you know either you have to put a number next to each one or something okay yeah i think ranking is better yeah and this is this is needed to classify your own experience yeah i think so okay and this one i think we need right we want to know what project so i think we're gonna take a contributing part out because we're only interested in people okay you're using okay and this one we need when they know that and how do you identify yourself yeah so okay so now we have how many questions we probably still have 10 yeah i would i would remove um most of q1 the question one um i think i think essentially what we're saying and again this is suggestion i'm not mandating anything but i'm i'm we're only targeting people who use uh cloud storage and so okay so we don't really need this they don't just they can't answer it but we assume that they are using it uh well i don't know what what other people think if if we said something along the the lines of we're only interested in your answers here if you use cloud native storage so you know if you're implicitly if you're a storage vendor and you use stuff then we're including you as a user not as a vendor and if you're a project distributor same thing and if you're a cloud provider same thing um and all of the yeah same if you're a project developer so basically people so basically it will just be this one category basically exactly okay yeah that's fine yeah we're just saying that we are talking in cloud native storage user yeah yeah they can still the vendor they can still be a user in their like dev development even you know exactly but but we're only interested in them as their capacity as a user as a user okay yeah i'm i'm fine yeah okay so now so now we have dramatically cut it down now so we only have one two three four five six seven eight nine ten eleven yeah i think that's good so uh just off thought like an additional question ask for the biggest pain points they have around uh using container native storage chain points okay points doesn't have to have a selection but more like a that will be that can be the optional part that we can ask them yeah i think it's actually very important so if we can make it simple to answer and non-optional i'd be in favor of that because i think this is you know one of the main things we want to know is okay what do you not what do you not like about cloud native storage so we can pick i don't know to what extent we can make that a multiple choice uh with another category yeah so what do you so it's about hard to hard to install or hard to hard to deploy they are user that they still have to they still have to deploy it somehow so you're assuming they are using cloud storage then yeah uh something puts on that one right like the failure or something like some failure happened performance this is tied to the it's actually this is actually tied to this one's more see yeah like the attribute question but this is more abstract and then that one is more concrete yeah so um okay so we give when you give some choices right so then they can easier for them to answer so what other things that you don't like about is hard hard to deploy hard to manage hard what um they don't have a solution to yeah i think i keep interrupting somebody somebody trying to talk in the background did i interrupt you uh i was this Kiran again so i think this kind of putting it next to this q5 made sense uh kind of answered so you just leave it as a blank just let them answer or do we want to we can do i was just adding two things to number five so one is cost and one is uh ease of use you want to add this exactly oh okay and then use exactly the same list for your next pain points question oh okay well that's a good idea okay yeah that's i think that's a good idea yeah yeah so basically they cannot achieve they cannot achieve this so yeah it's a kind of a here but i think what i have is probably a little bit more difficult to read so that's basically what that means you know have you run into any problems trying to meet those requirements that's what so yeah so can yeah i can use this because you add this one as an example yeah maybe again i think your accidentally didn't cut the ease of use you you didn't copy the ease of use item to the bottom of the second list oh okay i think it's just a question yeah okay and maybe this is also a ranking question so like if any of these are one of the most problematic of these for you in in ranked order okay and i think don't know it doesn't apply okay well if they have this one and a little difficult to rank because they could answer more than just one one thing yeah i guess if they have some terrible pain point we haven't got on the list they can write it in there and say this is no right yeah okay okay so they okay so um other comments on this looks final we have 12 questions basically i think we've done very well if we can if we can narrow down to 12 useful questions like this i think that's something people can answer okay it looks like they're useful questions okay so i will move some of those to the optional you know that's depending on if they want you then they can but they can skip them yeah okay then uh so we're okay with the survey now um queen and don't we have another item that you want to discuss right do we have you running out of time it's already end of the yeah i think we're out of time okay how we engage with the end user community yeah yeah yeah it's a pretty straightforward thing to have a thing uh and we can go there and we can ask a panel of end users questions okay i can send the details out to the mailing list it's pretty straightforward i think we should do it okay question is just when when we want to do it and i guess okay once we finalize maybe in a few weeks time once we finalize this questionnaire we can maybe use the opportunity to one point end users at the questionnaire and maybe even discuss some of these questions you know ask them these specific questions and get some debate going okay i thought i thought alex also has a concern that i think he said because he talked to uh shara and she was saying then maybe just need some representatives from the vendor side or something but not not like everyone on the list join that he was saying something like that so it's a thing that he uh wanted you to help make a decision over there but we can talk about that later i mean yeah first first finalize the questionnaire yeah let maybe send that out uh or yeah we can chat with alex uh either on this or or uh some other way cool okay all right thanks everyone so i will um modify this and then i can send to the can send it to the mailing list for more feedback sounds great thank you for putting it together okay thank you so bye now bye