 Go. All right. Okay. So again, this is the October 25th share call have call and They've got a few folks on already and here's notes from last time recording on YouTube as well Pretty Tight agenda or slim agenda here today, but I think we have any newcomers How you don't need to do roll call since we know everyone Terms of looking at the action items from last time So we did have a call When the few folks kind of looking through Getting the share read environment set up and then Going over how the harvesting framework works etc. Talked a little bit about How to start getting folks ramped up I Think everyone is pretty much Set at least at Virginia Tech that was on the call and getting their note They're no red share read environment set up, right? It's silence is acceptance Silence is a yes, I assume I Think I had something an update to show Today, I think Ryan Cam youth youth said you had something to share as well Right. Yeah, I came out the demo and he should be joining us shortly. I'm in here. Oh, you are okay, and They almost have anything else to mention before we go Did you want to assign a note taken first? Oh, yes, thank you Willing to take notes here Ideally not me since I'm facilitating I can take notes. Thank you harsh. All right So One thing that I made progress on even since the call earlier last week was Looking a little bit more at the harvesting Remember, so some of this is Repeat some of it is new essentially what is set up now is looking with cross ref as a Example of this essentially it takes a configuration From this YAML file here and it feeds at the URL that it should use and then names of the various properties to pull a few different things from and so far I am pulling creators and titles So kind of looking at how that works. It's taking in the YAML file here pushing that into the environment configuration Then it's making a call on the on cross ref splitting up the items here and Then actually pulling the Creators and titles so I'll so let me just try running this really quick so you can see how it looks at the moment Make sure everything's there. It looks like it is you can clear out About windows here Okay So just kicking that off it then generates all these messages and the very first one Shows the actual configuration being loaded Into an object So you've got you know all these things created with author So doing this mapping given name is to author.given and the name is author.family Etc. URL there and Then on down so all these is actually just pulling out a debug message at various points in the process So this is the next one here actually Actually for some reason it's like splitting the order of these but This one is actually this one here So this is what the work looks like coming out of cross ref so it's got Things like a created object a title array An author array so like this one Samuel Sharp as well as Joseph Bonomi So then looking at how it's actually parsing that it is getting the creators that into an array and getting given name Samuel Sharp Etc and also getting the The titles as well and just to prove to you that is actually using the configuration. I'm gonna swap These really fast Save this So we've got Samuel Sharp there. So This works though it should be swapped And I didn't have to redeploy the code or anything It's just looking at that that one Yama file is the configuration again We'll see that in here They are indeed swapped by accident and then looking at the creators It does then have Samuel's family name sharp is given names. So So the idea here is that for any time kind of mappings that we do For pulling things out of cross ref, etc Would then define those mapping files it would then be fed into The workflow the workflow would key in on whatever approved property was and then split that out and to pull Out the code the way it looks at the moment. It still Needs improvement, but it's at least functional. It doesn't handle every case necessarily, but essentially it's grabbing the like configuration For given in family name, it's interesting where it's actually Pulling from a few of her a so I'm doing a little bit of messy code here that needs to be cleaned up still but essentially It's doing a And that's to do a parsing this author dot family just so it's doing a split on that Then it's saying okay. I want to get the author. It's assuming that's an array at the moment and then it's actually saying get me Family here or get me actually get me given get me family there The titles looks much simpler at the moment since it's just one thing But it is also something similar where it has to grab an array of various titles so there are I think there's at least some titles that have more than one Object in them. I'm trying to remember But it is being returned as an array anyhow. So like here's another example of the title that's being returned and Sure, sure. So the The the file that you added the workflow file you added to GitHub the last time last week Does that contain this code too? It does not because this code is actually new as of 10 minutes ago Okay, so then all you could have to do is like just update that workflow to for someone Yep, so I will push that up To get to this perfect Yeah, but but again, like I've I've been Working working working to get time to work in this and I actually just added these improvements in the last hour or so So also just another testament to how how easy this is how this is this is going to be is what we're Overlooking at the other thing that you notice here is that at the moment these are being split up in separate results and the next thing that I was going to be looking at was merging those into One object so then the same object would have like the creators and titles and then The idea there is that we would feed that into whichever persistence layer is at the other end for sure So any questions on this? Hi, I have one question. Sure. Well, I was writing one harvest. So I started with unpaved all API and Unpaid all API requires so you can only fetch Using a DOI so it could be one DOI at a time So one problem I faced was like if I have say a list of a thousand DOI How can I make my harvest to like run one at a time like? Like and like I have a list of hundred you are also do is I want the entire flow to run one by one for each Like this this list could be a external input also to the system, but sure sure thousand in one node Yeah, I think this splitter node What is is is the is something that would do it? And I think you know kind of what this so there's so looking at this so I'm just thinking out loud right now So like the right here, I'm I'm saying split on message payload and it seems that it's an array So then it actually does Each of these following things in the workflow one time for each So like if you put the splitter earlier in the process and said okay, I'm going to Say the DOI is this and then run through all of the whatever workflow you have I think it would work to do that. You would just need to Have it being float in as some kind of Object and and like this that probably I was also using like a property thing like this where it's a Right right here. I'm setting the URL that is coming from that Yemo file to the URL that It's expecting Okay, so like in this one It's then using like it's it's just keying off of that particular property. It's the one that it happens to be looking for So because I know that I'm sending it beforehand But like depending on whichever property you're doing that would I think that would be the Thing so you'd have like the splitter plus like then the setting the property. Okay So is the question that you're trying to find a way to sort of like sleep in between iterations so so I found that this inject API does do the periodic thing but Cause the same thing every time at every time period. I wanted to cause a different URL at Equal interval of time like every minute. I want to fetch one URL, but that URL would be different, right? So my suggestion would be to Write the function yourself. I think I there might be some sort of like Q that you could use it would help But I'm not sure if that's readily available in red But somebody may have certainly written it. I look through the sort of like the The library or the node modules or so the node red modules available on the NPM, you know Library and see if there's anything that works for the use case we're looking for That's pretty much what I had for that one Yeah, I've been of I I'm trying to capture as much detail as I can on the meeting minutes But some of them are very new to me. So please add that's a lot of undetailed as necessary So is it a good time to hand off the Rainer cam then I will stop sharing Yeah, definitely I can take over in a second. Let me okay was on paywall I Can write this part. Okay. Go for it. Yeah Okay, so Okay, so everyone see my screen Yes, and can you see the terminal that just appeared above it? No, okay It's not super important. I can just describe what's happening. So basically I'm just running a so the way that Decentralized networks usually work is that it's a common It's like a whole bunch of nodes that are talking to each other, right? But when you join that network, you need a way to learn about a lot of nodes really fast So what's usually used for that is a bootstrap node, which is basically just a very highly connected node They're almost every other node is connected to and that facilitates a very easy process of discovering a lot of other nodes really quickly it's kind of like like in directions or You know looking through the index of a book or something and The idea though is that if that bootstrap node goes down then everything keeps running Everything's still connected to each other and you can still retrieve information from the network because it's not stored on that node The node is just the the glossary the directory for where all the other nodes are So with that in mind This is a node that's spun up in the browser and so what we can do is actually Open up a whole bunch of these And you'll see one second Oh Maybe come back to me in a few minutes Something's not quite working right here. That's how it always happens when you demo isn't it. Yeah, okay Sorry about that I'll stop sharing for a second and figure out what's going on right? You want to go ahead demo was the same for both of us Okay We could always you can always share all your screen. We could see your tinkering. I got it. I got working Okay, so let me show my screen again Yeah, sorry about that. Just take a second. Okay. So these are the list of nodes Which are the other browser windows that are currently open that are connected to this one So one of these is the bootstrap node It's right here. And then the other one is the The other browser window that's open that's connected. So if we open up another browser window with the same URL, so it's basically just pulling the same JavaScript You know, we'll see another one pop up so this is connected to two other browser tabs that I have open and also the The bootstrap node currently so that's all well and good, but you also have to be able to input and retrieve information So let's say we want to put a value in And so it stores that value as a hash And then if we take that hash We can retrieve it from any of these other nodes and it'll retrieve that value This is pretty cool, but what's really cool is if I stop the bootstrap node So the central thing that everyone's connected to I Can still input more information and retrieve that information from other windows. So basically the the I don't know the the moral of the story is that we have a working demo of multiple in browser I guess like torrent nodes essentially talking to each other and They can talk across networks and if the central node goes down Like if the central like essentially a quote-unquote server goes down They can still talk to each other. So it provides that level of persistence and of Decentralization that we've been looking for that's basically it cool. And then like To to take the step from like a browser tab Versus like a server like what is the like transition for that? Oh, so a server is actually easier It's it's very easy to run a similar sort of thing on a server But I couldn't make a demo of that too The idea is that people would be able to do either or I think eventually and they could help facilitate things Just by having a browser tab open where they could at least download things by having a browser tab open and then Be able to post files by keeping a server open like a server node Cam are you volunteering for doing a server demo on the next call? I could I would definitely give it a shot You don't have to I'm just making notes. Oh, yeah, um, let's say I would say possibly There's a it's definitely on the table, but okay, I would need to push on that And then it has to do with how much time I have available to me and where Where is this code going at the moment somewhere in good hub yet, or I'll throw it on share soon It's not a good right now though. Okay. Okay, but you'll put in that share research Or as a new repo or I probably yeah, okay All right, so maybe you can just shoot a Something to discord and share Dev when that's That's up most definitely So I think that was the bulk of what we had Planned so far for today I Think the other thing to mention so like harsh referenced Some things on github I think that's everyone I'll just pull it up since I don't have it open. It's like So all these things are on github share research the stuff that I've been working on is currently in the API schemas repo So if you go into there There's a few different things going into the mappings. There's Here's one version of that YAML file and Then the others are pretty much not in play there and The actual flows we pushed those up to a different spot So it's on this share read flows so This is then a version of what I just demoed but it doesn't have the latest updates So I will push up So the demo you gave Rick. Yeah So if I want to do the same thing on my machine I would have to get the share read flows github repository and the schema repository to work together with it Yes, that that was so the so really the You technically at the moment you don't need to to get stuff from the crop from the harvest flows All you really need is the API scheme is one Because you can actually just copy and paste what's in the file here and Then push when you're in Node-red let me go there You can just import to there so if I say Import and if you have it in your clipboard then you can just paste it and it will actually we'll put it all in there But I think we want to have a better mechanism for that as we're going to be doing some real version control around this stuff So I think those are the things that we're we need to figure out about how to actually be Using that flow directly from where it's pulling from from github So is that an action item for that is that that is an action item? Yeah, so that and actually that's a pretty good action item for someone else to take if they want to try to figure that out Could you frame it for me in a sentence sure sure so so it is taking the flow that is in github and then being able to Have Share read look at That file that flow configuration directly from the file that's coming from github as opposed to having to Push it to Node-red In its cone in this internal configuration. I'll show you where it's actually Getting it now. Let me just open the shell so I think it's in this see this Not no red directory. Yes, so whatever your root is for your user So I'm on a Mac with you know Linux Linux based right and if I do Here it is looking at this flows lib 2218.json as the main thing feeding that I Think there is like a configuration For that maybe it's in settings To you change that so that there is a configuration telling it where to actually grab That the whatever flows to load so that would be the other thing to to research is how how to update that configuration to point to The flows that we want to load from github. I can run with that Rick That sounds great Based on what we discussed earlier about like not having a like a place to do like chair project management for this is that thing you want to Think we should like discuss now or come back to it later or I'm trying to determine if there's an action item there Sure, sure. There is actually we do actually have a few issues on the share read repo Where we had put some things. This is this is it. This is right place, isn't it? I'm not seeing it for some reason Cameron is this right place Yeah, it is I'm trying to I'm trying to see it too. It's not showing up for some reason. That's weird. I mean, let me see if I can find the right I'm not quite sure. This is the right repo. Is it not the repo? I mean, let me go back. Let's see Think just share. Oh, maybe oh, it's just the share one. That's right Yeah, and there are some issues in here now Yeah, see if you so if you want to go ahead and create that Issue in here harsh I Can make sure you have rights to do that if you if it will not let you okay, we can take that offline Yeah, there's definitely some some stale tickets in here now that need to be looked at Yeah, the so at the moment we have I think it's You have the one project and we haven't done anything like With waffle or anything like that we certainly could any opinions on that I Feel like we We could just continue using this issues page for now and then once we get into a more collaborative set where Be multiple people from different institutes are working together on things and like we can formalize more like what projects and what miles Sure, that sounds good. Yeah Wait, where is what was that? I just I agree that I think Okay, go back to the All right, so good. See already got some action items on there And I have the action I'm to push my coat up as well Anything else for today doesn't happen up to go the whole hour Um, I can just like want to bring attention to like three action items and one of them is tentative for cam based on how much time he is able to You know find in the next month or so But I would expect like that we would discuss these things in the next tech call And just make sure we close out these tech action items on the next day call. Sure. Is there anything else? I missed based on what anybody sees in the meeting minutes. Please feel free to add And thank you for taking notes So our next call is set and normally we've had these scheduled Every four weeks, but this one got pushback week. So the next one is actually in three weeks And I was also just thinking about the ask the question of whether we should have another call in between The next one or if we should have these be slightly more frequent than every four weeks, but since the next one is in three weeks, we can probably punt on that one This one's good to me But if we but if it does make sense to have like something similar to what we had on the past week where we just did a little bit of More deep dive into getting a configuration set up, etc We could have something like that Any call for anything like that at the moment? Okay, the one thing I want to ask is like so as we as we're going to be working through Before in the next couple weeks, what is a good way to just add something to the next call agenda thing? Hey, I thought about this and I was doing this Let me just add it to the agenda instead of place I should go and create a document or look at an existing document and just add it to the agenda I Will create a skeleton of the agenda for next time. So and then send it out That sounds good. I can be another action item for me Yeah And I'm looking at the notes on this one itself and there was an action item for Leo to follow With colleagues at just on open-door licensing. Is that something we would? A little bit further today Leo Yeah, I can I mean I can briefly mention What I found out it was not a massive task obviously So I mean last time we were discussing that the open-door service has been more or less recently been updated to a sort of V2 version which is also more heavily branded as a Gisk service and previously what isn't stated clearly on that service though is However Remains that also the V2 service is the same Non-commercial share alike license that the original Open-door service has used and as far I know my understanding was that I think your Rick your and Jeff's kind of concern was I mean you've you've basically got their interest quite strongly to harvest data from open-door But you see the limitation that In the sense that you don't you yourself don't want to limit Your reuses to non-commercial purposes that that's the core of the discussion if I understood it correctly right that that's correct, so we wanted to want to Maintain that share can be used for commercial and non-commercial. Yeah, yeah regardless Okay, I mean it's again. I think from I mean in the indication from from our team from the Sherpa team who Managed also the open-door open-door service Is that you know we can discuss and see basically what we can do and I think that's It's probably valuable to at least do this I guess it and that's really sort of slightly speculated from my side It will partly I guess also depend on you know more basically is Are the chances of people using it? Reusing the data for heavily commercial purpose and I would assume they're relatively slim at this stage right right now Again, just yeah, that's just my argument sure But I am Remembering now that this is not the only example of data that we would have been in chair though as well if we're thinking about They loading data from Web of Science for example That fits the same pattern and maybe it actually is not a big and not a Big deal so so the way we have envisioned the usage of Kind of the peer-to-peer notes like the stuff that Cam demoed earlier is that a peer would request from another peer data Within a certain scope and then they would say yes, you can have it or no you cannot yeah, and I Wonder if the open-door data would just kind of fall as one Example of one particular use case that would that would happen there You mean in the sense of what would happen there right did like like so so this like there might be a you know This is this for a commercial and on commercial purposes. Mm-hmm. If non-commercial you can have the data. Yeah, it's commercial You can you can not yeah, yeah Yeah, I mean if you can yeah, I guess if you can kind of embed that somehow into your framework I guess that's To me that sounds sounds like a quite interesting case because I mean on the one hand I certainly share the idea that you know that data should be as open as possible And that in a strict sense again, that's my personal opinion It includes the non-commercial license, but of course that license is still Relatively widely used in some area. So the problem will reoccur necessarily Mm-hmm Yes, yes, yes So, yeah, I mean as I guess as to yeah, I'm probably from a static correctly basically to two ways to approach is one is having kind of the licensing discussion with us and The other one is basically looking out whether what you can do about sort of evolving your note framework to have the capacity to differentiate between different Um Basically access scenarios, right? Right, right? Yeah, so it feels like there's some kind of action item there Yeah I mean what I what I can definitely do is again follow that discussion up internally And and see what Um Well, I don't for example really whether we've had any sort of precedence cases precedent cases Where you know, we had some initiative or project using it While leaving basically a door We use this You know, right Yeah, yeah, this is description Good, let me just go to my browser because it's larger there Yeah So so I'll create a Ticket in the the GitHub issues just we haven't on our list. Yeah, so that we get to it when when it makes sense Good. Yeah Okay Good and there are some action items at the bottom of this meeting minutes document, which I don't know if they are like Necessary at this point Even for tonight. Yeah, we'll have been carried over Yeah, these I Think these are all Stale now just give her these Yeah, they're not they're not useful at the moment anymore. Oh wait there was the Yep, we did that. Yeah. Yeah, thanks for checking Okay Anything else for today? I was just gonna add that Taking making these recordings is a great idea and I find them very useful. I Just was watching the July call and I was able to come back to with questions for some people So we should keep on doing that Good. Good. Awesome. All right. Well, I will Stop recording now and we'll get this one pushed up to YouTube as well Thank you and I'll send out some notes to the share dev list about the link to these notes as well as Link to the agenda for the next call when I've got that. All right. Thank you all perfect