 Good morning, I'm Ryan Colson, and I'm Max Kirkpatrick. And today we will be presenting our research from the last eight weeks for the Continuum Robotics RU here at UW-Stuff. And the topic is bendy arm. User testing of a Continuum Manipulator for Assistive Technology. And the ARM stands for Assistive Robotics Manipulator. So first of all, what is a Continuum Manipulator? Continuum Manipulators are robots that are inspired by the limbs of certain biological organisms. For example, the trunk of an elephant or the tentacle of a squid. Now what's special about these particular limbs is that they can bend continuously along their entire length. This is as opposed to, say, my limb, which can only bend at certain joints, or the limb of a traditional robot that you might see in a factory setting. Now in order to achieve this capability of continuous bending, these robots need to be constructed of compliant materials. And so, well, you can imagine that you need a material that can bend a little bit in order to achieve that bending. And additionally, because they can bend continuously, these robots have infinite degrees of freedom. And so these two aspects, the infinite degrees of freedom and the compliant materials of which Continuum Manipulators are made, imbue them with this quality of compliance, which is very important to them. And by compliance, what I mean is that these manipulators will conform to their environments. And so that means that when they collide with something, instead of maintaining their original shape, they yield and conform to the shape of whatever they collide with. And this compliance is really the most important quality of Continuum Manipulators. By the way, that picture in the top right is an example of a Continuum Manipulator, which you can see looks very similar to the one that we have constructed. So like I was saying, the compliance is a very important aspect of these manipulators. And it really is the key aspect that differentiates them from traditional robots. And it's the aspect that makes them very suitable for assistive technology. And the reason for this is because, as you can imagine, in assistive technology, you experience a very close human-robot interaction. And when there is this close human-robot interaction, one of the main concerns is the safety of the user with regards to the robot. And so you can imagine that when a traditional assistive robot, like the one shown here, the Jacob arm, is being used, if it were to collide into the user, it would have potential to cause harm to that person because it's a hard, rigid robot. However, when a Continuum Robot collides with an individual, as I said, it just conforms to them, and there's no real opportunity for harm. And so this is why Continuum Robots make so much sense in the assistive technology field. So I've told you what a Continuum Manipulator is and why it makes sense to use an assistive technology. Now, from a more academic standpoint, what are the knowledge gaps in the academia that we're filling? Well, Continuum Manipulators are a fairly well-developed field. However, while many of these robots have been designed and many applications have been proposed, hardly any of these have really been tested. In fact, in the assistive field, there's only really one robot in the entire field of Continuum Manipulators that has been used for an assistive purpose before, and this is a robot that was used to demonstrate, well, it hasn't actually been tested yet. It's been built but not tested, and this is a robot that's meant for helping people in vaping. And so that's really the gap we're trying to fill, is using our robot and actually using testing for a specific application, which is assistive tech. Okay, so I'm going to start and walk you through some of the design aspects of Ben Neon, and I'm going to explain how we went about building this and how it actually functions. So Ben Neon is built of a flexible backbone. So these have, like, this backbone material here. It's low density polyethylene. It bends with a relatively low force and is relatively good at resuming its shape after being bent. And it's actually made by eight tendons. So there are eight cables. They're very hard to see on this picture that run the entire, well, four of them run the entire length and four of them run half the length of the arm. And these tendons pull on this here and here, and in doing so they can push and pull the arm in different directions. This causes the arm to bend, and then to ensure that this arm bends with sort of constant curvature, which makes it a lot easier to model and to understand. We have these disks here that are spaced at uniform distances, and all the tendons are out through the disks. So that sort of ensures that you get a much more uniform curve as opposed to just attaching a string to the end of the stick and pulling on it. That'll cause it to bow out in the opposite direction. So these disks help make the bend much more uniform. So there are two sections to the arm. First section ends here. So this is the proximal section, and then this is the distal section. Each section is driven by four tendons that are actuated by 125-ounce inch NEMA 23 stepper motors. And as I mentioned before, all the tendons are out through these disks. You can see these four stepper motors on the top, they're offset by 90 degrees each, so that is how we get actuation along two different planes. So this is one plane, this is the other plane, if I can do that. And that causes our ability to actuate these in pairs. So what we do is we actually actuate one motor in one direction and the motor on the opposite side in the exact opposite direction at the same speed. And that causes the robot to bend along the plane defined by those two motors. We can then move the robot out of that plane by using the other set of motors. So we can get to positions that are variously placed within sort of a sphere around the arm. Combining that with the motion of the second stage as well gives us even more flexibility in where we can position the arm. So now we have a total of four bending pairs that can move us around in space. This gives us a total of eight controlled degrees of freedom because there are eight motors actuating the tendons. And as such we can reach very complicated positions, sort of S curves, wrapping it around itself, wrapping around the objects, that sort of thing. So we can reach some pretty strange configurations. One other aspect about this and its design and this is sort of a drawback is the fact that the two sections are mechanically coupled. So as I mentioned before, the tendons run through both sections. So here's the tendons running for the second stage. They run through the routing for the first stage. As such, when you pull on those tendons you actually affect the motion of the hole on, not just the top stage. And so there's sort of a mechanical coupling between the two segments. And we actually, one, we inherently affect the position of the other and vice versa. So it's pretty clear to see like when we move the bottom section to keep the top one stationary, the top cables are static and they pull back against the motion induced by the bottom segment and this causes the top segment to sort of maintain a constant orientation under that move. So it sort of like continues pointed in the same direction as the bottom segment moves underneath it. And sort of we get an opposite result when we actually hit the top segment without actually hitting the bottom. We bend the entire arm as opposed to just bending just the top section. So that's one of the problems that we look to address through user control and such like that. So there are three different ways to control bend the arm. There's dual joystick controls and dual joystick segmented control and single joystick compensated control. In the top right picture you can see kind of our control panel. So in dual joystick control it's probably the most straightforward of the three. You use the left joystick to control the bottom segment of the arm and the right to control the top. Don't use any of the other switches except for the electromagnetic switch to turn it on and off. We also have single joystick segmented control. The main difference here is that you're only using one joystick instead of two. So here you only use the right joystick and then there are three modes to switch in between. In the first mode you move the entire arm. So it moves both segments at the same speed at the same time in the same direction. So that's good for quick general positioning. Then when it comes to like the fine tuning of the position of the arm you're going to use the other two modes which control the bottom and the top respectively. Then of course as Max mentioned the robot is mechanically coupled so you will get some undesired motion in the segment that you aren't currently controlling which is where the inspiration for the third control scheme came from. The single joystick compensated control scheme only has two modes. The first mode is the same as the first mode in single joystick segmented with the whole arm. The second mode allows you to move just the top and the bottom segment will move the bottom segment will move in the opposite direction at reduced speed to kind of after compensate for that undesired motion you get in the bottom segment as a result moving the top so it makes control of the top segment a little bit easier, a little bit more intuitive for the user. In any of the three control schemes you can click the joystick and it will send the robot into your centering protocol basically just returns the robot to a neutral position and once it is in that neutral position it goes through a slack removal procedure you can see our setup here we have eight limit switches one for each tendon so all the slack removal procedure does is check all of the limit switches if any of them are open meaning that one of the tendons is loose it will coil that tendon until it is taut and some of the slack has been acquired as a result of over extending the arm in any direction. So that's the design of the robot now we move on to really the meat of our research which is the user testing before we describe the user testing that we performed with the robot it makes some sense to describe precedent in user testing from previous studies so on the assistive tech side of things as I pointed out earlier in the bottom section in the intro slide the Jacob arm manipulator is an example of a currently commercially available assistive robotic arm that is in use and so really that's kind of like the ultimate form of user testing if you think about it because it's moved beyond just being tested and now it's actually being used by people in their everyday lives on the continuum manipulator side of things user testing is much less developed as I was describing previously and in fact there's really only two designs that have undergone any kind of user testing in the literature and that can all be summed up in three papers really one is the oct arm which has been used in field trials in a very non-formal sense where they didn't really record any kind of quantitative data it's also been used in a more rigorous study that did record quantitative data however this study involved using the oct arm on a computer screen as part of a simulation and so it was lacking in that it wasn't using the actual or a lot of it was using a simulation of it the air octer really it was just kind of part of one of the oct arm studies testing the user interface that they came up with it for it and again this was very informal testing that didn't record any actual quantitative data and so you can see how this gives us a big void to fill by generating user testing that reports rigorous data being collected and so as you can see that's kind of our third goal is to record significant quantitative data both to fill this void in the field and because inherently quantitative data is better really than qualitative data because it provides a more objective measurement some of our other goals for user testing were to get an idea of how we might be able to improve the robot just based on any difficulties the users had during the testing and to get an idea of how well tasks could be performed with the robot which we didn't really have to do any extra work for because user testing itself necessitates that certain tasks are accomplished however by designing these tasks such that they were able to either mimic real world tasks that disabled people might accomplish using the arm or to at least demonstrate the potential for being able to accomplish these kind of tasks we were able to kind of make a statement on how this arm a bendy arm might actually be effective as an assistive manipulator in the real world and so for the testing we split it into two separate rounds in the first round the goal was to test as many people as possible a very simple task and go through all three of the control schemes that we devised with the main goal of determining what was the most effective control scheme so that we could move forward into the second round and do more in-depth, more complex testing with a smaller number of users so just like Ryan just got done the same the goal for our first round of testing was to go through a lot of users have them use all three of the control schemes and hopefully determine which one was more intuitive and effective so the task for this one was very simple all we had to do, you can see the setup here all we had to do was pick up the nut and drop in the cup on the other side of the robot so our procedure entailed first introducing the three control schemes we varied the order each time just to counteract the learning effect and after we introduced each control scheme we allowed the user one minute to practice to get a feel for how it works and after all three control schemes had been introduced we had them run through the trial three times once using each control scheme and we of course timed them as they did so and then at the end we gave them a brief survey it asked for their age and any relevant choice to experience they may have had just to kind of control for any of those confounding variables we also had them rank the control schemes based on intuitiveness just because intuitiveness doesn't always correlate with effectiveness so we wanted to get their actual opinions on them and at the end there was just an open-ended survey we took into consideration for improvements to the robot okay so we went up testing 14 subjects and the data is presented here so we can sort of see a trend that single-dose decompensator has the fastest average completion time and the highest well lowest ranking but number one is the best so the best ranking for intuition score these standard deviations are actually fairly large so it's not enough data to make a very conclusive statement about which control scheme is actually the best but what we can do is see sort of that in general we observe a trend that people complete the test faster with compensative and they sort of notice that single-dose decompensator control is the easiest for them to use at least as a new user so we came up with some ideas as to why this might be and we basically think that it's because single-dose decompensator is the simplest of the control schemes to actually like just pick up and use so it requires the least amount of user input you don't have to control two joystick simultaneously and it sort of does the work for you so you don't have to like try to manually compensate for the motion the mechanical coupling in two segments it tries to do that for you and it makes these motions where precise positioning is more important and just makes this a little bit easier now the cost of this of course is that you actually get some reduced functionality because the robot tries to solve these problems for you that you might be able to solve faster so in dual joystick mode you could simultaneously control those segments with both hands and probably do these tasks faster than if you were using this compensative mode but in general we found that people were not very good at doing that they didn't learn that fast enough they didn't grasp that concept quick enough and it requires a decent amount of understanding to do that so you have to know the robot pretty well to be able to make use of that we also did control for things like age and joystick experience and that sort of thing we actually found that there was no significant impact of any of those on the testing results so people just did none of them did any significantly better or worse and basically that just sort of goes to show that with the data we have it seems to be the best to proceed on with the single joystick compensative mode additionally this joystick scheme has the advantage of being a single joystick mode which in assisted technology having a minimal number of inputs is very important if you have someone that doesn't have function in one of their arms or say has very minimal function in one hand that has another functional arm they want to still be able to use that arm so you let them control the robot with their weaker side and then they regain a lot more functionality than sort of probably even losing functionality if you require them to use both hands to control the robot so that's one of the main reasons that we went for these single joystick schemes one thing I'd like to point out real quick when the histogram is included on this slide is just to kind of bolster our conclusion that compensative was the most effective control scheme as you can see I mean we said that the standard deviations are very high which they are but as you can see there's this massive outlier in the compensative that really drags both drags up its completion time and raises the standard deviation a lot and really if you look at the histogram it's pretty clear that compensative is in fact the most effective so for round two of user testing we used the single joystick compensative because that was the term most effective in round one the goal was to see how much or to what extent people were able to improve their performance and the understanding of how the robot works cost a series of trials so in this round we had two separate tasks that we had users perform the first one was the pegboard task it was the simpler of the two you can see the setup in the top right corner we had to pick up the first peg and move it into a different hole on the same pegboard and then pick up the second peg and move it into a different hole on the pegboard on the other side of the robot so we had them do this three times each in the first two sessions and six times in the last session then there was the drawer task a little bit more complicated we had them open the drawer remove the metal nut from inside the drawer drop the nut in the cup on the other side of the robot and finally close the drawer and of course time ended when the drawer was finally closed so since I was a little bit more intricate took a little bit more time to complete we only did two sessions of three trials each and the results from that are presented here so this is again with only three users because the time requirement was much higher and we can sort of see the percent improvements across trial session one which consisted of three trials in session three with a pegboard test so we can see fairly large percent improvements actually almost 50% anywhere from 25 to 50% to sort of time improvement in the average time screen session we see a similar trend with the drawer trials although not in years higher of a percent improvement anywhere from 8 to 35% improvement and again we only had two sessions here I suspect that maybe this trend would continue if we did more testing the time was a serious constraint the raw data from these trials is presented here in these graphs so we can see with the drawer trial we have a fairly nice downward trend the bolded line is the average the red, green and blue are the three individual user scores so that sort of gives us a nice clear indication of a decreasing time to completion for these and then in the whole test, the pegboard test we can sort of see a similar trend but it's a bit more messy especially without highers like this point here one thing we did notice is that after trial number six and this was on a day split so this is where we took a break and we came back the next day and continued testing users tended to get significantly better there's a very large drop off here and we hypothesized that this is due to sort of the user gaining proficiency in a task they've done it a few times, they start to understand what they need to do and then they realize suddenly something inside just sort of clicks and they understand like a much more optimal way to do the test and that's sort of our hypothesis for this so before I continue with this slide I just did want to point out that the slides that we just discussed we were able to carry them out by testing subjects from within our group so I'd just like to give like a quick acknowledgement I was having Jerica and Quig so they really helped us out carrying out our research and with that I'll move right forward so beyond the tasks that we completed in our user testing there are additional tasks that we've both demonstrated with our robot and proposed that our robot could accomplish either in its current state or with small changes to the robot so some of the ones that we've demonstrated are being able to pick up a set of keys and put them on a hook and take them off and you can imagine this actually being a useful task for someone using the robot in an insistent setting just as the opening and drawer task that we completed in user testing would be and pictures are shown here of accomplishing that task we also were able to accomplish some back scratching with the robot which may seem like sort of an esoteric and maybe trivial kind of task but you'd be surprised by the results that come out of surveys of disabled individuals how often they mention things that seem kind of trivial like this like scratching their nose or scratching their back things that you take for granted as an able-bodied individual but that actually make a difference to people with disabilities in addition there are other tasks that we have not actually demonstrated but proposed that the robot could be capable of such as unlocking doors we could probably do that maybe not it could maybe not do with a key in a lock right now because that's kind of a more complicated twisting motion but if you can imagine the kind of door where it's bolted and you move over the bolt we think that in its current state Ben the arm could probably complete that kind of task you can imagine a multitude of different pick and place tasks that the robot can accomplish and this is actually in surveys the most important task that disabled people would like to be able to accomplish with robots is just picking something up and putting it back down of course right now the robot is limited by its end effector which is just an electric magnet and can only grab magnetic things but you can easily imagine that by swapping it out with a more versatile end effector the robot can pick up all sorts of things within its payload capacity another task would be bathing as I said there is already a robot in the academia that is intended to be used for bathing but nonetheless continuum manipulators are good for this kind of task and bathing might be kind of hard task to evaluate on a statistical measure but we propose that it could be easier evaluated by wiping whiteboard which is the same kind of wiping task but this could be more quantitatively measured because you could measure how much of the whiteboard could be erased so obviously bendy arm is just prototype there is certainly plenty of room for improvement so there is just a list of some of the most notable limitations of the robot in its current state first of which has been mentioned a couple of times now but it is the mechanical coupling of the robot how you get undesired motion in one segment even though you are controlling the other so it makes it a little bit harder for the user to control for us to come up with more sophisticated control schemes which is why we kept them relatively simple for the purpose of this research second major limitation is the backbone material it is low density polyethylene which is very flexible which allows us a great range of motion but it is not entirely elastic so it does not always return back to its central position when you go through the re-centering protocol so that kind of offset it as performance went on and additionally because it is plastic the payload capacity is pretty limited we found that it was below half a kilogram third thing of course as Ryan mentioned is the end effector it is just an electromagnet so you can only pick up light metallic objects with the robot in its current state so hopefully its future duration will have a more adaptable more versatile gripping mechanism in place of the electromagnet another thing to mention is that it is tethered you have to plug into a power outlet so that restricts where you can place it and finally it has a pretty bulky base which is an ideal for mounting it on wheelchairs which would be a good application of it as we are thinking of the field of assistive technology so the base and all of the other limitations mentioned would mean to be addressed in future generations so those are limitations some of the highlights of the robot include its low cost our final bill of materials was $574.85 and although this does not resemble production quality cost of the robot you can imagine that when you compare it to commercially available arms which cost somewhere in the tens of thousands of dollars range that this robot even in its production stage would still be relatively cheap the simplicity we also consider to be a highlight there are no sensors attached to the robot minimal amount of programming is required due to the fact that it has inherent safety and so you don't have to program in safety measures and it can be built by unskilled individuals using widely available parts for this reason we believe that the robot represents a potential open source platform that other people could build upon I'm just going to really quick run through some of the future research here we're looking at iterating the design of the prototype user testing with an actual target population so actually people with disabilities all of our user testing was limited to fully able-bodied people and also an actual comparison between the existing tech and what we're proposing so right now we have no comparison between arms like the Jacob or Manus arm and this continuum technology that we're looking at it's going to be a lot of work before we can get there because there's things like mainly as the fact that it is nowhere near as sophisticated as the Jacob materials need to be beefed up all that stuff that was mentioned before but if those changes are made you could potentially see running actual comparisons with existing technology actual price comparisons, that sort of thing to get to the point where this could potentially replace some of that technology okay so what's the big takeaway basically Bendy Arm is a continuum manipulator which we believe could be applied to the assistive technology field its continuum design might make it more optimal than currently available assistive robotic manipulators due to its inherent safety and low cost we back up this claim with our user testing in which we had subjects demonstrate the ability to complete a wide range of tasks with minimal training and quantitative evidence showed that users could improve using the arm in a relatively low amount of time the future research that should be completed with this arm would include iterating it testing it with the target population which is actual disabled individuals and then having a direct comparison between this arm and commercially available arms to kind of give an idea of whether this might be a better alternative