 This talk is little odd but for two reasons actually, first is the domain which is machine design not machine learning at least initially then we will come to machine learning and deep learning later. And second is I am being a teacher also rather than you asking me questions I will be asking you questions. Is that ok? Later you can ask me questions but initially I will ask you questions. In a mechanical engineer in the audience, oh boy, I thought there would be none, still I will give you some background. So the domain is machine design and use of neural network in some help to that. So what is machine design? Say designing products, I will explain that, I will take the questions last. So if you are designing a chair like the one that you are sitting, basically what are you doing? You are deciding the shape. This is the shape, you are deciding the material, that is what your aluminium, iron, whatever and then deciding the dimensions, how thick the base would be. This thick, this thick, this thick or this thick, traditionally how do you decide the minimum thickness that sustains your weight 80 kg, 90 kg, 100 kg, how do you do that? You basically make 10 chairs, different thickness, half people sit on it, whichever the minimum one breaks, that is the design, right, traditionally. But of course once the computers came, you do not do that, at least initially, you design those shapes on computer, this is what is called as CAD computer array design, you design those shapes on computer and then you analyse also them on computer. Strength analysis, putting some load, will it sustain or not? So because I mean, you can make 10 chairs but you cannot make 10 aeroplanes, right, so you have to design and do strength analysis on computer, this is a special case in this analysis. If the shape is little thin, like sheet metal shape, have you got sheet metal, something like this, sheet metal, where the thickness is constant but the other dimensions are far more than the thickness, so this is what is thin wall or sheet metal shape, like car bodies or aeroplane bodies or sheet metal, there is a trick, as the thickness is constant we assume that the stresses or the deflections are same, so at least one dimension goes off for us to worry for the strength analysis, that is the idea. These things may look a little alien but I will come to the shape problems slowly and quickly actually. So what is CAD, designing shapes on computer, CAE doing strength analysis, you cannot take shape just like that and then put it to CAE, if the shape is thin, what you do, you create a surface within the thickness and what is the advantage, instead of doing very complicated 3D analysis, it changes to a surface analysis, so from solid you can convert to surface and then the computations are reduced a lot but the results are equivalent of deflection and all. So if you had a product like this, what you do, can you see there is some thickness, it is a sheet metal like or plastic sheet, you make a surface first and analyze that. So if you put some load here, it is equivalent of putting load here and doing same analysis but as you can see this being a surface, number of nodes points to calculate deflections or states are far less. So this is what is called as made surface, the surface that is in the middle of the thickness, I will come to the domains quickly, don't worry. So the idea is given this shape, can you calculate this shape by some means, it helps because if you are doing solid analysis, the results and if you are doing a surface analysis, results are almost similar but the timing is reduced because the calculations are less, that is the idea. This has been in use for 30, 40 years, if it is a thin ball component, you have to create a surface in the middle, that much everybody is okay, we need to create a surface in the middle. Let us take simpler example just for us to assimilate, if this is the solid shape, this is the surface shape, the only condition is it has to be morphological, it should look similar, if you have shape like this, it cannot be L, it should look similar so that the analysis will be similar. Are we okay with this so far? Given the shape, can you calculate this surface, what is called as made surface? So this is a function, this is a transformation from solid to surface, this is a dimension reduction okay, but dimension is less. Is it easy to calculate this made surface, just look at the solid, what is the solid made up of, this phase, this phase, this phase, you have list of phases coming to us as an algorithm, can you compute this? So this is the dimension reduction research or the algorithm that we want to develop. Getting made surface is not easy, it is not a trivial problem, it is going on for 50, 60 years, still the problem is not solved, because of the complication, I will show you the latest results, are we all okay so far? Many CADs CA packages, those who are mechanical engineers may know AutoCAD, parametric, geometric others, they do that and we will see how they do that or what is the result, can you see some issues, this is the input shape that has gone in and this is the calculated made surface almost latest result, given surfaces of this shape like this, it is not possible to calculate made surface, so this is the original problem, the one that I am going to talk about is far diluted one for the easier understanding, okay, this is the original problem, so forget the mechanical engineering part, now you are in the shapes domain, given this shape, can you calculate a good made surface, is it okay? Because we cannot tolerate gaps, if your chair had a gap, what the analysis would be, it will just break there and there itself, so we cannot tolerate gaps, let us take this simple example, letters model as a solid, what its made surface would look like, we can at least intuitively understand, it should look like Yogesh only, the current output that we get is this, can you see issues, this is one of the latest output from a highly commercial, highly expensive package today, so there are issues, I will just zoom in to one of the things, if you have a output shape like this, the made surface that automatically comes is like this, it does not get connected properly, so this transformation from solid to surface, this function, we always call it as a function, input to output is still to be learned well or at least heuristically or even algorithmically, so this is the current quality, what people do, do they abandon this method, no, made surfaces so computationally nice or less compute intensive that they still do it, let there be gaps then people are put on the job to patch up the whole thing and the patch up goes for 2 weeks, still the whole aeroplane or the car body is made surface first, patched up really nice and then the analysis is done, so made surfaces are important, the computation is far faster because actually it is just not one computation, you can do iterative computation, change the thickness, again run, change the thickness, once we are made surface we are done, then we can do experimentation on thicknesses, so even if the quality is bad, even if the manual patchwork is needed, we still do it and uncorrect it manually, so this is the problem, original problem, let us look at a very simple shape, a shape like this, a plate, how a shape is designed in a CAD package, you take a profile rectangle, you extrude it or sweep it, so that the solid is formed, is that okay, I will repeat, how a plate is formed, you take a rectangle and sweep it, so this plate is formed, what would be the mid surface, instead of the rectangle getting swept, you can put a middle line and sweep it, are you all getting, so instead of rectangle being swept, a line if it is swept, it is a mid surface of the same solid, now the problem is reduced to it, so initially we wanted to do mid surface of a complicated shape, now we are coming to the plate, for a rectangle, can you calculate the middle line, now my second question, the first one was, any mechanical engineers here, the second question is, given four lines, a long one, short one, long one and again short one, can you calculate the middle long one, I will give you 30 seconds, by the way this research is going on for 30 years, you have 30 seconds, can you think in mind what would be your algorithm, just think, I am not going to ask you, and whoever cracks it just raise the hand, do not spell it out, okay, what is the input, four lines, one, two, three, four, and the output is middle line, think, whoever is cracked it, I will give you my solution which is dumb solution, and then we will develop on that, okay, who has got it so far, the problem is at least input output wise very clear, right, forget the computation, my solution would be, find the biggest one, find the shortest one, because we just get four lines, right, nothing else, find the biggest one, find the shortest one, shortest half, the biggest one, offset it or translated, you are so same, same wonderful, one of the, I should not say dumbest solution, that is possible, now you have this solution in mind, apply it to T now, apply it to T, how T looks, this, this, this, this, this, this, are you able to imagine how T looks like, get the biggest one, get the smallest one, is it a T problem, no, so we are developing an algorithm for any profile, thin profile, can you calculate a mid curve now, we had initial algorithm, longest one, offset it by the thickness half, is not working, can you develop this little further, now we can split, so there are various ways people have tried and I will show, this is the first paper came I think in 1967 of doing this and the algorithm is called as grass fire algorithm, if you have any shape, assume that it is grass, it is a lawn and you have started a fire from outside, as the fire comes in, wherever it meets that is the mid curve, this one of the conceptual under, I will show you few more examples, so for us the problem is given a shape like this black, can you calculate the red shape, say for y, say for l, for any shape like this, thin shape, is the problem clear, forget mechanical engineering now, even solid to surface, now we are in 2D, complete 2D and we are going to solve this problem, given outer profile can you calculate the middle one, is it okay, think about it, there have been quite a few approaches, I have put them very simplistically here, but this is of course, 50-60 years of work of people, other people, what they do, I will just quickly explain this is called as medial axis transform, they assume that the ball is rolling, whichever is touching the profile and you take the loci or the collection of the centers, that is the mid curve, now we will see it is not exact mid curve like the middle line, it has some branches, but still this is commercially used, people will remove the branches and still use it, because this method works, another method that you may not have got, mesh it, get all the midpoints connected and this will work for any shape, not just the rectangle that we saw or T or any shape, third one is thinning that I just mentioned, offset from every side, whatever is left, that is the thing, the last one is pairs, you find opposing pairs and get the mid curve, there are far many approaches I have just listed to you, is speed okay, mechanically okay, I will come back to the original problem, if the part is thin, we get mid surface, because the computation is less, but we get equivalent results, that is the original problem, so solid to surface was for that, number of nodes to calculate stresses are less, instead of doing 3D, too many degrees of freedom, if you have surface, far less number of freedom and we have reduced that problem to now profile only, 2D profile, 1967 till now, everybody is adding something to this, how to do dimension reduction of a shape, with as much less error as possible and it should be applicable to any shape per se, this is my PhD work, I am subtopic, actually my PhD is on mid surface and one of the subtopic is of course profiles, given a shape like this, my implementation would calculate like this, so you can say at least for simple shapes it is more or less okay, but things are not all rosy, it was fully rule based and algorithms running into pages and the one that I use was called cellular decomposition, you cut the shapes, make standard shapes as standard mid curves and connect them together somehow, okay that is the idea, if any new shape comes, they are stuck, new logic, not scalable, T was two rectangles, L was again two rectangles with a slanted line, I think it is fairly easy to understand, fully rule based, now this transformation from 2D profile to a 1D curve, can neural networks learn that, so that is the topic today, so people have been trying for 50-60 years, can we put a dent in this research, can we do it completely different way, forget rules, that is what machine learning is all about it, forget the hand coded rules, heuristic rules, we will just provide what, inputs and outputs and let somebody figure out, we need neural network figure out, so that is the idea we have been trying, how supply lots of examples, we know actually intuitively we know given a shape how its mid curves should look like, supply them as much as possible, can it learn the dimension reduction, okay, but things are not that easy also, so what is the idea, supply lots of training data, input output, let it learn, what is the use, given any unseen profile, it will calculate the inferencing, this is training, this is inferencing, right, that is the idea, what would be the input to the problem, anybody, I am going back to the basics, what is the input to this problem, dimension reduction, 2D profile to 1D curves, what is the input, what are the inputs, for a profile what is how profile is defined, point set of points, so for a polygon 1, 2, 3, 4, assume that they are connected, so they are lines, what is the output, one line, so four lines to one line, first pair, what would be for L, 1, 2, 3, 4, 5, 6 lines and two lines, are we all okay with this, input and output, supply as many profiles that you know, let it learn, so supply set of points or connected lines, non-intersecting, so these are some caveats, some mathematical things, assume that this is a closed polygon, we are aware of these terms, right, and the output is polyline, okay, simplistically, so it is a dimension reduction problem, like PCA, PCA people have done, PCA is used here as well, but little differently, so given a shape like this, black shape, can you calculate red shape, only condition it should represent the original shape, why should be why, why should not be L, the whole purpose will be lost, is it like a translation, can you see what is the input, number of lines on the left hand side, like English, number of lines on the output side like French, we have seen a lot of machine translation talks in this workshop, input some sequence output, so is it a sequence to sequence problem, in a sense yes, at least initially, I will give you counter examples, so is it a sequence to sequence problem, can we tackle it in machine learning way, anybody, LSTN, bidirectional LSTNs with attention, all those things, it is not really a sequence to sequence problem, I will give you some counter examples, consider O, what is O, two concentric circles, what is the output, one circle, is it a sequence on the input side, you started with having four or five points representing a outer circle, how do you jump to the inner one, if you have the next point of the inner one, the line gets connected, so it is not really possible to represent even input shapes, are you getting the difficulty challenge, if you have loops, you just cannot represent the data itself, got the challenge, output challenge is why, what is why, is it a polyline sequence, one, two, should I go here, as a next point or should I go here, so there are branches, so input is not a sequence, output is also not a sequence, ideally it is a graph to graph translation, graph disconnected graphs this is very difficult problem to solve, to my knowledge I did not find any solution on GitHub, so I said it is not solvable, that is what my limited understanding of deep learning, if it is not there on GitHub, it is not done, is it an autoencoder problem, what is autoencoder, input and output same, here is not case, input and output are different, but it is an encoder, decoder problem, we will see, some graph structure here, graph meaning network, some network here, we want to do transformation, translation, so there are variable size encoders, decoders, what do they do, if you have in machine translation, you have five words English, say seven words French, what do you do, anybody, you are machine learning people, you do padding, you add extra minus ones just to make the lengths similar, can you do padding here in case of geometric points, so you decide minus one is to be padded with, that minus one becomes a valid input, because minus one is a valid point rate, so you cannot pad with anything, again a different problem, so I mean too many problems, input cannot be represented, output cannot be represented, padding cannot be done, sequence to sequence, graph to the graph network is not available, variable size issue is there, what to do, and any solution, any spark, can we use images, forget geometry now, forget mechanical engineering, forget solid to surface transformation, now we are boiled on to the profile to curve, both things are not representable, so easy way out for which GitHub solutions are available is image to image translation, are you getting it, what has been solved by image to image, fixed size, 28 by 28, 100 by 100 input, even 100 by 100 output, inside figure can be anything, some pixels are black, rest everything is white grayscale, so we have converted the original geometry problem to image to image translation problem, 2-day image, images are 2-D to my knowledge, the 3-D things are called walk cells, so if we supply forget the geometry problem now, forget mechanical engineering, given this shape and this shape as an image, 28 by 28, 100 by 100 whatever you decide, can we learn the transformation, now this is in a solvable domain, right, we are in our land now, that we are familiar, no sequence to sequence nothing, so what are inputs, given an input like this, image, output image like this, input image like this, output image like this, is this a mid-curve, if you supply hundreds thousands of such images, input and output, can neural networks learn at least this, image to image, so what would be the idea, once you give unseen shape whatever, thin shape, can it give me mid-curve, in image, forget from image to geometry that part we have forgot, are we all okay, now I will just mention few solutions that we have tried, the biggest problem is, so input what are the inputs, shapes like this and outputs are also known but in the image format, it solves as I mentioned all those problems, we will try encoder decoder network, given a shape like this, it will have some network here, simplistically few nodes, maybe CNN, convolutional nodes, layers, something else which is far fancier, it gives you to circumference state or hidden state and then you expand back, how does it learn, this pair this pair, whole thing goes forward propagation, some error, back propagation, over some epochs, weights are adjusted, weights are stabilized, once the model is ready, meaning weights, values are ready, any new shape, same weights, that's it, simple, problem has been reduced dramatically to a very achievable state, in neural network we need lots of data, this work was actually only personal in some sense, so how many profiles can I make, 10, 20, 25, not like million, right, so if I have five profiles, just like this, how can I populate at least hundreds of thousands, any ideas, I want very simple, I'm a mechanical engineer, GAN is, so we'll have, I'll probably talk to you later how to generate, I would need such solution but just simply given few profiles, input profiles and output, how can you have variations, not actually it's a 2D profile and a 1D profile, curve, it's a profile and a curve, as he rightly said, can we transform both of them similarly and you can have many variations as much as possible, so both are there, 5 degrees, 10 degrees, 20 degrees, so can I populate at least 36 types for each, 10 degree rotation, 5 degree rotation, more, 2 degree rotation, translation, both translated, so geometrically points would be different but shape-wise both are same, so this way I populate it, so there's, so original data is like this, picture comes like this, this has been taken as an image now, so the original problem, image, original output, image, what can you do for data augmentation, making many variations, original data was in vector format, if you are aware of image technologies, vector is actually a curve equation, then you do rasterization, you put pixels from the image and then you can do translation, reflection, these are actually different operations, mirroring, rotation and scaling, all these operations can be leveraged to populate different variations of same L, thin L and the profile L, you do variations and then you can generate lots of data, which is supervised, actually correct, a simple python program with given transformation matrix, those who are, those who have done 12th probably know transformation matrices, you can generate lots of samples, now these can be fed to neural network, is that clear, so far, mechanical doing solid to surface, forget the solid part, profile to 1D, geometry is not suitable, gone to images, in images to create variations, we have transformations, now this is a routine work, like scikit-lan you have three lines of actual modeling code, rest is all data preprocessing, this is all what we have done so far, data is ready, many options are available, simple encoder, decoder network, if you have less data simpler would be better, if you have more data then you can add layers, so what is simple encoder, decoder, whatever input size, if it is 100 by 100, what would be the input size, 100 into 100, 10,000, output is again 10,000, but with different pixels colored, is that it, so simple one layer, encoder, decoder network, that's what I'm showing today, some results, but you still can have dense encoder, decoder, multiple layers in the input, multiple layers in the output, you can have CNN nodes, layers and then there is something called as Pix2Pix, probably which is more popular in this moment, Pix2Pix have some theoretical issues with that, but these are basically input to output image translation networks, this is what, these are what we can leverage, so I've used this, this has been, of course this has been stolen picture, this is called as image segmentation, original photo and a segment, the portions photo, if you feed such pairs it learns the segmentation, why it is used if you want to block, say trees only or the background, this can be taken, taken into different color and then you can superimpose, so it's used for blanking, generally, so very popular in image domain, so this is what I've used, did not want to show code, but still if you don't show code, go into conference, so simple Keras network, one layer and these are the results, I'll wait here for a moment, there is input on the top and these are inference results, not training results, inference, given a shape it gives me shape like this, given a shape it gives shape, obviously they are not crisp and clear, there is noise, there are few dots here and there which are misclassified pixels, but the good part is that simple one layer encoder decoder, you can at least positionally correct, if this was the line, if the line did not come here, at least the spatial position, it understood well, the learning has happened, simple encoder decoder, you can see the learning has happened, this source code is available free on GitHub, if you type this mid-curve NN and Yogesh Kulkarni, you will get the GitHub thing, I did try a couple of others, the other solutions could be, you have this output and have a denoiser on top of it, which removes the noise and that could be better and picks to picks, so this is what I got for the last few slides, not very perfect but encouraging, to my knowledge this is the only work that I have seen, as I have done doctoral work, I have done enough literature survey that anybody else is doing it or not, because that is the point of the work, to my knowledge nobody has done it, so this is at least start with a simple encoder decoder, people can come up with, those who are familiar with this or more expert, can come up with different architectures, data is available free, you can populate more data, start doing that work and contribute to that GitHub, if you have fantastic results we can co-author the paper, co-author meaning your name will also come, so that's it, so ok results but this is just a start, so that's what I wanted to talk in this conference, a new domain altogether, machine design to so-called machine learning is what we have done, of course deep learning and what can be done, more profiles, more data and different architectures and I'll take the questions now, many I mean you can tweak the error functions and all also, that's it, just some summary, not just for machine design, I thought I should apply few more examples, it is used in animation also, if you have 2D profile of a man, a skeleton is actually given moments and the further the muscle part is actually rendered on the, so you don't have to transform everything in animation, only this skeleton has to be transformed, so this is skeletonization also, given the skeleton you can add a mass, you can do that, this is a reverse medial axis transform, you can do shape matching, if you want to retrieve shapes instead of going for matching every surface you match the skeleton, right, so it's easier to get, are you getting that, so many applications but the one that I showed was strength analysis, many approaches going on for 50-60 years and this is the beginning, doing the same transformation by deep learning, am I okay on the time, I think perfect, 435, I do, just one note, I do have a paper on it, a brief paper on it, on Vixera, that's opposite of archive, it's public paper, if you want to study little more, you can just take a look at some theory part is mentioned there, anyway yeah please, okay, accuracy is actually very simple here, if you have a L-shaped pixels, you have to have L here at that location only, it cannot even deviate, so the error function, if the pixel was there or not, simple, very strict error function, in case of GANs or whatever, picks to picks, it actually, the second network is actually a cost function adjuster, so it allows what is good output, I don't want good output, I want perfect output, so a simple error function is actually better than going far little sophisticated, but good sort of output, so and something else is completely inaccurate, so it has to be there at that position and those pixels only, because by definition, if you deviate a bit, that's not L anymore, and it's not in the middle, so it's a little strict in that sense, yeah. So this is definitely increasing because you have come up with something in the mix of mechanical and world networks, but do you think that this sort of approach will lead to better results than what you conversely do, do a measurement and then come to the mix of phase? One cannot predict, it's a new technology, at this moment no, the rule-based ones, though they figure very miserably as you saw some examples, they still work better, even my doctoral work was far better than this, I could solve many shapes, even the non-later shape also, horse and other animals also, so that's a 60-year-old work, a three-month, four-month work won't compete with that still, but some progress will come, the potential is you don't have to worry about rules anymore, just good examples or good too many examples and a decent matching network, that's the way it will take it further, don't know whether it will be good or bad, that's the start, yes, it doesn't matter now, can you see, it's an image, you can have three, four O's whatever, doesn't matter, it's an image to image, all that sequence problem is completely gone, only problem is if you get output like this, I want, I do not want image, I want surface, so somehow I have to vectorize it back and extrude it, removing the noise and as I said removing all the errors, it has to be a perfect connected dots, then I'll be able to extrude it well, so the post-processing is heavy as of now because the output is not good, stitch, that domain is completely because that algorithm itself is different, so they have not catered to some cases, no, they are rule-based, this is what I also did in my work, so given 10 surfaces what do you do, take the top two surfaces, make a pair, another top two surface make a pair, put an offset patch and extend and trim, this is the normal approach, this is called as mid-surface by pairs, I've worked as an internal software developer for that algorithm as well, 20 years back also and and recently also for a different company, that's the approach, completely rule-based, no training data as such, I did not, it's a short brief of my work but once an extended proper journal paper will come, I will have to put benchmarking, I will have to pitch the current state of the art, no this was highly rule-based heuristic rules, wonderful, for all these you need data, you need to have a gap surface and a filled surface, if you have millions of such data, give me a call because somebody has to prepare the pairs right, as you rightly said for a shape of a gap shape and a fill set, if you have many such examples then we can get that training data somehow and start learning similar way, so that's possible, the problem there is it's a 3D shape, we have to sample all the points, so 3D is a very difficult problem that's why I showed you a 2D thing first, if 2D works fine then we'll go to 3D, thank you all, thank you for coming, any other question catch me offline, thank you.