 All right, well, welcome back everyone again. It's my distinct pleasure to introduce the moderator of our panel, following the fascinating lecture by Professor Freeman. The panel will be moderated by Professor Stanley Chan, who is an Elmer Accessory Professor in the Elmer Family School of Electrical and Computer Engineering, and also the Department of Statistics at Purdue University. So Dr. Chan does research in computational imaging. He is particularly interested in the photorevigor imaging, imaging through atmospheric turbulence and robust machine learning. Dr. Chan is the recipient of the Best Paper Award in the IEEE International Conference of Images Processing in 2016, and he has also received the number of distinguished teaching awards at Purdue University. He is currently serving as an Associate Editor of the IEEE Transactions and Computational Imaging, and was a former Associate Editor of the OSA Optics Express. So with that, thank you again, Stanley. Okay, so I know Bill has a very interesting, interactive activities, which I'm really, really looking forward to. So maybe I'll just pass the time. Bill, let's do some exercise. I'm gonna raise my hand if you ask a question, okay? Fedders, Ways, and Future of Computer Vision Research. And then after that, we have a couple of faculty here. We can recheck about these decisions. And just to, how much time should I, I can really tell you whatever length of time I should spend on the slides. What would you like? Sure, how about 15 minutes? Great, perfect. Okay, thanks. Hi again. So the prompt was, you know, what's ahead for the future and I have, I wanna share with you this, so there's slides that kind of talks about the future of computer vision and my viewpoint on it. So the first thing to say is a disclaimer. That this is a cartoon you're gonna see. This is just a cartoon story. It's, you know, there's a lot of subtlety that I'm just not gonna go into. So please don't get upset if I sound like I'm being kind of black or white. It's just, that's gonna be the style of this presentation. A cartoon is something that maybe looks like the original, but it's just very simplified. And that's what this is. Okay, so, you know, as we all know, there's been a revolution in, well, in science really, but in particular in computer vision and image processing and a sort of cartoon description of how we generated papers between 2013 and 2021, is you could say that you take this standard textbook on computer vision by four sides and ponds, open it to any random page and look at the topic on that page and put the word deep in front of it or add the word GAN behind it or something. And then, you know, collect a training set, build a model that implements that and publish. Again, a cartoon, you know, forgive me, but that's a lot of what happened. And you can actually almost verify this. You can go look for, you know, say the topic is texture and go Google deep texture of CEPR and you find a paper on a deep epipolar geometry, CEPR, you find a paper on it. Just anything in the textbook on computer vision, you put deep in front of it, CEPR behind it, you find a paper. And what's, let's see, and what is it that worked so well? Well, you know, there was this really very simple model of processing developed by Hugo and Wiesel and of course others of, you know, convolutions with a non-linearity following it and then repeat that in many layers. Again, the cartoon version, Jan LeCun implemented this in a tastefully done network of convolutions, non-linearities, multiple layers and, you know, to emulate the biology. And what was the result of this kind of very first step in emulating the biology, unprecedented impact and academic success? Just unprecedented, you know, just amazing and, you know, touring awards to the leaders in the field but so much revolution from this, just taking the first step toward doing things the way people do. So if taking one step gives you these incredible results making, maybe taking two steps in that direction is gonna be even better. So that's the kind of point of this talk. And so let's use something else from human vision and visual neurophysiology and incorporate that into our vision systems and maybe that'll take us to an even better place. So there's a lot of things you could use, you know, from psychology, from perception literature. So then here now is therefore a model for how to make papers and computer vision for the next five years. Again, it's just a cartoon, you know, forgive the flippantness of all this but you could take this, there's a really wonderful, it's getting old now, but a wonderful vision science, human vision textbook, vision science and open it to some random page and look at that concept and develop an architecture which implements that concept into your vision system and then publish it. So that's the model for the next five years. And if you don't wanna use that older textbook you can use the most recent VSS, Visual Sciences Society Conference abstracts. Okay, but there's one tweak and that is there's a lot of kind of human and neurophysiology which may be idiosyncratic to biology. And my image for that is kind of related to flight. Like of course, you know, all birds fly and birds have wings but they also have feathers and it seems to me that, you know, feathers are kind of idiosyncratic to the biology. I'm sure they're aerodynamically very important but we've got a whole aerospace industry built on winged featherless flying machines. And so somehow we've got the sense that the wings are important and the feathers are kind of side effects of the biology. And so the game is when you look at that human neurophysiology textbook to decide what of these concepts are feathers and which ones are wings. You know, which ones are idiosyncratic aspects of the biology and which ones are really fundamental things. And just as a mnemonic to help us remember that, we note that if you take a human and add wings to them, you get a flying human. And if you take a human and add feathers to them, well, you know, not so much. So anyway, this part is interactive. I don't know if it's gonna work so well on Zoom. Let's try it. So here's a list of human psychology or neurophysiology features. And I'd like you to tell me whether you think they correspond to feathers or to wings. You know, are they something we should put into our computer vision algorithms or can we kind of ignore them? Okay, here's the first one. And I don't know how to do this, but it's fine with me if you're on mute, if you're allowed to. I don't wanna see the chat in my stream so I'm not sure how I'll do that. But okay, first one is dorsal and ventral path, visual pathways. So the visual processing kind of breaks into two streams, a wear system and with handles motion and things and what system which handles kind of static imagery. Any ideas whether this is feathers or wings? And how do I get feedback from the audience? Hmm, so I can't hear anyone. Maybe you're not able to unmute yourselves. And I don't really see the chat in my, when I'm presenting. So. So I can click the reaction button. Okay. Can you see the reaction here? I can, yes, I can see that. So how about we use green check to say this is wing and then a red cross to say this is feather. Okay, okay, good. So I see green wing. So there's a feathers and I see two votes. One, okay, a couple of greens and then green wings. Okay. And then in the interest of time, I there just kind of go through some of this. So I think it's feathers. I'm not sure really that it's essential for vision. Of course, some people disagree with this, but I just think you can do it other ways. Okay, next one. Sorry. Explicit representation of border ownership. So here's a picture from a segmented image database. And if you look at all the horse pixels, it looks like this. This one, sorry, I'll just, I'll run through this one because I really enjoyed the time. I think if you don't really explicitly represent which ones are the edge borders and who owns the border, you'll just get, it'll be so hard to learn what the shape of a horse is and so forth because this shape on the right of horse pixels includes the occluder in front of it and because you haven't represented it properly by properly saying that the border between her leg and the horse is owned by her, it's not owned by the horse, whereas the border from the horse to the background is owned by the horse. Anyway, so I think therefore that this is wings. I think it's essential. And this is part of my little program to try to put the vision back in vision learning. I think we need to recruit some of these concepts from human psychophysics and put them back into our machine learning algorithms. Actually, really interesting time. I think I won't do the interactive version of this one because it's kind of hard for me to pull the people but I'll go through these and you should think please on your own, think about the answer to this one. Feathers are wings, counter completion. I think it's essential. So there's a little video on the top left is a thing of a chair and we could only get the correct contour as if we really analyze the grouping properly. If you just look at motion alone it just totally fails on this problem. So I think that counter completion is an essential piece which we're not really putting into our algorithms currently. That's why I can see. Foviation, we see the world in this really strange way where there are foveas, high resolution, everything else is low resolution. Is that critical? Well, I think for consumer applications it's critical. You know, it's got to be efficient. And for, but for academic research I just think it's not essential. I think it's feathers for that. And Jeffrey, I'm trying to infer what you're thinking by your expression but I'll hear from you in the panel. Perceptual grouping, I just, I'm so in favor of that it's just got to be wings. Otherwise, how do you put things together? Trichromacy, do we need RGB to perceive the world? So sometimes I think wings, sometimes I think feathers, I think people are persuading me it's probably feathers. You know, you can get by with black and white camera. The brain is amazingly power efficient, low heat dissipation, portable. Is that a critical part of vision? Well, again, I think for consumer applications yes but I think for academic research perhaps no. And then finally, having both feed forward and feedback connections. That's something that's very prevalent in the human visual system and it's present in some artificial systems but not many, but I think that's also a critical piece in terms of kind of validating your internal representations of what's out in the world. I think you have to have feedback connections. So that's my little spiel and I don't want to take up more of the panel time and I really look forward to what other people have to say and so I'll stop sharing now and thank you very much. Sorry that the interactive part is harder over Zoom. Great, thank you, thank you Bill. So let me just introduce our panel today. So we have Professor Kathi Kramani from Mechanical Engineering and we have Professor Jeff Siskine from Electrical Engineering. We also have Maggie Zhu from Electrical Engineering. So I really like this exercise. I think this is really, really interesting in a sense that I guess we're looking at a problem, right? We're looking at a problem of what would be the interesting things to think about in the next five or 10 years or 20 years ahead of time. So I want to hear the perspective of the panels, right? We hear some perspective from Dr. Freeman. Maybe I'll just open up the floor and ask what do the panels think about in the future of computer vision? What are the interesting problems that in your opinion, we should spend some effort on? Anyone? Jeff, do you want to start first? Okay, so I share Bill's general view that we're lacking a lot of incorporation of what we know about human vision in our computer vision research, both at the psychophysical level and at the neurophysiology level. I think there's a fundamental difference between engineering problems and scientific problems. And a lot of the approaches that people take might be good from an engineering perspective, but they don't actually exhibit human level behavior and they don't actually work the way human vision or biological vision would generally work. And one critical difference is that machine learning systems are trained on absolutely huge data sets and humans are not. Humans, when they're growing up, don't see a million or 14 million images from ImageNet flickering in front of their eyes at five hertz for the first few years of their lives and they don't actually see images, the same image more than once and the images don't come labeled. So how human vision works, I don't believe that object recognition is a process that is learned. I believe that that is almost completely innate and we only learn that to it in the linguistic labels that we assign to categories that the innate human visual system can differentiate. And I think that whole approach to vision is something that largely the community is not addressing right now and it might not be relevant from an engineering perspective, but it's definitely relevant from a scientific perspective. Maybe I should ask Kati from an engineering or science perspective. Yeah, it's a really good question. I tend to sort of inform the basics from perspectives that you gather in the real world in some sense and to Bill's point of view, I was just looking at some of his recent papers, what can you learn when staring at a blank wall? It sort of was analogous to some of the examples he was giving, but I also was looking at some of his patents. One of your most cited patents built from 1997 was this hand gesture machine control system with Wasteman. I still work in that area, but of course, and being publishing in computer vision and ECCV, CVPR and ICCV, all of them. But of course at times change, the sensing modalities change, the context changes and we want to solve problems with fewer assumptions about the environment and context. So I think to that point of view, your analogies of feed forward, feedback and perceptual grouping, I completely agree with this control completion, it's not used, but I don't think that you have to be a wing or a feather, you can be both. So I sort of buy into this perspective of A plus B equals C that combining applied and basic together can actually feed back into the basics as well as feed forward into the applied, partly because I'm biased, I kind of do that all the time. So my perspective there is, you can be both a wing as well as a feather, depending on where you want to go with the publications. But yeah, I kind of buy into the notions that both Bill as well as Jeff point out to that a lot of computer vision problems, of course, learning consumes so much of energy, we do so many things in our brain with extremely less energy and we are able to make out a lot of things just by looking at shadows and so on and so forth, right? So certainly inspiring computer vision problems with the kinds of ideas that Bill actually very nicely presented. I think it's really good for science perspectives. Wonderful. How about Maggie, do you have any comment on that? I think it's very interesting exercise and thanks Dr. Freeman for doing that. Zoom is difficult, but I think we still see some consensus, I guess, based on these set of exercises. So I think I sort of agree with the concept of depending on what perspective we're looking in terms of whether it's a wing or a feather problem. But sometimes that can be ambiguous, right? So for example, I think one of the question we little quiz we had was looking at efficiencies, right? Of these models, but I think in terms of efficiency, there are also different ways of interpreting this, right? It could be, do we want to build a very efficient model architecture from the point of power consumption and so on as Dr. Romani pointed out. But also what about understanding the design itself, right? So are we trying to build just a black box design type of efficient network or so on? Or are we really thinking about the theoretical side, the physics foundations of doing this design and then making this a white box and maybe that is more efficient. So I think there's a lot of interesting perspective and thoughts behind how we look at whether or not the problem itself is a feather wing, right? So another example I think is important is, we're trying to at least attempt to mimic or have a better understanding of modeling of the human vision system. But a lot of times the problems that we encounter do need to include human as a input, right? And then how do we do that by engaging the human but also provide that level of autonomy to a lot of these practical problems? So these are all things, you know, I'm very interested and I think these are kind of, I hope the future direction of where we're going. That's a good point. So Maggie, you're actually a very good point. So human eye, right? So I want to ask another question. It would be human eye, right? So when we think of a computer vision research, one of the really, really ultimate goal is that I'm gonna make an artificial eye that can do as much as a human eye or maybe more. Should this be the goal of computer vision or actually that should be something else, right? Should we ask the computer vision system to do something complementary to human or do we want it to just replace human eyes? Well, I know there's a sort of domain specific but I want to just hear comments from maybe Maggie, the folks here. Well, I mean, maybe we have a lot of answers coming. The comments. Hey, well, I'm not allowed to turn on my video so I guess I'll just turn on the video. Can we make Ed turn on the video, Maria? Let me see. We're working on it. Yes. There we go. Oh, here we go. That was about the question I was gonna ask and I was gonna make it much more provocative. So I'll address this to our guests. So Bill, why does it have to work, a computer vision system work like a human vision system work? In other words, you can say you get insights maybe from human vision but is that the only way to go? I mean, is there another approach that you can take? And this sort of dovetails off a Stanley's question too. Yeah, and it's a great question. I guess I'm, let's see, I mean, so one answer you might say, which is not my answer, but it's the first really reasonable one. An example of that is like the AlphaGo or the chess programs that totally beat humans and do it in a very different way than humans do it. So why should perception be any different? And certainly that's a great point. I guess I feel like right now human vision is just so far ahead of computational vision that you just can't go wrong by trying to make it more like a human vision system. But I do certainly acknowledge that eventually when we get our act together and understand things better, surely it's just gotta be the case that machines have all these different set of constraints than humans have. And machines don't have to grow up from an instance into an organism. And that's surely gonna be a big constraint that changes things. So in principle, no, it doesn't have to be like people, but we're just so far away from what people can do that I think a very reasonable kind of step right now is to just try to copy what people do somehow and work on that. But I understand that that's not in the long run, maybe what we need to do. Well, I mean, and I agree with Jeff's point. I think object recognition is probably inherent and we just put labels on it. And I think there's some evidence of that, maybe even to a certain extent in animals and certainly also, you know, 3D vision. So, but yeah, I always worry that maybe we want the neurologists to solve our problem for us. Yeah, now I see what you're saying. And I also wanna jump in that, you know, if you look at other biology inspired, you know, camera sensors and so on, they don't necessarily look like human eyes, right? But they have very, you know, specific unique characteristics and then, you know, we have a lot of, I think, faculties, researchers in the field that are exploring those possibilities and they may be designed for specific type of applications, right? But I think, you know, definitely taking inspiration from biology is, you know, one way that, you know, to go as the future direction. Yeah. Oh, you're muted. We can cut it, you need to unmute yourself. I have a comment, you know, tied onto Maggie's comment and also a question to Bill. You know, it's how we think about AI, right? Is it going to, you know, how is it going to be used eventually, right? One is doing the science, but also how does the science get used? You know, is it going to extend humans or is it going to relieve us of doing some work or is it like a backup or is it being designed to replace some of our things, right? So I think there's complementarity of what we want to do that has a lot to do in terms of how you might think about it. I don't know what your perspectives there are, everybody in the panel, but also Bill. Well, just to mention a point that's in the opposite of what I was kind of saying before, you could imagine if you were really successful at building a vision system like a human that then your airport security vision system would also get bored and get distracted and not be able to just stare at something for 10 hours straight as you would like our machines to be able to do and which is so hard for humans to do. So indeed you want to diverge from, as you're saying, you want to have a complementary set of strengths. So I do computational imaging, design sensors, algorithms and so on. I see Charlie Bowman is also in this room. Maybe this question is more towards specific things. As we design algorithms, what we are doing today is you give me a sensor, be it spare camera or event camera. So you provide me that kind of device. And then I think really, really hard for a couple of months, okay, how am I going to use this sensor? So I want to fit this question around and I want to ask maybe Bill or Charlie or other folks in this room who have some experience on this topic. If you can talk to a sensor guy, talk to a sensor guy, I want a new functionality. Can you make it for me? And you can only ask for one functionality. What functionality would you want? And assuming that person can make it for you. So what would you want? Well, I've been looking for time, so let's let him to go first. Hi, Charlie. Oh, okay. Okay. Am I supposed to be answering this? Yeah, you can, Charlie. Okay, yeah. You know, one thing that comes to mind, I don't know if it's an answer to your question, but I think a lot of people know Sergio Goma from Qualcomm. Oh, hi, Bill. I think maybe you know him. And he's, I think they were, he's talked about, and I'm not an expert in this area, but integrating sensors with computation, local computation. They have this technology where they basically just fabricate two devices that are so flat that when you put them together, they effectively bond and they can communicate. You know, maybe so, I mean, maybe the broader question you were asking Stanley is that if we had control of the design of the sensors rather than just the algorithms and we wanted code integrated code design, where should we be going with that? And you know, I think there's a lot of potential along those lines. And maybe that's feathers rather than wings, but maybe it's important feathers. It's sort of like foveation. But what kind of functionality would you want, Charlie? Right? You have a cell phone. You have cameras. You have spray. What do you want, Charlie? I'm not sure. But maybe what I want is probably in real applications, you're always or in future real applications, you can be limited by data rates. So maybe you want a lot of sort of like these normorphic cameras where you have a lot of summary information that you can provide. And then you use that to extract the kinds of information that Bill was talking about doing reconstruction in ways that you might not expect to be able to. You know, one other thing I want to throw out here, not to change the subject, but is I'm not a, you know, I'm okay. I don't consider myself really a vision researcher per se. So, you know, maybe I don't have the depth of experience, but one thing that I, when I think about vision, it seems like there's a lot of problems where we've made a lot more progress than I expected to make in the last decade or in the last 15, 20 years. That you can really solve some really amazing vision problems in terms of having a static image or you have a video and you can tell me a lot about what's going on there with high reliability using automatic methods, right? But somehow that doesn't seem really intelligent in some sense. Like it's doing something. It's giving you the answer to your problem. It's not insightful. I don't really feel that the computer has a lot of understanding. Okay. And so I don't really have the expertise to be able, I can ask the question. I don't have the expertise to answer it. What does it take for a vision system to have really understanding of what's going on? And I kind of feel like it involves a certain amount of dynamic process where, because understanding relates to action and intent. Okay. So I guess my question to Bill is this. Is dynamic vision like where there's interaction, not just a video in which you process it, but interaction between the vision algorithms and system and the ascending device? Is that wings or feathers? That's a great question. Yeah. So, well, it's something I think a lot about. So I love the simplicity of a passive system. You just set up your algorithm in front of a TV set and let it watch for a year. And then you come up with a good vision system. That would be really cool. But I talk about those with my friend Rich Sutton, who's big on reinforcement learning. And he thinks, no way, no way. You can't do that. You've got to interact. You've got to poke and prod. And that's the only way you can learn how to see or understand the world. And that might be true. I sort of, I'm sure it helps. I'm still somehow feeling like you could just do it by looking at, you know, looking at all the YouTube, spending a year watching YouTube, and you can really understand a lot about the world. So anyway, I'm sure it helps to be interactive. So is it feathers or wings? I don't know. You know, maybe it's wings. You know, it could be that you just got to. Prop hope the world in order to understand the causal structure of the world. And so therefore maybe it's wings. I guess it's as an as a researcher, it's so much easier to compartmentalize everything. If it's, if you can, if you don't have to be able to robot in order to understand how to see. By the way, I just wanted to go back and answer Stanley's question. I would love to have a differentiator as in my sensor that comes comes back with a high dynamic range derivative of what's falling on the sensor. I mean, obviously you can do it digitally, but you, you're, you're at the mercy of your quantizer and signal noise issues. And I should say I have to credit that. Jack Tumblin wrote a paper on why he wants to differentiate in camera. And I just, I thought that was such a great idea. I want to credit Jack for that. And you can build a really nice analog differentiator. Okay. Yeah. Doesn't have to be digital. Yeah. Okay. Yes. So talking about this dynamics, I wonder if Ed or Jeff, you have any comment on that. I think Charlie asked a very interesting question. Right. So, so you want to understand not just from the static scene. You also have this dynamics that's involved in the scenario. And I know you guys, Ed and Jeff, you have been working a lot on this space. And any thought on that? Is it feather or wings? From your perspective? Maybe Jeff. So, I don't know so much about feathers. But what people lose track of is that biological flights can do things that human aircraft or human-designed aircraft can't do and vice versa. So human-designed aircraft can fly supersonically and birds cannot. But a bird, which has a flexible wing, a non-articulate or complex articulated wing, can be at a high altitude. It could pull its wings in. And it could enter ballistic motion phase, fall ballistically. And just at the right time, spread out its wings and attenuate its motion and catch a prey. And that's something that human-engineered aircraft are not able to do. And each have optimized for a different kind of functionality. The same thing holds true getting back to your initial sensors. There are sensing modalities that we can engineer that can do amazing things that the human eye can't even begin to do. Like MRI and CAT scan and the like. No matter how much you try, a human staring at another human body can't see inside the body. Whereas ordinary everyday medical imaging does that to very, very good purpose. So there are sensors that if I could have in my wildest dreams, if you could provide me, it would be great. Generally, if you can increase the spatial and temporal resolution by several orders of magnitude of brain imaging, that would be amazing. If you could image directly the neurologic activity, not indirectly to both, that would be amazing engineering capability. And it would have nothing to do with computer vision, but it would be tremendous for neuroscience. I don't really have the answer to what kind of sensor I would want if I were trying to do computer vision. Just a completely different question with a completely different range of possible answers that I haven't thought about very much. Thank you, Jeff. Thank you very much. I see a hand raised. Chi, do you want to comment something here? Yes, yes. Hi Bill, hi everyone. So I just joined here this month and all the way from Boston. So very nice to see you from Boston. So regarding, I first want to echo with what Bill has said about the differentiator camera. So I was thinking maybe like building such kind of sensor that outputs the differentiation of images can be a combination of both the optics and the electronics. So if we are able to make the optical design such that the pointer function obtained for the images follows certain kind of properties, maybe like it will be very easy for a like, maybe it's very easy for electronic operation to obtain a robust image that is a differentiation of the original ones. And so that leads to my imagination of a future camera. So I really hope that the optics side of the camera can perform much more type of computation than what we currently can do. So currently I see a lot of people using diffractive optical elements or some people are doing nano photonics. So they are able to manipulate, for example, the phase and intensity of the light wave that pass through it. I'm wondering if we are able to do non-linearity operations on the light that goes through certain kind of optics. If so, the kind of computation we are able to enable for a piece of camera can be much, much more expanded. So those are my comments. That's very exciting. That's very exciting. Yeah. I will just add a couple of comments from my very selfish perspective. I've been working on sensor for a couple of years and I know analog processing. I know digital processing, 14-bit, 12-bit, 16-bit processing. I want to have a sensor that is 1-bit, 1-bit binary. 1-bit binary, as soon as the signal comes to your sensor, I want it to just dump your binary response, 1 or 0. And then what do you get? Well, you get speed. You get resolution because the pitch can be a lot smaller. You also will be able to sense the signal in a completely different way. You are still digital, but then you are binary digital. And so you can do motion if you can track them. You can do dynamic range because I can fuel stamp. So in my imagination, some kind of sensors along the slide. Well, we've been doing something along the slide. We just quantized the sensor with Eric Foster. But I think that's still a really, really long way to go, right? This binary sensor that's replacing the whole sensor today, not analog, not digital, but it's a binary sensor. That would be really, really, really cool if we can fuel it. So that's why I've been thinking. Sounds great. So, all right. So I want to switch gear a little bit to another question, which is, I know there are lots of students in this room and they are really exciting about computer vision and they want to publish in CPI and so on. Maybe I can ask Bill and also the other panels for their opinion. What is publication? This is really big. What is publication? In your mind, what cost to become a good publication? There are different metrics. I know schools, people, employers, there are different metrics. But in your mind, what is a good piece of paper? What is good? That's a great question. So I'll just jump in and I really want to hear what the other people say. So I'll be brief. So this is something I wrestle with all the time. Also in evaluating faculty candidates and so forth. And I have to say, I've really been thrown for a loop by the neural network revolution. It's much harder for me to evaluate things now than it used to be. You know, it's possible to get in and make contributions with a much earlier in one's career than it used to be possible because the tools are so accessible. Which is great, but it's just finding confusing. So I'd love to have some crazy new ideas, some new way of looking at things. Something that you'll do differently now that you've read the paper than you would have done if you hadn't read the paper. Or something you're excited about. Those are all things I love. And that's of course in conflict with what's a great way to get a CVPR paper in. Which is to have a table with all these rows and your rows have all the bold numbers in them. You know, that's a great way to get a paper in. But it's not necessarily the one with the great new ideas. And so I'm really torn about that. And I really love the great ideas papers, but they're hard to evaluate. And somehow even harder to evaluate now. Anyway, I want to hear other people say too. Yeah, yeah, yeah. Maybe I'll start with Ed. Ed should have a lot of experience with advising students. Well, I'll just label myself part of the walking wounded here. But, you know, my concern is many times people that are jumping in and using all these neural network tools, for example, they lack understanding on how they really work. I don't know how many times that I've been at CVPR in the last couple of years where I walk up to somebody and listen to their poster. And then I asked them, explain to me how it really works. And you get a lot of people that are sort of choking when I do that, when I ask those questions. So I think it's important to have insight. You know, one of the things I teach a, Bill, we have this course here. It's called vertically integrated projects where we form teams of students from freshmen all the way up to seniors. And they work on research problems. Okay. And, you know, 99 out of 100 students, they want to do something with machine learning, whatever that means. Okay. And, you know, I ask them, I say, well, you know, you want to fly on an airplane that was designed by engineers that use the neural network system. And they didn't really understand how it works. We want your grandmother on that plane. And you can figure out what the answer is. So I think that's what I would see. I'd also like to see students doing computer vision image processing. I learned a little bit more about, you know, statistics and probability theory. Like your introductory thing where you say, okay, let's just take a Bayesian model, make some Gaussian assumptions and let's see where it goes. I bet there's a lot of people that say they do computer vision or image processing. I couldn't do that or understand it because everything has to be solved by a neural network. You know, why did you take 100,000 samples and put it into ResNet or something like that, Professor Friedman? What's wrong with you? You're swimming in the opposite of a side of the river. You know what I mean? So that's what I would like to see is more understanding about how things work. Yeah. I'm there with you. So let me ask the other folks on the panel. Maybe Katik, do you have any comments? Yeah, I mean, you know, I'm a basic geometry guy, you know, just points, lines and triangles. I like to build up foundations for spatial geometry in computer vision. So I like to see a lot of more geometric content based on the foundations, you know, can you solve certain higher order problems, especially guiding it with these structure that you can garner from outside the vision area. So I kind of believe in more special purpose AI algorithms in computer vision and also one more human input into the AI system. So there's a lot of gap between what the AI systems can do today and what humans are capable of doing. So even if we think about AI as helping or aiding humans, you know, the gap is substantial and leads a lot of issues such as trust, as you point out and so on and so forth. Right. So in many of my colleagues in EC, like Saurabh and others are looking deeper into those types of issues. I think there are a lot of important aspects. But I like to think of AI and vision also being used for, you know, non-damaging AI versus more things that can cost more damage. There are a lot of areas where vision can be used but not be harmful if it fails. So and, you know, this is a mix of these problems. And of course we are entering this autonomy age and major questions of reliability and trust all come in. So there are a lot of interesting cans of worms. This AI has opened as Bill pointed out. We are in that world, unfortunately, unfortunately. So, yeah, I still like to inspire my problems with basic geometry. Yeah. Maggie. Yes. So I think, you know, there's another point sort of to bring to this discussion related to your original question of how do we advise students, right? Because, you know, we're sort of in this era and then there is, I feel like a larger environment in which whether you are applying a job, you're applying a school, you're evaluating a candidate, it's about where you publish, how much you publish and how reputable those publication venues are, right? And sometimes it becomes difficult to, you know, see, you know, the potential of the work. And then I don't know about, you know, those of you in the panel and the audience, but, you know, I've had experience where, you know, we work on something we feel is interesting and then, you know, it's a little bit immature, but then, yes, the reviewers agree it is an interesting problem and then, you know, it's something worthy to explore. However, we're not seeing enough of these big tables with bold numbers. So, I mean, I feel like it's a struggle, right? We want our students to be successful and then, you know, a lot of them come in and say, hey, I want to be successful, I want to publish, you know, CBPRs and all of that, and then how do I get there? So, how do we advise them? I mean, how do we, you know, I don't know influence change, you know, to cultivate a better environment for kind of educating the next generation of students, right? So, you know, we want them to understand how things work, right? We want, I mean, they're going to be engineers, you know, research scientists, so they're going to influence what comes next in the future. And that's more like a question for me, because I struggle with that too. That's actually a good question. It's actually a good question for us as faculty to think about, because we sort of, on a different side of the river compared to students, right? Because because students, they really in a rush to graduate and to get jobs, they need those numbers. But on us, we sort of, okay, now, if I have, I have a, like an e-gap, I can do sabbatical, I can just do work on something else. I think, Maggie, you're raising a really, really good point. And I can see the tension, but I don't have a solution. All I can do is just to encourage students to think big, to think big. But as it goes to the publication, we also want to face the reality to create big tables. I don't have a solution either. I'm not sure if you say it's a building culture already to this community because it's getting big, because it's getting competitive. I don't have a solution, but I can sort of just simplify with all our students. Maybe I can hear more from our other faculties or anything that we as faculty nowadays can do to help our students. On one hand, they want to get a job, I understand. On the other hand, don't lose their great talent of just spending so much time on the table. What can we as faculty do to help our students? I think that's such a good question. I'll just jump in. I don't have an answer. I wanted to hear what other people said too. But as you say, we have so much more patience because of our different career stage than the students do. And so for us, for a paper to get rejected or even to get rejected twice, it's not the end of the world. There's paper deadlines coming all the time. Just use the valuable feedback from the viewers and make it better. But that's a really hard, it's just very hard I think to be at an early stage in one's career and receive one or two rejections on a piece of work. And I don't know the solution to that. I try to tell people, the students, don't worry about it. Sometimes it goes from rejection to a best paper. It really can be, you just keep working at it. But it's a hard message to hear, I think. Do other people have any, so what do other people do? I wonder what other people think. Ed or maybe Charlie, any input on this? Yeah, I would just say my philosophy, I don't know if it's the best philosophy, okay? But I just try to ignore a lot of the hubbub. And I know it sounds kind of naive, but I try to do the right thing. Like, I try to just say, you know, what is really important here? And if you have a paper rejected, you know, take the feedback and try to incorporate it, try to look at it from the reviewer's point of view. And really try to keep focus on making a valuable contribution as opposed to hitting some specific goal. I mean, I feel like particularly as the success of these fields in computer vision, you know, image processing, computational imaging has led to the participation of a lot of people. So I think that the communities tend to naturally create barriers to entry. So you have to know the secret handshake and they pretend that the secret handshake is really important. But I believe many times the secret handshake is silly. It's just something they've created so that they could cull down the number of people they have to consider and reduce their competition. So I try to just really focus on doing valuable work that I think is important. And I try to have a small group of friends that are, that I, whose opinion I trust, and I ask them to give me honest feedback. And if they think of what I'm doing is jump, they go, you know, Charlie, we like you, but this is just isn't very interesting. So I move on. Okay. I have another question I want to ask, not to change the subject, but Bill, you have a really unique perspective because you've been in industry really deeply embedded in industry, and it's related to the previous question, but also deeply embedded academia. You know, as an academic, I tend to think that academics have more silly bean counting, but I'm sure there's a lot of silly bean counting in industry too. Probably, you know, because the grass is always a little greener on the other side. But I'm just curious, and but there's another dynamic that's occurring in the past, maybe for other fields that in areas like machine learning, let's say specifically, there's an argument that being made that industry is ahead of academia. Maybe others will disagree, but at least some people believe that's true because, you know, it's really core to their mission. There's a lot of resources. You know, okay, let me, the salaries are dramatically higher. Okay. And so they attract really enormous talent. So, but you know, academics, we like to think that we're free thinking and the dark side of tenure and so forth. So do you have any perspective on that on the relative balance between those and also how we can better get synergy between those, because maybe they have different things to bring to the table? Thanks. Sure. I'm happy to comment on that. But then I want to put it on the stack that I want to ask other people to, you know, how to protect students from the CVPR steamroller. I mean, we have, you know, we have a lot of experts on industry and academia. Yeah, well, industry is exploring this notion that everything bigger is better and still even more bigger is even more better. And they're exploring that and we'll see where it goes. But my friend, yeah, your wife, you know, reminds me and others that this whole thing, this whole juggernaut, frenzy of neural networks started with one academic and one graduate student working on something for two years in isolation, you know, that's where it came from. And surely the next big thing is also going to come from that kind of environment where, you know, academics do have more freedom than people in the industry. On the other hand, yes, it is my I always wonder whether the right model for what's going on is what happened with the semiconductor industry where, you know, industry has the best semiconductor manufacturing and academia dozens. And is that the way it's going to go with machine learning to I hope not. I still think there's a lot of room for creative, crazy ideas to change to change the industry coming from academia. But I fear that that I don't know. That's that's that's what keeps me hopeful, actually. Okay, good. Can I chime in for a second? Yes, yes. Because I'd like to follow along the lines of what Bill quoted yesterday, which is something that I try to remind my students, which a lot of young, entering PhD students these days, see the deep learning revolution and think the world started without extending 2012. And what they don't have the perspective that some of us older folks have is that the nature of the entire field of computer science, not just computer vision, not just AI, but all computer science, is that there's a progression of what you might call paradigm shifts, what you might call facts. And every paradigm shift, every paradigm and every fact lasts for a while. Some last longer than others, but they all come to an end. They necessarily have to come to an end. Because if you think about it, if you have a topic and an approach that a huge community, a community as large as the combined computer vision, machine learning, AI, national language, robotics, community is at this point with many tens of thousands of people, it can't go on for very long until one of two things happens. Either we solve all the problems and then there's nothing left to work on, or the methods run out of steam, they ask until and the field ceases to make progress. And the only way people make progress is by a paradigm shift that creates a discontinuity. So if you look at the way things have worked, not just with the current paradigm, but with all paradigm shifts over the past 50 years or so since the dawn of computing, in fact, there are these discontinuous increases in performance or ability functionality that happens when there's a paradigm shift. And then there's a slow march up from the curve until it asks until and then there's another discontinuity. So I have no doubt that whatever we're working on now is going to, asymptote is going to dry up at some point. It might be two years it might be five years, it might be 10 years, I don't know exactly how long, but it will. And at some point, there will be a new paradigm shift. I don't know what it is. I wish I did. I'd be a head of the curve at this point. But what I tell my students is when you graduate with a PhD, you have another 30 or 40 years in your career ahead of you. And over the course of that career, there are going to be five or eight paradigm shifts. And you have to prepare yourself to ride all of those, to persist through all of those. And so it's not so important to become an expert just at the current time. You have to get the good sense about how to run your entire research career to persist across all of the paradigm shifts that are going to happen over the next 30 years or 40 years. Great. That's an excellent point. Yeah. So maybe I should ask Maria, how much time do we have left? Is Maria here? I don't see her here. So I'm just looking at that gender. I think we are at a time, 4.45. So I wonder how the audience feel about us? Do you have more questions we want to chat about? Or we should also, or we should honor this time. And should we get a stop here? I'm open. I have a lot of time. Okay, this afternoon. So I just wonder what do you guys want to do? Dr. Chan, I just need to just I want to just let everybody know that we were actually out at time. So okay, okay. Sorry to do that to you. I'm so sorry. Sure. Sure. No problem. No problem. Thank you. No problem. Thank you for monitoring this. But I guess so let me just say a couple words to conclude this panel. I just first of all, I really want to thank Bill for for spending this afternoon with us. I think this is really, really, really stimulating. The the talk is just so stimulating. I just cannot believe at least I'm learning a lot, right? How to become persistent on the problem for a long time and then make some real progress. Although, though we see a lot of failures, never mind, just keep doing this. This is a really good lesson for me. And there are lots of great ideas that I'm learning. Lots of great discussions in this panel. Thank you, everyone for participating, sharing your comments. I guess one challenge I will just bring it to everyone is really how do we help our students. I think this is really a burden for me. Okay, how do I help our students? How do we encourage them to rethink back? While at the same time, we also only do acknowledge the need for those job markets. That's a challenge for us. Maybe in the near future, when we meet in person, maybe we will have some ideas. I hope Bill can visit us. Hopefully later this year, I really want to show you around campus. I'll have a student show you some interesting images through turbulence work that we can talk about. So I really look forward to that. Again, thank you everyone for coming. Thank you, students, faculty for joining us. Thank you, Bill, for virtually coming to this event. Thank you, everyone. Thank you. And thank you so much for inviting me. I will take you up on that invitation. I will come visit you. And I look forward to it. Thank you. I've learned a lot in this panel. And thanks a lot, everyone. Thank you. Thank you very much. Have a wonderful evening. Look forward to see everyone. Hopefully on campus very soon. Thanks, Bill. We really appreciate it.