 Right. Hello, everyone. I'm Dimitri Perollis, the head of our Elmore family School of Electrical and Computer Engineering And it's my pleasure to actually introduce the next speaker, Professor Maggie Zhu Maggie actually has been in our school for some time now. She's now an associate professor It's a distinct pleasure that she actually is also an alum from our school. So we got in here undergraduate and PhD degrees from us Maggie is working in a lot of different areas. I think she will have a really exciting talk today I'd like to highlight things like image processing, computer vision, video compression and digital health. So lots of things going on In addition to her experience at Purdue, Maggie actually also has worked in industry She actually was with Future Weigh Technologies for three years and received the certification of recognition for core technology contributions So Her work has received a lot of attention from the NSF Industry like Google and also she has received the Purdue Seed for Success Award twice So thank you, Maggie. It's a great pleasure to have you here Well, thank you Dr. Perollis for the introduction. So it is a very exciting opportunity for me to be here today and to share with you guys a little bit on my journey to today and then To actually look at some of the work that we have done developing our lab So the title of my talk today is Towards Unified Visual Representation and Understanding So to start So just a little bit about my journey and as Dr. Perollis mentioned that I've spent quite a lot of time in my life Here on the Purdue West Lafayette campus So I feel like I can you know call myself a true boiler maker and this is you know really home for me And I appreciate along the way The people that I have in counter who have you know being my professor's teachers and some of which are now my mentors and the collaborators to have sort of put out different lenses And then the interactions and the wisdoms I have received from them All right So here is a little bit overview of my research So I work on the intersections of visual understanding and representation So from a visual understanding point of view Videos and image contain very rich amount of information, right? So they and then the lot amount of information presented are from different level of granularity From you know a whole video down to a single pixel in the image Okay, so now there is no free lunch, right? There is a cost that's associated with these rich set of informations and this cost is typically measured in bits, okay, or in terms of the data rate, right? So then the problem of you know, how we represent these visual informations is basically an optimization problem, right? So how do we minimize the cost? We wanted to spend yet trying to preserve or trying to maximize the quality and then serving the different purposes, right? Whether it is for storage it is for real-time transmission over network or For the different tasks may be down the road whether it is you know for human to see and to appreciate the beauties That are being captured or for some machines to be able to interpret the data that it receives So here is some examples of the work that has been developed in our lab And then a lot of these again are Works that I've done by my students and inspired by our collaborations with other people's Here outside of Purdue and the industry So on the top here is a work that we did by looking at now We have a lot of these deep learning models and then to train these models are very expensive and costly, right? But yet we're continuously receiving data from the real world So how do we make these models to adapt to these new data without going through these costly Expensive retraining so we did this work in the case for image classification That is to understand what are the different objects presented in the images So in another word we're interested in quantifying the size of objects In the scene without having to go through the complete 3d reconstruction Okay, and then using commercially available sensor devices like RGB cameras like depth sensors So in another word My student and I were interested in looking at if we wanted to understand What are the same in objects in the scene in the pixel level? Can we do this without having a full understanding of what are the different objects in the image right by leveraging a pair of related Images to be able to do that Here we have I Can see my point a little bit. So here let's suppose we have some string of data or images or videos that are captured under different Settings environment context, right? So how do we be able to tell from them the relationship between these images and to be able to cluster them into similar context? In another word here, we're leveraging the human vision system where it is very difficult for the human vision to tell or to distinguish the fine details within some parts of the image that contain high-frequency components such as texture like Content, but yet these high frequency content are very costly to code where we have to spend all these bits to encode them right so by leveraging to human vision system characteristics, we were able to improve the Efficiency of coding these regions are much better. Therefore, we can retain still retain good quality while spending less bits on those regions and more recently Our lab has been interested in looking at new network based approach to perform lossy compression Okay, so one interesting finding we see is that we could actually achieve these progressive decoding in the sense Where we can spend very few bits, but yet we already start to see the semantics that are represented in this image So and then by adding more bits to it. We can fully reconstruct these images So my interest in research is always to think about what does all these algorithms methods that we developed has a Real societal impact. Okay, so one way to do this is partnership. So we partner with academic collaborators researchers as well as industry partners and Here are some examples of what we were able to achieve So for example here, we have a very long Successful collaborations man with many different nutrition scientists and practitioners all over the world to Use our tool which to do better assessment of dietary intakes in studies So this is the image-based approach that we have developed for dietary assessment On another example, we have partnered with manufacturers of headset devices where The device captures on the left a what a real person as facial expression looked like yet We wanted to re-enact the same facial expression in the virtual avatar So there is this mapping that we have to do our texture-based coding tool is one of the many new coding tools available in the public Available open-source loyalty free AV1 Kodak that has been developed by this alliance of different companies Most of which are large streaming and content sharing companies and platforms So very we are very grateful and happy to see that Collaboration come to fruitness Here we have a collaboration with the Purdue infant speech lab where we're developing tools to help them understand What is the interaction between a caregiver an infant that's Simulates or stimulates, you know the infant speech development So they were able to use our tools to better understand that interaction We also work with a startup companies So here's an example Where this company is interested in this photo realistic redesign of Interior surfaces like you know, what a new product will look like, you know on the floors as a carpet or are you know The countertop without you even have to go in and do this So this is an example for that And the last example that I'm showing here is we're interesting looking at edge cloud systems, right? so particularly looking at what are the complexity for the edge devices and Then the type of analysis that needs to be done in the in the cloud and how to best optimize that type of information for images and videos Okay, so What's the next step right so moving forward? I'm still very much heavily vested in these Two directions. Okay, and here are some examples Of that direction that I see moving forward So as we have many of these mobile and wearable System sensors that are collecting these visual data along with other data Well, how do we use that information to help make? predictions for the personal responses of For example the foods that you're eating okay, or the environment you're in right and Combine that with other type of monitoring like activities or even sensors inside of a body to collect microbiome Informations how do we use that to model all these responses to help improve health of all? So on the other side as we have a lot of these data collections We need to think about what are they using for what's the best way to represent them right are they be used? To reconstruct the pixels so that we as humans can take a look at those or are they used for machines to do further? Interpretation's analysis then the reconstruction should be tailored to the features that are in these images and videos And as always I'm looking forward to expand my collaborations Particularly with domain experts because they have very neat challenges that I Working alone in my lab will not be you know realize those challenges so on these cross-disciplinary research and applications are sort of The foundations of many of the work that we develop in our lab so here is a little bit about Activities related to education and mentoring so I've been very privileged to be able to teach both at undergrad level and graduate level courses most of which are related to signals and systems and I've be able to mentor many students who I'm happy to see that are here today also PhD students master's postdocs and be able to also serve on a large number of PhD student committees And then this is also thank to the large department we have as well as the many collaborators across the campus To learn about their students work on some of which I Do a little bit more mentoring than just serving on committees talk about their project Provide my perspective and learn from them. So I'm very grateful for those opportunities as well I've been also being involved in many of these flagship Project courses within the EC department at a college level And then also in cases where I do independent study courses with some of our undergrad students too Over the summer or throughout the semester Um, here's just a summary of some of the services on both internally at Purdue within our department serving on various Department committees at a college level as well as externally be participating in different organization committees and then to kind of expand my collaborations in the field So here I really wanted to thank all my students It always amazes me of their creative mind, but also and their perseverance because sometimes You know as I was just really is that you don't just get your beautiful results in the first shot, right? So it takes time to come up with great ideas But also there is a level of dedication and then perseverance to be able to really get To that beautiful end results that You know you're proud to present And then many of my formal students they are quite successful In their job as well So I'm grateful to be able to have such a good team to work out these interesting and challenging problems So they said well, you know, what about your mentor collaborators this I think is the hardest slide for me to put together Because as you can see I've you know work with many people's both at Purdue some of which have you know moved away But also, you know externally as well as with a lot of industry partners, right? and then And then when I think about back my journey even when I came back here for my interviews I met people who have given me Advices or just ask me a question which I would have never thought of right and then that inspires me to continue that Conversation with them even before I start my journey here as assistant professor at Purdue and along the way I would say that even though we have a very large department in EC But I always bump into my colleague in the hallway and this becomes a little bit difficult in the past couple years But you know it's it's coming back and then just you know having a casual conversation with them Sometimes inspires a little bit in depth discussion of research and sometimes, you know It is about what is your projection of your career, right? What do you take in the next step? So I really appreciate those small talks just you know stop say hi type of conversation that I have With you know the colleagues in our department And lastly we have a group of amazing Women faculties in the College of Engineering So this is a group of people where they're they're very generous in sharing their wisdoms Okay, and they're at different levels different Part of their career to not just give you wisdoms But also provide the support that will need because there it is a challenging job And as a mother we have different roles to play too, right? So I really appreciate all their mentoring and the support they have given to me throughout this journey So lastly I would like to thank my support from both federal agencies from various Partnerships with industries as well as other universities throughout for Supporting our research. So with that I would like to thank you for this opportunity to present My work and my journey Thank you so much So yeah, let's go ahead and have some questions for professor zoo Yes Great presentation. Thank you for sharing some of the things that you have done and your background Given that you have friends you have had so much time at Purdue and then you worked in the industry as well how has your mentoring style changed over the years and What's your overall summary or the philosophy of mentoring students? Yeah, great. Yeah, great question. Thank you So I think, you know, I Have a great mentor During my PhD and master to professor Adele. I think he may be online here And I think I learned a lot from him in terms of how to do research And then as well as you know, we have a very large research lab at a time So I also kind of learned about you know, how do you interact with students? But this is something where no one really can truly teach you how to mentor, right? So I work in the industry a little bit I also have a great mentor during my industry work as I work in a research lab So it's sort of you one foot in the academia one foot in industry you still publish and all that But when I come back to Purdue It's a bit of nerve-wracking to have your own student to start with right and then I think at the beginning a lot of mentality is there It is less of I teach them but we sort of teach each other, right? So we work very closely Kind of looking at problems how to solve problems and then you kind of just kind of learn from You know interaction with other colleagues, right to see how they you know mentor their student So slowly you sort of developed To a mode where you feel comfortable with interacting with different type of students, right? And then I think what I learned most is you know, everybody is unique There is no I'm single style that could fit you know each student And you need to have the ability to see where their strength lies right and where you know They they could you know be benefit from perhaps working with a different student and so on so I think it's the ever-involving and changing Process and then I'm still learning from that So I saw one of your directions very interesting you take a photo of the dish and you can get calorie count Yes, and how would that you imagine the ground truth come from? Yeah, that's a great question. So So typically what we've done this right now Before you know we have sufficient data to come up with a pretty accurate model is to work in dietary studies, right? So many of these studies are done in a somewhat of a more controlled environment So they wouldn't know working with dietitians. So they prepare the foods and then so they wait everything So we have you know the ground truth that are available, you know for those and then from the image point of view That's right So we would still be asking the domain expert to see what their you know assessments are And then that would be kind of the ground truth that that we would use to evaluate our approaches Like then you have to have ground truth from different countries Um, yeah, so that is a good good very good question. Yes So that's what we're looking for is to expand. So most of the collaborators We're working with are still more on like a Western type of diet But I know as you pointed out when you have very different regions people, you know prepare difference different They're different cuisines and so on. Yeah, so it is a very challenging problem And then but luckily I'm not the only one who's working on that So there are other people around the world that are also working on similar problem coming up with you know Very novel and interesting solutions to that Yes Yeah, I got it the metric so Maggie really interesting, you know what what caught my attention was You know these collaborations you're having with other domain experts and and That's actually leading to a lot of downstream impact of your work as well So I guess the question is based on your experience You know are there some things that we might be doing better in the College of Engineering or beyond that promotes more such I mean was this random? Did you or are there some things that you think actually helped seed? Some of these things. What advice do you have? Yeah, very good question. I think I was lucky because of You know my PhD work is also have is also across an interdisciplinary So I was sort of trained in the modality of how to work with people who are not from your field Right and then because people will speak different languages, right? but I think in terms of The opportunities here I feel like since I you know came to Purdue I've seen a lot of different opportunities that are provided for people to Kind of talk to have very short talks about and this is I feel like become an increasing trend too Because you know sometimes I go to these NSF workshops where they have people giving lightning talks of like under a minute Right, so you get to see a little bit of what people are doing So I really like that because then you get a sense at a high level Is this someone that you would like to work with right and then Purdue also have a lot of big Strategic partnerships right so as part of that you get to learn you know with other people as I listed here Some of these collaborates their work right and think about you know how you can Come up with you know great ideas with them together So I think that's the benefit for the being you know at a very large university Because there are a lot of opportunities For us to learn and then to just sort of come out and then kind of talk with these different people's yeah Have a question not from online. Dr. Alabaq has a question and he asks What is the most difficult career decision that you've had to make as a faculty member here at Purdue? difficult decision uh I Don't feel like there is the most difficult but I think You know the the decision to take on this job is Is something that a little bit unexpected for me because I you know being back in academia be a professor with them You know my always sort of my plan. That's why I worked in the industry a little bit But that experience actually helped me to make the decision to coming back because I I thought about what would I like to work on right so and then so so that is not really that difficult a question, but a A challenge, but you know it did take some you know time to process But I think a lot of time I feel it's difficult to decide you know Which student that you wanted to work with me or research lab? Because it takes a little bit trying to understand them trying to know them a little bit better And sometimes I've have cases where it didn't quite work out, right? So a student may try they think they wanted to do a PhD, but they end up well, you know This may not be and the way to go. So these are times where I have to face difficult decisions. So But you know, it's always a learning process and then Kind of to to get to know everyone that comes away. So they they are all very unique individuals Mike you one question from me as well. You're working in a really interesting Interdisciplinary area with a lot of unique angles I was wondering as we're thinking about investments in this area, right for Purdue to Become a bit stronger, right? What advice would you have for Purdue to think about for the next? Let's say decade what kind of investments do we need here? I Think you know as sort of looking at The area and the field that we're working on I I think that you know We have been hiring a lot of excellent faculties So I would you know really love to see you know that trend kind of moving forward and then and then you know, it's it's I was serving on the faculty, you know a search committee Last year and it is difficult decisions You see Bill is laughing to you see all these excellent candidate He said well, they will be a great fit, you know for this and that but we can't you know have everyone so So, yeah, so I think that is it's always the people right that brings the strength To already a very excellent program that we have All right, so and without I think we can thank Maggie once again. Thank you so much. Thank you