 What's going on everybody welcome back to another video today we're going to be discussing the difference between a data engineer versus a data analyst. Now in this video we're going to be going over four main categories the responsibilities qualifications skills and salary of both the data engineer and the data analyst and then at the very end we're going to talk about which job may be right for you. Now let's jump right into it with responsibilities and let's start off with the data engineer. The first thing that the data engineer is going to be responsible for is developing and maintaining data pipelines. Now if you don't know what a data pipeline or an ETL pipeline is is basically an automated process to get the data into your databases from a source. The data engineer writes all the code to develop that process but they also need to maintain that process and so if something breaks down the line they are going to be the ones that need to go in figure out what's wrong and fix it to make sure that the data is coming in properly. Another thing that data engineers do is ensure the data accuracy and quality and this is not something that they need to do just one time this is something that they need to do as long as they're getting data from that source. Now it would be near impossible to check every single data source manually either daily weekly or monthly and so typically they're going to set up some type of monitoring system that checks the data and sends them a flag if anything looks off and one of the other big things that data engineers do is they design develop and maintain database architecture. Now as part of the ETL process they need to extract the data transform the data and then load that data somewhere and typically that's going to be a database a data warehouse or a data lake and so before that data is loaded some type of database and table schema needs to be created in order to store that data properly. So let's move on to the data analyst. A data analyst is going to use static data to both describe the data and gain insights. So the data engineer is going to create an automated process that pulls that data in and stores it and then the data analyst is going to go and pick that data up wherever they stored it and they can use that for their further processes. Now the next thing that a data analyst does is ensures data acquisition and maintenance. Now I think this is one of the biggest things that people miss in the data analyst job description. Oftentimes you're going to be doing data acquisition as well but it's not acting as a data engineer you're not creating the automated process but you may be working directly with a client to better understand their data at the source and then mapping the data from the source into your target databases on your side and this is called data mapping and it needs to be done because typically your client is not going to format things or keep things in the same columns as you would in your database. So when you're taking a table from the source you're going to tell each column exactly where it's supposed to go in your database. The very last thing for a data analyst is you're going to create dashboards and reports and this is no surprise I think this is what a lot of people think of when they think data analysts. You're going to be creating dashboards that include all of your insights and KPIs that your company wants to see and track and you're also going to be creating reports whether those are internal or go out to clients that give them information or give your team information that they can use and act on. Now let's move on to qualifications and let's start off with the data engineer. Almost every single job that you'll find for data engineering you will need a bachelor's degree for even some you're going to need a master's and some of the degrees that are highly recommended are degrees in computer science statistics applied mathematics and computer engineering. For a data analyst it is somewhat similar a lot of jobs are going to require a bachelor's degree some are even going to require a master's degree typically at larger companies some really good degrees for data analysts to get are degrees in computer science statistics mathematics economics and finance now when I was doing my research for this I found that a lot of data engineering jobs were requiring a very technical degree and I think that's because this job is much more technical than a data analyst position and so they didn't care as much about the domain knowledge as much as they did the technical side of things whereas with the data analyst a lot of jobs prefer the technical degrees but they were more than willing to accept some type of domain degree like a finance degree or an economics degree as long as you have the right skill set. Now let's look at the skills section and let's start off with the data engineer some of the skills that you'll need to know are sql and no sql and if you don't know what no sql is it is kind of like sql except you can store unstructured and semi-structured data in it a few other skills that you'll need to know are things like hadoop java and python and within python you'll need to know things like pandas and airflow for example you'll also need to know apache kafka and apache spark as well as hive and then some type of cloud platform like aws or azure so for the data analyst you're going to need to know things like sql r and python and within python you might need to know things like pandas numpy and that plot lib you'll also need to know some type of data viz tool like tableau or power vi you'll need to know how to do data modeling you may also need to know sass or sbss excel and then some type of cloud platform like aws or azure when i look at these two skill sets the first thing that jumps out to means that the data engineer is a lot more technical they're going to need to know several programming languages very in depth in order to get their job done efficiently where the data analyst may only need to know one like r or python or they may not need to know one at all if their company doesn't use one of those now let's move on to the salary section let's start off with the data engineer for an entry-level job you can be looking at anywhere from 55 to 80,000 for a mid-level anything from 80 to 120,000 then for a senior level you can look at 120 to 150,000 for a data analyst for an entry-level job you can be expecting anywhere from 45 to 65,000 for a mid-level job anywhere from 65 to 85,000 and then for a senior level job anywhere from 85 to 110,000 as you can tell the data engineer is going to make a little bit more money at each level and that's probably due to the technical requirements that you're going to need in order to become a data engineer versus a data analyst now who do i think these jobs are for i think the data engineer is going to fit really nicely with somebody who has a technical background as well as somebody who absolutely loves databases and building databases so if you know a lot of programming languages and you love building and designing databases this job might be a perfect job for you for the data analyst i think it's going to be somebody who's very curious and likes to solve problems they're going to enjoy digging into the data to find trends and patterns and then visualize those trends and patterns that they found the data analyst job is also going to be a little bit more entry level or beginner friendly for people who are trying to get their foot in the door in the data world so there you have it that is the difference between a data engineer and a data analyst i hope it was helpful for you i hope that that gave you some type of insight into what you may want to become in the future for your career thank you guys so much for watching i really appreciate it if you like this video be sure to like and subscribe below and i'll see you in the next video