 Slynyddu i gweithio a fyddwch i ysgolwlad uchydig yw'r ysgolwlad! Mae'r fwynhau ar yw'r ysgolwlad yw'r ysgolwlad yw'r ysgolwlad, rwy'n meddwl Nadia Kennol, ac mae'r ysgolwlad yw'r ysgolwlad yw'r ysgolwlad yw'r mewn ffais ym Llywodraeth Cymru, wedi'n wneud ym Mershyn yw, mae'r eisiau bod ydych chi'n fawr i'ch gweithio ar y 3-part yw'r ysgolwlad yw'r elfennyrch. For eich sesiwn, ydych chi'n gobl i ddechrau yw'r pasuniau yw'r gweithgwmmu. Y gallwn ichi'n gobl o'r fynd Ieg sydd y ddechrau yw'r gweithgwmmu. Efallai'r modl i fod yn amlwg i nhw? Felly, rwy'n iawn i ddefnyddiaeth lluniau yn gweithgwm telefonol, a'n olygu'n gobl cyhoedd maen nhw'n ymgylch meetw sydd gyntaf estasig. Ond yna wrth gwrs eich Eistudon, byddwn i'n gennym ei dweud ym 7Even toio maen nhw. I'r sefyllfa yng Nghymru wedi'i gael y Llu, ar y 26 o fawr o'r Ffawr, os ydych yn ymddangos y clystau a'r methau sy'n gyffredin hwnnw i. Rydyn ni'n gwneud i'r llawnod yma wedi'i ddweud y clystau, ac mae'r ddiwrnod hynny'n gwneud i'r llawnod hefyd, ac mae'r hyrach oescyddol. Y ddweud 3, ac yna ym mhwynt, oeddwn ni'n meddwl cyllideb yn ymdangos cyllideb, ac mae'n gweithio'r ddatblygu, bydd ydych yn gweithio y llwylliannol ar gyfer ac yn gweithio gyda'r cyffredinol. Mae'r unrhyw ym Llyfrgell yn python a'r gweithio'r gweithio'r gweithio'r gweithio'r gweithio'n rhaiff, felly bydd ddweud yn cael cyfnodd o'r llwyniadau ar gyfer y llwyniad cymryd yma. OK, dyma'n addysg. Gweithio yma'n mynd i'r Llyfrgell? Gweithio'r Llyfrgell yn ddiddordeb yn oedol ffyrdd ymgylch. yw'r cyfnodd ar gyfer cyfnodd ymlaen, yw'r cyfnodd yn ychydig ar gweithio'ch cyrthafell yw'r cyfnodd ymlaen yn ymlaen. Fy wneud o'r maen gweld yma, yw'r cyfabiliad ymlaen i wneud ei wneud y gallu hyffordd. Fy fydd yw'r cyfnodd ymlaen, ac mae'n cyfnodd ymlaen yw'r cyfnodd yn ysbytiau hynoddi. Mae yna maen nhw'n fydd yw'r cyfnodd ymlaen yw'r cyfnodd. It was defined in 1950s by an AA pioneer named Arthur Samuel. He described this as the field of study that gives computers the ability to learn without explicitly being programmed to do so. See, this definition still holds true today. Machine learning takes the approach of letting computers learn to program themselves through experience. This means that machines can recognise a visual scene, understand a text written in natural language, or perform an action in the physical world. Machine learning is behind chatbots, it's behind predictive text, it's behind language translation apps, the shows that Netflix suggests to you and how your social media's feeds are presented. In order to understand exactly what machine learning is, we can explore some of its functions. I would say that there are three main parts to a machine learning system. The first is the descriptive. This uses the data to explain what happened in the past. So this may start with either numbers, photos, texts, time series data, and then this data is gathered and prepared to be used as a training data. All the information the machine learning model will be trained on. So this descriptive analysis looks at data statistically to tell you what happened in the past. This descriptive function is very fundamental machine learning, as it allows you to explore the variables and attributes within your data set. The next function is known as the predictive analysis. This is where the system uses the data to predict what will happen in the future. Programmers choose a machine learning model to use, supply their data, and let the computer model train itself to find patterns or make predictions. We then move on to the third function, which is known as the prescriptive analysis. So this system will use the data to make suggestions about what action to take. Prescriptive analysis kind of takes predictive analysis to the next level. It basically asks, like now that you have an idea of what will likely happen in the future, what should you do? And it suggests various courses of actions and outlines what the potential implications would be for each. It's interesting to note that the output of a machine learning system is a model that can be thought of as an algorithm for future computations. The more data the system is presented with, the more refined the model will be. And the quality of the learn model is also dependent on the quality of the data used to train it. So if the data is biased, for example, then the output of the model will also be biased. But we'll discuss this a little later. So why is machine learning important? There is a lot of data in the world generated not only by people, but now also by computers, phones, and other devices. Traditionally humans have analysed data and adapted systems to the changes in data patterns. However, the volume of data has grown so large that the ability for humans to make sense of this data is incredibly hard. So this is where we turn increasingly to automated systems. These automated systems can learn from the data, but also they adapt to the shifting landscape, so they change over time. There are also multiple uses that machine learning can be applied to in order to do things for businesses like cut costs, miss gate risks, and generally just improve the overall quality of life. With greater access to data and computational power, machine learning is becoming more important every day and will soon be integrated into many facets of human life. We can have a look at some applications that machine learning has. The first example is image recognition. Image recognition is the process of identifying an object or a feature in an image or a video. Image recognition has helped the healthcare industry, whilst improving diagnosing diseases like cancer with greater accuracy. Another application of machine learning that we see quite regularly is fraud detection. Fraud detection basically through the analysis of images of banknotes, machine learning algorithms detect patterns in financial operations and decide whether a given transaction is legitimate. Another application that I find really interesting is recommendation systems. Things like YouTube, social media, your Netflix account. These machine learning algorithms basically segment customers based on their user data and behavioural patterns. Things like what you purchase, your browsing history, your likes or your reviews, and then targeted them with personalised products and content suggestions. Another application is text-to-speech systems, which can be used to improve accessibility for those with hearing impairments, disabilities or age citizens. Text-to-speech systems also has another name known as speech synthesis. Using a variation of machine learning called deep neural networks, which we won't discuss today, but we can use deep neural networks to produce artificial speech from text. These are just a few machine learning applications. Through applying descriptive, predictive and prescriptive analytics, we've been able to apply these methods to other fields such as transportation or retail. We can move on to have a look and discuss how machine learning is different to classical statistics. Before moving on to discuss the methods of machine learning, I think it's important to elaborate how machine learning is different, but also similar to classical statistics. I've summarised what I think are the important distinctions. Looking at the approach from the machine learning perspective, the focus is on limiting assumptions. However, in statistics, these assumptions are made from algorithmic models, also known as statistical modelling. Statistical modelling refers to the data science process of applying statistical analysis to datasets. A statistical model typically is a mathematical relationship between one or more random variables and other non-random variables. The application of statistical modelling to raw data helps data scientists approach data analysis in a more strategic manner by providing visualisations that aid in identifying the relationships between these variables and making predictions. On the other hand, machine learning programmes need large amounts of data and proper insights in order to function. They use this data to then build their models. This moves on to the focus. In a machine learning approach, we basically try to limit the assumptions to a minimum, and it will let the data almost speak for itself. Statistics, however, usually requires the statistician to make assumptions about the structure and or the distribution of the data, trying to guess the relationship between the variables in order to write an appropriate model. The next difference, which I think is quite crucial, would be between the inference. Machine learning models are designed to make the most accurate predictions possible, whereas statistical models are designed for inference about the relationship between variables. A main difference between machine learning and statistics is indeed their purpose. However, same machine learning is all about accurate predictions, whereas statistical models are designed for inference can be argued to be a bit of a limited statement. This is because there is a difference between comparing statistics and then comparing statistical models. Yes, we can still examine the parameters of machine learning, and yes, we can still focus on the predictions and statistics. The inference would be dependent on your approach as a researcher. It would be dependent on the research questions that you have asked, your goals, your aims. And this kind of brings us on nicely to the overlap. It brings us onto this gray area between the two. Personally myself, I would not put a label on every single algorithm to fit within machine learning or to fit within statistics, because they are deeply intertwined and use many of the same tools, the computational techniques that I've mentioned. So in the end, when you're in the gray area between the two, the fact of doing statistics or doing machine learning often depends on the mentality that you use when approaching the problem. We can explore this gray area by looking at an example of a machine learning method known as regression. Through a statistical lens, through a statistical lens with regression, you would typically examine the variables, and based on the meaning, try to understand which ones might interact, which ones have some sort of linear dependency, and then build a model. However, through a machine learning lens, you might use a backwards elimination process of features starting from a model containing every interaction. So this is letting the data decide which ones are relevant. So regression proves to be an example of both a machine learning method, as well as a statistical method, because it uses not only the same computational powers, uses the same skills, and also has this fundamental understanding of regression in itself. But this moves us on to discuss some of the machine learning methods in a bit more detail. In general, most machine learning techniques can be classified into supervised learning and unsupervised learning. In supervised machine learning, the algorithm is provided an input data set and then is rewarded or optimized to meet a set of specific outputs. For example, supervised machine learning is widely deployed in image recognition, which was mentioned earlier, and it utilizes a technique called classification. Compared to unsupervised machine learning, the algorithm is provided an input data set, but not rewarded or optimized to specific outputs, and instead trained to a group of objects by common characteristics. One example could be recommendation systems or recommendation engines also mentioned earlier. And this is typically done using a technique called clustering. But we can expand on this a little bit further, but I think it's first important to discuss the differences between models and algorithms, because I have used both terms quite frequently in the last 15 minutes in itself. An algorithm in machine learning is a procedure that is run on data to correct a machine learning model. Machine learning algorithms are procedures are implemented in code and are run on data. Machine learning models, on the other hand, are outputs by algorithms and are comprised of model data and prediction algorithms. Algorithms typically tend to come before the model. Moreover, a machine learning model is a program that can find patterns or make decisions from previously unseen data. Again, for example, in image recognition, a machine learning model can be taught to recognize objects, whether this be cars or dogs, and then the machine learning model can perform such tasks by having it trained with a large data set. During this training, the machine learning algorithm is optimized to find certain patterns or outputs from the data set depending on the task. The output of this process is named a machine learning model. So how exactly does a machine learning algorithm work? There are three really important functions to a machine learning algorithm. The first step is the decision process. This is the steps that take in the data and guess what kind of patterns your algorithm is looking to find. In general, machine learning algorithms, as discussed, are used to make predictions or classifications. The second function to a machine learning algorithm is the loss-slash-error function. This measures how good the guess was by comparing it to other examples. An error function basically serves to evaluate the prediction of the model. So, if there are known examples, an error function can make a comparison to assess the accuracy of the model. The last function is the model optimization process, and this is simply a method in which the algorithm looks at the miss or the error and then updates the decision process. So, basically, if the model can fit better to the data points in the training set, then weights are adjusted to reduce the discrepancy between the known examples and the model estimates. This algorithm will repeat itself and repeat the evaluation and the optimization process until an accurate threshold has been met. Just a little bit more about models and their implications. Learning models are established based on data-driven principles, so a typical process is to learn-slash-discover generic patterns based on a training data set. It's important to know that machine learning works in an adaptive learning manner. This means that it learns-slash-discover a generic rule from a number of examples generated by such a rule. The nature of machine learning always limits the training data, so only a specific sample of a population to be modelled. The most important fact here to know is that the ultimate goal of machine learning is towards inductive bias or generalisation. So, a learning model can predict output correctly given input that has not been encountered during learning. In other words, a learning model should be able to generalise the rule learned from observed data to unseen data. We can move on now to discuss the differences between supervised learning and unsupervised learning a little bit better. What is supervised learning? We can understand what this means by looking directly at the words that make it up. Supervised means to observe and direct the execution of a task or an activity. Using this definition, we will be supervising a machine learning model that might be able to produce classifications. But how do we supervise a machine learning model? We do this by teaching the model. We load the model with knowledge and then have it predict future instances. So, how do we teach this model? In short, we can teach a model by training it with a labelled data set. Supervised machine learning algorithms functions with data that is already labelled. Each sample is typically tagged with one or like several labels according to the category of objects. And then corresponds to specific characteristics and properties which allow the identification and classification of objects. Now, according to the labelled data, the system is learning from, it will be able to use this piece of information to identify and classify new images. The example on the slide is from a data set called Iris, which is, you may or may not be familiar with, but it's a very, very common data set used for machine learning and statistics. So, there's another overlap. My thing is important just to talk about some of the components because there are overlaps in the names of these variables. So, if you look at the rows, we have the sepal length, the sepal width, the petal length, the petal width and the class. So, these names are its attributes. These are also known as its variables. So, if we look down in the columns, that this includes the data and are typically known as features. These are also known as your values. So, it's just useful just to provide a bit of context to the overlaps in names. So, there are two types of supervised learning. We have regression and classification. Now, a regression algorithm can predict a discrete value, which is in the form of an integer, whereas a classification algorithm can predict a continuous value if it is in the form of a class label. So, classification in itself is a predictive model that approximates a mapping function from input variables to identify discrete output variables, which can be labels or categories. So, an example from the iris data set might be that you're attempting to predict class against, for example, sepal width. Compared to regression, now regression algorithms tend to predict a continuous value based on the input variables. The main goal of regression problem is to estimate a mapping function based on the input and output variables. So, if your target variable is a quantity like income, scores, height or width, or if the probability of a binary character, then you should use a regression model. Here are some supervised algorithm examples followed in the classification and regression. In classification, some examples of the algorithms include a decision tree, random forecast or a cane nearest neighbour. A decision tree is, in this algorithm, a classification model basically is created by building a decision tree where every node of the tree is a test case for an attribute and each branch coming from the node is a possible value for that attribute. We then have random forests, so this is a little bit different, and this is a tree based algorithm and includes a set of decision trees which are randomly selected from a subset of the main training set. We then have cane nearest neighbour, which is another example of an algorithm in the supervised learning, and this basically assumes that similar things exist in close proximity to each other. We then have some example algorithms from regression, so this includes simple, multiple and polynomial regression. Typically, this is where we estimate the relationship between one independent variable and another dependent variable or multiple dependent variables if you're using multiple linear regression. But yeah, these are just some examples of algorithms in the supervised learning. These do not include all possible algorithms. So now we can move on to understanding what is unsupervised learning. In short, we do not supervise the model, but instead let the model work on its own to discover information that may not be visible. Unsupervised learning uses machine learning algorithms that draw conclusions on unlabeled data set, so this is the opposite to supervised that uses labelled. Unsupervised learning is said to be more complex because we have little to no information about the data or the outcomes to be expected. Unsupervised learning has fewer tests and fewer models that can be used to make sure the outcome of the model is accurate. Therefore, we have less of a controllable output. Compared to supervised learning, unsupervised classification or clustering algorithms attempt to discover structure in data by organising into groups whose members are similar to each other. The raw dataset being used in the unlabeled and then the algorithm identifies the patterns and relationships within the data without help from the user. Here in unsupervised learning there are three subfields that is clustering, association and dimensionality reduction. Now clustering involves the analysis of patterns and groupings of unlabeled data as previously discussed. We then have an association. An association rule is a rule-based method for finding relationships between variables in a given dataset. These methods are frequently used for market analysis and they allow for companies to better understand the relationships between different products, for example. Then we have dimensionality reduction. It is a technique used when the number of features or dimensions in a given dataset is too high. It reduces the number of data inputs to a manageable size but also preserving the integrity of the datasets as much as possible. It is typically commonly used in pre-processing data stage and there are a few different dimensionality reduction methods that can be used as seen on the screen. We are not going to discuss too much about these algorithms as they can be a little bit complicated but we will be discussing some of these in the later sessions this week and next week. Just for some reference, there are also some other machine learning methods that move from supervised and unsupervised. We have semi-supervised learning, reinforcement learning and deep learning methods. Semi-supervised learning refers to a learning problem that involves a small portion of labelled examples and a large number of unlabeled examples from which a model must learn and make predictions. It falls between the two. We then have reinforcement learning and this happens when the algorithm interacts continually with the environment rather than relying on training data. One of the most popular examples of reinforcement learning would be autonomous driving. We then have deep learning which was briefly discussed but this is a newer area of machine learning that automatically learns from datasets without introducing human rules or knowledge. So this requires massive amounts of raw data for processing and the more data that is received the more the predictive model improves. But as of recent, supervised and unsupervised learning methods make up a lot of the applications that we see day to day. So now we've discussed the big definitions and the big questions surrounding machine learning. I think it's important to then discuss some of the steps of machine learning and this was discussed by Eufeng in 2017. So they described that there were seven steps to machine learning. This involves collecting your data, data preparation, choosing a model, training, evaluation, parameter tuning and then prediction. In order to understand these steps I've come up with a case study that allows us to create our own model. So I'm going to be talking through a case study and explaining how each step is applied to our case study. So let's just say that we have been asked to create a system that answers a question of whether a food item can be classed as an apple or as an orange. Now this might be a really simple question but this will help us to understand how we can go about choosing a model and how we can go about training our dataset. So this question answering system that we build is called a model and this model is created by a process called training. The goal of training is to create an accurate model that answers our questions correctly most of the time. But in order to train a model we need to collect the data to train on and this is where we begin. Step one is to gather the data. For the purpose of developing my machine learning model, our first step would be to gather relevant data that can be used to differentiate the two fruits. You can use different parameters to classify a fruit as either an orange or an apple, but for the sake of simplicity I've taken two features. The first being the colour and the second being the shape. So using these features we would hope that our model can accurately differentiate between the two fruits. Now typically you might have a model that includes way more features but yeah we'll be just using two features for today. So the data that we have collected here, the data that you see on the screen would be your training dataset. So we can then move on to step two which involves preparing your data. So once you have gathered the data we then move on to the data preprocessing stage and typically this does involve the biggest step. This can take up the most time consuming step depending on the complexity of your dataset. A key focus of this stage is to recognise and minimise any potential biases in our datasets for the two features. So typically the first step you would want to do is randomise the order of your data for the two fruits. And this is because we do not want the order to have any bearings on the model's choices. Furthermore we would examine our datasets for any scunas towards a particular fruit. This again would help in identifying any potential biases as it would mean that the model would be adapt at identifying one fruit correctly but may struggle with the other fruit. Another major component of data preparation is breaking down the datasets into two parts. We have the larger part which typically is around 80% roughly and this would be used for training the model while the smaller part typically 20% is used for evaluation process. So this could be 80% to 20% it could also be 70% to 30% but this would always depend on your aims as a researcher. This is important because using the same datasets for training and evaluation would not give a fair assessment of the model's performance in real world scenarios. Apart from the data split there could be additional steps taken to refine the datasets. This would be dependent on what type of data you have but this could include removing for example duplicate entries or discarding for incorrect readings or removing missing values. But luckily we have a very simple data set so there's not much that we need to do. Now step three would involve choosing a model. Now there are various existing models developed by data scientists which can be used for different purposes but these models are designed with different goals in mind. For instance some models are more suited to like dealing with text or other models maybe better equipped to handle images. But with regards to our model I think that simple linear regression would be suitable in differentiating between our two fruits. So in our case the type of fruit would be our dependent variable while the colour of the fruit and the shape of the fruit would be the two predictors or the independent variables. So this means that our features represent our independent variables. And just to recap linear regression model the linear regression model chosen here is an example of supervised learning. And this is because the outcome is known so we can continuously refine the model itself until our output reaches the desired accuracy level. So now we can go on to step four which involves training. So here we use the part of the dataset allocated for training to teach our model to differentiate between the two fruits. If we view our model in mathematical terms which you can see the equation on the top right of the screen. Then the inputs so our two features would represent the coefficients or would have coefficients apologies. And then these coefficients are called the weights of features. However it's not too important to discuss the mathematics behind a regression model. So once you have taught your model to differentiate between the two fruits we would then have to take a step back and do some evaluation. And this is where we draw our attention back to the functions of an algorithm. And this looks at the decision process, the loss error and the model optimization. This is where we would have to reevaluate our model and ensure that decisions taken at the start. Yeah that the decisions taken at the start have the highest like optimization. We then can move on to our step five which is the evaluation. And this is a very important step as well. So with the model trained, it needs to be tested to see if we could operate well in real world situations. And that is why the part of this data set created for evaluation is used to check for the model's proficiency. So this puts the model in a scenario where it encounters situations that were not part of its training. So in our case it could mean trying to identify a type of apple or orange that's completely new to the model. For example we never identified green apples in our original data set and this might need to be included in order to improve the model's efficiency. This also leads us on to our next step which is the parameter tuning. So if the evaluation is successful then we proceed to the step of hyper parameter tuning. So this step basically tries to improve upon the positive results achieved during the evaluation step. For our example we would see if we can make a model even better at recognising apple and oranges. And there are two ways we can go about improving the model. Firstly you could revisit the training steps and use multiple sweeps of the training data set for training the model. This is because it could lead to greater accuracy as the longer duration of training provides more exposure and can improve the quality of the model. Another way to go about this is to refine the initial values given to the models. Random initial values often produce poor results as they are gradually refined by trial and error. However if we can go up with better initial values or perhaps initiate the model using a distribution instead of a value then our results could get better. I think it's also important to note that there is not a definitive approach for parameter tuning as it would depend on the learning approach you choose but also what type of variables you have present. And this leads us to our last step which is prediction which is what I've been honing in and out for this whole talk. But yes the final step of the machine learning process is prediction. This is the stage where we consider the model to be ready for practical applications. So our fruit model should now be able to answer the question whether the given fruit is an apple or an orange. It's kind of interesting to know that a model like this would gain independence from human inference and draws on its own conclusions on the basis of the data sets and the training. I guess one thing to think about. So the challenge for this model remains whether it could outperform or at least match human judgment in different relevant scenarios. But yes I hope those seven steps have been able to provide some more context about how to run your own machine learning models and hopefully be able to apply this to your own work. So let's have just a quick recap about the differences between supervised and unsupervised learning. As discussed, supervised learning learns from the training dataset by making predictions on the data and adjusting for the correct answer. Supervised techniques deal with labelled data whereas unsupervised data deals with unlabeled. The goals of supervised learning is well known before the training sets to the type of output the model was expecting is already known. We just need to predict if unseen new data was an unsupervised learning algorithm. The goal is to get insights from large volumes of new data and there is no particular output value we're expecting to be predicted, which makes the whole training procedure more complex. In terms of its applications, supervised learning models are ideal for classification and regression in labelled datasets. Things like spam detection, image classification, weather forecasting could have price predictions among some of the most common applications. Compared to unsupervised learning, fits perfectly for clustering and association of data points used for things like detection, custom behaviour prediction, recommendation engines, even including noise removal from datasets. Supervised learning is comparatively less complex than unsupervised learning because the output is already known, making the training procedure much more straightforward. In unsupervised learning, on the other hand, we need to work with large unclassified datasets and identify the hidden patterns in the data. The output that we are looking for is not known, which makes the training harder. I realise I've been talking for quite a while, so I've just got some scenarios on the screen that I'd like you guys to think to yourselves about. I'll discuss whether these scenarios are basically best suited for supervised or unsupervised learning methods. Let's have a look at some of the examples and I'll give you 30 seconds to a minute just to think about this to yourself and have a think whether a supervised or unsupervised training method would be most appropriate. In our first scenario, we need to predict the number of vehicle purchases in a city for historical data. Do you think that we would need a supervised or an unsupervised learning method to do so? I hope you've been able to think about that a little bit to yourselves. In this instance, it would seem that we have already labelled a dataset because of this historical implication. In this instance, a supervised model might be more appropriate. A further example of an algorithm that falls within the supervised learning that might be appropriate could be, again, a simple linear regression. Because with simple linear regression, you can estimate that relationship between one independent variable, which could, for example, be the historical data, and then compare this to the dependent variable, which would be the number of vehicles. We'll then move on to scenario two. In this scenario, we need to identify if a potential customer in that city would purchase a vehicle given their income and community history. So, again, do you think that a supervised or unsupervised learning method would be most appropriate to build a model for this situation? Again, I'll give you 30 seconds just to think independently. Initially, when I first read this question, I thought that this would be quite suited to a supervised learning method. This is, again, because we have a known number of classifiers in the question. We know that the known classifiers would be the income and community history, which would identify some sort of labelled dataset. An algorithm that might work well within the supervised learning methods could be a decision tree. So, in this algorithm, a classification model is created by building a decision tree where every node of the tree is a test case for an attribute, and then each branch coming from that node is a possible value for that attribute. So you could build a decision tree to decide if that customer in that city would purchase a vehicle depending on the income and community history. And just the last scenario before we round off, we want to determine different segments of customers, and this includes gender, age, income, education, etc. Do you think that supervised or unsupervised learning method would be best to build a model for this scenario? So, in this instance, we're presented with a little bit of a different question, and this is because we isn't quite clear if we have a labelled or unlabeled dataset again. But I would suppose that this would need an unsupervised learning methods because we're trying to determine different groups of customers, so something like a clustering model would work well, for example. Now, as this class clustering is an unsupervised technique where the goal is to find natural groups of clusters in a feature space and then interpret this data. So, we could be interpreting different segments of these customers. I know that also clustering is commonly used for determining customer segments in marketing data, so being able to determine different segments of customers helps markets teams approach these customers in a more unique way. But yeah, that draws our conclusion to the three scenarios, and I hope they've just given you some time to think. But that draws conclusion to this talk. I hope I've been able to teach you what machine learning is. We then talked about some of the functions of machine learning and how these works. And then we had a quick discussion about how machine learning is different to classical statistics. In the second half, we looked more specifically at some of these unsupervised and supervised methods using that case study. We were then able to identify the seven steps to machine learning.