 Yes, good morning, good afternoon, or good evening, depending on where you're logging from. It's my pleasure to welcome you to this second session of the Glossel and Soil Spectroscopy webinar. My name is Isabel Verbeck from the Global Soil Partnership Secretariat. Our today webinar will introduce you to a very important topic, soil spectroscopy for accurate measurement of soil physical and chemical soil properties. Before starting, I would like to remind you that the session is organized in a webinar format in which participants cannot activate their audio and camera. The meeting is recorded and the recording and presentation will be uploaded on the Glossel and Soil webpage. Excuse us for the delay of uploading the recording from last week, but we were very busy at the Global Soil Partnership with our annual plenary assembly, but we will do this within the end of this this week, this today webinar and the one from last week. So some technical information you are encouraged to post your question in the Q&A box, which will be moderated by my Glossel and colleagues. In addition, you can see also a chat box available that can be used for interacting between participants and please use the chat responsibly. If you have any technical issues you can write to me directly on the chat, I will be very happy to have them. Finally, I would like to invite you to join our new Facebook group, so Glossel and Soil Spectroscopy for my colleague, you will put the link now on the chat. So, before digging into Soil Spectroscopy with our renowned speaker, I would like to give the floor to my colleague Yip Bang, who will provide you with a bit of background on the Global Soil Partnership and Glossel and it will be moderating this session. Yip, over to you. Good morning, good afternoon, good evening, depending on where I log in. In the next few minutes, I will just briefly introduce a little bit to let us know give some brief information who is organizing this webinar and why we are organizing this webinar. This webinar is organized under the framework of the Global Soil Partnership, we call GSP. GSP is established in 2002 to position soils in a global agenda through collective actions. Our main objective is to promote sustainable soil management and improve soil governors to guarantee health and productive soils. The activities are downscaled through the R7 Regional Soil Partnership, also supported with our partners and our international governmental technical panel on soils. This ITPS is the highest body of our expert panels. Regarding the working area, we're working with a range of, a wide range of topics as you can see from a screen. On both of these topics, we also have different technical networks, for example, Glosselon, and for example, the international network of the black soil. If you have any information, you can find it in our website. Talking about the Glosselon, Glosselon is a global soil laboratory network is established in the 2017 to build and stress in the capacity of a laboratory in soil analysis and to respond to the need for harmonizing soil analytical data. In 2017, Glosselon, we started to work on the wet chemistry focused on a training harmonization and SOP standard operating procedures and execution of internal laboratory comparisons. In last year, we launched the Global Glosselon Initiative on Soil Spectroscopy, we also call dry chemistry. The main objective of this initiative is focused on the national capacity training. So that is why we organized the first series of the webinar to invite all the scientists, well-known scientists to share their knowledge to the world and let our colleagues and labs research groups around the world to know and to learn this technique. Last year, we also launched the international network on fertilizer analysis. For more information, you are very welcome to visit our website, also write email to me, write email to my colleague Lucrezia. In the end, I would like to introduce our coming webinars as some of you already joined the last webinar and second webinar and the third webinar will also focus on the general information about the soil spectroscopy. Our guest speaker today will be the next webinar speaker will be Alex McBrenney, professor. After that, because after some communication with countries, we realized that one of the most often asked the question is how to build a global spectral library and how to use the global spectral library. So we invited our guest speaker from Brazil and from France to give some good example from their country. In the last webinar, we will invite Iovendor from Eastern to talk something about the measurement. But please be noticed, this is just beginning of our webinar and more webinar will be coming with different languages and more interesting topics. So, thank you. Thank you very much again for joining this webinar. And now, and now it is. Now I have, now I have the great honor to give the floor to Professor put him on ministry from the University of Sydney, a very well known worldwide soil scientist. He's passionate about the role of soil in managing climate food, water, energy security and maintaining biodiversity. You can find more information about his research interest from his webpage. My colleague Isabel will soon post his webpage in the chat box. He's also former chair of international pedometrics working group. He and his team was one of the first research group who started to develop some function and packages for soil spectral data analysis using our program. I remember the first time I started to use our program for telemetrics modeling was in his team. Recently published a book soil spectral inference with our analysis digital analysis digital soil spectral using the R program environment, which is a very good material to learn how to use our program for soil spectral modeling. I highly recommend you to use this book for learning purposes. I'm currently collaborating with the University of Sydney to record a series of video course together with the code and the make it available for free on the gloss along website. By end of this year, this product is also part of our gloss no national capacity development program. In the meantime, I would also like to introduce another panelist that today, Professor Alex McBrenny, Dr. Alexander Wodox, Dr. Jose. Karen and Dr. Edwin John, they will be, they have, they all have intensive experience in soil spectroscopy and spatial modeling. They will be here help to us today and answer some questions in the QA session. So please feel free to post your questions in the QA box anytime. Without further ado, I would like now to give the floor to put him on please. Okay, I think you can share your screen now. Okay, thank you, Isabel. Thank you for the introduction. All good. Can you hear me now? Okay, and thanks to Global Soil Partnership for the invitation to present this webinar about soil spectroscopy. So my name is Abudiman Minasini. I'm from the University of Sydney. With me presenting this work, this work is a collective work that we do together, I'll acknowledge Watini who's not here today but Edward Jones is in the panel. Alex McBrenny is also in the panel, Jose Padarian and Alexander Wodoo. So good morning, good afternoon, good evening and good day in Australia. So before I begin, I'm here in Sydney. I would like to acknowledge the country where I'm presenting from. So I would like to acknowledge the Gadigal people of the Euro nation, which are the land I am standing today as the traditional students of Australia. Their long and rich history is about carrying the land and which we are doing today is about soil, about the land. So before I begin, I'll just clarify some terminologies about infrared. So this is about soil spectroscopy. And what I'm talking to you today is about infrared. So there are, as you see the slide here, the picture here, this is the electromagnetic spectrum. As you go from left to right, you have an increased wavelength, but also if you go from left to right, it's also decreasing the frequency of the electromagnetic spectrum. And within that electromagnetic spectrum, from the lowest wavelength to the highest wavelength in the middle, there is this infrared, which are very important for soil because it provides analysis of the soil. So it starts from the visible wavelength until what we call as the mid-infrared wavelength, which I will talk about today. So there are different definitions, but mostly what's agree is around 300 to 700 as the ultraviolet and then the blue and then the green and then so on, and then the red. So the 700 to 200, 2000, sorry, 700 to 2500 nanometer is what we call as the wavelength, what we call as a near infrared. And then the mid-infrared, what we define here in this presentation is 2500 to 25,000 nanometer. There are different definition as I say some literature divided into near infrared short with infrared and so on, but for the presentation for this presentation when I'm talking about near infrared, it's around 700 to 2500 when I'm talking about mid-infrared is 2500 to 25,000 nanometers. The structure of my presentation, this webinar is I'll be very general, I'll give you about what is the visible and NIR applications. And because continuing from last week, last week, the presenter has a Boston book has presented about some of the near infrared, so I'll continue it about the near infrared applications in soil science, and then how it's going to be used in the field. And then some words about how we calibrate the infrared so that it can predict about the soil data. And then lastly, I will talk about mid-infrared for accurate measurements. So let's begin. So first I'll talk about the near infrared applications in soil science. So, as we say before that near infrared is visible to near infrared is from 500 to 2500 nanometers. And the instrument, the standard instrument that people use is something like this one, or some other brands won't worry about it. In this study we've evaluated if we use the full length of the visible to the full length of near infrared that is from 500 to 2500 nanometers. So the top two, these are what we call the top of the range, the standard, the top of the range visible to near infrared spectrum. In this case, we have a more low cost spectrometer that for example this one, which only, which is only part of this infrared range, and another infrared spectrometer, which is only around from 900 to 1700. This one is shorter, shorter, shorter, shorter, shorter near infrared range is from around 1300 to 2450. So this is on top of it. The top one is the state of the art, the most complete spectrum of the soil. The last two are the low cost infrared spectrometers because nowadays you get more and more low cost spectrometers that are available. So can we do the same thing with the comparing the top end of the spectrometer. The first two that is the top end of the spectrometer the first two column is the top end of the spectrometer. And the last two are the limited range or lower cost spectrometer. So what you're seeing here is we are predicting different properties from soil carbon clay content pH CC of the soil the sand content of the soil and the exchangeable calcium. And what we are comparing the first two column is the top the standard spectrometers visible to me and refer that spectrometer. And the third column is the more low lower cost and the fourth column is a much lower cost spectrometer. Right. So and this is related to the error, but it's called the root mean square or the error of the model. So what we can see is that the first two, which are the standard spectrometer near infrared spectrometer, they will perform best, right, it will have lower error in predicting soil carbon clay content pH and so on. But the two, the two spectrometer, which are lower cost and but a limited range also perform as well, not not as good as the standard spectrometer but also can perform in some cases as well as the, the full standard spectrometer. So in terms of measurement of pH, they are almost perform as well as the full range spectrometer, for example, exchangeable calcium and also sand and CC they perform as well. Right. So that means that instead of using the standard the high of the end the one that is very expensive one, we can use a portable one that the small one that we can use. And lower cost so that we can use it in the field or in the lab for estimating soil properties. Immediately so we knowing that this one the one in the middle here the one called new spec. It's a lower cost spectrometer. We now can use it to as a tool for predicting a fertilizer recommendation. So this is working together with colleagues from the Indonesian in the Indonesian Center for Agriculture Development. They say they want to come up with a tool that they can give to the extension officers or the so that they can go out in the field and then measure the soil and then it will give them an indication about the fertility of the soil and for the fertilizer recommendation. So, the first thing that we do is that we calibrate the spectrometer the spectra with the soil properties that is measured in the lab. So don't worry too much about it so I don't want to highlight this the one on that is in this red box is that for clay pH organic carbon and total nitrogen we got the reasonable calibration 50 to 60% of the variance can be explained by the spectrometer. And in terms of the nutrients what we can estimate quite accurately is what we call as potential phosphorus or potential potassium that's all in other words they are total potassium or total phosphorus and also the how much phosphorus that is can be returned absorbed by the soil P retention exchangeable calcium exchangeable magnesium CEC and best base saturation. But the one that cannot be estimated accurately is available P or available K available phosphorus or available potassium will come back to it again. Later on but but having to know the basic soil properties having to know the properties of the soil that is related to how much it can absorb nutrients how much of is the capacity the CEC how much is potential available. We can develop a tool that give the tool to the extension officer that if this is the soil this is the spectra and then it will give you a estimate of the predictions of the soil properties. And using this kind of tools in in a in a in a mobile phone or a smartphone app, then you can do a fertilizer recommendation because over in the Indonesian Center for Agriculture they already have lots of trials fertilizer trials that fits this how much fertilizer and it's added and how much is the yield. This relationship already established that gives them an idea that given this information, we can use this infrared technology to estimate the fertilizer recommendation. This is for for farmer application. So this is one application that we that we work together with the Indonesian Center for Agriculture so that the tool is not just for our research but how we can translate from the research to application. And the next I'll talk about the near infrared for soil inference, because as you can see that the infrared spectrum of the soil. Although you can see that it's a spectrum, but it is sort of interpretable because it contains information about the minerals of the soil about the character of the minerals of the soil. About how much clay is in the soil. What we can do for research and for pedology purpose is that we can take this infrared and then go out in the field and measure it directly. So this work is by at Jones what we call as proximal sensing in soil profiles. So if we have a soil profiles, we go out and take a spectrometer and then we scan it and then it can give us an idea directly in the field about how much is the organic carbon distribution. And how much clay is distributed in the soil. In this example, this is in Australia, what we call as a duplicate soil, a texture contrast soil, you can see that on the on the on the surface soil up to 40 centimeter it's a sandy, but it hits this layer where it has a clay so infrared can give us directly in the field. So the actual amount of clay and the actual not the actual amount of the estimate of the clay and the estimate of the carbon in the soil but we can go further. It's not just a basic property, it's not just texture, not just pH, not just CC, but we can go further what we call as a spectral inference system that means that we have relationship that we, we, we know about if this is the texture, if this is the CC, then how much we can expect the field capacity would be, and so on. In doing that, we can also estimate the mineral composition that about what is the make up of this soil because, as we show earlier that the infrared contains information about the minerals composition of the soil so, for example, in this in this location in this soil, most of the top soil is sandy material so it's mostly quartz. And the clay that it has is a kaolin kaolinite clay. So as you go down the profile as it's increased the clay then you get a proportion how much is it due to quartz how much is it due to kaolinite and other minerals as well. In this example, the soil here on the B horizon has a calcium carbonate we can estimate that how much calcium carbonate it is, and the different types of clay that is this is not a pure kaolinite because it has a mixture with elite and and so on, a spec type. And the last one, this is a vertical soil, and then we can estimate that this the main mineral of this soil is a spec type and so on. So, using this infrared technology in the field we can be more certain we can get more information about the composition of the soil and about the, about the mineral of the soils. And finally, we can also estimate how about the quality of the soil, given this kind of texture given this kind of mineral, what would be the expected amount of solids amount of water that are available amount of water that are not available. And this is an estimate of the available water capacity of the soil, this is all based on the inference from the, which our colleague will talk about next week about the inference about how much water is in the soil given this, this, this, this input, this action and so on. In the summary, for example, if we want to use this soil as a function to grow crops, we can say that what is what can we learn from this soil set A, the top soil is sandy is a low available water capacity is low CC. And then by the, the B, B horizon 60 to 100 centimeters a heavy clay, but the heavy clay is kaolinite is not expensive. It's infertile that or it's not, not suitable for cropping potential, but slightly, it has moderate available water capacity has high calcium carbonate. The pH is neutral to high. And so it's quite fertile and the site see its clay, it's has a expensive clay, it is a spectacular clay is a moderate amount of calcium carbonate high available water capacity that is using this kind of information we can build a complete picture about the soil. So this is the second example. And the third example is, can we use this in the field, but not just over here we say that we dig a profile or take a soil core and then we scan it. But what we say that can we go out in the field and then if we embed the penetrate and bet the infrared in the penetrometer and we push it on the ground in the ground in the soil, can we get a direct measurement. And in this case that in Australian condition that we want to estimate soil carbon because a soil carbon is tradable in Australia. Here, we can give an example for different soils that with penetrometer, it goes it push it in on the soil and then it collects the spectra every centimeter and then each centimeter it predicts the carbon content of the soil. So we get a complete picture of what the distribution of the organic carbon in the soil using this instrument. So this is this is just an example. These are just examples of what we can take the near infrared spectrometer that is from the research base to the field doing application that is either for farmers or either for research or more for general analysis. So that's about near infrared and now I'll talk more about mid infrared for lab soil analysis. So the difference is that as we say before that the infrared is from 700 to 2500 nanometers. And then for for for mid infrared it's it's the other ways it should be from this way from 2500 to 2500 nanometers but it's usually expressed in terms of the frequency one over centimeter or the inverse of the of the wavelength so we won't worry about that too much. So what we can visually we can visually see that although these are different cells that the curve of the spectra or the for the near infrared is very small and the peak of that, each of this individual peak of this identified peak is very broad and very diffuse. But for mid infrared what is called it's the this is a fundamental molecular vibrations with well defined pics that within this big within this area, we can say that this is due to Calanite. This is quite, quite prominent, quite pronounced that this is the peak for Calanite. And there are some pics that's related to organic matter. So this pic is related to what's this picture. It is quite because it's a fundamental vibration mid infrared provides a well defined pics that so that it can provide more information about the soil solid. But the difference is that the near infrared is robust for field use. That means if you take it in the field and then you measure the infrared spectra will be clean. It's not going to be affected too much. Sorry, I mean that the spectra is not going to be distorted. The spectra is not going to be noisy because it's a the soil in the field is not supposed to you have a grounded and it can record the spectra because of the of the shorter wavelength. And it's suitable for field analysis but the mid infrared although it contains more information it can then the molecular vibrations fundamental pics, but it's not robust to fill use because it's quite sensitive to the environment. The surface roughness of the soil, and so on so it's not well suited in the field. Right, so that's why we're saying that mid infrared spectroscopy is suitable for lab analysis, that if you take your samples back in the lab, and then grind it and then homogenize it and then you use the mid interest in mid infrared spectroscopy or spectrometer, you will get good results for that will show you always share with you later on. Right, so this is just an example of the mid infrared spectrum where I say that it's very clear is that if you look at this peak here around 3700 this is quite quite prominent for kaolinite that you got this peak that is a triple. It looks like the crown of a king that triple triple is it's once you see that in the spectrum you know that this is kaolinite. And then you like you can see that this the shape is a is completely different from the killer that and for the memorial that or spectra. You can see the shape is quite different so they are fundamental fundamental vibration fundamental characteristic then you can see you can visualize except that you can visualize on your spectra. Right, so because of this fundamental peaks, then people have been questioning have been asking the question this is already 23 years ago in 1998, lesionic and colleagues in in South Australia they asked can meet infrared diffuse reflect the analysis replace all extractions. I mean if I take my samples in the lab and do the mid infrared diffuse reflectance analysis can I get the same results as if you take your samples and and do the lab analysis, I will going to answer that so this has been asked 23 years ago this has been researched 23 years ago, but the take up is still quite slow because I think, first is about the instrument about the which type of instrument about the instrument is still not widely used is still expensive. The analysis of the spectra which now I think it's not going to be a problem, because now you have a free software and which the, which there will be a cost later the year that will be given that you taking this spectra you can now do analysis that are that will be quite difficult 23 years ago, 23 years ago you make good use special software and so on. So before, before we begin about the soil, which soil properties can be used and which, which, which can be can be predicted. First I will talk about calibration and accuracy so this is one, one thing that he from FAO asked me that can you talk a little bit about calibration and accuracy, because there could be some misunderstanding. So what we have is the spectra. And what we want is soil properties. So the spectra looks something like this, although they are fundamental pigs that they tell you that this is claiming the role this that this is due to the alkyl or for the organic matter so this is organic soils and then there are some quarts and so on. But this are not, you cannot use that pick directly like in chemistry. So if you measure this this height of this peak or the area of this big lesson going it's not going to give you organic matter because it's a carbon content of the soil because the spectra of a soil. Because soil it contains a lot of different minerals and and organic materials and they interact, interact with each other so that is, it's not straightforward to interpret the, the pigs of the spectra. So we need to have some calibration that given this spectra. The calibration will calculate and then give you the properties so for example what is the texture what is the pH what is the CC and so on. To do that we need to go have this calibration function so how does the calibration functions. Right, so the calibration functions if it's a simple system for example, concentration of nutrient in water because there's no other. Information there's no other elements in the pure pure water then you can do a univariate calibration that means the height of the peak will tell you something about the concentration. But because the soil is a mixture of many other things many different things. Those pigs doesn't work why because it's a it's a combination even if this mixture of organic matter and different and clay it will have interaction interaction between the elements the soil have lots of elements, which will see about it's not going to give you a pure response or pure smooth. So we need to use the whole spectra but what we can do we need to use the whole spectra and use a calibration model to that will tell us that if this is the spectra this would be the clay content and so on so which is called a multi varied calibration. It's a multi varied it's not just one peak or single peak but the whole spectra and the whole spectra it's a people call it ultra spectral. Why outer spectral because the whole spectra is about 2000 variables right so in a in a in a on the drone or on the plane people call about hyperspectral hyperspectral image hyperspectral image contains around hundreds of bands. And multi spectral in the Lancet the satellite images contain multi spectral image, because it contain multiple bands for bands five and 11 bands or 20 bands but hyperspectral contains hundreds of bands but in the lab analysis lab and I am lab and I am we got ultra spectral you got thousands of wavelengths right, which are difficult to handle if you're using a standard linear regression models because you can use a linear regression models then then it's just a matter of what's your soul variables your interest and this this spectra you multiply by regression coefficients then you get some kind of calibration functions. But this kind of model doesn't work because the spectra as I say we got thousands of variables. Sometimes you got more predictor variables than the more than the samples, for example, we can get 2000 spectral values, and you only have 100 soil samples. So the linear regression breaks down. And in addition that the spectra highly highly correlated that means between one band to the other band. If I'm here and I'm here, they are correlated that means that the really the conventional linear relationship breaks down it doesn't work. So we need to do some kind of treatment to the spectra one solution some of the solution is called variable selection some of it reduction dimension reduction. And the one thing that people mostly use is the principle component analysis you might have heard about it. That means that this spectra if we multiply by some transformation matrix, it will give you a scores that means that this the spectra is now become variables that are reduced in dimensions but then they are not correlated and can be used in directly in modeling. So this is one way but there are different ways that we don't, I can't explain everything to you but I'll show you some examples of the results and one of them. One of the standard way that people call this partial v square regression, we won't go too much about it, but it's just taking the spectra and then it does some transformation about the spectra and give you the prediction. Right. And since it gives you a prediction you need to know about how accurate it is how accurate is a prediction it is. And that's why we need to do some accuracy measurements we need to plot. What is the measured value versus the predicted values. We need to measure some kind of statistics. The statistics that's in in the literature is called root mean square error that means how much error we expecting in the near infrared literature is called standard error of prediction, which maybe in statistics is kind of misleading but that's what people call it in the near infrared literature. It's called SCP standard error of prediction, don't ask me why but that's it. And the other one is about the mean error or bias about how much bias is the prediction so this is just an example this to is the measure this is the predicted. So this is one to one line so this is the ask where it's a point eight three two, I'll talk more about that ask where, but there's no bias because it's almost, if you plot one to one line, they are fall fall, mostly around the mean, but this one. There's some bias that means that all the prediction, all the measured values predicted higher than what it should be it has some bias on it. So this is what we call as a bias. And again, if you see that the ask where maintains the same that means that the linear relation there is a real linear relationship, and the linear relationship explained 83% of the variance but that is a bias. We need to take account of this bias in the prediction, while the root mean square error tells us about the spread, the spread of the, the error so this is measured value predicted values. So if we assume that it's normally distributed, then around four times of the standard error or we've been four times as proven square error will be two times standard deviation plus or minus this will be the spread of your, of your error which will, we'll talk about it later. So don't worry too much. This is not about statistic and then ask where I'm sure you've heard about ask where that is how much of the coefficient of determination that is how much variance is explained by the model. But so there's other prediction quality that people say that because the ask where if you compare these two, it will give you the same ask where but one is bias, it's not going to the ask where is giving a true indication, then people call it a concordant correlation coefficient, which measures the degree of the correlation. One to one line. So this is just an example that the Pearson correlation is 85% and 85% but the concordance this one is 81 and this one is 62 because this one has a slight bias towards it. So won't talk too much about it, but the one thing that that is that he asked me to tell to make a point to you that that when you read a literature when you read a paper, if you just rely on ask where you're going to, you can get a, you can get not true relationship unit to see the relationship and this is just an example that that famous statistician wrote that this for relationship or data set they all have the same mean the same variance and the same correlation. The ask where is 0.67 but obviously you can only trust the first one but not the other the other three. Right. So, so the, don't just trust the ask where value when you read the literature when you read the paper that says the ask where is such and such. If you look at the book look at the relationship before you, before, before you make a conclusion, right, and ask where because ask where the formula is that it's, it's one minus the residual over the variance of the data. So if you want to increase the ask where what you can do is that you can just increase the variance of the data that means that you'll, you'll have a better ask where this is just an example if this is a random value see the ask where it's just a point three. There's no relationship, right, there's no relationship, but if they are too highly leverage point highly this is the same data set this is the same data here, zero to three, but if you include some two high values here, you suddenly get a very high r square which is meaningless. The, what you need to do is that beside the ask where values beside the concordance very you need to look at the relationship you need to look at the error. You'll see that the error, this is point seven for the error, this is one point six, right. So the error ask where is, it's just one of the measure that you have to be careful about. So that's, that's the message. So, and now we're going to look at the mid infrared. How can we use how accurate is this mid infrared for measuring soil properties. We need to go through few soil soil properties and extensively soil properties and we can discuss about it. So as we say before, to make a prediction, we need a spectra but also we need a standard lab measurement, right we need the spectra and also let standard measurement to do this calibration functions. Fortunately, our colleague in the USDA, so survey laboratory. So they have a archive of more than 1717,000 soil profiles from the US with well documented and precise standard operating procedures, a lot of soil analysis. So they have the mid infrared spectrum they have the mid infrared spectra of each of those soil profiles, and then each accompanying it is the soil analysis of standard lab analysis. So with this uniform, with this uniform techniques in the laboratory and spectra we can know now we can be assured that what we're getting is not due to the random chance or due to and so on. So to do that we use a method called memory best learning or it's a local PLS method don't worry about it's just a method. So we take this the soil sample the whole 17,000 sorry, it's more than this around 50,000 samples. And then we divided 75 as a calibration 25 as a validation that means that 75% is going to be used to build a model and 25% is not going to be using the model so. So that we don't have a chance that we get to pick a good one or pick a bad one we repeated this procedure 10 times so that we can get a clear understanding of what it is right. So the first techniques also that we we try is this called deep learning. It's called a convolutional neural network so it's a new technique that takes spectra and then sort of do a scanning and then fit it into a neural network and then predict the functions. And CNN is mostly used in a in a image analysis is just like how Facebook recognize your friend. So that asking this how can that that algorithm, given the spectra how can it recognize our soil. So it turns out it did performance quite well. So in terms of total carbon organic carbon CC clay sand and PS. So this is around. So this is calibrated around 30 something 40,000 soil samples and it's validated around 110,000 soil samples right so this is a this is quite a complete data set. But it's it contains a different a range of a total carbon from zero to 10% organic carbon the same CC from the load to a high CC and from clay from zero to 80 plus percent and pH from around three to around nine. So it contains a huge range of values, because it was collected for the whole of us contains a range of soil and what we can see that we can get an R squared. Although I say that be careful when you use R squared. But if you look at the plot on the right one, the right head corner one the one that is called the CNN deep learning one. We can get an R squared or point nine five point nine six point nine seven point nine eight that means using a standard method of a mid infrared and calibrated against standard standard measurement we can get a very accurate measurements of a total carbon organic carbon and the organic carbon CC clay, sand, silk and pH directly. So that is a big plus right. So that's the basic properties that we are we know that the spectra is responding to but there are also different other properties that can be that have been mentioned and then we ask, can it be predicted. Again, the US laboratory they have about 200 soft physical chemical and biological properties. So we run some simulate modeling and then we want to figure which one is well predicted and which one is not well predicted. To do that we group it into four categories of assessment ABCD. So this is just for the so that we can understand it later. A that means that the performance to ask where it's very high. It got very low bias. It has high concordance and so on. So a and the the second rating is be it also get a very high R square. And the C is a moderate R square and the mean R square is around 0.67 or 0.7. And then the D one is the R square is lower around 0.4. Right. So we say ABCD so ABC is a high quality performance. C is the medium quality and D is low performance right so just remember ABCD and we'll go through it. So for so chemical properties before we begin we have a hypothesis that we say that properties because infrared what you're doing is getting the soil and you have infrared signature characterised trade, because it has to do with the mineral components with the surface chemistry of the soil. It will be well predicted, but properties that are related to a soil solution or so extraction chemistry should not be well predicted. Why because it's a characteristics even for the available P. For example, Bray P, Olson P or other types of extraction you still don't have a good correlation. That means that the soil it's not related to the soil surface because the infrared is only looking at the surface chemistry. And then elements in the higher concentration and related to the soil minerals should be well predicted. So this is our first, this is our hypothesis. And first we look at what is called the melee extraction, which is a mild acid that people use that with this extraction we can analyse a lot of elements. What we see is that elements that such as calcium, aluminium and magnesium and barium, because it is responding to the chemical element of the soil mineral albumin and is in a higher concentration is in high concentration it is well predicted because it is based A and group A and B, while the others like silicon, potassium, silicon and potassium is not well predicted. What we see is D and but like an iron sodium and magnesium is only mildly predicted so in other words it's mostly related to the range of the data. That means if it's in high concentration, such as this one, the calcium, aluminium, magnesium and barium, you can get well prediction because it's concentration is high enough and also it's related to the mineral content of the cell. So melee extraction is an extractor of the of the soil. It's only related to the high, the one that is with a high concentration. And for phosphorus, people want to know what's the available phosphorus. Unfortunately, available phosphorus such as water soluble phosphorus, very extractable phosphorus, all since extractable phosphorus, mainly extractable phosphorus cannot be predicted well because it's related to the soil solution, the extraction of the soil. But what can be predicted very well is the phosphorus retention or phosphorus retention capacity. That means that the measure of how much phosphorus that can be absorbed or retained by the soil, which is very useful in some tropical soils with a high iron and aluminium oxides. This is an expensive measure but mid-infrared can predict it very well. So phosphorus absorption, yes, but phosphorus in the solution, no. And then we look at the elemental concentration or the total concentration, total element concentration of the soil. And what we can see that it's quite amazing that the mid-infrared can characterize most of the abundant of the soil. For example, like something that's abundant is to do with aluminium, potassium, sodium, silicon, vanadium, burlyium, which is a minor trace elements but it's related to the mineral. The major element, potassium, aluminium, silicon, so this is the total concentration of the soil, well predicted, but also magnesium, iron, so the one that is coloured the red and the green, they are all very well predicted. And some of the ones that are not well predicted are the ones that has a low concentration, such as antimony or tungsten, which is low concentration that's a very tiny amount and is probably not related except that it's probably related to contamination and so on. We can see that a huge amount of major element, a huge amount of macronutrients, total macronutrients can be predicted reasonably well. And some of the heavy metals like nickel, copper, lead, cobalt and tin, even though these are not contaminated soils, these are natural agricultural soil, mostly agricultural soils and natural soils, they can be well predicted. And we look at silica, for example, this is silica, if we look at the prediction, using mid-infrared and we predict, we compare it if we just use the basic soil properties that we know is related to silica, the clay, sand, we can see the pH of the organic matter, we can see that we get much better information from the mid-infrared because the mid-infrared also contains information about the mineral of the soil while this one is a sort of indirect prediction. So mid-infrared itself is a good predictor in silica, now there's an interest because it's found that in some plants, especially rice, this is a very important macronutrients. But in terms of, again, in terms of extraction, this is in terms of saturation extract, electrical conductivity and all the elements that are in the saturation extract, we can't predict it because it's to do with the extraction method. It's something that mid-infrared cannot predict and organic matter, different types of organic matter, total carbon, organic carbon, inorganic carbon and carbon, labial carbon due to a potassium permanganate extract, we can predict it very, very accurately. The last square is 0.92 to 0.97, total nitrogen reasonably well. The particulate organic matter, which I'll explain later, is also predicted well and glucose, the beta-glucosidase is an enzyme activity in the soil, we can also predict it reasonably well. So something that we found that is not well predicted is still a sulfur, we still don't know why we can't predict well the sulfur. And nowadays, it's not just about total carbon, it's not just about the organic carbon, but different forms of organic matter. And if we differentiate it, there's a particulate organic matter that means that small organic fragments that has been decomposed less than two millimeters, but it's still not incorporated in the mineral of the soil. This is called particulate organic matter, and the lifetime is shorter because it's not protected, it can be consumed by microbes and so on. There's a mineral associated organic matter that is organic matter that is complex by the mineral, and then there's a resistant organic carbon such as chow or other organic carbon that has become resistant due to the process. Now people have sort of tried to differentiate it into particulate organic matter, mineral associated organic matter, resistant organic matter. We can have evidence that we can use it as well using mid-infrared, this kind of not just the total carbon, not just a total organic carbon, but also the different types of carbon, the particulate or the mineral associated and the resistant organic carbon can be well predicted. And finally we'll talk about soil physical properties. So the proposition again is that properties based on soil solid composition and surfaces can be should be well predicted, but if it's post base relationship it shouldn't be well predicted. We'll talk about it later. And people want to know, well, can it predict bulk density or not? And the answer is maybe an indication, but we have to realize about what is mid-infrared actually measure. So this is a field, this is going to affect the bulk density, right, about the aggregation, and in the mid-infrared lab analysis you take those soil and crush it and make it into this palette and scan it. Does it make sense that it can predict bulk density or aggregate stability? Probably not. But what it can give is an indication of that given this type of mineral, different types of clay and different mineral, this would be the likely bulk density. So although the R-square is 0.71, what we can say the error is 0.11, that means four times of this, that you're probably looking at an error about 0.4 or 0.5 of a unit of bulk density. So you just want to use it for a rough estimate that this is the given soil, what would be the approximate bulk density. Probably you can use it or estimate the bulk density and also aggregate stability in this measure is not that well predicted. I think it's to do with the aggregation. And the people what want to know is can it measure field capacity, can it measure water retention, can it measure available water capacity? It depends, right, so we have to know a little bit about soil physics. So when we measure field capacity what we say is, sorry, what we say is that this is to do with clod when we want to measure water retention. That is a wetter than one third bar or 33 kC, or PF2.5 in some measurements, is that we need to use clod because these are water that are available in between the aggregates. If you use crush samples or sieve samples you get different results. And you can see in this lab result we have clod data that are measured with clod and data that are based on sieve and dry samples, right. So if this is sieve and dry samples that look similar like this, we can have a high accuracy in measuring the water retention at 0.06 bar or 6 centimeters, or sorry, yeah, 6 centimeters, this is 100 centimeters, sorry, 60 centimeters, 100 centimeters, 300 centimeters. We call it the one third bar, the field capacity in the US. It's very accurately, but this is remember this is a sieve samples, right, sieve samples, but if it's a clod sample that looks like this, and you crush it and you measure it, the accuracy is relatively low. That means that it's a sea level, accuracy at the sea level. Similarly, if it's water retention at a wilting point, which the clod doesn't, you can use sieve sample, you can measure it very accurately, right, so depend what is, what is your purpose, right. So again, this is an example of field capacity, reasonably R squared, but if you like to look at the error, so the error is around 9%. If you look at the field capacity, the relationship is tighter, the error is 4%, so depend how much error you want to, if it's just a, if it's just a petro transfer functions that you are asking, you can probably use that. So in other words, one scan, this is all come from one scan, one scan, one sample you take the mid-infrared and it can give you all these properties, microbial, biomass, particulate organic matter, and so on, pH and so on. So one spectrum, you can get a lot of different properties that are well estimated and what we calculated that's about 50 soil properties that are well estimated and there are 50 properties that are reasonably well as reasonably estimated. So that's a good thing that means that you can, once one spectrum, you can do a lot of assessment, for example, for crop production or soil fertility, knowing the texture minerals and calcium carbon and pH will give you about the condition about the soil and about the soil carbon or pH that's the effect of management. What we can't do well is that we can only give you an estimate about the density and plant available water, what the MIR cannot do well is give you the effect of management but it can give you an indication about the physical condition and the ideal condition. But still, looking at if you want to assess it for soil production or fertility about nutrient cycling, carbon storage and so on, all these properties that are estimated from the single spectrum of the mid-infrared can give you a lot of information. So that's about it. So in summary, mid-infrared offers a rapid and highly accurate measurements of many physical chemical properties. So we counted that's about 50 properties that are well estimated and 50 other 50 properties that are reasonably well estimated or mediumly estimated. So it's related to the soil mineral components, surface chemistry. It is well estimated. If it's related to the extraction chemistry available nutrients, there's no reason why it can be predicted well. You can see it as an alternative to petrotransfer functions. And about the detail about how you process from the spectra into properties. This is in the book, a bit of promotion of course, but there will be a course later training course offered by there will be available at the FAO website very soon. And this is some acknowledgement to my colleagues in Indonesia and in the US for the collaboration. And thank you for your attention. Happy to take questions. Thank you very much. And thanks for this great presentation. Give us a very comprehensive review about using the soil spectroscopy to measure the different soil properties. And we have, we had quite a lot of questions and answered by colleagues from your team and totally more than 50 questions already. And we also saw some interesting questions would like to invite you to answer it live. The first question is the soil profile in many, many slides show a higher resolution in terms of the depth point on the order, or few millimeter. If I'm not wrong, how these readings were obtained in shoot or at the lab after collecting samples from the field. Okay, so I think it's it's not every. So it's not every millimeters. I put the question in the chat box so you can read it again for sure. Okay, thank you. So I think it's it's I think, maybe, maybe it looks too high risk, but it's not, it's not every centimeter. So the one that's on the profile I think it's every five centimeters that are that we do it in the profile. But now, using the penetrometer, which my, my colleagues have done it's a you can do it a continuous measurement so it's taking measurement every one centimeter. So manually manual, if it's manually. It's very tiring to scan every second, but now with the with the with the penetrometer automatic collection of penetrometer. So we can get measurements of every centimeter of the spectrum. So the second question I just posted in chat box would be possible to estimate the concentration of the soluble component in extracting solution. I think I think yeah I think that's one one possibility so because it's it's, if you can measure. So if you can measure, if you can measure the concentrations of the solution the extracted extracted solutions as well I think that could can be that that can be an option as well. Using different, different, different kind of. That's a different kind of spectrometer because that's to do with the liquid liquid, the transmittance of the. Yeah, I think this is the last question and we found it's quite interesting can be very interesting for many audiences about the estimating soil function. No, I don't think so. Okay. Why should it be. I mean, I think it's a, yeah many. Yeah, I think it should be, I think. I think it is to do with the biology, what, what, what biology signal because the biological signal of the bacteria or the, the, or, or fun guy or other other other organism. I don't think that the spectra can, can separate out. Thanks. I think that there are some new questions coming from QA box, Woody, please feel free if any of them you, you feel good to answer. Yeah, so one of the slide I think one of the slide I said, I said that these are the qualities but my chorizo is the one that cannot cannot be well cannot be estimated. So there's a microbial biomass. I think microbial biomass either they have different studies already that show that it can be estimated, especially the, the, the, the total biomass, the total microbial biomass the PLFA. I think there is a paper that already show that it's reasonably well estimated. And then the, and then it's RPD a good option. There's no, there's no one, there's no one measurement because RPD is exactly the same as R square so there's no one measurement that I think the, the, is that don't just pass one, one single index but look at, look at, look at the, the different instruments about the error about the, about the distribution and so on. Could we get a state areas but about the instruments, I think the instrument doesn't, it's not affecting too much I think if you're using one instrument that it should be well, but converting from one instrument to the other that's another, that's another issue because it could produce a different, different spectral signal but there are ways of trying to, trying to handle that. So, can we say, and I have all the information which is included in the visual and I would say yes, most, most probably, except for the, for the color which are, which the mid infrared that's not measure. Oh, that's the number of points affect the test for accuracy. Yes. Of course, you've got more points more data will get more, more representation and more accuracy. So how, how do we define a good prediction model so I don't think there's a one universal or one universal, some people like to say, or if we ask is greater than 0.6 or 0.7 then you get a good model but it depends I think it depends on the context as well. I mean, we shouldn't try to say that if it's the R-square or RPD greater than this then it's excellent or it's good if it's less than this is, it depends on the error and the uncertainty and what is your application. It's rapid but much time to spend in processing the data though this potentially negate this. So the mid infrared I think in the lab, I think one of the limitation is that we haven't, we didn't talk about this that the infrared is, it needs to be in the lab in the dry condition in the standard condition and it will perform better if it's also a grind in the finer, finer, finer soil. Yeah, there could be a trade off because the processing the, but I don't think the data analysis nowadays I don't think it's a, it's a limitation, because nowadays you can program it and it's it could be available so any any spectra is, it shouldn't be a problem. I think there is a continuous question, Tommy. So what is the preferred method of data calibrating and. I think that one, Alexander is responding by the text. Yeah, we can go next maybe. Okay, do you think MIR is going to replace this MIR future proximal sensing tool. I think it, it, I think the MIR is still has problems to use in the field I think it's still affected too much of the, they are portable mid infrared but I think it's, it's still. It's for soil is still challenging. Yeah, so there are values of using in this MIR in the field and there are values of using mid infrared in the lab. Do you need to build a regional calibration. It depends on your application so if you think that you want to use it as a as a lab that you take sample from the whole region I think it will be, it will be a good idea. But a lot of places, a lot of, even a lot of commercial labs. When they analyze the soil samples, they don't throw away the samples they keep it at least one month right, because of for for checking or for other issues I think if you're working with the if you're working with a commercial lab I think that's a that's a good resource that you can type into them that you can ask them to, to give them samples. They already analyzed that people already pay for them that you can build a library. Okay, I guess this will be the last one. The last one about ESP and EC. Yeah, yeah I'm not sure I think. I think most of the most of the data that we look at it's it's not. It's not well but maybe because of ESP they are because it changed some of the properties and if in case of extreme I think it could be in some cases it could be so it's it's not universal but in some cases. In some cases where I see that ESP and EC are predicted. Okay, okay. Thank you buddy. And in the last I just want to add that because from the chat box, and then many colleagues asked about this book, and I would like to say I would like to mention one more time is this book. It's not completely open access. So if you can find a way to borrow this book from your library or your university or your lab or can, can, can pay for this book, we will encourage you to use this book. Otherwise, we are currently working together with the University of Sydney and the, and the, the big team with the boogeyman and Alex, and we are currently recording some video course based on this book, and so give a training for the spectrum modeling using our program. And this video course will be online in our Glow Sloan world page by end of this year so everybody will benefit this video course and repeatedly, repeatedly watch this video course and then learn how to use our program for the spectrum modeling. So, a very big thanks. Thank you to our today presenter. And the, the today I think we reached almost 400 participants to join the second webinar on soil spectroscopy. My colleague is now posting the link to the other four webinars scheduled on soil spectroscopy and you all invited to join and register. I would like to check this page irregularly as another series of webinar on what chemistry health and safety equipment person quality assurance and quality control and the laboratory management will be organized in next few months. And the certificate will be issued, and it will be sent by email automatically after two weeks after the webinar so please be patient. Dear participants, the recording of the webinar will be shared with you all together with a small report and the presentation of our speaker of the day. Thank you all once again, and I wish you all a pleasant end of the day, or even. Thank you, Woody. Thank you. And all the panelists are helping with the questions. Thanks. Thank you. Bye.