 This is the Lesson 4 lecture and this is about pixel-based and object-based image classification. The objectives for Lesson 4 are, define the eight classic elements of image interpretation, explain the seven categories of tasks performed with image interpretation, apply the elements of image interpretation to data collected from a variety of sensors, construct an image interpretation key, perform pixel and object-based supervised classification using ArcGIS Pro, create a workspace and project in eCognition and execute rule sets, and apply segmentation algorithms to remotely sensed imagery in eCognition. So therefore, basically the objectives are to get into the idea of image interpretation and pixel-based image classification, a pixel-based supervised image classification, and then to transition over to object-based supervised image classification, and that will be our segue into object-based image analysis. So this slide highlights the fact that high spatial resolution mapping needs are not met by the free moderate resolution satellite data that is easily available to us. So in other words, in moderate resolution is about 10 meters to 30 meters, 10 meters for Sentinel, 30 meters for Landsat, and that is not sufficient to do for example urban mapping where you want to map the roads and subdivisions and so on and so forth. And so the two graphics on the right of the nape image on the left, so we have a one meter nape image on the left where we can see the features on the ground very clearly. If you look to the on the right-hand side on the top, you see a pixel-based classification from a Landsat image and you can see all you can discern is the presence of the woods, but that's about it. You cannot discern the line features, the roads, the subdivisions, and so on and so forth. So here's another case where we have a comparison of Geobia-derived high spatial resolution map on the left, and this is derived from using one meter ortho imagery and LiDAR data. And on the right, we have a map that is derived from a Landsat image. And once again, you can see that all you can discern is the vegetation patches, the patches of the forests, but you cannot see the individual buildings and the roads and so forth. And typically townships and small cities and large cities need these high spatial resolution maps to create value-added GIS products. And this once again points to the efficacy and the power of object-based image analysis techniques. So this slide highlights the comparison between a pixel-based image classification and object-based image classification such that on the top row, you see the nape imagery that was used for this classification. So the nape imagery has a one meter resolution. And then look at the, in the second row, the pixel-based classification and you can see the salt and pepper effect on it. If you look on the last map in the pixel-based classification, the last map, which means the one that is further to the right, you can see that salt and pepper effect in the woods, in the forest class. Whereas if you look at the object-based classification classifications in the third row, you can see that the maps are much clearer, cleaner, and do not have the salt and pepper effect in it, which is a distractor. So what is Geopia? It stands for once again geographic object-based image analysis. It is essential for analysis of high spatial resolution imagery. The recent advent of high spatial resolution remotely sensed data like multi-spectral imagery and LiDAR has shown limitations of traditional pixel-based classification methods as you will examine in lesson four. High spatial resolution data is now becoming available free. So for example, now we have nape imagery is available for the entire nation free. Increasingly, we have these high spatial resolution satellite constellations in the commercial space that are crunching out terabytes of data and that in special cases they will give you free imagery for education and research purposes. And the amount of this imagery available for both teaching and research is going to keep on increasing in the coming years. So what has happened is that in the past couple of decades, in the past few years, we have become data rich and technique poor. So Geopia methods can simultaneously analyze data from multiple sensors with data of varying resolutions and incorporating physical and social vector data. And along the same lines, Geopia literacy has become an important workforce development issue, a workforce for remote sensing applications because the remotely sensed data sets are converted to actionable information and these remotely sensed data sets that are becoming available at higher spatial resolution, higher spectral resolution, higher temporal resolution are massive and they all need to be converted into actionable information such that this information can grow legs and walk in a public policy sphere and lead to betterment for all. And with the advent of Geopia techniques, the elements of image interpretation have once again become very important and as we will come to see, provide a very essential framework for object-based image analysis applications in which we try to replicate the cognition of a human being based on these elements of image interpretation. So data is not equal to information. So you can have stacks of data, layers of high spatial resolution data that needs to be then converted into information which is where the data becomes very useful and you can begin addressing questions like how much tree canopy do we have? How much tree canopy do we have per parcel? Or how much more tree canopy can we potentially have per parcel? How much impervious surface do we have per parcel and so on and so forth and all of that can only be done effectively with high spatial resolution data and object-based image analysis becomes a critical tool to convert this data into information. So here's another slide that kind of highlights this idea of conversion of data to information that data plus OBIA is equal to information that object-based image analysis techniques are critical to converting this data to actionable information. So we have a NAPE image over here and this NAPE image was converted into a land cover map using actually both NAPE and LIDAR data that gave a very accurate high spatial resolution map which was then queried using the property parcels that were available as to how much tree canopy do we have per parcel? How much impervious surface per parcel? How much additional tree canopy we can have per parcel? And Charlotte O'Neill Dunn has done a lot of work in developing these metrics that I have shown to you on this slide. So going back to elements of image interpretation, here's a picture and what do you see? And if you stare at it closely enough, different people might interpret this image differently. And so I'll just let you look at this image and make your own judgments. Do you see a snow field or do you see horses stand in there or do you see both? So go into human cognition. The word cognition comes from the Latin word cognizir, meaning to know, to conceptualize, to recognize. And the human faculty for processing information, creating categories, applying knowledge, and changing preferences is very special, is what makes us human. And cognitive processes can be conscious or unconscious. And going to human cognition and image interpretation, we can see that anytime we are looking at an image, our eye is automatically assessing the proximity of the objects that it beholds, the similarities, the continuities, like for example in the case of a road or a river, symmetries that might exist in the picture, and other elements of image interpretation. So how do humans see and classify objects? So basically the sense organs gather and sort the raw data. So we've got rods and cones in our eyes. So basically each one is going to create a pixel of data that captures the brightness and the color. So we basically have a pixelated image in the back of our retina, which is upside down, and the brain processes, filters, and classifies the data and presents it to our conscious mind with information. If our consciousness were presented with everything that our senses collected down at the pixel level, we would be inundated and unable to function. So basically the millions of pixels, let's say, lots and lots of pixels are rapidly converted by the brain to a few objects, and the information presented to our consciousness is qualitative, and our brains work from this complexity of the incoming information to abstraction where it puts it in categories. So remember that our brain creates these categories and land use and land cover are categories as well of phenomena that occurs on the surface of this earth. So what our conscious mind sees is determined by the categories that we have created ourselves and the categories that we are aware of. They may be categories that we are not aware of. We tend to find what we are looking for. Forrests, once again, a generalized forest class can have subclasses and can be alpine or bottomland, wild, managed, old growth, regenerated, or the back 40. Trees can be a forest, tree farm, orchard, windbreak, arboretum, or your backyard. A tree can be coniferous, deciduous, ornamental, native, invasive, or your favorite shade tree. So what this highlights also is the importance of geontology, which means standardizing definitions of observable geographic features, particularly at high spatial resolution. And what this standardization does is it allows for a seamless sharing of workflows in the community of remote sensors looking at particular problems. So once again, how does our brain go from complexity to abstraction? Abstraction being the objects, the meaningful objects that we perceive, for example, in a high spatial resolution image. And when we look at an aerial photograph or the world around us, we are presented with great complexity in the scene that we behold. And we are naturally wired for our very survival to make abstractions from the images that we see around us. In other words, we are constantly classifying the imagery that is coming into our senses. And we are also conditioned by our experiences, our past experiences, our surroundings, and even by our training as Earth remote sensors to make certain kinds of abstractions. Now bear in mind that qualitative concepts are difficult for computers. If a computer, let's say, is looking at the image to the right or the left, it cannot tell where something is not right here, where a human being can say, well, wait a minute, something is not quite right here. And that's because a computer, once again, cannot deal with qualitative ideas, but can deal with quantitative ideas. Once you have classified the image, now the computer knows what every pixel stands for, what type of a land cover does it represent. So how do computers see and classify objects? So computers see digits and process quantities, numbers, and computers require rules or software or programs to process these digits, these digital numbers that imagery is made up of, and image objects are made up of. So these rules compared to human cognition are relatively simplistic and deal with specific quantities employing mathematical or logical operations. This will become clear as we develop rule sets in e-cognition based on an interpretation key to classify high spatial resolution imagery. Again, so what is Geobia? It's an approach to image processing which employs an attempt to make image processing more like the human cognitive process. And it begins with partitioning remotely sensed imagery into meaningful image objects or segments based on their spatial, spectral, and temporal homogeneity. And then there is the attribution of properties, quantities, characteristics to image objects based on the parent remotely sensed imagery as well as other data sources. And then it is followed by the classification of these image objects using rules which analyze object attributes, relationships with other attributes, with other object attributes within the image or relationships with objects external to the object under consideration or other objects internal to the object under consideration. So the complexity to abstraction process for Geobia workflows involves the following. So first you need to have an idea of the classes that you are trying to extract from the remotely sensed data and therefore you need to have a class hierarchy. They need to be the major land cover classes that you're looking for. And then there will be subclasses as well. For example, a forest can be divided into deciduous and evergreen. And then each one of those could be divided into the different species of the tree stands that exists and so on and so forth. So delineating meaningful objects or segmentation is the next step and is a very important step in object based image analysis workflows. Then there is always some image the object exploration in which you get an idea of how the object characteristics are quantified for objects. What's the mean blue reflectance? What's the NDVI for an object? What's the standard deviation of the right for an object? And so on and so forth. Then you select relevant attributes for categories and categorization. That means to say that is the process that you go through in developing certain characteristics that define particular objects and they get encoded in the rule set. And then there's also morphological operations that you can do in e-cognition even though this is an idea that is at the intermediate or the advanced level in Geobia and that is reshaping objects, merging similar small contiguous objects into bigger objects, growing objects by attribute thresholds, breaking big objects into smaller objects and shrinking objects by attribute thresholds. Then objects are assigned to a category, let's say a land cover class. Then there's some filtering of noise, some cleaning up, they are cleaning up algorithms in e-cognition and then the classified map gets exported for further analysis to a geographic information system. So here is a more formal definition of image segmentation. So commonly the term segmentation means subdividing an image or entities within an image into smaller meaningful partitions. And yet segmentation is any operation that creates new image objects or alters the morphology of the existing image objects according to specific spatial, spectral or textural criteria. This means that segmentation can be a subdividing operation, emerging operation or even a reshaping operation. Here is a graphic that shows you the process of segmentation. So we have an image of a painting of Mona Lisa and we have an image of it and you can see that it was segmented using a particular criterion and then you can see the segmented image that is of a lower resolution than the original on the right. Always keep in mind that well-defined image objects or segments have an objective meaning that they comprise what are known as meaningful objects. So in this example, objects correspond to a desired subset of the overall image. Like for example, the skin objects, the hair objects, the objects comprising the eye, the objects comprising the smile, the background shadow and so forth. In eCognition software, perhaps the most effective segmentation algorithm is the multi-resolution segmentation algorithm and it works bottom up. In other words, it begins from each pixel and grows outward. It consecutively merges pixels based on spatial and spectral homogeneity and the resulting size of the segments is based upon what is known as the scale parameter and the shape of the resulting segments is based on the shape, color or what is known as the compactness parameter in eCognition. There are 30 computer-intensive algorithms and can be somewhat slow and memory-intensive, but as time is going by and more computing power is becoming available and is getting to be cheaper, it works very well on contemporary laptops and personal computers. It yields very good image abstraction. The meaningful objects such that if you had a geographic scene as you will come to see, it can break out the roads and the forests and the built up areas and the pastures and the agriculture into meaningful objects. It can create slivers and noise sometimes and often can be used as a secondary segmentation process where the whole segmentation is done with the chess board algorithm or the quad-tree algorithm and this will be clear as you start working with the lesson for lab activities in which we are going to do some object exploration. The image segmentation parameters in the multi-resolution or the MRS segmentation algorithm in eCognition is as follows that you have three of these segmentation parameters. The first one is the scale parameter which begins with a value of 10 and larger the value of the scale parameter the larger the object, so it depends if you want to capture the individual three crowns as objects you'll have a smaller scale parameter if you want to capture the entire forest one object then you're going to have to have a larger scale parameter and then you have the other two parameters for multi-resolution segmentation as well the shape parameter which is related to the spectral homogeneity of the object and the compactness parameter which is related to the shape homogeneity of the object in the ArcGIS Pro we will mainly utilize the mean shift segmentation in fact in ArcGIS Pro as of yet there's only one segmentation algorithm available and that's the mean shift and within which you will have two different parameters to vary to get different types of objects and we will be doing an exercise in the lab in the lesson for lab activities in which we will examine these ideas by exploring both multi-resolution segmentation in e-cognition and the mean shift segmentation in ArcGIS Pro for example here we have a case where the multi-resolution segmentation operated upon the image of Mona Lisa and when the scale parameter was 30 which is relatively smaller then the image objects end up being the bright skin the shadowed skin, the eye components the mouth, the curly hair, the straight hair and so on and so forth whereas if you did a multi-resolution segmentation of Mona Lisa with a scale parameter of 500 notice that the image objects are much larger as they average over the spectral and the spatial homogeneities and the basic idea being the larger the scale parameter in e-cognition, multi-resolution segmentation the larger the objects just depends on what you are looking for. OBIA or Geobia also allows for multi-sensor data fusion so for example over here we have a lidar intensity layer then we have a color infrared nape imagery with four bands and we've got the lidar data, spatial data here as an NDSM and all three of these layers can be queried to produce a very accurate land cover map you can also have class hierarchies in e-cognition as well so you can have land cover classes and then you can drill into a particular broad land cover class let's say like forest and get into the sub classes as well such that you have a hierarchy of classes available to you in object based image analysis and after the process of segmentation is over in e-cognition you will be developing rule sets and these rule sets reside in the S tree window and here's just an example of a rule set that you will be working with in your lab for activity and the rule set will be provided to you. Once you have your rule sets developed then you get very consistent output as long as you are dealing with the same data layers of the same sensor taken at the same time you will get a very consistent output of your map products. The workflow that will be emphasized in e-cognition software applications in this course will be as follows bear in mind that you always have to do significant data preparation before executing a Geobia project and that involves image clipping mosaic in, reprojection and so forth getting all your layers in the same coordinate system is extremely important for e-cognition applications because e-cognition does not reproject the data on the fly like ArcGIS Pro does surface models are created if you have LiDAR data available such that you can get the normalized DSM on the NDSM the heights from the ground then rule set development on e-cognition on just a subset of the data and once you've got the rule set ready for a representative subset then that rule set can operate upon the entire image over the entire study area and then you get a land cover map that is further analyzed and accuracy assessment will be performed on it to assess how accurate the land cover map is. Upon executing the rule sets in e-cognition are image data that can be optical imagery like nape imagery along with spatial information from a LiDAR derived NDSM you make the rule sets operate upon this data and you get a land cover map and once you have a land cover map now the computer knows what is where and this layer can go into your GIS for further analysis. So here's another example of further value added GIS analysis of a high spatial resolution land cover map and in here the percent impervious surface per parcel has been calculated for the city of Gainesville for the year 2010 and this information is needed to estimate the storm water utility fee that is going to be instituted sometime in the coming future. Similarly the land cover map can be used to estimate the percent tree canopy by parcel within the city of Gainesville, Georgia as well and that's yet another value added product to look at aspects pertaining to urban forestry, how much forest cover do we have in the city and it turns out the more the tree cover the urban tree cover that we have that starts driving down the water treatment costs because trees act like natural filters to the water precipitation that occurs. In the lesson for lab activities you will conduct a pixel and object based supervised classification in ArcGIS Pro and this includes some object exploration with varying the two different segmentation settings that exist on ArcGIS Pro for object based image analysis and they are the spatial and the spectral parameters. We will also have an introduction to eCognition and the lab activities that you will do in lesson 4 and in lesson 5 will be based on an OBI tutorial created by Jarlath O'Neill Dunn for America View. Jarlath is a faculty member at Vermont and also teaches Geography 883 at Penn State. eCognition software will also be utilized in these lab activities in which you will learn to create workspaces and projects how to upload and alias the data and executing provided rule sets for segment exploration and for water classification in this particular lesson. So the rule set in this lesson will be provided to you and as a resource for introduction to eCognition 10.0 which by the way was just released this December so it is pretty much brand new. Please view the Geography 883 lesson 2 review and the lesson 3 preview lecture on Thursday the 21st of January in which I have gone over the general eCognition interface and how to create workspaces and projects. Please look it over before you start working on the lesson 4 activities. It will be very helpful to you. So this is what the interface for eCognition looks like and as I mentioned in the previous slide eCognition 10.0 is brand new and has several improvements in it. I believe that you will really enjoy learning object-based image analysis. It's a very essential tool to get off a contemporary remote sensor and we will be using eCognition in lesson 4, 5, 6 and a part of 7 and some of you may choose to use eCognition for your project in developing a OBI application. If you have any questions or comments please post them in the lesson 4 general questions and comments forum.