 In this video, we'll be talking about the process of self-organization in complex engineered systems. We'll firstly talk about why self-organization is a key feature and characteristic to these technologies. We'll then go on to discuss the basic ingredients that are required and the process through which these systems self-organize. Finally, we'll wrap up by mentioning how this process leads to suboptimal but often robust solutions. Firstly, just to refresh our memory of what self-organization is. Self-organization is basically the spontaneous creation of a globally coherent pattern out of the local interactions between initially independent components. There are essentially just two ways in which a system or technology can achieve the global coordination that is required for it to function. Either this design pattern is imposed on the system from some external force, or it emerges out of the interaction between the different components internal to the system, and this is self-organization. With the linear technologies that we previously talked about, such as bridges, chairs or toasters, the system is designed and controlled by the engineer and operator who imposes an external pattern of organization on the system in order to make it operate in a predefined, optimal fashion. This is our traditional industrial paradigm that we typically try to apply to all areas from governments running a country's infrastructure to designing residential areas and parks. With this centralized design, the components are coordinated through a linear top-down design pattern that constrains the components and requires them to function in a well-defined, specific manner in order to achieve the global functionality. This paradigm is so familiar to us that the idea of getting global coordination without some centralized plan or design may sound impossible or like magic, but it's not, it actually happens all the time. Think about people crossing the street. There is no one orchestrating the process, but through the local interactions between people, they become differentiated out into a distinct pattern with the formation of coordinated lanes going in each direction. Without formal design and centralized coordination, components interact in a non-linear, networked fashion. The type of organization and patterns that result from this are defined by the type of interactions between the components and not the design pattern that was imposed on the system. Whether we're talking about the internet, a smart power grid, transportation networks or logistics networks, in these complex engineered systems, the elements have some form of autonomy to choose their course of action. For this reason, these systems are not the product of our traditional top-down method of design. The overall structure to the system is a product of the local interactions between the components, with global patterns emerging out of these local interactions. There are a number of basic prerequisites that need to be present for this process of self-organization to play out. There needs to be, firstly, some form of randomness in the initial state to the system, variation amongst the state of the components, elements with the Autometer Act, and local non-linear interactions between the elements. We'll discuss each of these conditions and why they're important separately. The first important condition is some form of randomness in the initial state to the system, or at least lack of centralized coordination. There is no possibility for self-organization when the system is already designed and held in some well-defined and orderly structure. When a city is centrally planned and regulated in a top-down fashion, then there will be little space for citizens to self-organize. But when immigrants move into unregulated areas around large cities, we get self-organization in the form of shantytowns, specifically because this space is unregulated. Wherever there's a lack of centralized control and adaptive actors interacting, we'll get some form of spontaneous emergence of order, such as the flow of automotive traffic in a city or the flow of IP packets on the internet. Secondly, the components in the system need to be heterogeneous. Heterogeneity of states is important. If all components already have the same state, then they're synchronized and there is no need for self-organization. If all the people crossing the street are crossing in the same direction, then they're already coordinated. It is only when there are people going in different directions, that is to say components occupying heterogeneous states, that we get friction in the system as they bump into each other, and there is the possibility to optimize and increase the efficiency of the system by synchronizing the components activities in order to reduce this friction. And it is often friction created by variation that drives the process of self-organization. Thirdly, the capacity for components to adapt and respond to events in their local environment is important. We can get self-organization without adaptation, such as can be seen in substances like boiling water. But adaptation is a major accelerator and key element within the process. If things can adapt, then they can easily and quickly alter their state in order to synchronize with other components. And this is part of why information technology is driving a whole new architecture to our technological infrastructure. Because of the lack of capacity to adapt, our traditional industrial technologies had to be designed in a top-down fashion, where everything is well-defined, predetermined and remains static over the course of its life cycle. But as we begin to embed communications devices in all types of technologies, and through shared protocols, they can synchronize their state and self-organize to find new, optimal solutions to dynamically changing environments. Lastly, all these systems need to be able to interact. It is only through non-linear interactions that the agents in the system can synchronize their states and coordinate their activities. Typically, this requires some form of protocol and interoperable platform. For example, smart cities are enabled by open-standard IoT platforms that allow different devices to share information and coordinate. Through open traffic control platforms, cars can communicate with parking lots in order for the system to self-organize. By layering a telecommunications network on top of the power grid, producers and consumers can adapt to each other and self-organize to optimize the low balance on the system. Dense, non-linear, networked interactions between components is another key ingredient in posturing the emergence of a globally distributed pattern of organization. So these are the basic ingredients that facilitate the process of self-organization, some form of initial randomness, the capacity for autonomous adaptation and dense interactions. But now let's think about the actual process through which self-organization takes place. Firstly, as mentioned, we have some environment without centralized regulation. We might call this an open platform. Examples could be a smart power grid where anyone can join as a producer or consumer, or it might be a land area that is unregulated by urban planners, or it may be an unregulated mesh communications network. Heterogeneous agents then join this network or platform and start to interact. Next, components that interact most frequently and intensely begin to synchronize their states in order to reduce friction, creating what is called an attractor. Meaning that because some small subset of the components have formed a pattern of organization and reduced friction, they will be more effective at performing their function and this will attract other components to this more effective pattern of organization. For example, once a number of people have decided to occupy the same location, a town or city forms, with its own set of specialized and coordinated relations that then enables it to be more effective at developing large infrastructure, conducting trade and other economic activities which increases its relative wealth per capita. This creates a positive feedback loop that results in the attraction of others to this particular coordinated state in order to gain the benefits of being part of this organization as the city becomes larger and more populous. Lastly, when all the different components are aligned within different local level attractors, then these different local attractors may stay stable for a prolonged period of time but ultimately evolution will act on the system and they will have to either cooperate or compete in order to create a global pattern of organization or the whole system will become vulnerable to being subsumed by some more efficient external pattern of organization. The net result of this cooperation or competition being that all elements will be aligned within the global pattern of organization and the process of self-organization will be complete as this new pattern of organization both enables the components to function more efficiently but also constrains them by making them comply to globally accepted protocols of communications and behavior. As examples of this we could cite the development of most technologies from videocassette tapes to bicycles and mobile phones. Different design patterns to the technology were initially developed and competed or converged with one design paradigm ultimately subsuming all the others to become the default. The net result of this process of self-organization is systems that are not so orderly and are often suboptimal. If we look at ecosystems which are self-organizing systems, they are hugely suboptimal with respect to the processing of energy. Only a minute fraction of the energy that enters the system is processed all the way up the food chain. The vast majority is wasted. The internet is another example. It often simply fails to deliver packets. It has a massive amount of redundant and virtually inaccessible information. Unlike our traditional centrally designed systems that are specifically designed to be optimal self-organizing systems are not designed to be optimal. They are typically not orderly or well behaved but they do work and are often robust and sustainable in the long term. In these distributed complex engineered systems none of the components in the system have a global vision of the whole thing. They are acting and reacting to local information. When a new person moves into a shantytown, a new computer connects to the internet or a new vehicle enters an urban transportation network they have no global vision of the whole system and they are only interested in their particular local environment. And this is all the system is composed of. These agents following their own rules according to information they have within their local environment. No one is in control. There may be someone creating the platform and protocols but no one can control or know what is going to emerge out of these non-linear interactions. Tim Berners-Lee may have created the World Wide Web but he has no control over how it will develop nor any great capacity to predict it. The inevitable result of all these agents acting and reacting to each other locally is going to be a far from optimal solution for the whole system. But self-organizing systems promote engagement. People feel more engaged because after all they truly create the system. People feel empowered and actively engaged in their social networks although someone has created the platform and defined the rules under which they interact they still get to create their network and shape how it develops. Self-organizing technologies are a whole new paradigm in systems design. The IT network-enabled technologies of the 21st century are increasingly distributed and adaptive as we start to place embedded devices in all kinds of things and then network these things. This provides the enabling context for the process of self-organization to operate on our technologies. Meaning that our traditional industrial paradigm that technology needs to be fully designed and managed will likely be less dominant and have to share with a new model where designers and engineers are the platform and protocol developers through which different technologies interact and synchronize in order to find solutions. And as always it is the interplay between the two the top-down formal design and bottom-up self-organization that creates the core constraint within these complex adaptive systems. To summarize we've been talking about how complex engineered systems create global coordination through the process of self-organization that requires an initial state of randomness and lack of control. Adaptive components and dense interactions as an enabling context. It is a process through which initially heterogeneous components come into contact, interact and synchronize their states to create attractors. With these attractors then cooperating or competing to give rise to the global pattern of organization that is typically characterized as being suboptimal with no one in control but also often robust and sustainable.