みなさんこんにちは、私はマサキ・キムラです。私はコネクティビティーの間のレガシー・システムとクーバネイティーを話します。ソーサイピー用のセンデルをアイデンティファインしています。まず、私をご紹介します。私の名前はマサキ・キムラです。私のギッハブハンドルはM-キムラMです。私はHitachi BantaraのOSSデビューオープンです。私はコミュニティーの2つのコミュニティーを活用しています。私はメインデビューオープンでローグロックボリン・システム・システムを作っています。私は最近、コミュニケーションのコミュニケーションをインプリメンティングしています。私はクーバネイティーのリーソーサイピー用のセンデルを活用しています。そして、コミュニケーションはクーバネイティーとインプリメンティングの中でコミュニケーションを活用しています。では、私はこのセッションのオーバービューの間に、私は簡単に紹介します。ほとんど全国のオーガナイゼーションが、ITシステム、システム、オンプレミス、オール、クラウスのために、クルバネティーズとクラウドネティーブロードを使用することができます。私はこれについて説明します。レゴシーシステムは、ソーシアイピーズの使用について説明することができます。つまり、ソーシアイピーズの使用について説明することができます。さらに、クルバネティーズの説明は、ソーシアイピーズのクラウドネティーブロードの使用について説明することができます。なので、ソーシアイピーズの使用について説明することができます。つまり、ソーシアイピーズの使用について説明することができます。そして、クルバネティーズの説明は、ソーシアイピーズの使用について説明することができます。ソーシアイピーズの使用は、ソーシアイピーズの使用について説明することができます。つまり、ソーシアイピーズの使用について説明することができます。つまり、ソーシアイピーズの使用について説明することができます。つまり、ソーシアイピーズの使用について説明することができます。つまり、ソーシアイピーズの使用について説明することができます。つまり、ソーシアイピーズの使用について説明することができます。つまり、ソーシアイピーズの使用について説明することができます。つまり、ソーシアイピーズの使用について説明することができます。つまり、ソーシアイピーズの使用について説明することができます。つまり、ソーシアイピーズの使用について説明することができます。スターブルとユニークソーサイピーの使い方をご紹介します。まずはファイオールの使い方です。ファイオールの使い方は、ネットワークバウンドリーと言われています。ファイオールの使い方は、ソーサイピーの使い方、プロトコールの使い方、 etc。ファイオールの使い方は、スターブルとユニークソーサイピーの使い方をご紹介します。このファイオールの使い方は、ネットワークバウンドリーと言われていますが、ソーサイピーの使い方は、スターブルとユニークソーサイピーの使い方をご紹介します。2.ソフトウエルの使い方は、ソフトウエルの使い方は、ソフトウエルの使い方は、ネットワークバウンドリーとユニークソーサイピーの使い方をご紹介します。ファイオールの使い方は、ソフトウエルの使い方をご紹介します。3.ソフトウエルの使い方は、アプリケーションの使い方をご紹介します。ソフトウエルの使い方は、ユニークソーサイピーの使い方をご紹介します。例えば、サーブルの使い方は、クライアントAとクライアントBを使い、ソーサイピーの使い方をご紹介します。クライアントの使い方は、サーブルの使い方をご紹介します。とにかく、サーブルの使い方をご紹介します。サーブルの使い方をご紹介します。ソーサイピーの使い方は、ユニークソーサイピーの使い方をご紹介します。次に、クブネティスの使い方については、サーブルの使い方は、クブネティスの使い方は、シリークなネットワークモデルの使い方をご紹介します。CNiの使い方をご紹介します。SUNSとしては、コンテナネットワークモデルの使い方をご紹介します。先ほど言った センスアニメの私たちとCNIで携 new ピラリーCNI プラグミンスCNI プラグミンスCNi プラグミンスネットワップモデルを見ることを確認することができます。三つの説明があります。一つ一つの説明を見ることを目指します。第一つはポッドとポッドコミュニケーションの説明です。ポッドコミュニケーションの説明を見ることができます。ポッドコミュニケーションの説明を見ることができます。二つの説明があります。中央のサクルドポッドを目指します。説明があります。ポッドコミュニケーションの説明を見ることができます。ポッドコミュニケーションの説明を見ることができます。ポッドコミュニケーションの説明を見ることができます。ポッドコミュニケーションの説明を見ることができます。ポッドコミュニケーションの説明はあります。このノードのメンシュールは一つのポッドコミュニケーションの説明を見ることができます。どこかが一つのポッドコミュニケーションに buried in a field is now able to set up nice spot in mid of the field.高うわががドッグだと需要高うわがーが天から落ちて comeuniversal ground will fall into a field that makesAgent like kubret can communicate with all the pods in the same node.The third one is about pod to pod communication.It is similar to the first one, but it is for pod in host network.Kubretis can assign host network to pods as described in the circle pod.definition of network model goes like this.pods in the host network of a node can communicate with all pods on all nodes with that node.So, even in the host network case, the pod can communicate with the pods in the same node with that node,and the pod can also communicate with the pods in different nodes with that node.From the previous slides, I copied all the marks to the diagram in this slide.So, it should show all the defined connections in the kubret's network model.Let's compare it and connection to outside kubretis cluster.If we add non-kubretis server to the diagram,sauce IPs when going into kubretis clusterand source IPs when going out from kubretis cluster are like these.We can see that both of these connections are not defined in the kubretis model.So, it depends on the implementation of each CNI plugin.In most plugins, these source IPs when going out from kubretis cluster iseither not it with node IP or pod IP which may change across pod listers.As a result, we are now sure that source IPs are unstableand might not be unique when they go into kubretis cluster or go out from kubretis cluster.With these in mind, I had a discussion in kubretis communityin the kubretis enhancement proposal or CAP.As a result of discussion in the kubretis community,more extreme use case to connect beyond the same network was found.The example in the previous slidesis just talking about the kubretis deployed in the same network to the existing systems.But this use case is about deploying kubretis cluster in a different networkeven in another cloud connected via the internet.So, we need to handle wider scope.Also, for implementation, it is asked to be implemented outside the core kubretis.It will be a fair decision that implementing it in core kubretis would break the existing kubretisnetwork model which would lead to break existing CNI plugins.Now, we understand what is the issue.Let's move on to how we solve it.Submariner is used to solve the issue.So, let me explain what is submarine first and continue how it solves the issue.Submariner is a tool built to connect over the networks of different kubretis clusters.It can connect kubretis clusters in a different network.Networks can be in other clouds even over the internet.You can connect more than two kubretis clusters.You can communicate from a port in one kubretis cluster to a portin another kubretis cluster almost as if it is in the same kubretis clusterlike accessing beer service that is not a road balance service.This is a very rough summary of how Submariner works.It only explains what is needed to explain the rest of this presentation.So, if you are interested in the details, I recommend you to check existing Submariner presentations or websites.Submariner makes one of the nodes in each cluster a gateway node as described in the diagram.So, there is one gateway node for cross-cruster connection in each kubretis cluster.It uses vx run tunnel to connect a port or service as described in the blue lines in the diagram.Also, it uses ipsec tunnel to connect the gateway between the clusters.as described in the green line in the diagram.So, the port in the on-premise cluster sends a packet to the gateway in the cluster to connect to the port in another cluster.Then, the packet goes all the way through the green line from the gatewayin the on-premise to the one in the cloud.And finally, the packet comes from the gateway to the service in the clusterand wired it to the port. In normal mode, port ip's and service ip's are used as they areSo, no overlapping cider is allowed because if cider overlaps, Submariner can't decide if the packetsare for the cluster or the other cluster.To allow overlapping cider between clusters,GlobalNet feature exists in Submariner.GlobalNet controller assigns a global ipto an exported service.GlobalIP is a kind of virtual ip that can be used across clusters.GlobalNet controller keeps the rule to forward a packet to the corresponding service.So, the port in the on-premise cluster can access to the service in the cluster.In the cloud, be your global ip, global ip can be regarded as a stable ip address across clusters.As you can see, Submariner provides features to connect different networks acrossinternet and provide a stable ip address.So, let's see how we can apply it toour outside cluster use cases.This slide shows the basic idea to apply Submarinerto external network use cases.The diagram in this slide describes fromnon-cluster to cluster connection and the one in the next slide will show the reverse direction.First step is to prepare an all-in-one Kubernetes cluster to connect fromnon-cluster server as a gateway.This cluster is only used to be a gatewayto other clusters.So, it is not intended to put some workloads.In non-cluster server,we will add a routing rule to send to the all-in-one cluster for global ip subnet.So,all the packets to the global ip from non-cluster server will be sent to the gatewayin all-in-one Kubernetes cluster.As a result,non-cluster server can accessto the service in the cluster in the cloud via global ip.Also, for access from cluster tonon-cluster,service wizard selector can be used in all-in-one Kubernetes clusterto assign global ip to non-cluster server.The access from the port in the cluster in thecloud will be done via the service in the all-in-one Kubernetes cluster.You might notfamiliar with the concepts like service wizard selector,so I will explain a bit about itin the next slide.Service wizard selector is a way to make service an abstraction forother backend than port.Normal service is a service with selector as described in the diagramin the left hand side.It is used to allow ports to be accessed via cluster ip that is assignedto the service.Selector is used to filter out a specific set of ports that should be accessed via cluster ip.Kubernetes updates the set automatically by updating an endpointfor the service.On the other hand, the service wizard selector is to allow access to non-cluster server via service from ports.Instead of specifyingselectors to select a set of ports,users create an update endpoint by themselves manually to allow access from cluster ip.NextKubernetes concept is headless service.Headless service is a service without road balancing via cluster ip.As explained,normal service is accessed via cluster ip but headless serviceisn't roadbalanced by Kubernetes.It is often used with state-of-the-set to providea unique identity for each port.Let's see the difference by using diagrams.In a normalservice,if there are multiple ports backed by a service,it roadbalances to port ip from cluster ip and it adds a DNS entry for only cluster ip.In headless service,it doesn't providea roadbalance and adds a DNS entry for port ip.Also,it adds additional DNS entries that are prefixedwith port name to identically access to each port.To achieve the same access for headless service with globalnet,we need some considerations.For normal service,global ip is assigned to each servicebut for headless service,global ip is assigned to each port.Let's see the difference in the diagram.Fornormal service,backend ports are roadbalanced via global ip so the global ip is assigned to the serviceand the dns entry is replaced to the global ip.For headless service,backend ports are notroadbalanced so we need to provide a way to access directly port via global ip.So global ip needs to beassigned to each port and dns entries are replaced to each global ip for ports.Final configurationfor external network connectivity with submariner goes like this diagram.For external to cluster connectivity,headless serviceand stateful set should be used.Submariner version 0.10 or later supports headless servicewith globalnet.For cluster to external access,service without selector should be used.Actually,headless service withoutselector is needed to make source ip unique but not supported yet as of version 0.11and it is still under discussion.So currently,what we can do is just useservice without selector.As a result,one direction of access.Source ip is global ip butnot unique.Now that we've covered the theory,let's see it in action.Let's first see the demoenvironment.There are three servers running in the same network.Test VM is a non-cruster serveruse to test access to Kubernetes cluster.Cluster A is an all-in-one Kubernetes that is used as a gatewayfrom test-bm.Cluster B is a Kubernetes cluster that will be connected from test-bmwith stable source ip.Cluster A and cluster B are connected by using Submariner withglobalnet feature enabled.Test-bm is configured to have a routing rule to useCluster A as a gateway for global net subnet.For DNS resolution for test-bm,DNS server is configured in cluster A and differenced from cluster sorry,test-bm.In this demo,Cluster B is also an all-in-one Kubernetes and exists in the same network to the test-bm.However,it can be multiple node cluster and exist in other network.Okay,so let's check cluster A andCluster B first.We can see that cluster A is a single node cluster with 26ip and cluster B is also a single node cluster with 27ip.Next,let's check that they are connected by usingSubmariner.Subtitles show all commands show that cluster A is connected with cluster Band the same command for cluster B will show that it is also connected with cluster A by usingSubmariner.And let's check that we deployed DNS server in cluster A.It is deployed as a deployment and exposed to our access from external server as 251 of242segment.Let's check that test-bm is running 142ip and gluting table is set toCluster AIP for global net IPs and the result is configured to point to theDNS server in cluster A.We can access to the DNS serverand finally let's deploy HTTP server on test-bm to test access from Kubernetes cluster.Thenlet's deploy state-of-the-set web in cluster B.It is configured to have two replicasweb0 and web1.We'll expose the state-of-the-set with heteroservice engine xssthen export the service to allow access via global IP.For allowing access to test-bmWe will create service-without-sector in cluster A and create an endpoint to point totest-bm.Then also export the service to allow access via global IP.Let's try in the actualenvironment.First in cluster B we will deploy state-of-the-set web with two replicas.We will expose itwith heteroservice engine xss state-of-the-set and service exists.Then export the serviceto allow access via global IP.Then global IPs are assigned for port web0 and web1.Forallow access to test-bm in cluster A we will create service-without-sector test-bmand create endpoints manually to point to the test-bm.Then also check that service and endpoints existand export service to assign global IP.Then global IP is assigned to test-bm.Now thatwe are ready to test access.Let's first test port to external connection.From port web0 and web1 that aremanaged by state-of-the-set web,we will access to test-bm via service test-bm in cluster A253ip is used to access to the test-bm.Let's check the state-of-the-set in cluster B.We have state-of-the-setand two ports managed by it.And first we will log into the port web0 and try access to253ip that is global IP for the test-bm.And the access succeeds and the log shows that it is accessedfrom the ports global IP.Next let's try the same for port web1.Log into the web1and try access to 253ip and the access succeeds and the log shows that it is accessed fromweb1's global IP.Finally we will test access from external to port connection.From test-bmwe will access to port web0 and web1.As we configured DNS in test-bm the service can beresolved by DNS.Web0 and web1 can be resolved and each of them is pointing tothe global IP for each port.Also access from test-bm to each portwill succeed.Due to lack of implementation in current submariner we use servicewithoutselector for test-bm instead of headless servicewithoutselector.The source IPfor these access are from global IPs but won't be unique.Let's first checkDNS resolution in test-bm.Engine XSS is resolved to global IPs for both portand if prefixed with port name it is resolved to each port global IP.Then we will check thatno access yet to these ports.Okay let's try access to port web0 from test-bm first.The access succeedsand there should be access log only in web0 and source IPshould be global IP.Okay so do the sameto port web1.The access succeedsand access log in web1should be added.Source IP is global IPbut not the same one.In conclusion,as for problem statement,some legacy systems require stable sourceIPs to identify senders.Source IPs aren't stable when going into Kubernetes or going outfrom Kubernetes.Such legacy systems and Kubernetes clusters can't work together as they are.And as for IDEA for solution,submariner is a tool to provide multi-cluster connectivityfor Kubernetes clusters.To connect clusters that have overlappingsiders,global net feature is available.Combination of global net and headless service,headlessservice with a selector provides a stable source IP to Kubernetes cluster.As for current developmentstatus for submariner,headless service with global net is supported in submariner version0.10 or later.Headless service with a selector with global net isn't supported yet as ofsubmariner version 0.11.We need more tests and feedback for this use case.In this slide,I'm sharingrelated issues and PRs.Please check these URLs,try this feature,and give feedback if you areinterested in this use case.The second to last URL is the one for missing feature,and the last URL shows how you can try this feature as shown in the demo.As our links are all discussions that will be useful to know the more detailed background and implementation around this feature.That'sall and thank you very much for your attention.