 Hello, welcome to the talk about nets. My name is Valdemar Quevedo working at Sinalia communications and in this talk I'm gonna talk you talk to you about the net speed 2.2 release, which is what we call the jet stream release much in the making since much of 2020 and 2019 we're very proud of With this release and hope you like what you'll see so in this talk I'm gonna cover about talk about the Get a brief intro to nets and then also want to cover how to get it started with just in what it is in some of its background Followed by Q&A after this session So if you're not familiar with that Briefly, it is a high performance messing system made for developers and operators so that they can focus more on building their applications rather than how to do the messaging so from the beginning Nats is a Already a 10 year old project is one of the earliest CNCF projects Since the beginning some of its rates was to have the performance have be very simple have a very secure and Always be available. I cannot always available dial tone has multiple qualities of service multiple communication patterns as I'm gonna cover in a bit and Thanks to the simple protocol has a multiple client implementations. It is a single binary. Just a 10 megabytes It's a TCP based plain text protocol. These are all the verbs from the protocol Which has stayed the same throughout many years takes very little to configure nets You know as a client you only need the URL and the credentials. There is a gossiping and a lot of things that make Nats deployments be very flexible and have a very long uptimes, so straight forward API and From the Nats v2 series. It has a security Multitenant accounts Isolations of the subjects namespaces and also it supports multiple network topologies Which is a very advanced powerful pattern that we call the Nats adaptive edge architecture. So you can have silo of Nats clusters and those Be able to be connected to another remote cluster in order another data center and expand into a super cluster types of setups and then you can have what we call the leaf nodes connections that can be outside of the super cluster or Create a home spoke topology so that you can have Like a centralized Nats cluster and then through the leaf nodes. They can all communicate all the clients can still Communicate so it can all of these three building blocks the Nats server super cluster gateways and leaf nodes You can use them to map into the your architecture or the needs of your organization Nats of these queries about the streams and services. So Nats In Nats, you have a flow of data which you can fan out and you can have services Which is basically do some work and return the result. These can both be load balanced In terms of the publish in the client API you can publish and subscribe Which are we call the streams you can also request And the request response and on services and both of those can be load balanced So that you can publish a message or make a request and then only one actor for himself clients going to handle that message request response is sending a Message inspecting a single one response publish you can broadcast into a number of members from a group and Two subscribers is the load balancer where only one of them will receive it. This is a very classic Nats since the V1 series Which is an at most once delivery so in terms of delivery guarantees Nats which Ensures that you you have to be connected into Nats table to receive a message so the Nats server means that We'll hold some of the message For a certain number of For some time, but you have to be connected and consume that message Otherwise, you can become a slow consumer You have to avoid becoming a slow consumer. So if In case there is no one when and we make a request In Nats very similarly you can make a request and then just retry making the request something that we do with HTTP services Something that is new in the v2.2 is Instead of what used to be a Nats timeout when making a request Now you get a very fast error called the no responders error because it's kind of like an analog of the 503 HTTP status Now you can have your client your requestors in Nats Very quickly detect whether there's Interest in the subject and then try another logic In terms of at least once delivery in Nats with initial support for for it came In It's not streaming the net stream project also known as a stand as they've been the opposite of Nats It has a lot of adoption and it's very easy to use the many people like the the API But have some shortcomings given for the because at the time when it got When it got developed Namely the horizontal scalability of the whole system was very difficult And also it didn't have very good integrations with the permissions and accounts features from Nats v2 And as a result and basically all the feedback from the community we got We have a released first quarter of 2021 the Nats jet stream Which is the next generation persistent engine for Nats. You can find all of the docs under the github Nats.io jet stream repo it was in technical preview preview in December 2019 the previous cube come and the clustering was finalized on the first version of its of its Of not just in clustering got released in March 2021 unlike Nats streaming it is It is built into Nats. So it is a subsystem that is part of Nats And you don't have to use protocol buffers and also another request response protocol to able to persist messages which was One of the pain puns from Nats streaming and also you don't you're actually the the connection represents the It is not like an a session like in that stream but the connection you It is no longer like a session in like it wasn't in that streaming it supports It is based on subjects. So it supports a multi-tenancy and that's V2 accounts isolation You can use either file or memory based storage So the key concepts in jet stream is that You have streams and a stream can be one or more Nats subjects that you decide to Have it be persisted the streams themselves. They can be mirrors from another stream There it can be sourced from multiple Streams to create a new stream You can also have consumers and the consumers that they can be either push or pull based And you decide from where and how you want to Be delivered those messages or in case of pulling how what is the Re-delivery time for example for each one of those messages and Unlike Nats you also you do have different types of acknowledgments You can have like an active negative act as a terminating messages and I'm gonna cover those in a bit as well but first I want to introduce something was fundamental into in development of jet stream That we now recognize like a key component for doing operations with Nats Which is a Nats CLI much like in Kubernetes. You have something like the Qtl of Qtl that You where you can point everyone to okay, so give me the output from this Qtl command and We have adopted this Nats CLI the developer one of the core maintainers from Nats and These gives you a lot of Utilities to be able to establish like a common tool set for the Nats community for doing operations on Nats so you can do various from the various things basic like pop-sobbing to doing some of the more advanced Monitoring and jet stream management all through the Nats CLI so it's you can for example this you can subscribe on foo and Set a number of messages on the bar on the full subject and These combined with the system account that got introduced with the Nats v2 you can also Liberage the system account functionality to have more advanced Reports of the status of the system for example, you can now query the number of servers and connections that are connected Do similar things that you can also do with the Nats top But all through Nats, which Nats about just now HTTP protocol This is all getting the messages through Nats. So in this case, I'm getting the server connections from this single server and how many messages have in are being sent and Just next CLI We can get started with jet stream because it has all it takes to Start creating or the streams and consumers So the simplest setup for jet stream is just to start the server Nats server height with hyphen js You'll get a cool banner saying that the jet stream got started and by default has a Dynamic limits, but for the production recommended setups we You should be using the allocating how many how many memory How much memory and storage you want to use for for jet stream Setting up jet stream is quite straightforward. It just takes Explicitly in case of a production setup, how much maybe you want to use what is going to be the Storage directory In this case, we're already setting up my system account named sis and Something that has to be noted is that you cannot make the system account use a jet stream. So at least You can enable it just from globally for all the accounts or you can Opt in Certain number of accounts so that they can only a certain number of accounts have the jet stream functionality enabled And in case of a clustering it is very similar to how you you you would set up a regular flow of Nats servers in this case the membership is determined by the cluster routes each one of the name the servers that are Part of a jet stream cluster or where to have a server name in this case We have the is the configuration for the server named a and we're part of the cluster that is ABCDE and in this case we have a five notes as part of the jet stream club cluster Under the hood is using a raft for the election there are multiple raft groups For the For the whole like API from jet stream, there's a leader there's also a leader for each one of the streams in this case We are monitoring What is the status of messages that are being persistent in the jet stream in this case the leader it is The server a and again all of these you can all of this info you can get as a not server report or raft Just in report using the net CLI and in this case we know that the note a is the leader of Of The jet stream What is the call the meta group meta wrap group? and there is one stream named s1 that is using the storage as This is file-based storage and the replica. There's the leader from this replica is the node named be named be Has a number of other at a Nazi like has another Set of functions to able to Manage the membership so you can ask Any of those members to step down so there's like a new election and There is a another member takes over the leadership so you can replace those notes in case there's a failure for example So again all through the net CLI Again through the Nazi allies how you can also make a streams in this case. We're making a stream name foo a lot of the defaults and these basically what dictates is that now there's a Stream name foo and any event that is published into foo will also be persisted So if now we just just regular nets so regular nets protocol. It's all it takes to able to Have those messages be persisted We send 10 messages into into foo and Then we can create a consumer That in this case it was a pool based consumer that is going to have those We'll be fetching each one of those messages And well in terms of applications what you would use is what you call the nats client just in context The first I'm going to show you I'm going to show you how it works in go So you as part of the nets and go library client library you will first start a connection With your credentials in this case. I'm just using User password and then call the jet stream function as part of the Of the connection and that will give you an enhanced type that you can use to Persist messages again, it's just regular nets. So classic nets on foo will also persist message, but they we didn't change the interface of classic nets and In order to get the published with the act responses. We have a new you we use the Producer setup that you can get from the jet stream context. So now you can publish a message on Hello JS and receive an act that acknowledgment that your message got persisted As part of the metadata from the act response, you would know that the sequence number or and also with The message that you publish was a duplicate The push based consumers is the net suggestion client context are very similar to regular net subscribers They're essentially the same. You're just reusing the Just in context to create on the fly and a female post base consumer This is a female. So if you stop this application as it is in this example, it will replay all of the messages from the beginning You can see that you can mix and match both Jetstream base subscriptions and classic nets in this case the just The jet stream one is being re-delivered all the previous messages And the classic one is only getting the new messages all everyone subscribes to through Push in order to make Those applications Resist replay on from where they were when on restart you need to make giving them a durable name in this case my app Becomes a durable name from this subscriber and also you can change from the Auto hacking that The previous example was doing so that you can manually act each one of the messages in this case It is a simple act that you're done with that message so that you can Go to the next consumer sequence. There are multiple types of acts in this case We're making a synchronous a subscriber push base subscriber You can simple act. You're done You can ask for the delivery of this message or terminate and stop delivering this message or just Seeing a signal to the server that you're all still working on the this message Pull base of Subscribers are a bit different. They have their own special API called pull subscribe and The durable name is a required field So do you know currently you don't have a female pull base subscribers? You always require a name So in this case you create a pull subs subscriber on foo using the durable name pull And it's a simple for loop that is gonna be fetching five messages at the time It will quickly get a set of five messages messages if they are There are any ready Otherwise, it will wait for a bit before Retrying Next part of the loop in this case. I'll give you know also new Recommended to check the change lock is that Well, so release the initial and QTT supports Websock as part of the release just stream and just enclosing of course There's an embedded Nats account server and you can use to avoid having the other using the HTTP based one Adjusting this is heavily the message headers, but regular Nats description color use those headers That's why there were two new protocols H pop and H message and other features such as traffic shaping recommend you to check the change look from what is new in The Nats documentation For the roadmap we'll have of course a new client API's We'll spend more time with a wasm ecosystem later this year and also Some requested feature like client cluster migration and Thank you for a stay for the talk and I'll be back for the questions. Thank you. Thank you