 So, rwy'n gweithio, yna, ond rwy'n gweithio eich cyfnod. Rwy'n gweithio i ymweld i'r ffocws ar y ffiltru, ac mae'n gweithio'r gweithi. Dwi'n fuddio'r gweithio i'r ffiltru, ac rwy'n gweithio'n gweithio. Felly rwy'n gweithio gweithio'r gweithio, ac rwy'n gweithio'r gweithio'r gweithio'r cymdeithasol yma, o'r cyflwyts cyllid yma, yw rhaid y gwbl cweithio'r cyfnod. Yn ymweld i'r llwyng. Felly, y gallwn gwneud hynny y bydd y dyfnod deffdes yn gweithio cyfnod o micro-serfoses ac cobinettys. Micro-serfoses wedi bod yn fyddai cyfnod o'r hyn yw'r cybinettys. Yn mynd i'r cybinettys, mae'n fyddai arall yn y stach. Fyll, stach, gweithio, y cwntain, y stach, ddweud. Yn mynd i'r gyfan, mae'r gyfan yn gweithio i ddim yn dweithio ar y syniwyd â'r cyffinod. Mae datblygu'r cyffinod, a fyddai'r cyffinod chi yn fwy o'r cyffinod ysgol cyst crying, a'i gweithio i'r cyffinod sy'n llawn. Yn dweud mwy cael y 5 a 10 sefydli, a ddweud 50, mae'r cyffinod ar gyfer cyffinod dyfnodol yn y cyffinod? Yn fwy o'r opeth, mae'n gwaith y lle hwnnw i milio mellu. Allan chi, phwy i'r gUNTOS, sy'n wneud gweithio. I love sharing stuff and my early on in my career I didn't fully grock how many things were out there techniques patterns tools and I made it sort of a mission based on feedback from my mentors to learn about these things So know you have options When you are testing there's mocking service virtualisation and we're also going to look at some remote remote local tooling as well Very briefly, uh, this is me at Daniel Bryant UK on most of the interwebs 20 year Darius it career in software development Started as a Java developer with a JavaScript move on to architecture Then ops and then platform engineering wrote a couple of books along the way with my buddies shameless plug I'm doing a book signing uh Thursday. So if you want a free copy of mastering API architecture come along to the O'Reilly booth on Thursday I have worked with ambassador labs. I'm kind of repping them today But I actually left them about a year ago So I'm doing freelance work now some of the Cintaso folks. I'm representing as well doing work with them But this is kind of an independent talk today Setting the scene so Most of the advice I'm going to give today is based around the idea you're building microservices that are running in a container And who knows if we're really building microservices some kind of small services bounded contexts good apis, right? Um, I think the advice will apply to macro services Maybe even the monolith if we dare to build monoliths these days But one of my favorite memes here is we are deploying onto kubernetes, right? Now I love kubernetes It is complicated though, and it's got better over the years But I still think it's not for the faint of heart and as a developer by kind of trade I had to learn a lot about kubernetes and and I definitely aged as I learned kubernetes, right? I'm also going to rip rip rip or talk a lot about this fantastic work by toby Clemson Uh, this is my gold standard. Can you believe it or not? It's 10 years old Now how many things in tech last 10 years like a cnr space, right? This is fantastic work by toby x thought worker doing amazing stuff I am going to use his definitions of unit testing integration testing Testing communication paths and the communication between systems or components component testing uh talks about uh subsystem testing testing the functionality at the api level And I'm going to mention end-to-end testing using toby's definition Sorry, I should have put the link up. There's the link right in my bad uh on the martinfowler blog I am not going to talk about contract testing. I think it's super valuable I have got blogs on packed and using spring cloud contract if you want to know more Many other people do a better job at that than I do as well Contract testing very valuable couldn't fit it in within 25 minutes To frame the discussion as I do want to mention the inner and outer dev loop A fantastic blog by Mitch Denney unfortunately disappeared now, but that you can find it online And it's kind of common parlance these days, but in the inner dev loop We're doing our test driven development. We're building our systems. We're modeling the world We're trying to capture right we're keeping that feedback loop super tight Eventually we push to the outer loop. This is where we do code reviews more verification security analysis all that good stuff, right? CICD Now, I think we can argue that the outer loop is kind of been captured by remote and that's totally fine Right, we pushed stuff to github. We build on harness. We use civo to spin up kubernetes clusters totally cool with that now I think The inner loop is still very much around the local space now I'm going to shout out the git pod folks they toner github code spaces There's a lot of folks doing remote ide's or cloud development environments And I am bullish on that future, but at the moment like say a lot of us are still around the local in this kind of experience We're running stuff on our local machines I think unit testing is primarily in the inner loop Integration testing primarily in the inner loop. You're knitting things together, right Component testing end-to-end testing more in that outer loop But sometimes you do need to do it in the inner loop too And there's a bit of a warning sign if you're doing a lot of component testing locally Or you want to like push a lot of stuff to remote to test it They are warning signs, which I'll talk about more in a moment One thing with service oriented architectures Microservices in general is there's just more stuff, right? Totally makes sense and my inner loop Can depend on your outer loop. I am you've given me this new dependency this new service I've got no idea how it works. I look at the swagger docs the open api docs perhaps async api I get an idea, but I want to poke and prod that service. How do I do that? And it cascades through too as well. All right, so there's just more things with service oriented architecture I think a lot of us start mocking things And then running as much as we can locally and this works, you know picked your poison, right? How you want to run things locally if you install it on your machine You can get quite far now not every developer is happy like installing docker and running docker That's something to bear in mind too, but you can get pretty far with this approach Coming from the JVM world I think I can pretty much say after we get a certain number of services This becomes quite tricky, shall we say, right? And my experience is like it's only a few for a JVM or a CLR based workload With go and other services you can get a bit further perhaps five ten services But you can't run everything on your machine at some point People then tend to do more mocking and then they rely a lot more on component or end to end testing, right? Again, pick your poison. These are some of the ones I've used along the way Once you get to this space, there's a danger that the inner and outer loop Can become the same and I'm sure a few of you recognize you're writing some code You're building a container. You're pushing it to a registry deploying it to a cluster Testing made a mistake and the loop begins again, right? And this can be quite slow reminds me in my old school days sort of like Compiling stuff and putting in print lines and then recompiling it and redeploying it And my Java HTTP days, I was doing this with the gsp pages, right? Like this really slow loop Automation can help hat tip the scaffold folks gardens get cube octeto casing tilt There is many others too, but a lot of the time that these can help you automate some of that build And deploy of the docker container in the background to a remote target So you're not constantly on the cli doing docker build docker push Like the you know these tools do much more than that I would just say But these tools are good jumping off tools if you're looking for automation Sometimes though you want that really fast loop, right? You want that you don't want to be doing docker build docker push or container runtime of choice You don't want to be doing that. You don't want to be doing any kind of build and push even if it's automated You want to be in that inner loop superfast, right? But you do want to minimize the differences between production and local This is the tricky thing you want high fidelity, right? We've got production running looking good. We've got local running looking not so good, right? We've all been there. I think we can all recognize that one But just bear this in mind Last two slides about that trade off of speed and fidelity. They are really really important in testing You're always trading off speed and fidelity I'm not sure what's going on there. That's not my laptop. I don't think Help anyone don't know any thoughts My laptop looks fine Turn it off Any thoughts anyone? We'll get some running. Sorry everyone I saw it flicker just a moment ago We do the turn it off and turn it back on again Fingers crossed or magic. Thank you. He's all good. Got another Microsoft approach, right? Sorry Microsoft folks Turn it back off There we go. It got those there So With that context set trading off the speed you're trading off fidelity bear that in mind again I'll hat tip people like charity majors Cindy should Haran. They do amazing blog posts on those topics too Options and trade offs So we are going to build up to this slide along the way again quite a tight timescale So I'll keep it snappy, but I do want to talk along the top We've got our mocking service virtualization and removal tooling along the side We've got the various types of tests, right and this inner loop exploration I'll just give a bit of context on that one That is where you've been given a service maybe a spec and you want to poke and prod and figure out what that service does I call that inner loop exploration I'm trying to build my mental model of a third party or maybe even an internal dependency That's why I don't think mocking it makes sense because I don't know what it does Right I can't mock it if I don't know what it does And I do think things like remote will make a lot of sense here because I can run the thing in a cluster And do a remote to local bridge and start poking and prodding the API to actually get feedback as well So that's the kind of context we're going to build up to If you want to be grounded in more in the tech show me the tech right options If you're looking for mocks, I come from a Java world, Mochito, Pyronwok, Jest for javascript go mock for go If you're looking for service virtualization tooling Love test containers, local stack I was a big fan of that when it came out of Atlassian many years ago For embedding AWS components in your test runners And I've also run things like H2, HSQL Embedded Kafka when I'm running stuff Like I want to simulate something rather than run the real middleware real data store in my tests There is a microx homofly wiremock and mounty bank more network level Virtualization tooling I'll cover that more in a moment They're great tools as well check them out and the final category of local tooling I put two splits dev space octeto scaffold tilt telepresence miradid and gaffire that is your full list I want they're all open source as far as I know. There may be companies behind them, but they're open source tooling I have put in bold the cncf sandbox projects My grocs dev space telepresence and in blue are the tools I have worked on On my bias right I worked on hoverfly about 10 years ago still going and I worked on telepresence part of the community still But I left the ambassador labs about a year ago where I was before that was working on it quite a lot So with the tech in mind Let's have a look at some mochs. So hopefully this you know App dev audience, I'm going to go pretty fast on this This is the kind of thing. I'm thinking of mochs, right? I can verify interactions. I can stub data over here Happy days, right? We will just just the level set what I'm talking about with mochs There we go The good with mochs they are idiomatic if I know Java and I'm working with the Java mocking framework the like the learning curve is pretty light Quick and cheap you saw that right super easy to do verifications super easy to do stub to calls as well. Love that And if you are building out a component that you're mocking, they can be a great way to build your mental model Kind of outside in TDD. You're literally prototyping the API prototyping the interface before you're building out the back end Of that a component within your application. So I've learned a lot sort of API design with mochs over the years The bad there is implicit assumptions coded in and I call this the three realities conundrum You'll recognise it when everything works like fine locally test pass push it to production Something falls over because there are tests the mochs sorry have drifted away from the actual implementation And the reason I call it the three realities conundrum is there's always a real world. We're trying to model some business problem There's always your code your writing to address that problem as soon as you've got a mock in the mix There are three realities there right not no longer just to there are three and if they get out of sync you can get confidence locally But as soon as you push it to prod stuff falls over And and you'll see this a lot with difficulty keeping up to date and sharing and coordinating I worked on a Ruby on Rails project. We were sharing our mochs as the project went on over the years And it became highly couples and we were like duplicating mochs and falling over each other with different data in the same mock And these kind of things just became a nightmare. There is commercial tooling available to help some of these things The final thing it doesn't need a test runner. I come from the job world J unit but end unit to pick your poison right. You need to run the mock via a test runner I can't just stand it up and poke and prod like I can with some of the other tools. I need a test runner So what to use then this is what I'm thinking for mochs. Love it for a unit all the way through to component End-to-end. I tend not to use mochs in end-to-end. I'd rather use of service virtualisation or I'd rather use a sandbox Like I'm testing against the PayPal API. I'd rather use the PayPal sandbox than some kind of mock of it, right? Just more realistic And you can't really use mochs when you're building your mental model with a third party thing Service virtualisation. Now this has been around for a long time. It's definitely more enterprise either I would say And this is some code from hoverfly I worked on. You can see probably recognise the kind of vibe, right? You can see I'm setting up a sort of fake mytest.com, putting a rest URL there, so a path And I'm saying when I hit that end point return some data and I can literally assert that down below, right? I'm just saying you know calling this API asserting it But notice I'm going over the wire. I am making a request To a synthetic end point of mytest.com. That's very different than just calling a mock within your code, right? You can test more of the networking stack, you can introduce failures to deterministically see how your application handles those failures Lots of really powerful stuff with this And why a mock and mounting bank do the same kind of thing The real cool thing with service virtualisation is the ability to capture traffic though I can make a request, put hoverfly in the middle, I can capture the request and response in a script database and export it as JSON I can then remove the network connection, so I'm just running locally or I'm running in CI, right? And replay any requests that I made. I can also change the data, I can introduce failures, really quite powerful This is an example code here. I'm no longer setting up the mock like you saw in the last slide I'm loading an adjacent file that contains request response pairs I'm making my call over the wire right and I'm asserting the values back, but I've not preconfigured that, that's being loaded in that file Really quite powerful God of shout out test containers, I love test containers, I've found the project, luckily know the founders One of them was very active in the London Java tech scene, so rich, I'll shout him out and say it again, crew I've done amazing work on test containers, there's a commercial entity behind it called Atomic Jar Which has now been acquired by Docker as well, so the future is bright for test containers There's plenty of good open source stuff in there, but you can literally run a service or a data store Within a container The beauty of test containers is it's super fast and it manages the life cycle for you within your test runner So tear up, sorry, set up, tear down, that kind of thing And you can run a real thing You can see here I'm literally putting in some data into a redis and I'm asserting on the get back No need to set up any orchestration, beauty of test containers, I can literally run the real thing So service specialisation, the good It's more realistic than mocks, it can be the real thing, albeit resource constrained If you're running, like say, test containers locally, maybe I can't have massive databases You can't have a container with a lot of, you know, with MySQL in it, with some data in it That's no problem, but if you want to do like gigs and gigs of data Maybe you want to explore other options You can do those wire level deterministic things, but you can't do that You can do those wire level deterministic failures This for me was really powerful on a couple of migration projects I worked on Where we had some cludgy old mainframe system with someone that bolted a rest of the API onto it We just couldn't get it to fail on demand and we couldn't understand how it failed But what we did is we started putting in what's called middleware and hoverfly That would corrupt packets coming back, mess with the data, stuff that we kind of knew happened And we could make sure in our code that we were defensive against those kind of practices So we recorded a request response against the mainframe thing But then we introduced failures in our test suite to make sure we handled that properly It is easier to use a larger amounts of data If you're mocking and you start writing and you're copy pasting lots of data lines into your mocks That's a bad sign, switch up from that If you're having to like do crazy updates to your mocks all the time Again a bad sign, maybe service virtualisation is a better look The bad, they are slower often by an order of magnitude than mocks to initialise Mocks are in process within your test runner, these are often at a process calls So it is a bit slower to initialise It can be tricky to configure, hoverfly does have a bit of a learning curve, I think on that Keeping the data up to date and shared is tricky This is where a lot of commercial tooling is being built around this space Making sure the data can be shared and version control, that kind of thing So what to use when? With service virtualisation, don't really think for unit tests Because it tends to be a bit too slow for unit tests Integration component end to end Again if that PayPal sandbox is super flaky or super slow Maybe I'll record it using hoverfly, wiremock and I can put that in my end to end tests In a loop exploration, maybe That migration example I gave, we had this end point of this legacy system We recorded the request and responses So we could understand a bit more about this service we were developing against as well So it can be useful in that scenario A couple of shout-outs, Michael's focus is actually in the front row here hidden away If you're looking at async API testing, which is very popular these days I think it's the gold standard in that space A lot of the tools I've worked on are more REST API based Open API, swagger, that kind of stuff right Do you love what they're doing in the async API space And not just Microsoft, a bunch of other tools are going to shout out the Docker folks In Docker desktop there's a series of extensions, plugins, these kind of things And you can often get started very easily if you've got Docker desktop running With one of these extensions We've got one for telepresence that I worked on too And I find it a really nice way to demo and play around with the tech So just to kind of keep on the shout-outs and I'm sure you can chat to the folks in the front here Yassine and team if you want to know more as well Final section, remote one I'm not going to dive too much into this I've got many other longer versions of this talk online where I do live demos I wasn't brave enough today with 25 minutes to do a live demo That involves the network, right? So check out my demos online But just no remote tools to local I think I stole that from the Influx folks But remote tools, you can take an application, a service running in a pod And you can kind of swap it out with your local machine Too far away, sorry That was my bad, sorry You've got to take a service and you can swap There we go And swap that service out with your local machine And basically there's a proxy mediating, bridging between the local and the remote Traffic can go both ways You can send traffic from your local machine into the cluster Pocon prod services You can also have traffic routed from the cluster Maybe from the ingress into your services To test your services running on your local machine Very powerful kind of modality There's other cool stuff you can do And imagine I've got a laptop there But you can also think of that as a CI runner So the CI runner can do a bridge between local to remote You can have a shared, say, staging environment And swap out the various services when you're testing Which is very cool Don't worry if that doesn't make sense I'll put some links up in a minute Where some folks have actually talked at KubeCon About how they use the remote tools But the reason I like these and this kind of testing context Is they allow me to reach into the cluster And test with the real thing That's the big thing with the remote Remote to local, it gives you that So for the good, it enables testing against And poking of the real thing You literally put your local machine Effectively into the remote cluster You can run various tests As if your machine was in the cluster Very powerful Datasets can be large scale When I was working on telepresence We saw a lot of use cases with machine learning Massive databases running in a Kubernetes cluster They could never hope to bring down locally So they'd use telepresence to bridge from their local machine To remote Run their tests, disconnect That was a real powerful use case There is no need to build that mental model Of a dependency You can literally connect up to the cluster Start calling the APIs With Postman, with the CLI Whatever works for you The bad You do need a solid network connection Mutating shared state As with most of our lives as developers Is the bane of existence So if you're sharing that staging environment But you're modifying the shared state That breaks someone else's tests That happens all the time So just bear that in mind With great responsibility Or is it great freedom comes with responsibility There is a lack of language bindings Mox clearly are idiomatic to your language Service virtualisation tools Have SDKs Like to believe for all the main languages A lot of the remocal tools do not yet Have language bindings I know something like that I think I saw the Metalware folks in the back They're doing stuff with MirrorD Which I think gets rid of some of that stuff as well So just bear that in mind If your developers, your teams, you Want a language binding, want an SDK Some of these tools don't offer that Quick shout out It was razor pay folks They're like a unicorn over in India Fantastic tech team Chacked them a few times Did a talk where they compared using Dev space and telepresence for testing Large scale services They've got razor players Like a PayPal type vibe right Massive scale, super interesting use case Sooner the event Venkat did an amazing talk In COVID times Check that out If you want to know more about telepresence Running that in GitHub actions Got your link there as well So what to use when This is what I think for a remocal Unit tests, no Unless you've got a special use case Where you are that massive Data engineering machine learning scenario You don't want to be running Remocal tools bridging into a cluster Crazy eye latency Then it may be integration testing But the rest of the test Is really good use case I'll give a shout out I did see the The Mirror D folks metal bear They're doing a great looking talk I'm going to go along to it tomorrow That goes into more of these things If you want to know more about that The remocal vibe I talked about I think they're going to talk about Some of the mechanics behind the scenes Which sounds fascinating Wrapping up So we went through what to use when Hopefully in this whistle stop tour And again there is a longer version Of this talk online from various conferences You can check out or I can give you the links And this is what kind of my thoughts are I'm not saying I'm 100% right I would love your feedback on these things But again early on in my career I really value just signposts Of what to use when And what kind of options Knowing what kind of options Are available for me as a developer Additional content I've mentioned I think Cindy Sharon already But Cindy's work is fantastic Learned a lot from Cindy over the years I've written some stuff about Coupling and cohesion with testing That I think stood the test of time About five years old now Hopefully that's useful for you And Sam's book And Alex and crew I've reviewed both of those books They are amazing authors Like they do They go a lot more deeper Into some of the architecture considerations And the testing considerations That we've talked about today So those are my kind of recommendations Wrapping up Hopefully you've seen today The inner dev loop can be painful With microservices and Kubernetes The more things you've got The more trouble you're in generally And there is those inflection points That are the inflection points So if around five services Five to ten And around say fifty services You just need to switch up your tools More often than not You can't run everything locally You can't manage the data in mocks You need different tools You have options Part of it if you're a tech lead Like a big part of your thing Is knowing your options right The CNCF landscape Does include several useful projects Check it out right I'd like to see more in there I think the CNCF landscape is very Well maybe I wouldn't like to see more CNCF tools There's a lot of tools right But I would like to see more testing tools Because I think these are really important To the development life cycle To developer experience And lastly Meta point Choose your dev test approach wisely Too many times as a consultant I rock up to companies And they've just kind of fallen into the tools They've used and fallen into the testing Approach they use And I get it I've been there right But if you can Choose your tooling intentionally And document why you're doing what That is just amazing for onboarding Amazing for longevity of the project At that point I think bang on time I'll say thank you very much Haven't got time for questions But I'm around the conference For the next three or four days Come and find me You can also hit me up hopefully On email or on Twitter Or LinkedIn as well Thanks for your time