 Thank you. Very good afternoon guys. Thank you for taking your time out and joining me here in this session My name is Anupama Tirvai Pati, and I am a senior software engineer at Bloomberg Bloomberg is a global finance and a media company It provides real-time finance data news and analysis to professionals across various industries I Am part of aim aim stands for asset and investment manager For buy side firms we provide multi asset solutions such as portfolio management compliance trading and other operations We are Enterprise scale we operate at enterprise scale, but we also maintain Agility that's that's us So that's about me and my company a nutshell Today, I'm here to talk about the power of specification testing Adding functional requirement to unit testing Let's start with behavior-driven development BDD. We all love BDD In the ever-evolving era of software engineering Delivering a product which meets user expectations is a must BDD helps us to achieve this goal efficiently The principles of BDD Bridges the gap between stakeholders developers and testers by fostering clear communication sharing knowledge and better understanding I will take you through rest of the slides using this example Consider that you have this gigantic trade-booking application Which does many many things and now we have received a new requirement Enabling short selling Here on the screen you can see the financial definition of a short selling But I'm gonna give you a simple one Short selling is you trying to sell something that you don't have I'll repeat myself Short selling is you trying to sell something that you don't have how you borrow it You borrow you sell you buy back and pay it back The key idea here is you trying to bet That the prices are gonna go low in the future and That's how you make the profit Okay, so let's try to apply our BDD principles to this requirement Here are a few of the scenarios for the requirement the first scenario Successfully borrowed stocks Consider user initiated a trade-booking it get books User sees a successful green message on the screen a happy case so two things happened over here first as User was user had like you know found a borrower Next is they had the sufficient funds So the next scenario is unsuccessful borrowing due to insufficient funds User again initiates a trade-booking it gets rejected and this time user should see a error pop-up message Seeing cannot book the trade due to insufficient funds and The third scenario is can't find a counterparty to borrow from That means like you know user again initiates a trade-booking gets rejected because user could not find a borrower So the last two scenarios are the failure scenarios, but then the error messages are different So these are some scenarios that we're gonna see in the in the you know how we can implement and More here's our beautiful trade-booking booking architecture We have our short trade-booking service, which is dependent upon many things such as Trader info stocks info real market data and much more The real system is much more complex than this However for the sake of simplicity we shall only focus on the trade-booking engine Here's our trade-booking engine. We have a lot of APIs which Eventually hits this part of the code That is validate and book Validate and book function essentially calls two more functions that is valid request and book trade They both have their own implementation But if you look at the validate request it does things like valid security valid user valid stock and more Everything looks good. We have our trade-booking system We have successfully added the short selling, you know feature to it. It is enabled all the scenarios are covered Everything is good. Our users are happy Now let's take a look at the trade-booking engines unit test Here are some of the sample So we have test validate price test valid quantity to sell Test stock exist in the market and few more We have also parameterized our test to cover all the permutations such as On the test valid price. We are checking price with positive negative and a zero value Good we have you know a good coverage Let me ask you a question Which user workflows are we testing with these unit tests? That means by looking at these unit tests Can you guess the user workflows? The answer is not too clear the reason is a unit test are only talking about What they are doing, but it's not about the user behavior our end goal is to ensure the system behaves as the user expects us to be and The unit is lacking expressing this user behavior a few more questions What happens if the stock does not exist? What state changes in the system if a trade was booked successfully? Can we answer these questions by looking at the unit test The answer is again not too clear this time because they are lacking the specificity They're only talking about the implemented code, but not the intent We would like to use our unit test to document the code's intent moving on One fine day Engineer had time to refactor What was the refactoring they moved the user validation logic into a new module Now what happens to the unit test which are using the user validation logic? Now we have to go back and update by mocking it Here's a sample. We have like test user is valid trader and you would write something like this The question is if I'm changing all the unit test because I updated the code How would that protect me from the regression defects? So this question is something that even we ask in our teams before we actually pick up any of the refactoring work Slash stories less tasks. You can say anything is We see the impact of the refactoring on the unit test we cater for that We see if there is an impact if there's not we do a proper analysis So so yeah three shoes now Imagine if you had test that Covers the required behavior Access the documentation of the code's intent and protects us from the refactoring That means some way the test would talk about the BDD acceptance scenarios very specific enough and Decouples from the implementation So any test which is providing having all of these three qualities We call that as a specification test Let's revisit our trade booking engine and write some specification test Here's a sample test trader Shot sell with insufficient funds returns an error message to user This is the second scenario that we have seen earlier User initiates a trade booking it gets rejected and they see your error messaging the insufficient funds cannot work This test talks about two of the qualities that I have just spoke first It's very specific. It's talking about the test the error message that the user is seeing and It talks about the behavior the user workflow the user behavior here Coming to the point number three how spec test helps us with the refactoring is This This is our trade booking engines diagram, right? So we often write our unit test quite low That is the smallest function or the smallest unit of your system you write it for that for example here So we have our valid security valid user valid stocks much more right you write the unit is entry point is here Whereas the specification test are written above it. They are on a higher level You need to find a level where you can actually form a user behavior and that's when you can write it out So since they are on a higher level It's it's not impacted by the refactoring The specification test matters about like what's an input and what's going on rather than what's happening in the middle so here and To be honest like when we were initially implementing the specification testing we had a challenge at this point because We had to shift our traditional unit testing, you know thinking to do specification testing. We have to See on a higher scale and and then form it over there and also like If you're doing a like a new testing you need to have the setup So obviously there's one time effort that we had to do and understand so that challenges with that Let's see some more specification test Test traders successfully borrow stocks. So the trade ID is generated. This is the first successful BDD scenario where it's a happy case It was successful and use a season message But with the message there will be a trade ID generator because it's a successful, right? So this test is talking about that bit where the trade ID is generated or not Let's look at the bottom two tests First is the test unknown trader not allowed to sell so error code 12 is shown to the user Next the test unknown trader not allowed to sell so error message with the trader ID shown to the user Both the desk are quite similar because they both are talking about the same scenario Except what they are trying to test that is one is talking about the error code is well It's shown or not and the other is talking about the error message and We wanted something like this and And Specification test actually helped helped our teams in many ways such as first is it was easy to read and That helped us to have an effective conversations with the product team We could literally pull this up and then talk with them about the scenarios and the behaviors What can you know go wrong or which is correct? second when we started writing the specification test we found In one of the surveys like there was an undocumented behavior Because now you have to check what you need to test at the or Output at the end of each test, right? So when we were trying to do that we found that there was one piece of code getting executed all the time and we had no clue So obviously we we checked and we did what correct thing has to do that we done next is I Feel like often engineers They understand the acceptance level or the acceptance criteria of a requirement if You have to deal like if the request and the response Deals with hundred fields or they place around with a hundred fields as an engineer You may not know the importance of all the hundred fields Again, there was like one time and we were writing the specters We found a conflicting behavior for one of the fields and we had no clue So we reached out to our product team we discussed what is a right behavior? And what's the right value should be going there and we fixed it So these are like the tiny takeaways and the learnings and that helped our team to you know With this kind of testing strategy some key takeaways Specification testing focuses on the testing the behavior and functionality of the software from the user's perspective It helps identify and prevent integration issues early on by testing the interactions between the modules If you write spec test before you implement Or before I do your actual code It will decouple the implementation for sure And it'll also help us to write or be on the right path of Effectively implementing TDD that is test driven development You can always write unit test. We are not never saying not to write If you ever find a piece of code which is missing out or not getting tested by Specification test which usually is not the case but still to be more assured and be more confident and have like a better coverage You can always go ahead and write the unit test. That's always a tool for you The main reason that we moved to specification testing was the efficiency we felt Somewhere that the efficiency could be done better. Our quality was great. It was always good and We wanted to be faster enough We didn't wanted to wait for longer time to tell our test so to what went wrong, you know, yeah That is nothing but the feedback loop We wanted to do the refactoring Work quickly. We didn't want it to update all our tests. We wanted our test being very specific and readable and You know what six months later if you look at the test, you have no clue what went wrong or in your documentation is old We didn't want to fall into all of these issues and this was a good solution for us and worked out Next is the specification test act as a living documentation Providing clear and comprehensive examples of how the software should behave Last but not the least Establishes the robust be feedback loop providing early and continuous feedback on the software behavior So we all know that the the testing triangle right at the bottom layer You have the unit test then the integration test and at the top you have the system level test So it's often said like at the bottom you have like many many tests That's why you have a lot of unit test and a fewer system level test as you go further on the top of the triangle there'll be fewer and The time taken to run this test is actually the reverse that is the unit test runs faster And if you go towards the top of the triangle, it is slower and slower Our goal was to do something on the bottom tier Where it's faster and yet, you know testable readable and more efficient for us and Yeah, definitely this this was our journey and you know, it was a successful journey That's a wrap from me. Thank you Thank you very much. This was very insightful So if you have any questions, you could move to the microphone and kindly ask she's to answer I have a question about testing more complex or complex systems when you could have like multiple Microservices working together. So I wonder how we can do that kind of testing in this kind of scenario when you Like kind of like what what would you do when you have like interact with five micro services to test some scenario or to Process some events messages How to solve this kind of situation? Would you have to like mock everything but then it's not like really testing in the end? That's a good question So, yes, like that is something that we do on day-to-day basis, but the thing is if you actually want to Do a proper scenario like end to end like where your request is flowing all the five Services and you want to see the responses and how it works. That's basically a system level test If you want to do actual hitting to all the microservices, that's the top of the triangle that I was just talking about The specification test that we went was for the bottom tier that is on the each service level We are still mocking out. Yes if there are multiple services included in it We have to mock it out But the question is at which level and to what extent that was a question because we again have interaction Test and you know you we all have like the testing suits that run on you create a PR Was this like the nightly runs and everything right? It's just the matter of that where do you want to do the changes and when how fast you want the results? So, yeah, I hope that answers your question Yeah, there's someone else over there. Yeah, so in our in our project we implement specification tests Like in cucumber so in a natural language as a natural language test and we find it really readable So it's really a living documentation. Yes, it's clear. It's comprehensive. It's good to communicate with the stakeholders But your specification test, do you implement them like with pie test, right? So these are just basically pie test test. Yeah, and do you have this cucumber also or something similar? We have behave For our integration test, but then I think yes, like when we did the specification test like in our services definitely we did it in python and Not too fancy, but like something what do you have seen a bit more complex with our actual things Okay, so we have also this level. So yeah, yeah, so when you talk about functional or spec testing, yeah So I've been in the industry for a long time. So we used to do unit test functional test module test system test and Finally non-functional testing and integration testing and still banks do do many of them Developers and have to spend so much time On testing you just have to think about test coverage. I think when you're talking about spec test, yeah, you're basically giving away the coverage focusing on the functionality A Bit more and yes, but one thing is that we are not giving away on the coverage. We still have a really good coverage I'll tell you the reason if you take the same scenario about the The happy case or the unhappy case where the things were feeling you were eventually hitting all those things It's just that you're not exclusively calling the valid user of the valid stock But if you say this Like valet and book this particular stock or whatever if you say Apple your code will eventually flown that die in that way So your coverage is still there with the specification test but I think Like you said you've been in the industry and slowly the standards have been changing Yes, there was once upon a time We had like a proper Q&A team who would do actual testing and we still do that We still have our own Q&A team But then it's more about the readability aspect as well because in finance things are really complex and we are so we are software engineers and So to bridge the gap and to enhance ourselves and not to be lost all the time of what we did a year ago or six months ago or three months ago This was really good or someone who's like newly joined our team if we want to explain them what we are doing It was next to impossible unless until they have like a good grip on the finance But we still did that even before the specification testing we had our documentation everything But that is not the case in most of the time So this it will be really easy, you know So that was the holy we still have a good coverage But we are trying to do better in our aspect. We still have the Q&A on all the testing there. They're there Yes, yes Hello, thank you for your lecture and I wanted to ask you how often do you mock some stuff in those Spec tests because as far as I understood if you do a lot of mocking in those Spec tests then those spec tests can become another type of unit tests that basically do not test anything so Just want to know maybe from your practice if there is some kind of a rule of thumb How not to give a lot of those mocks and stuff so hope this Understandable I think yeah, I think I understood like what you're trying to ask so I'll tell you from our experiences that when So basically for us there is obviously to some extent we are mocking because I'm not saying we are actually hitting all the other services No, we are mocking but they are a bit higher level. So You need to define your interface as well I mean your system needs to be pretty good only then your specification test journey is going to be easy Or else you're going to have a lot of troubles defining where to add this or where should your entry point be? So so that's actually all together different aspects. That's a different talk about the interfaces How clean it should be how good it should be what like that was like the one thumb rule Where your services should have good implementation or the designing? I think yeah that you can see Okay, I think we have a few more questions. Thank you for your time guys