 As you can see my name is Ivan. I'm not even going to pronounce my surname. Some time ago I started working in KDE and that's mostly my background as far as Qt is concerned. I love giving talks, I love teaching and obviously I'm a functional programming guy. If you ever saw any of the presentations I had at Qt Dev Days or Meeting C++ or somewhere else. So first a couple of disclaimers. I usually start my presentations with... What the hell? So at least you can read what's written. So the first disclaimer is that you should make your code readable because somebody who can read your code afterwards might be a psychopath and you know where you're located at. That doesn't mean that you should actually write the code like you're writing for a high schooler. You should write advanced code and proper code, just don't use stuff that nobody will understand ever. The second part is that these are slides, this is not production code and blah blah blah. So why I decided to talk about the Qfuture. Quite a long time ago, the first time when I was at Qt Dev Days, I was talking about something related to futures and I mentioned Qfuture and everybody started laughing because we all know that Qfuture is a really crappy class. And because of that, from that point on, I always put some code snippet of Qfuture so that the audience can laugh and always I say okay, we are not going to go into a rant about the Qfuture and this time I decided to actually do a rant about Qfuture and you will see why. So first, let's talk about the future concept itself. In essence, the normal way to call a function is you call a function, when the function completes whatever it is intended to do, it will return a value and then you can use that value. But what if that function takes a lot of time to complete? What if the function is, for example, you're asking the user to type you're reading something from the drive, which can be slow, you do network communication, which again, you cannot guarantee that it will happen quickly. Instead of having a function which returns an actual value, what is the future? Okay, we usually build with this with callbacks, signal slots and creating a new thread and do the actual work on the thread. And our code starts looking really crappy. Instead of having a normal flow of logic like the diagram on the left, you have actually to split every single thing that you do into a call and callback, call and callback and then you end up with something that looks like one of my favorite things. This is a spaghetti code by George Hart and this is actually how a lot of cute projects really look like. And although this is really nice as art, it wouldn't be really nice as project design. So instead of having calls, callbacks and directly calling functions that return values, we want to have some advanced, simple but advanced concept called the future. What is a future? It's a handle that will eventually get a value. So you're getting a handle like a small box in which somebody will at some point in the future put the actual value. And that's a really, really simple concept. Now, how do you get the value from the future? The most obvious way when you see the API is to call.get. Who can see the problem in this pattern? Get is going to block. Essentially, somebody gave you a future for a reason. It's an expensive operation which will yield its result much later. And then you call it .get and you're making a synchronous call. Like a normal function, you're blocking your main code. So this is something that you absolutely should never do. There are use cases for .get, but we are not going to go into those. Some of the libraries that you will be able to use with futures don't actually allow you to call .get on a future that is not completed. You will get a segmentation fault or an exception or something. Unfortunately, both Qt and STD send a library block when .get is being called. So what you should do instead of calling .get, when you get the handle, you don't care about the result. You just want to tell, okay, this is my box which will eventually get a value. And that box, you should explain to that box what it should do with the value when it gets it. So you don't anymore, you don't care about the value that you will get. You're just explaining the future, what it should do with the value once the value is available. So in this case, some set of the code. We have a handler and we want to, when the future arrives, we want to call a lambda on that value. But it will look like this in C++ 17, the middle line, handler.dan. You can pass, similar to this, not exactly, you can pass a lambda that will be able to use the value once it's completed. And fortunately in C++ 20 something, we will have a rate which will be a magical thing and it will be inspired by mornads and stuff like that. So we are going to skip that unless somebody is interested. If somebody is interested, you can ask at the end of the talk what the rate will be like. Okay, one quote from John Carmichael, we don't care. So in his blog post about functional programming in C++, he said that usually most of our bugs are because we don't understand the state of our program and in which different states our program can be in. And when you actually add threading and synchronous computations like futures and everything else, you just create much worse problems for yourself. So in C++, we have a couple of libraries that provide us with futures. One is the STD future, which is not so great at this point. In C++ 17 it will be nice. Boost future, which is what STD future will be. So it has .dan, it has everything that you wanted to do to have. And we have Folly future, which is a Facebook library. Folly future is designed after the same class in the Scala programming language. And this is the one that will throw an exception if you call a .get on an unfinished future, which is a beautiful design because it will force you not to use .get. And obviously we have the Q future and here begins the rant. So Q future started its life as a part of the Qt concurrent. And the idea was if you need to process a large collection of data, for example, some huge list of numbers. Qt is helpful enough to spin a few threads, give a part of that collection to each of the threads. Each of the threads calculates its part and then it returns the final value, for example, if you want to get a sum of a collection of numbers. So you had operations like filtering, mapping, so map reduce and everything else. And the main use case, especially with the .get method was the fork-join pattern. So you fork your process into a few threads. When all the threads finish, you join them and you get back the result. And that works really well for the use cases that Qt concurrent does. Unfortunately, in Qt 5, the idea was to make the Q future much more useful. So Mark from KDAB argued that Q future should not be in Qt concurrent, that it should be moved to Qt Core because it's a really useful concept. And it can be used outside of the Qt concurrent. So on the road to Qt 5, this is one of the mails Mark wrote. And finally we have the Q future as a part of Qt Core and all is well. At least that was the idea. The problem is that Q future is still meant to handle only thread-based asynchronous operations. So for example, you need to have a thread pool in order for it to work correctly and everything else. And still Qt has a lot of classes inside or a lot of methods that could have used Q future, but they're not because Q future is actually not meant to be used as a proper future. We have a Q meta object invoke method, which is something that can invoke Qt's meta methods. And from the documentation, you can see if the invocation is asynchronous, so you pass the Q invocation argument. The return value cannot be evaluated. Why? Why doesn't evoke method return a Q future? The second thing we have at least one class that is similar to Qdbus pending reply, which is who knows Dbus? Okay, so Dbus is IPC protocol on Linux. You call something, then you get the result. And because it can be an expensive operation, it can take some time. The pending reply is something that will allow you to register and know when the reply actually arrived. What is the difference between Qdbus pending reply and the concept of futures? Absolutely nothing. It's again, it's something, it's a handler, to a value that will appear sometime in the future. Then we have Q network reply, again, you're asking for something, you're getting a value at some point in the future back. And that thing also is not a Q future. So we have at least, I've listed at least three items from Qt, one item from Kd frameworks, and all of those essentially are futures. But we cannot use them as futures because Q future was not meant to be used like that. So why it would be really a nice thing to have a normal future that can model everything that we already have seen. Sometimes you want just to implement your own stuff, and you don't care whether your process result function will be called on a network reply, on a Dbus reply, on a user input, or anything else. You just need to pass it and tell it, this value will appear sometime in the future. You can't do it, you have to write process result for each and every of the Qt classes. That can return you a value sometime in the future because Q future is not the super class of all of those. And Q future cannot be legally used to model all of those. So it would be really, really beneficial if we could use Q futures in a normal way. So the next thing, if we kind of see for what we can use Q futures and we told that it's now Qt core and not Qt concurrent, how can we create a future, an instance of it? First thing, you open the API, you see there are two constructors. One constructor is the default one and one is the copy constructor. The default one creates an invalid future and the copy constructor obviously copies the future. So from the normal part of the API, at least as far as the Qt core is concerned, the only future that you're ever going to be able to create is an invalid future and copy it around. Which is again completely useless, right? Then you open the API documentation and what do you get? From the docs, to start a computation, use one of the APIs in the concurrent framework. So in a class that is in a Qt core, the only way to construct it is by going through Qt concurrent. So it's not really separate, right? And that's one of my favorite pet pills with Qt future. How do you get the value? You can call dot get, which we already said it's a bad thing to do. Alternatively, if you want to chain a continuation to a future, instead of calling dot then, which we don't have in Qt, we need to get this snippet of code. Because Qt future is not meant to be used like this. It's meant for the pork joint pattern. So let's continue. There are quite a few peculiarities regarding Qt future. Usually when you say this is a future of integer, you mean after a while an integer will pop up, right? With Qt future, it's not really a single integer. You can get a list of integers. A single future of int is not a single future value of int, but a series of values of integer. Again, why? Because it was useful for Qt concurrent. So whenever you see Qt future of t, you should just think for yourself in your head it's a Qt future of a list of t's. Okay? The second thing is that it can store an exception. So for example, you called the Qt concurrent run and the function that was run asynchronously throws an exception. It will be stored in the future so that you can access it somehow. The problem is that you don't have the API to access it. The only way to access the stored error inside the Qt future is to actually call .get to force the C++ runtime to actually throw the exception again to catch it with a tricad block, which is everything but efficient. And that's something that I actually tried to push a patch to Qt and got rejected because who would ever need to actually check for an error inside the Qt future? But who cares? The last strange thing is that when I tell you, you ask me for the time and I'll tell you later, I'm just doing something else. Is it normal for you to in the middle of my work to tell me, okay, pause that? No, when you say I want a Qt future it's not something that you should be able to pause. It's not a job, it's not a task, it's not called Qt job, it's called Qt future. But Qt future has set pause, it has canceled, it has a lot of methods that are just not meant for futures. And I think this is the end of the rant. So since the Qt future is a really nice concept and it's ABI stable so you can use it in your APIs and everything else, it's really a shame that you can't really use it. So I decided to take a dive into the code and see how the Qt future is actually implemented and whether I can count on the internal implementation to be API and ABI compatible. So every Qt future that you have is instantiated by a class called Qt future interface which you can implement for yourself. So we are now going to see a couple of examples. Obviously, if you read the first paragraph we are going to pretend that we are legal programmers, we are not going to subvert C++ to actually access private parts of Qt future. We are going to do just the things that C++ allows us to do legally. And legally, I say... Okay, so what is the simplest way to create a future with a predefined value? You create a new Qt future interface, create a future from it, then say, okay, this future, the thing that calculates it has started, it reported the result which is specified by the value variable, and you report that you have finished. Now you have created a future which already has a value. Unlike the default API which only allows you to create a canceled future, now you can even create a future that already has a value, not much use for that, but it's a start. If you want to create a future that has an error, it's almost the same just instead of reporting a value, report an exception. Okay, what if you want to create actually something a little bit more advanced? For example, you want to create a delayed future which will be able to return you a specified value after some amount of time. For example, you want to get, you're contacting a server and you want to get a default value after a minute if the connection broke or something like that. You want to have a delayed future with default value. You would inherit from QObject in order to be able to use Qtimers properly and then you inherit the Qfuture interface of T. How do you implement the method that actually returns a new future called start? Simply the same code that we already had, you say report started and when the timer hits off, you report the result, you report it is finished and you delete later the delayed future interface. So now we have a little bit more advanced block. Instead of just having the rudimentary, we already have a value. We are now able to create a future that will actually return us a value after a specified amount of time. And because I was bored at some point in my life, I decided to start writing a better library just as a playground thingy that kind of digs through all of the Qfuture backstuff in order to provide an API that could actually be used. So how do I create a Qfuture when you have wrappers for a lot of stuff? If you want to create a future that is already here, you just write make ready future. This is a pendant for the make ready future that will be in STD, the standard template library, for the STD future. If you want to create a canceled future, you can create the void one or the integer one or whatever you want. You can create, instantiate the thing that we saw earlier, make delayed future. It will return the value 42 after an hour and 30 minutes has passed. And by the way, this works obviously only in C++14. Now, these futures might have not been that useful. You can create wrappers for a lot of different things. For example, debas as in call. Instead of returning a debas pending reply, it will return a proper future which you can pass on to an API that requires futures. Then you have process get output. It will wait for all the whole process to finish and it will return your result. Again, exec and similar stuff. For everything that you actually need, you will be able to create a wrapper that in as efficient as possible way creates a new QFuture. How do you get a value from a QFuture you don't? One of the benefits of the strange implementation of the QFuture proper is that if you don't base it on threads and thread pools, when somebody calls .get on your customer made future, he will get a segfault. That's one of the things that I actually like because that way anybody who actually uses my library that returns him a future will have to avoid calling .get, which is so freaking cool. Some people actually thought it was a bug, but I took an hour to explain that it's a feature. So you never should call .get on a future. What you should do is pass it on to something else. So what have I started implementing? Most of these things are functioning properly. Some of the things need to be continued. So one of the things that you can do on a future is pipe it through a transformation. So for example, you want to get user input, some block of text, and you want to have length of that text when that text appears. You just need to do input, pipe, transform, transform with what function with Q string, column, column length. Who has seen the boost range or the range proposal for C++ null 20 something? Okay, so the approach is the same. You have a collection and you pass it through a series of transformations. Something like SQL, something like shell programming and stuff like that. And a future is again, it's a kind of a collection. It's not a collection that immediately has all the data, but it's a collection that will sometimes, at some point in the future, have data. So it's kind of obvious that we should be able to use it as if it were a collection. Alternatively, you can pass the input through a filter. You got an array of lines from the user. You want to validate and remove all the invalid ones. You just pipe through a filter function. Okay. And just remember, QFuture of string is essentially QFuture of list of strings. So the filter here will get a list of strings and just remove all those that are not valid. Now the next thing. Both Q string length and input validation functions are fast functions. They are not functions that will block your main thread or anything else. What should you do if you actually have a couple of functions that you want to chain and all of those functions should be asynchronous? So all of those functions should return a QFuture. For example, I want to send a request to a web page. That web page will return HTML code to me. I will parse the HTML code and then send a synchronous request for all the images that are inside that web page. So all of those are kind of asynchronous functions. You can't do anything apart, maybe from parsing and extracting images. Everything else needs to be done asynchronously in order not to block the main thread. So what can you do? You can just pipe it directly to something else. And this is something that in functional programming is called the monadic bind. So you have two functions that return a QFuture. They ask for a value and return a QFuture. And you want to compose them so that when the result of the first one, the future arrives, it automatically passes to the second one. If you use the transform like in the previous slide, what would happen? Input transforms some function that returns a QFuture. The whole type will be QFuture over QFuture over value. Okay, let me repeat. So is it obvious what transform does? Okay, so you have a QFuture of some type. Some type T, T1. And you have a function that takes a T1 and returns T2. Okay? When you call transform on a QFuture of T1, it will return a QFuture of T2. Okay? But if that function actually doesn't take a T1 and returns a T2 but takes a T1 and returns a QFuture of T2, transform would transform from the QFuture T1 to a QFuture, QFuture T2. Okay? It's a little bit sick, but bear with me. And QFutures of futures are really useless. So there is a method called flatten, which would get an array of nested futures and just give you a proper future of T. Instead of always chaining transform and flatten, the API just provides the ability... Okay, this is the wrong slide, sorry. The ability to directly pipe through some QFuture returning function, and it will automatically do the transform and the flattening. Okay? No? Questions? Yeah. That's why I said monadic bind. And usually I do talk about monads. Apparently you should have said that in the microphone. I'm recognizing monads all over the place. Okay, so a member of the audience said that he was recognizing monads and they said that, yeah, this is all monadic, monad-based. Essentially QFuture is... general concept of futures is one of... aspect, one of the different types of monads. But I didn't want to actually talk about monads this time as well. Okay, so the idea here is that you should be able to use a QFuture and compose as many functions as you want to it until you get the result that you actually want to have. And when that result arrives, you just process it and pass it to the handler. Unlike the normal Qt signal slot stuff, in signals and slots you have a signal and you connect it directly to a function that will process the result. Here you can chain as many things as you want before actually connecting it to the handler. Okay? And the usual things that you should always have in all libraries like this, if you have a collection of multiple futures, it would be nice that you have the API that will return you a future of the collection. And others. I think these few should be quite obvious what they're doing. Okay, so this was really fast. I thought that I'll have half an hour for my talk. Now, what are the limitations, even with this idea that you can compose futures as much as you want? The limitations are that although the future of T is a future of list of T, you can't really use it in a normal reactive stream kind of stuff. For example, one of the useful things to have is to map all the user input as a stream of events, right? So you have every time that the user moves a mouse, you want to have a stream that returns you coordinate, coordinate, coordinate, coordinate. Okay? And for the first coordinate, you can use the QFuture because the first time the user moves the mouse, you'll get the value for the future that you created. The problem is that you cannot model the whole stream with QFutures. Why is that? Because QFuture keeps everything in memory. Every single value that you have in the list will always be in that future. So it's completely unlike, for example, standard input streams. Just imagine if the standard input stream remembered all the lines that you read before, the line that you currently are reading, which would be completely useless, right? The QFuture unfortunately doesn't have the ability to forget some of the previous values. So you can't model a lot of advanced stuff with QFutures. You can essentially model just the simplest ones like I have a single value or a couple of them. Okay? I don't think that anything else is worth mentioning. So, now questions. Okay, before the questions, one thing that I didn't talk about is how would you design your software so that you do these transformations, filtering, and everything else, and to design huge programs with it? I didn't talk about it mainly because I talked about it in the previous year, and because you can't use QFutures for that. One of the great ideas of functional programming is that you have some input. You transform it, transform it, do whatever you want with it, and then you have the values that you want to use. And that's something that I was hoping I would be able to do with QFutures. Unfortunately, because of the simple limitation that, for example, this thing with the memory, you can do really, really rudimentary stuff with a QFuture. But the idea of having something like a QFuture that you can pipe through transformations, that you can chain, that you can do whatever you want, that is a really, really powerful and useful thing. Unfortunately, QFuture will not help you with that. And now questions. Here's your hand. Actually, why you based it on QFuture or not on the next standard future? I don't see 17. The reason why is that I maintain one of the frameworks in KDE and it is really D-Bus Harry. And the first version was synchronous and it blocked everywhere because D-Bus is freaking slow. And I decided, okay, I need something like a QFuture. Should I create my own, which will be ABI stable? Probably not because that framework is not for creating the futures. Then I considered STD-Future. Unfortunately, it's not ready. And unfortunately, KDE frameworks are still pre-C++, C11. We need to remain compatible and everything else. And STD-Future doesn't really guarantee ABI. At least GCC's implementations sometimes break ABI from major versions and stuff like that. And the reason is that I use QFuture because it was in Qt. I was able to hack through it to actually achieve what I wanted without much overhead like spinning off threads are necessary and keeping the ABI. But yes, I would have probably gone for the Facebook one. First, then Boost and then in the future I would choose the STD-Future. Are STD-Future supported? Sorry? Are STD-Future supported? Share. Honestly, I hate anything that has the share name inside. Essentially, okay, I'm not in the real world. I'm after all a free software guy, right? But anytime that you have concurrent stuff, anything that is shared is bad. So you should either not share and keep things to yourself or you should not have concurrency. Anytime you try to mix those, you need to lock. You need new taxes, you need all of the things that will just slow your software down. And you should all, if possible, you should always avoid it in your design, software design. Next. Just a quick one. Could you do me a little favor and just share the presentation please, if possible? Which part? What I want to say is share, the presentation, the file. Sorry? I think... Sorry, I heard you. Could you show? Okay, let's go. Okay, okay. There was a question in the back. Although I would recommend not looking at this one. In the past year I was talking about reactive queue. And this year at Meeting C++ I've been talking a little bit continuing on the reactive stuff. And usually I talk about these things but in a more useful way than just talking how bad queue future is. But I will post it. I have a question regarding this get how you hate it so much. Because I think get gives you synchronization. And that's sometimes good thing, I guess. Synchronization is sometimes necessary. I wouldn't say it is good at any point. There is one really nice comment about New Texts in general by David Butenhol. He said that in one of the mailing lists he said that we shouldn't have called the mutex mutex because it's a cute name and everybody likes it. We should have called it bottleneck. So bottlenecks are sometimes necessary. They're useful. But you should never put them in your code if you don't actually need to. If you design your software in a proper way you should never be... You should never need to do ecstasy synchronization on a top level. In the library it's okay to build to something but I would say that most of the time unless you're doing parallel processing graphics and stuff like that which is the fork drawing pattern what it's made for I would say .get should be destroyed. Essentially, if you want to have a system asynchronous as soon as you call .get it's not and that destroys the idea of having the asynchronous system in the first place. So useful, yes, sometimes. Good, never. It should work in a reasonable predictable way. For me, the reasonable way for .get is to crash. For example, you have a pointer that is invalid. When you try to dereference it, what does it do? It's a segfold. When you try to dereference a future it's not yet here, it should be a segfold. I'm not saying that this is the only way to think about it but it is as valid or more valid than let's block our program until the pointer gets valid. In this way, yes, but if we get as a synchronization point it's different. I do agree. But as I said, I don't want to treat it as a synchronization point. The last question. But even if you don't use it as a synchronization point it gives you access to the exception. How would you handle the exception in another way if you only passed the value to the receiving function, to the handler function? You wouldn't think this way. For example, when you have a queue future and you do a transform something if it's a value it will get transformed. If it's an error it will produce a queue future that returns an error. So that's one of the things again, I didn't want to mention more nets but the idea is you have a box. If you have a value, do whatever you want with it. If you don't have a value you have a box. We'll inherit all the backlog from this from this box. So whether it's an error, whether it's an a log how that value was produced or anything else. So you would handle the errors in that way. Thank you. Thank you for this talk. Thanks to Yvonne. I'm not going to try my surname, right? You don't want to try to pronounce my surname? Oh no, I tried to avoid it. We have 15 minutes pause and the afterwards is going to be the talk from last in this room. So come back. Thank you.