 Let's estimate what is the success probability for a payment of a certain size to be successful on the channel, and then select the route from me to Andreas that has the highest probability to settle. This change alone was an incredible step forward in getting more speed on the Lightning network. It seems like an understatement to say it, but it's been quite a year. With Bitcoin near new all-time highs, with the first nation state having adopted Bitcoin as legal tender, the explosion of NFTs continued growth in DeFi, and a lot more than that, it's incredibly hard to keep up. So on today's show we're going to check in with one of the most exciting projects, which might have fallen off your radar. Bitcoin is still in development, yet rapidly growing Lightning network, arguably the most anticipated scaling solution for Bitcoin to date. Although there are still some hard problems to solve, things are really starting to heat up, and to help us understand it, we're joined today by a special guest who co-wrote the book on it, Mastering Lightning Network, published by O'Reilly Media, and available for order now. But before we get to that, I'm Adam B. Levine, and this is Speaking of Bitcoin. For today's discussion we're joined by the other hosts of the show, Andreas M. Ansonopoulos. Hello, Stephanie Murphy. Hi. And special guest, Rene Picard. Hi, everybody. Rene, I really appreciate you being here, so I'm looking forward to this conversation, but in the introduction I neglected to mention that Andreas is actually another co-author on the book, which is his third O'Reilly book. He's also the author of Mastering Bitcoin and Mastering Ethereum, and now this most recent book, Mastering the Lightning Network. So to kick us off, Andreas, can you help some of the members of our audience who haven't been with us for years and years and might not be familiar with the Lightning Network? Just from a high level, what is the Lightning Network? The Lightning Network is a second-layer payment network that runs, let's say, on top of Bitcoin, and it enables fast, secure, private, and cheap payments that were previously not possible. A nice way of putting it, which I think Rene came up with this way of putting it, is Lightning is just a different way of using Bitcoin. So Lightning is Bitcoin, Lightning is Bitcoin payments, but it's Bitcoin payments done with a type of smart contract that allows you to execute them really, really fast between two people and route them across the network for very, very low fees or negligible fees, really. OK, great. So I think that that's a great succinct kind of way to think about this. Again, like it's a way that we can take kind of the limited in terms of scalability, right, the number of people who can be using kind of base layer blockchain at any given time, the higher that goes, the harder it is to run those nodes, which makes it so that you can compromise some elements of decentralization by pushing up that transaction throughput too high. Basically, it comes at the cost of some decentralization. So the Lightning network aims to kind of take that reality and then to build layers on top of it that then allow for people to use kind of that base ultra secure layer as sort of the settlement layer, which kind of eliminates the need to on top of it use, you know, a service like Coinbase or something like that, who's actually going to custody your stuff. Ultimately, you're going for the lower prices in terms of the lower cost for higher throughput, while at the same time, you're keeping the non custodial characteristics that makes Bitcoin so kind of compelling from so many kind of different angles. Is that a good summary? Yeah, let's look at some specific numbers. Now, people who are familiar with Bitcoin payments know that when you make a payment with Bitcoin, if you need it to be confirmed because it's something you're buying that requires the security of a confirmation or several confirmations, then you're looking at waiting on average 10 minutes per confirmation per block until that payment is finalized. And you can make a payment as small as you want down to one Satoshi with Bitcoin and as big as you want up to 21 million Bitcoin. But you're going to pay a fee. And with today's network fees can range anywhere from a dollar to five dollars, depending on how busy the network is at the moment. And of course, that changes a lot with the price of Bitcoin and the capacity of the network and things like that. But you don't generally make a three dollar cup of coffee purchase on Bitcoin on chain anymore, because not only are you going to need to wait 10 minutes, which for a three dollar cup of coffee, you don't really need a confirmation, but you're going to end up paying about a dollar in fees. And your transaction is going to be public for all the world to see your payment is going to be visible on the public ledger. Now, let's compare that to the Lightning Network. Now, you can make a payment as small as a thousandth of a Satoshi. That's the smallest payment you can do on the Lightning Network. Although for practical terms, that gets rounded up to a Satoshi. A Satoshi is one hundred millionth of a Bitcoin. So it's a fraction of a penny. And instead of taking 10 minutes for full final settlement with Lightning, you have your funds, your payment completely cleared anywhere between one and 10 seconds after you send it. So a second, two seconds is very common. A common experience for people using the Lightning Network is you're standing in front of a beer dispenser, like Renee and I did at the Bitcoin Citadel a few months ago. If that takes Lightning payments, you press pay on your wallet and you'd better be holding the cup underneath the spigot because beer starts shooting out within a second or two. So the payments actually are surprisingly fast. The point that beer was spilt, the fee you pay for that is somewhere between a hundredth of a penny to a tenth of a penny for a payment maximum I've seen was a couple of pennies on the US dollar. So just a few hundred Satoshis, a few thousand Satoshis at worst. And then that payment is completely private. It's between you and the recipient. And no one else really knows is coming for you or going to the recipient and no one sees it as a permanent recording on the ledger. I'm simplifying, of course. There's a lot of nuance to all of these things. But it's a radical improvement on the user experience of Bitcoin. So it's faster, it's cheaper and it's also more private. So what are kind of the reasons why everybody isn't using this today? Well, mostly because it's still new. You need a special wallet to use it. And the wallet interfaces were really quite clunky. The other thing is that until now, routing your payment across the network is sometimes challenging. But that's one of the things that Renee is here to talk about today, which is a way of making payments even more fast and effective with an invention of his called Picard Payments. So we'll talk about kind of some of the challenges of the Lightning Network, as well as some of the solutions and how it's very rapidly evolving. You got to remember this is new. It's still a bit clunky. The interfaces are still maturing. But already it is very usable. All right. So this is all new stuff. It's definitely one of the things that's new to me because, you know, I don't have a lot of technical experience. So how would someone like me, who's kind of more of an average user than a power user, use the Lightning Network right now? I would say the first thing would be to get a wallet. Renee, which one would you pick? I would pick anyone. I mean, I run my own notes, so that's probably not the suggestion that I would make to Stephanie. I guess for the casual user, there is a lot of either custodial wallets that run out of the box or semi-custodial solutions that work. I think Phoenix Wallet usually is a good choice for Android users. I think they're coming out with an iPhone version. I'm not sure if it's out yet. Yeah, I would say Blue Wallet or Phoenix Wallet are great first choices. These are wallets that either do just Lightning or they do both Bitcoin and Lightning and they're semi-custodial. So you have control of the keys, but they may help you with routing and other facilitated services. OK, yeah, that was going to be my next question is what is the level of custodialness, I guess, and explain that. OK, so first I have to get a wallet and then I don't need to run my own node to be able to do this. I can just start using a wallet. That's correct. The wallets are lightweight wallets that use the neutrino protocol to talk to a Bitcoin node and they may also use some services such as routing services by the wallet vendor or the wallet maker in order to assist wallet payments, but you don't need to run a node. It's just an app on your smartphone. OK, and then do I need to put in any the channels need to be sort of greased, if you will, right, with like funding? Is that right? Well, in the early wallets, you had to fund a channel, make a channel and do all of that by yourself. But one of the nice things about the newer wallets is that they make a lot of that complexity disappear. Part of that is because they offer some services in the back end to help you do that. So both Phoenix and Blue Wallets, for example, and you should look there, there's lots of wallets out there now. You don't have to even see channels. You don't look at channels at all. You just fund it. So you move Bitcoin to it, generate a Bitcoin address, move Bitcoin to it. And now you have Bitcoin in your lightning wallet, and it manages the channels automatically for you. OK, and so what if I didn't want to send at first? What if I had nothing in my wallet and I wanted to receive what I still have to fund it, or can I just receive? Well, that's where the service comes in. You can just receive and they'll fund a channel for you on the other end and charge you a small commission. I think last time I checked it was something like a 1% commission. That's if you don't have any funded channel already. OK. And it depends, of course, on the size of the payment. I mean, if you wanted to receive one Satoshi, it doesn't make sense to open the channel with you and to charge you 1% of the commission, but it's reasonably cheap. OK, well, that sounds pretty good and easy. What are the problems with it right now, if any? Really a lot about learning and how smooth these wallet interfaces are. The other problem is, of course, the network is still growing. And as a result, it's not always easy to make a payment. So when you try to make a payment, sometimes the payments will fail and they'll fail because your wallet isn't adequately connected to the network. You don't have funded channels. It's unable to find a route to your recipient, et cetera, et cetera. But those problems are gradually going away. So you see less and less of that as time goes by and the network gets better connected. Just to give you for reference, right now the network consists of some 12,000 nodes that have some 50 or 60,000 channels between them. OK, so please forgive me if these are like kind of basic questions, but you can tell that I haven't had much experience with this. But I think some people want to know. So sometimes if you try to send a transaction over the Bitcoin network and you don't include a high enough fee, let's say, it can get stuck and it'll be stuck until it gets unstuck, which can be for hours or days. So what about in the Lightning Network? If you try to send a payment and it fails, does it fail right away? Or does it tie up your transaction for a while? Hopefully it fails right away in most cases, it should, right? Because if you do something like not including enough fees, the routing note will basically immediately reply to you and say, hey, I want out the payment you have to add a fee or if the routing note doesn't have the liquidity, it will tell you, I cannot forward this payment. There are situations and circumstances because it's a decentralized network where a note is just unresponsive and you just don't get feedback. The time that you might have to wait is much more limited than Bitcoin because in Bitcoin, I think I heard cases where people literally waited for three months until the mempool cleared. This cannot happen in Lightning, but you might have to wait quite a bit. OK, well, that sounds better that it should fail right away instead of waiting potentially up to three months. OK, I think one of the interesting things about the Lightning Network is that it opens up all of these new application possibilities that happen because you're kind of exploring this new dimension of time and space. So in terms of space, the size of the transaction is so small that you can effectively do transactions that are infinitesimal amounts. So you could pay, let's say, a tenth of a penny, which is an amount for which there is no even copper coin in the United States. Right? You can't buy anything for that kind of money. So we've seen applications, for example, where people have to pay that kind of amount to post a message in a form to participate in a chat. So then it becomes purely an anti-spam mechanism. You can't sock-pop it and spam the form because while for the average person, a tenth of a penny to post the message is nothing, if you wanted to do it at scale, then it's not nothing. It's something. And so it becomes an anti-spam measure. Or you could use it to play a very, very simple game where you could have tiny Lightning payments as part of the game. So it opens up these possibilities with micro payments, not just micro payments for content, micro payments for gaming, but also micro payments for various applications and services that you wouldn't really consider payable, where it becomes almost like a spam control measure. Yeah, to me, it almost sounds like an improvement in every way. Like why would you not use this, except that the usability maybe isn't quite there yet for someone like me who doesn't have a lot of experience. No, I fully agree. That's the question I've been asking myself. Why would you not use this? That should be the slogan of the Lightning Network. The Lightning Network. Why not use it? So the onboarding might be a little bit rough. But other than that, once you are just out paying, I think the people in El Salvador right now where, you know, they had this adopting Bitcoin conference, I think most people paid a lot of things just with Lightning Network all the time. And I basically didn't hear anybody complaining that it wouldn't work for regular payments, like doing a cab or buying coffee, buying a meal, even paying the hotel. Conference tickets could only be bought with a Lightning Network. And these things, I mean, this is how money should work. You see somebody, you want to pay them, they show you QR code. You hit pay and a couple of seconds later, the payment is settled. Everybody's happy. It's just incredible. And the more widespread it is used and the more people they use it densely in a specific environment, the more usable it becomes. So where in Bitcoin, if everybody tries to jump on the network at the same time, it all goes to shit in the Lightning Network. Once you've onboarded and funded the wallets, which requires a Bitcoin transaction to do the channel open in the beginning. Once you've done that, then the more people actually using Lightning and moving Lightning funds back and forth and creating a closed loop economy, the more usable it becomes, the easier it becomes, the more easily the payments are routed, the more liquidity is available on the network, the more connectivity is available on the network. So we've been talking about the Lightning Network for a long time. I think I first heard about it in 2016, which is five coming up on six years ago, which doesn't feel like it was that long ago on the one hand. And also, yeah, it feels like we've been waiting kind of a long time for this thing to roll into reality in a way that's as impactful as the sort of technology of Bitcoin and cryptocurrency broadly that sort of underlies it. One question that I have is in the past, I feel like we've talked about this and it's been like, well, the network isn't ready for it, not the Lightning Network, but the Bitcoin network, the Bitcoin protocol needed to have upgrades that were made to it that would then make it so that we could really kind of see the full vision for Lightning roll out. Now, I think, and correct me if I'm wrong here, that there were some pretty significant elements of that that came into play with the recent taproot upgrade over the summer, notably Schnorr signatures. But am I correct about that? And I guess where I want to go with this question is on the one hand, what have we accomplished already, right? What upgrades have come to Bitcoin that have resolved some of those issues we've talked about in the past? And on the other hand, is there anything else that we're waiting for? Are we waiting for the next generation taproot? Or is it really just kind of an implementation and maturation, right? As these improvements sort of propagate out to the network, as new wallets are built? I mean, that's kind of what it is. Like talk about where we've come from and where we're going from a technology standpoint, please. So I guess the most important upgrade that happened to the Bitcoin network was the sacred upgrade. Because without going into the technical details, it would just not have been secure to open a channel if we didn't have the sacred upgrade. So basically, we couldn't really bootstrap the lightning network without it. And then you mentioned the taproot and Schnorr signatures upgrade. And that's a great one, because it enables us to have a much cleaner way of routing payments and of delivering payments with more privacy with a technique that is called PTLCs. That stands for point time lock contract in contrast to the HTLCs that we're currently using the hash time lock contracts. So this is really great. And everybody who is a lightning developer is really excited about this because things are just getting a little bit easier for us once we have implemented it. But right now, lightning developers actually really struggling and discussing how to use this Bitcoin upgrade to actually implement that stuff. So the concepts are out. Everybody knows how to do this on principle, but we haven't specced it out yet. One of the discussions on that particular point is what do we do first? And I think the recent conference where you participated, Renee, correct me if I'm wrong. From what I read in the transcript and notes, a lot of the debate was, do we do it all in a big bang upgrade where a whole bunch of things get upgraded at the same time? Or does this get rolled out incrementally with smaller changes that can be absorbed more readily by the network? That was kind of the essence of the debate. Yeah, that was a huge part of the debate because one thing that is problematic with this upgrade is that as of now, PTLCs and HTLCs are not compatible in routing. So if you want to send a payment along a route, either every channel supports PTLCs or every channel supports HTLCs, but you cannot mix them. So you really have to think about how to upgrade this, how to make this backwards compatible. And these are just engineering challenges. I wanted to also mention with respect to future Bitcoin upgrades. There is actually a lot of Bitcoin development right now being driven by the necessity of Lightning Network. A lot is related to fees and how the mempool is being managed. But there is another consensus level upgrade that the Lightning developers are wishing for, which is the any-pref-out upgrade or previously known as Secach No Input, which would allow us to have the L2 payment channels, which is a new channel design that is much cleaner than the current payment channel design and would allow us to not have this penalty that if something goes wrong, we lose our Bitcoins on the Lightning Network, which is one of the downsides that some people might have experienced in the past. Let's be more clear. If something goes wrong, the person who tried to cheat most likely loses their Bitcoin, which is the penalty mechanism. So within a payment channel, in order to ensure that someone can't cheat in the payment channel, there is a penalty mechanism. So if they try to cheat, you can publish a penalty transaction and take the entire balance of the channel. That's how it currently works. A cleaner mechanism would be that you simply don't let them cheat in the first place. A more of a can't cheat rather than cheat and you'll get punished. And that's what L2 would enable. And the problem with this cheating is that a lot of this cheating doesn't happen on purpose because somebody wants to try to cheat, but just because the software is so newer, so software might sometimes by accident be incompatible to each other in certain situations that developers haven't seen. And then this is considered to be a cheating attempt and stuff can go horribly wrong. I don't want to spread fear. A lot of these bugs nowadays are completely fixed. But I remember in 2018, it happened to not only me, but also a few friends and yeah. Well, back then it was hashtag reckless to use the Lightning Network and it was part of the design. In fact, that's actually one of the things that I like about the Lightning Network is that innovation on the Lightning Network moves at a much higher speed than in Bitcoin. And that's because the Lightning Network is effectively decoupled from the consensus layer in such a way that you don't need lockstep consensus and interoperability between all of the clients. You don't need them to have identical consensus implementations. They need to be interoperable. And that allows one client to implement a new feature without anybody else implementing it. Kind of incremental improvement, decentralized innovation, opt-in capabilities. And that's created quite a lot of competition between the various Lightning clients and node implementations. Unlike in Bitcoin, where 98% of all nodes are run Bitcoin Core on the Lightning Network, there's diversity. There's three very actively developed clients, node software, one called LND, the Lightning Network Demon developed by Lightning Labs, one called C Lightning written in the C language by Blockstream and one called Eclair written in Java or Scala, rather, by a French company called Async. So there's really quite a lot of competition. And as a result, there's quite a lot of innovation and things move very, very, very fast in this space. And just to add for completeness, these are only the OGs. There's also the Electron wallet written in Python that nowadays has a Lightning implementation. There's Rust Lightning, which I think nowadays is called LDK. There was a very early implementation by a Japanese group called Patar Gaman, I think. And then there is basically forks of other implementations. So there's actually a lot of people playing around with it. How does that landscape compare against the landscape of Bitcoin clients? It feels to me like the Lightning Network ecosystem that you just described there in terms of like interpretations of the protocol is actually more than we have in Bitcoin, at least that I'm familiar with. I can think of three or four off the top of my head, but that's pretty much it. And obviously, Bitcoin's been around for as long as any cryptocurrency has been around since it was the first. So I mean, like, how would you compare the two ecosystems outside of the fact that Lightning is clearly moving faster? Honestly, I would say that, and this is going to irk some people, but I'm going to say it anyway, the development culture and pace of development, the willingness to take risks, the degree of competition, the degree of diversity is much more similar to Ethereum than it is to Bitcoin. The Bitcoin core protocol is conservative by its very nature. It has to be. And it requires lockstep consensus agreement. Otherwise you fall out of consensus and risk losing or being cheated out of funds, which has resulted in almost a monoculture. There's one client, honestly, 90 plus percent is Bitcoin core. There is a go implementation called BTCD, which has a bit of traction. And after that, it's mostly libraries and frameworks that may or may not be very usable. And there aren't that many nodes running these alternative implementations. So it's completely different from Bitcoin. I'm not sure about the Ethereum comparison because I have never looked into that ecosystem. Andreas can say much more about that. But being active with the lightning developers, I have the feeling that still there is a lot of ethos from the Bitcoin community in there. So people are actually cautious about critical changes. And oftentimes as a developer, I have the feeling stuff could move forward a little bit faster. I think the main difference is that the lightning network is driven by a formal specification. So there is the Bolts repository where all of the functionality and behavior of the protocol is being specced out. And then if you have such a document, it's obviously much easier for people to just implement against this document. Whereas in Bitcoin, it was from day one that Satoshi said on the mailing list, I don't want to spec out the white paper. I have a reference implementation and the reference implementation is basically defining the consensus of the network and the protocol. Yeah, the spec is written in C++ and it's called Bitcoin core. So that makes a lot of sense to me because if you look at, again, like at the point that Bitcoin core in its earliest forms comes out, that is the actual proof of concept. That is sort of the idea that was rallied around because to do it hypothetically, people would have said, well, that won't work. So you do it in real life, right? You prove the concept works and you're off to the races. That is in contrast to something like Ethereum. Again, there was an extended period of public discussion, public interest, right? Many, many people participating and as a result, an ecosystem emerges. And it seems like that's much the same with lightning, is that it started from an idea that was then discussed extensively where people can have different ideas about how that idea should actually manifest in reality. And so you create this ecosystem that isn't there if you start from kind of the software as a basis. And again, with the really heavy consensus element, too, that actually makes a ton of sense to me. And it's kind of neat to see that happening on top of Bitcoin. My sense has always been that the lightning network, although it's not a Bitcoin specific system when it's finally done, really does represent a very interesting way for Bitcoin to kind of have its cake and eat it to our Bitcoin users, anyways, and again, like that competitive ecosystem, I'm always in favor of competition, right? It feels like ideas that are conflicting and competing with each other, bouncing off of each other almost always results in kind of more vibrant products that do better to address. It's simply because by nature of having that competition, you get from here to there faster. So I appreciate kind of going down that rabbit hole with me. We talked at the very beginning of the show about Picard payments, and I would like to explore that idea a little bit more. I think we have a decent idea of what kind of the challenges are that we still face as we move forward. So can you kind of talk to us about that specific problem that you're solving there and how you've solved it? Sure. So if you want to make a payment on the lightning network, you have to choose basically a route on how you want the payment to flow through the network. So let's say if I wanted to pay Andreas and I don't have a direct payment channel with him, I could say, hey, I sent something to Adam and ask Adam to forward it to Stephanie and ask Stephanie to forward it to Andreas. And this is in general how payments are being conducted from one peer to another. The problem with this mechanism is that if I select this route via Adam and Stephanie to Andreas, I don't know if Adam actually has enough funds in his channel to forward the payment to Stephanie. And the reason why I don't know this is because I can't often privacy is given as the main reason like we want to protect the privacy of the funds and the channels, but there is a much more important. Well, not sure if it's much more important, but a much more severe technical reason. And that is if I were to know what the channel balance in the Adam and Stephanie channel was, I would have to know it every time the channel updates. And this means I would have to know about every payment that is taking place in the network. And that would have the same poor scaling mechanisms like Bitcoin, where we also broadcast every transaction to every participant, right? So when we create the lightning network in order to scale Bitcoin, we wanted to get rid of this mechanism where we notify everybody about every payment. And for that reason alone, we can't know how the money in a payment channel is actually being distributed. So when I select this route, I don't know if this route is going to be successful or not. That being said, the lightning network as a protocol is by design unreliable, right? I will try to make a payment. It might fail. And then my note software will try another route. Just like the internet protocol is by design unreliable. And the term unreliable from an engineering perspective means something different than the plain English understanding. It means that reliability is built at a layer above. And so failed payments are simply retried until they succeed, just like failed IP packets are simply retried at the TCP layer. Yeah, and that makes sense because you can't know which is the correct channel beforehand because you don't know the balance. Exactly. And I just want to reiterate, I am so happy that Andreas is the co-author of our book because this is literally how the entire writing process went all the time. I was going with some very sophisticated technical detail like the typical researcher nerd and Andreas was like, Renee, we need to break this down. Yes, so the problem is, I don't know if the channel is going to be usable for my payment or not. And if you a year ago or two years ago looked at the scientific literature, you would find basically in every single paper about the Lightning Network, this problem being described. But in every paper would basically say, look, the smaller the payments are, the higher the chances that they succeed. They would always talk about success rate. And I was the first person to turn this around and say, hey, let's start with the success rate. Let's estimate what is the success probability for a payment of a certain size to be successful on the channel and then select the route from me to Andreas that has the highest probability to settle. And this change alone was an incredible step forward in getting more speed on the Lightning Network. That's not the full solution about what pickup payments are about, but that's the first very crucial ingredient. And I'm happy to say that C Lightning is the first implementation that implemented this probabilistic pathfinding. There's a block article written by Christian Decker. I helped them a little bit to evaluate this. And basically the number of attempts reduced by half, the time to complete also almost reduced by half, and the failure rate also reduced by half. So this is just incredible to see. Before you go on, let me just pick at that for a second. Given that we can't know the balances, what metrics are you using when you are estimating the probability? Yeah, so that's an excellent question. I mean, there's obviously an infinite amount of choices that we could do to estimate a probability. What I did is I actually participated in a research work together with my former co-author Sergey, where we tried to probe the Lightning Network and estimate what is currently in a channel. And from that data that we collected, we realized that the probability distribution is basically uniform distribution. What this means is that if there's a channel that has, let's say, 10 satoshis in them, every distribution, zero satoshi on one side, 10 satoshi on the other side, or one satoshi on the first peer and nine satoshis on the other peer, and so on and so forth, has the same likelihood. And we were very surprised to measure this. We expected that this distribution would look very differently. But for all the experiments that we have done so far, this is actually the distribution that we basically measure. And the nice thing about it is that's very easy to quantify in terms of mathematics, and it's very easy to use this as a cost function in our path-finding algorithms. Okay, in the example that you gave, you were listing a hypothetical distribution of 10 satoshis where it could go from zero on one side of the channel and 10 on the other to one on one side and nine on the other and so on. But not every channel has 10 satoshis in it, right? Exactly. They have different amounts, so how do you take that into account? So the main assumption here is that in order for a payment to succeed, it must not fail, right? So a payment, I mean, I know it's obvious, right? But it's easier to start with a failing case. So for the failing case, a payment will fail if on the channel between Adam and Stephanie, there is not sufficient funds, right? So let's assume Adam and Stephanie have a channel of, let's say, 10 satoshi. And I wanted to send, let's say, five satoshi. If Adam has zero satoshi, one satoshi, two satoshi, or four satoshi, these five cases, then the payment fails. But in the other six cases, where Adam has five, six, seven, eight, nine, or 10 satoshi, the payment does not fail, right? So basically when I want to make a five satoshi payment on this 10 satoshi channel, the likelihood for this payment to be successful is roughly one half. I think there's one point that's important to emphasize here, which is we know the initial capacity of the channel because that is created by a Bitcoin on-chain transaction called the funding transaction that is public. That is how it is announced. So when Adam and Stephanie create a channel together, they fund it and they fund the multi-sig address. So we know what the maximum total capacity of the channel is, which is that initial funding. What we don't know is after that initial event, how is that amount distributed between the two channel partners? So we don't know the split, but we do know the sum. And then the probability we have to calculate is what are the probabilities of different splits within that sum? And then the research that you did, Renee, shows that it's basically a flat distribution, that it's equally likely that any distribution occurs. Yes. Okay, and then based on the size of the payment you're sending, you can calculate the probability that it will fail or succeed based on that. Exactly, and one obvious result from this is that the larger a channel is, the higher the success probability is for a payment of a fixed size. And conversely, the larger the payment is, the lower its probability of success on this current lightning network. Exactly. And this leads to a very interesting side topic, which kind of unveils the second big part of Picard Payments. In the last six months, a very, very important capability has been enabled on the lightning network that changes the game. And that capability is called multi-part payments MPP. What this allows you to do is take a payment, split it into parts, and send it across multiple paths simultaneously, but atomically, and then have it essentially reassembled on the other side. That changes the lightning network from almost a stream network to a packet switch network. And if you combine the research of probabilities with the ability to split things into parts, now you have a different problem, which is what should the parts be? And how do you choose the best paths if you're splitting it into parts? Okay, so the flowing word is really a good word because now you're ending up with a flow that's like drops of water that could then coalesce back into a glass at the end. We're ending up with a flow. And actually, even in technical terms, we're ending up with a flow. We're ending up with a so-called minimum cost flow. So we're trying to find a flow through the network that has a minimum cost. And the cost here is the uncertainty of the channels to be able to route the payment, right? The success probability, we can translate this into a cost function. And yes, it's actually historically speaking, very funny. I was very critical of multi-part payments for a long time because while it was always obvious that if you split the payment, the smaller payment becomes more likely to succeed, now you have more channels involved, right? And well, you have to be successful on every channel, right? So you multiply this out, right? And this also lets the probability drop, right? So there is this kind of like sweet spot of how heavily do you split, right? And do you split into equal chunks or do you split into like not so equal chunks depending on which path you use? And yeah, I spent quite some time in the beginning of this year researching this problem. And together with my co-author Stefan Richter we found the provable optimal solution to this problem. Wow, that's really cool. One of the interesting things is that the result was rather surprising. At least it was surprising to me. So in my mind, because the more paths and parts you use, the more complex it becomes, I would assume that if you had a big payment, you'd probably just split it two, three times, maybe four and send it across two or three or four big channels. But Renee, why don't you tell us about your first production experiment and how many parts you split the payment into? Yes, so what Stefan and I did is we basically spun up a new lightning node and we implemented the method and we tried to send a payment of 0.3679 Bitcoin which is quite some amount. That's $20,000. Roughly, yes. And this payment was split into 345 different paths which were sent through the network. And that sounds really like a lot, but what we can see from this probabilistic approach, this is actually the way how we send it with the highest likelihood to deliver this payment. So if we would split more conservatively and have larger chunks, we would actually put more load on the network. Or if we split more aggressively into smaller chunks, we would also put more load on the network because with these larger chunks we would probably fail more often, right? So we would have to do more retries. And also in this split we had payments that were just like 10,000 satoshis and other payments were of size, I don't know, something like 4.2 million satoshis, right? So there was this huge range in which we split the payment to optimally deliver it. And that was just crazy to see and experience because until then everybody was basically like, yeah, the Lightning Network works for a few million Bitcoin, but that's about it. It's like for small payments. And we basically showed, no, it's actually possible to use it for large payments. That's a game changer right there. And it's useful to also understand that when you take a big payment and you split it into 300 and some parts, it doesn't mean that you have to succeed on all of them. It was one of the beauties of the multi-part payments is you spray it out. You basically create this missed flow of payments that go out across 300 plus paths and then you wait. And a few seconds later, you get what, 70, 80% of them get through and then 20, 25% of them don't. Well, great, you take the remainder, you split it again and you send it out. And not only that, but now from the error returns, you now have a better picture about the actual capacity and probability of the channels in between you and the destination than you had just a few seconds ago. So you use the information from the first round. Oh, so it's also like a probe to see, like you get more information through sending the transaction about which channels were successful. Yeah, exactly. Every failure is a learning opportunity. And then you split it up and you just send the rest. The other 20% and 80% of that gets through. And then you send the rest. And finally, after maybe three rounds, the whole amount goes through. This takes a few seconds, maybe even a minute, but you just sent 20 grand over the Lightning Network. So a minute is okay. Yeah, I was gonna ask Renee, did you do like time tests for these larger payments and what did you find there? Like how long did it take basically to send $20,000 over the Lightning Network? So I think for that payment took something like two minutes, but we had very poor conditions. Yeah, but it's better than 10 minutes, right? That's true, that's true. So one thing that is problematic is the optimization problem that we have to solve is computationally very heavy. And we just don't have a very efficient implementation of this. There is no open source code for this. So we basically implemented the algorithm ourselves in the not most efficient version. So there is certainly some way to improve this, right? So we spent quite some time on computation actually and not on sending out onions. And the other thing is the Lightning node that we spun up, we just randomly connected to some nodes. So we didn't particularly try to connect to well-connected nodes. So we had one or two peers that basically had really poor liquidity. We really had to do a lot of retries because as Andreas just explained, we're learning from our prior retries. So this entire thing is basically presented in two papers. The first paper is about probabilistic pathfinding that came out in March this year. And there we also explain how we can update our knowledge and use all the information from prior successes or prior failures. And then the second paper is the optimally payment flows where we only discuss the question about how to split this, right? So all these things together really present the entire system of how to use the Lightning network in this massive way. Are the implications of this requiring a lot of computational power that this wouldn't be something you could do with a smartphone? Is that right? Well, if you wanted to send 20 grants, maybe not. The computational power gets reduced if you want to send a smaller amount. So the computational power grows with the amount. And also there are... I mean, this is the classical thing about science, right? What we now have with this mechanism is we know what is the absolute optimal solution, right? And it was great for us to test that the actual optimal solution is really good enough, right? Because the result could also have been that the optimal solution just is not good, right? And then we would know that this entire technology would have been a nice endeavor to program, but it wouldn't solve our problem because, you know, I mean, we could prove that this is the most optimal way to do this. So what you usually do now is you do integration where you try to approximate the optimal solution in a much more computational, cheaper way. And maybe you lose a little bit of optimality, but therefore you gain a lot of computational complexity. When Renee talks about reducing optimality, basically what we're saying here is, okay, we now know that we can route this amount with this amount of computation and this amount of time. What if we could get 95% as good a result, but 20 times faster? Exactly. Well, that's a great solution, right? Because it's only 5% less optimal, but it's actually 20 times faster. At that point, you move it to your smartphone, right? It opens a whole different door for you. Just to add, this is exactly what I'm currently working on to figure out what are the good trade-offs here to make. I have a question about privacy. Like, is there any way that this lessens privacy on the Lightning Network because there is information that's able to be learned about the channels? And, well, you already know the balance of the channel because it's linked to a blockchain payment, but the distribution of the coins inside the channel, the Satoshis. So I do know the capacity of the channel because it's linked to the blockchain. What I learn while doing a payment is for certain channels, what is the minimum balance that they have or the maximum balance that they have? But that's a principal property of the Lightning Network, right? The Lightning Network in general has this uncertainty about the capacity split into balance values. And whenever you made a payment, even with the old system, you were trying a route. And if the route was successful, you would know for sure that now all those nodes had a certain amount of Satoshis on the other side. So from the very beginning of the Lightning Network, the payment process was always based around this trial and error loop where you basically learn some information. The only thing that I propose is that if we already have to learn this information, we learn it in an optimal way, right? So we quantify this information or the information gained properly. We measure the entropy and then we choose a route where we expect to have the highest information gain, which corresponds usually to the route that has the highest likelihood. So yeah, there is that. But that's a principal thing about the Lightning Network. Every time you make a payment, you learn something about the network. The question is how useful is it that you learn this because obviously the Lightning Network is a very dynamic system and a second later, the channel could already route a payment in the other direction, right? So this information doesn't stay constant. And a certain ground truth named the capacity of the channel is something that you already knew on the blockchain. So yeah, but there's certainly privacy implications here. But what I want to emphasize, they are not coming from my mechanism. They were there before. I'm just using that. Yeah, and very importantly, you're not down to the base layer of privacy where everything is on the public ledger. Keep in mind, if you know that money has moved on a channel or that balances have been split in a different way, that still doesn't tell you who sent it and who received it. Whether that payment was a local payment between Adam and Stephanie, or whether it was simply a payment being routed through Adam and Stephanie's nodes by other people and then routed back in the opposite direction. It doesn't tell you anything about where the payment came from or where it's going, who's paying who, none of that information is visible because the entire network is onion routed. So it uses an onion routing mechanism to hide senders and recipients from everyone, including the intermediaries. So the intermediaries in the Lightning Network, they see a payment coming in. They don't know if that payment is coming from the person who they received it from or someone else sending on behalf. And when they send it out, they just know the next hop. And they don't know how long the path is or where they are on the path. Let's make this more concrete, Andreas, right? I suggested initially to make the payment via Adam to Stephanie to Andreas. So if Stephanie receives the payment from Adam, Stephanie doesn't know it's actually from me. Whereas when Adam moves the payment forward to Stephanie, Adam doesn't know that this payment is actually for Andreas. And when Stephanie sends it to me, Stephanie doesn't know if I'm the final recipient or simply another link in the chain. And I'm just gonna forward it to someone else. You have no idea how long this path is. You have no idea where you are on this path. You have no idea if this is the entire payment or just a part of a payment. So really you learn very little, even about the payments you yourself are transporting or your node is transporting, whether or not you know the balances. Well folks, that is all the time that we have for this episode of Speaking of Bitcoin. A big thanks to Renee Picard for speaking with us today about the Lightning Network. We look forward to having you back on as this continues to develop. Yes, so I'm really happy that you invited me to the show. It was really a pleasure talking with you folks about it. And I want to use the opportunity to thank all the people who have supported my work this year because as you know, I have tried to stay an independent researcher. I open sourced these results even though there was a high commercial interest in them. And it's just amazing to see how generous the Bitcoin community was. I think there was 1.5 Bitcoins coming to my fundraiser. So I have quite some possibilities now in the next months to continue working on this. And in this sense, also thanks to Andreas and Peter who made two very large donations. And this is really just great. So shout out to everybody who is supporting this work. Thank you very much. Today's show featured Stephanie Murphy, Andreas M. Antonopoulos and myself, Adam B. Levine. This episode featured music by Jared Rubins and Girdie Beatz with editing by Jonas. If you have any questions or comments, send us an email at adamatspeakingofbitcoin.show and we'll be back next week with another episode of Speaking of Bitcoin.