 Come to the next session on homomorphic encryption. And our first speaker is Nathan Manohar. He's telling us about science series approximation of the mod function for bootstrapping of approximate efficiency. Hi. I'm Nathan Manohar. And I'll be presenting science series approximation of the mod function for bootstrapping of approximate HE. This is joint work with Charangy Jutla. I hope you enjoy this presentation. In this work, we focus on the CKKS homomorphic encryption scheme. The CKKS HE scheme is unique among FHE schemes, since it natively supports approximate arithmetic over real numbers. The upshot of CKKS is that it is ideal for applications, such as privacy-preserving machine learning, that naturally lend themselves to computation over real numbers. For these kind of applications, using CKKS is much more efficient, because one doesn't have to first try to represent computation over real numbers via monpy arithmetic or Boolean circuit over bits. CKKS homomorphic encryption inherits many of the properties of BGV, but has the clever idea of mimicking fixed-point arithmetic directly. Since fixed-point arithmetic is inherently noisy, the LWE error present in FHE schemes can be added to the approximate arithmetic error and only affects the least significant bits of the message. It can be shown that CKKS can perform approximate arithmetic homomorphically with only one additional bit of precision loss over plain text fixed-point arithmetic. Let's review the structure of a CKKS ciphertext. A CKKS ciphertext consists of a ciphertext modulus q over here, and as you'll notice, the message and its associated error lie at the bottom, and there are many excess ciphertext modulus bits that are 0 at the top. And what happens in the CKKS scheme is after you perform a homomorphic multiplication and rescaling, some of these excess ciphertext modulus bits at the top get consumed, and the ciphertext modulus decreases. What this means is that after several multiplications, all the excess ciphertext modulus bits have been consumed, and no more homomorphic computation can be performed. So what happens if we want to perform high-depth computation? Well, a natural question is, can we simply increase q to support the depth of the computation? Now, unfortunately, we can't because not only would increasing parameters decrease concrete efficiency, it's actually worse than that because increasing q decreases security, and we would need to increase the ring dimension, which corresponds to the degree of the ring polynomials to compensate. So it becomes completely impractical very quickly for large-depth computation. So what we need if we want to be able to perform large-depth computation is a method of taking a ciphertext where all the ciphertext modulus bits have been consumed and bringing it back to something close to a fresh ciphertext so that homomorphic computation can continue. And this is where bootstrapping comes in, and CKKS bootstrapping refreshes the ciphertext as desired. Now, because of the nature of approximate arithmetic, the data loses precision every multiplication, and bootstrapping will not be able to bring back this precision, so for large-depth circuits, it is necessary to operate at a high precision, including bootstrapping at a high precision. This has proven to be challenging, as we will see, and it's the focus of this work. Right, so in this work, we ask, can we support high-precision bootstrapping? And we show yes up to arbitrary precision, and in fact, our implementation achieves 100 bits precision. Let's review the CKKS bootstrapping procedure. Bootstrapping begins with a ciphertext with no more excess ciphertext modulus bits, and the first step is to simply view this ciphertext as a ciphertext with respect to the largest modulus Q. Now, what you'll notice is that this ciphertext now encrypts a different message, namely little Q times I plus M in the error. And now the goal of bootstrapping is to homomorphically compute upon the ciphertext to recover a new ciphertext that encrypts an approximation of M, but where there are a lot of excess ciphertext modulus bits left over so that homomorphic computation can continue. So bootstrapping for CKKS essentially comes down to performing coefficient rounding modulo Q. Now, there are two challenges here. The first is that the CKKS scheme operates on the ciphertext slots and not on the coefficients of the ring polynomial. However, this can be handled by applying homomorphic linear transforms. The second challenge is that the mod function is not easily represented by a polynomial, and this is the focus of this work. So in this work, we're gonna be focusing on approximating the mod function by some polynomial. We need to approximate it by a polynomial because that is what CKKS can compute homomorphically. And our first observation is that we can assume that M is significantly smaller than Q so that we only care about approximating the mod function in these red approximation regions. Now, why can we assume this? Well, we can simply begin bootstrapping when there's still a few excess ciphertext modulus bits at the top. However, there are several challenges and not any old polynomial approximation will do. The most obvious challenge is that the polynomial better be a good approximation because if it's not, then the error of approximation will be large and most of the precision bits of the message will be destroyed. The second challenge has to do with polynomial degree. Because we have to evaluate this polynomial homomorphically, if the polynomial degree is too high, we may either not be able to compute it homomorphically at all, or after we compute it homomorphically, we'll have consumed so many ciphertext modulus bits that we'll basically have to bootstrap again immediately. And the final challenge has to do with coefficient magnitude. You see when we evaluate this polynomial homomorphically, we have to treat this qi plus m as the message. And if the coefficients are small, we can operate it essentially this evaluation precision. However, if the coefficients of the polynomial are large, we have to operate at this larger evaluation precision and this corresponds to losing more ciphertext modulus bits per depth of the computation. So we need to find polynomial approximation of the mod function that simultaneously has small approximation error, has a low degree and has small coefficients. So what were some of the prior approaches to approximating the mod function? Well, there were several works that used a scaled sine approximation of the mod function. However, this had the downside that it can only achieve low precision since the sine approximation has a fundamental cubic error from the mod function. There are also a couple other recent approaches. The work of LLL plus 21 approximated the mod function by performing algorithmic search on a composition of functions. And the work of JM 20 gave explicit formulas for direct polynomial approximation of the mod function using a technique called modular grunge interpolation. However, one of the issues with these works is that the coefficients of the polynomials can be too large, which as we saw before causes more ciphertext modulus bits to be consumed when the polynomial is evaluated homomorphically. So how do we go about approximating the mod function? Well, our first observation is that there's a well-known technique to approximate periodic functions called the Fourier series. However, if you zoom in on the red approximation region we care about, you'll notice that the Fourier series performs quite poorly. In fact, it performs even worse than sine at approximating the mod function in the red approximation region. Now, why is this? Well, in essence, it's because the Fourier series is attempting to approximate the mod function everywhere instead of only focusing on the red approximation region. So a takeaway from considering the Fourier series, right, is that it performs poorly in the approximation region and thus cannot be used for bootstrapping. So what do we learn by considering the Fourier series? For simplicity, suppose that we want to approximate modding down by 2 pi so we don't have to worry about any scaling factors. Now, the Fourier series was a linear combination of sine x, sine 2x, sine 3x, et cetera. Now, observe that all these sine terms all have periods that divide 2 pi. This means that any linear combination will automatically mod down by 2 pi as desired and perform the same in all the approximation regions. So in this work, right, we ask the question, can we use a different linear combination to approximate the mod function, specifically one that only focuses on the approximation region? So just to quickly recap, right, in this example, right, we want to approximate modding down by 2 pi on inputs of the form 2 pi i plus x for small x. And since all the sine kx terms have periods that divide 2 pi, this equation holds and thus it's enough to consider behavior for small x because how the sine series performs around zero, it'll mimic the same behavior in all the other approximation regions. So in particular, we want to find some linear combination of the sine kx terms that's approximately equal to x for small x. Now it turns out it's easy to find such a linear combination however it is harder to prove that such a linear combination gives an exponentially good approximation to the mod function. So now just as an example, consider the following order 2 approximation, right? So a linear combination of sine x and sine 2x. So consider the following, 4 third sine x minus 1 6 sine 2x. Now what happens if we plot this approximation? So going back to our picture, right, at the red approximation region we care about and in purple we have sine for comparison, we plot our sine series approximation of order 2. You see it already does much better than sine and it's very close to the red approximation region already. So why was this? Why did this work? Well, just quickly reviewing why was sine a good approximation, right? Well sine was a good approximation because if you look at the Taylor series of sine to analyze its behavior near the origin you'll see that you get the x term as desired, right? That's the mod function. And then you have this alternating series just gives a cubic error. And now if we take our approximation of order 2, right? You just look at the Taylor series expansions of sine x and sine 2x. We add the coefficients and then add these two terms. What you'll notice is that we get the x term as desired and now we get a quintic error as opposed to a cubic error. Right, and this worked because we could sum the linear terms, they summed up one which gave us x and the cubic terms canceled. And thus we got this approximation with a quintic error. Right, and so our natural question is can we cancel higher order terms by introducing additional sine terms? And the answer is yes. And so going back to our picture, right? This was the sine series approximation of order 2. If you add an additional sine term, a sine 3x term, you see the approximation gets even better. And by the time we add a sine 4x term you can't even see the red line anymore. The approximation region is completely covered by our sine series approximation. And now zooming out, what you'll notice is that as expected our sine series approximation to the mod function is a very good approximation in the approximation regions we care about but as expected is not a good approximation to the mod function outside these approximation regions. So what are some of the challenges to formally proving strong error bounds for our sine series approximation of the mod function? A first attempt would be to simply use the Lagrange remainder term of the Taylor series expansion of our approximation to bound the error. Unfortunately, this gives a poor error bound. We are able to exponentially improve on this error bound using more sophisticated techniques. We show a stronger error bound using Leibniz's alternating series test. However, proving that our approximation satisfies the alternating series test is not immediate. And in order to do so, we have to calculate the determinants of generalized Van der Maan matrices. These are Van der Maan matrices where the last column is replaced with much higher powers. And to calculate the determinants of these generalized Van der Maan matrices, we use the well-known generating series of the complete homogeneous symmetric polynomials. Now what error bound are we able to show? Well, we're able to show that our sine series approximation with n terms has an error of approximation of order epsilon to the two n plus one. And thus, by adding additional terms to the series can be made arbitrarily small. This also matches our intuition, right? Because each additional sine term we add in the sine series has an additional free variable and it's coefficient. And thus, essentially is canceling an additional term in the Taylor series expansion of sine and reducing the error further. Now in contrast, approximating the mod function by just sine had this inherent cubic error and could not be made arbitrarily small. Furthermore, we also prove that the coefficients, these beta k's of the sine series, all are small. In particular, the first one is less than two and all the others are less than one and they alternate sine. And now recall that one of the things we wanted was for our polynomial to have small coefficients. And now, because the beta k's are small and the Taylor series approximation of sine kx has small coefficients, the resulting polynomial that we obtained from our sine series approximation also has small coefficients as desired. Now how do we go about homomorphically computing our sine series? Recall that ckks is only capable of homomorphically computing polynomials, so it can't compute the sine series directly. But here, we use the fact that ckks actually supports computation over complex numbers. So the first thing we can do is we can use a Taylor series approximation of e to the ix and then by extracting the imaginary part using this standard identity, we can obtain an approximation to sine x. And now, having computed an approximation of e to the ix, this has this really nice property that by simply squaring, we can obtain e to the two ix and by extracting the imaginary part, we can compute an approximation to sine two x and we can proceed in this manner by repeated squaring to compute in general e to the kix, an approximation to that and then extract the imaginary term to obtain an approximation to sine kx. And in particular, we can compute our sine series approximation to the mod function via a low degree polynomial. So what do we do? We wanted to approximate the mod function by a polynomial that simultaneously had small approximation error. We saw that by adding additional sine terms, we can obtain arbitrarily small approximation error. And our polynomial would have low degree, which it also does because it doesn't have very many terms in the sine series to obtain a good approximation. And we also wanted to add a small coefficients, which it does because the coefficients of the sine series are small, as well as the coefficients of the Taylor series expansion of sine kx. So to evaluate our sine series approximation to the mod function in practice, we implemented this approximation on top of the Heon library and we were able to obtain 100-bit precision bootstrapping using a sine series of order six. And thus, high precision computation is supported. Thank you. Questions? Oh, questions. If you have a question, can you use the microphone? Thank you. What? Does it work? Yes. So thank you for your talk. My question is the FHE schemes are well known to be kind of slow. So how long does it take to actually compute your methods? Oh, yes, yes. Okay, cool. Yeah, so it is a bit slow and this is just because, well, first of all, the Heon implementation library that we used is not the most advanced. But it's a good, we used it for like just to compare to prior work because a lot of them use this library. And so if we wanted say 45-bit precision bootstrapping it would take about 30 seconds to run. But for higher precision it takes, so it starts to take around two minutes. And the reason for this big jump is because around this point, when you increase the input precision you need to increase the, so you know how I mentioned that if you increase the modulus Q you have to increase the ring dimension to compensate for security. And this was sort of the jump point where we needed to increase the ring dimension from two to the 16 to two to the 17 in order to maintain security, right? Because if you're operating at higher precision every, you know, every operation, every multiplication is gonna consume, you know, at least the input precision. It's of the side modulus. So you need a bigger modulus to evaluate the same function when you're operating at higher precision. Okay, you finished it? Yeah. Hi, thank you for the really great talk. I'm wondering how this technique compares to the double angle formula method, which seems to try to just like shrink the range that you're trying to approximate as opposed to your technique which like improves the accuracy over at the larger range. Yes, well, so, I mean, so I mentioned that we first have to approximate sine, right? Like, and then from sine we get approximations of sine two X and et cetera. And we do sort of do this like double angle electric because we take e to the ix and like I mentioned, we approximate e to the ix. And the way we actually do that is we divide e to the ix by this variable capital K which represents like the number of intervals we need to approximate. And so in that sense, we are like shrinking the region and then like we're doing these squareings. So I didn't mention in the talk what we are doing this internally. This is what was some of the prior work did also. I see, I see. So you're like doing both at this, okay, yeah. Yeah, we're doing that to get a good approximation e to the ix. From that, we get, we can use e to the ix to easily get approximations of the other terms. Now, of course, like how much you have to do this depends on how many intervals you need to approximate with. That depends on all sorts of things like the, you know, the hamming way to the C or T. And also if we wanna operate like, so if you take an approximation sine X, then from that you get approximations say sine two X, sine three X, right? The error is gonna blow up a little bit. So you need a little bit of a better approximation of e to the ix to begin with, then you would have needed if you were only computing sine. I see. Thank you. Yeah. Thank Nathan. I don't know if we're supposed to clap if they can hear us clapping, but I think we should clap anyway. Thank you.