 Hi, everyone, I'm Brinley Zabriski and I'm going to walk through how to use the REMR package. So REMR stands for Rare Event Meta-Analysis and it's a package that implements the method in this article here, so it's a permutation-based approach for heterogeneous meta-analyses of rare events. So this method can work well when events are binary and rare adverse and when there's heterogeneity in the dataset. So the main benefits of this method is preserving the type 1 error rate and having good confidence interval coverage. So I'm not going to go into detail about the method and about the results but I did want to show a few things. So here's just some plots. The first one is showing confidence interval coverage on average and this yellow method is REMR. And so we can see that that comes really close to what we want, in this case 95% coverage more so than other methods and then similarly we preserve the type 1 error rate more so than other methods and kind of these challenging cases where we have rare events and heterogeneity. So those are the benefits of using this method and obviously there's consequences. So we're going to have larger confidence interval widths than other methods but again that comes with the benefit of maintaining coverage and so forth. So how this method works, it's based on conditional logistic regression where we condition out the nuisance parameters instead of trying to estimate them but instead of using asymptotic inference we use a permutation-based method for inference and that is useful when we have rare events or sparse data. So what we get is something that looks like this here which is the permutational distribution of the test statistic and the way the test statistic is computed can be found in the paper but for this example data our observed value of the test statistic was 37 and then we permuted our meta-analysis data set a ton of times with some constraints to get all possible values of the test statistic so we get a range from 27 to 53 here. So then to get like a p-value we could just say okay well everything more extreme than 37 for a two-sided p-value would result in our two-sided p-value there. So that's kind of the gist of the method so again based on conditional logistic regression but we used a permutation-based approach for inference instead of an asymptotic-based approach. All right so with that let's jump into the actual package REMA. So we're going to use this very clean data set so it's a smoke pain cessation devaculatization drug and the goal of this meta-analysis was to determine if there's serious cardiovascular adverse events with the use of very clean so very clean uses is here and then a placebo is here and so you can see the the patients who received the placebo total and then the number who experienced a serious cardiovascular adverse event and so forth so there's 22 studies here so for REMA we can go ahead and load the package and then if we look at the documentation we can see there's just a few arguments here the only ones that we need to specify are these four so just the data information so we're going to start with that just in a really basic meta-analysis so because we're doing a ton of permutations this can take a while to run depending on the number of events so the number of events is what largely drives computation time here we're looking okay this you know set of 10 here adds add some computation time so this takes just over two minutes to run so I went ahead and ran that in advance so we don't have to wait for that so if we look at the results we see that we have a combined odds ratio of 3.4 95% confidence interval that is pretty wide right that's a pretty big range and so that's just again one of the you know potential downsides of this method is that we're going to get a little bit more conservative results but I mean we're pretty much guaranteed that we do maintain 95% coverage which is nice and our P value that we got is 0.046 so that's just kind of a really quick meta-analysis using Rema but I want to go through a little bit more detail about what's going on so if we plot this Rema object we're going to get a plot that is the permutational distribution of the test statistic so in this case where this asterisk is that is where our observed test statistic is so we have an observed test statistic of 34 and then a permutation distribution ranging from 20 to around you know the over 40 and so to get the two-sided P value we can add up everything that has bars of heights less than this value and that's how we get the two-sided P value so a benefit of this is that we we're not assuming normality here in this does look fairly normal but you can see that the tails are not the same right and so you can imagine you know different distributions that look a little bit differently than this one that are more skewed and we kind of maintain that exact distribution if you will which is it's kind of nice so let's look at some of the details here so rare event tether genius analysis method great our two-sided P values returned and mid P equals true so mid dot P is saying that we're going to use the mid P correction when we're competing our P value and what that that does is instead of taking this entire bar and adding that probability to our two-sided P value we just take half half of that bar and that reduces conservatism and it's been shown to work well and so that's what we implement here in this package and then we'll go over the conditional maximum likely destination in just a second I did want to show you everything that's in this remote object so the you know traditional things that you put in are in there as well as your arguments T is the odds ratio CIS is 95% confidence nobody can pull out P value is the P value and then the distribution contains the test statistic and the probabilities that we're in that plot so you can get the actual data there and then T side is the observed value of the test test statistic so that's what to kind of just straightforward Rema implementation looks like because it takes so long to run let's say we decided we actually wanted a you know 10% alpha level instead of 5% so instead of having to run this whole thing over again and take another two minutes in this case we can just pass Rema a Rema object and it can almost instantaneously update the results if we change let's say an alpha value so here you can see you know our confidence intervals a little bit narrower because we changed our alpha level alpha level there so that's nice we can pass a Rema object in instead of the whole data so we don't have to recompute the permutational distribution we already have that and so we can just change alpha and then similarly if we didn't want to use the mid P correction for some reason we could turn that off without having to rerun everything and then you can see that does make our results you know more it's conservative in this case. Okay so you really just need to put in the data and let Rema do its thing but again you can actually pass a Rema object if you need to change things afterwards. Okay I did want to show you just two more things but that's really the basics of this package we have a few more obviously updates we want to do but it's it's very functional now in it and it maps well to the paper so you can get more details there. So here's just a made up data set just for illustration so if I run this you'll notice that we get an error and it says that there's only one value in the permutational distribution of the test statistic and so this method can't work so that same as in that you know picture there'd be only one bar here and so that's just the trivial case where we the value is the only value in the distribution so in case like this Rema simply won't work and so you'll need to use a different method to analyze your data set so here's an example where it doesn't work and then I also wanted to just show a quick example where we use a different estimation method for the odds ratio so again I just made this data set up if I run this and then we can plot it take a look so this only has two values in the permutational distribution so we have a test statistic and our observed values 24 so whenever we have an observed value that's at the minimum or the maximum of the permutational distribution then we cannot use conditional maximum likelihood so if we go back to this first one we did you can see that we use conditional maximum likelihood to get the odds ratio compensating great but we can't do that in this case where we're at the in this case the maximum of the test statistic distribution so instead of using conditional maximum likelihood estimation we instead use the median of bias estimate so when we do that you'll see get some slightly interesting results so here it says we're using the median of bias estimate great and then our odds ratio we actually get an odds ratio which is nice and we get a lower bound in this case for the confidence interval and then we go to infinity for the upper bound and so that'll always be the case when we're at the minimum or max we'll have infinity somewhere in there we do get results we get a p value it's just a different way to do that estimation and again more results are in the paper but that that'll be the two different options you'll have for this method part and that's that's basically it that's REMA so if you guys have any questions please feel free to reach out if you want to kind of check out more of REMA here's the CRAN and the GitHub repo this is the article which the package is based off and the data set that we use for illustration is linked here as well so again please feel free to reach out I hope that was helpful.