 Yeah, so welcome everyone to the art tables workshop for our in medicine. We're going to be talking. And, and teaching about the art tables framework for creating both exploratory and sort of regulatory submission style tables. Using our and the next slide. Yes. So I am a statistical computing consultant. I'm the architect and lead developer of the art tables package and related packages. And my education is that I have a PhD in statistics with a focus on statistical computing. And my co host is Adrian. Go ahead. Yeah, so I'm a principal research software engineer at Shannon tech here in the Bay Area and previously I worked in Russia in Switzerland, and I was the technical need of a nest of approach called nest where we developed software in order to perform exploratory and regulatory clinical trial analysis and we're going to talk a bit more about that. And also my by training have a PhD in statistics. Good. Yeah, so maybe I'll take this one here so the trade so the slide so here I think gave you for the share them in the chat. Yeah, in the in the q amp a so if you if you guys go to the q amp a under answered. Hannah Hill asked about links. And so I've posted both the code repo and the slide deck there. So you can follow along as desired. And I see people are joining right now so once you're here please jump to the third slide. And so this time we have not set up and also with your clown session. We have please use your local or installation also we have less. It's less of an exercises workshop, like we give you the code we run through it. We take time to discuss it with you. We will receive receive that is a good format and we test this format this time. So you can go you can clone that repository the repository I have it here on my desktop. Once you have cloned it you can go to your studio and you can open project and then it should look something like there's a supplementary code. It should look something like this. Yeah. So even if you have our tables installed do be sure to do that uncomment and run that remotes install our tables, because some of the features that we will to be talking about are very new. And so they will be available in that on the main branch in the GitHub repo, and they'll be available in the next crayon. But they are not released in the current release on crayon. Our other people. It looks like there are some attendees that are having trouble accessing the Google Docker. It looks like other people are able to do that that correct. Yes, there's 34 people. I think for those who you doesn't work. Try maybe to use Chrome or different browser. I'm not, I'm not sure, but we will be we will be showing the slides as we're going through them. And so she will still be able to sort of see what's going on. And then perhaps we can, we can try to troubleshoot that after the fact. Because it looks, it looks like it is working for most people. I don't have access to the Google doc that is very unlikely because anyone with a link. Yeah, they should be able to do it type answer at the official link here. That's, yeah, and that's the same length that I that I put in the original. So, yeah, so Steven go to answered questions and that that link should work, but I think we should go ahead and move forward, Adrian. Good. Yes. And so maybe then you can do that for us. So, I think very important is that we keep coming back to that. We have a very, very exhaust. It's a big project. It's used by many downstream. We have lots of documentation both of all the functions and there's lots of them. If you have browsed it and then we have many, many vignettes. So, yes, currently the version we use for this workshop is our tables six zero six one nine thousand and two. We have installation instruction in the script, but also later there's a slide. So I'll pass the ball back to Gabe. Okay. Right, so what is our table so our tables is a general table generation and rendering framework that particularly targets ASCII but does support other output formats like our TF and PDF. And that is the sort of feature space is informed by the need this regulatory needs for pharmaceutical companies that are conducting clinical trials. But our tables itself is not actually specific to clinical trials is just built in a way that supports the types of really complex multi faceted tables that that are involved in clinical trial submission. And so they can, it can be used to to generate the overwhelming majority of tables that are involved in a clinical trial submission, and any that they that it can't those very very few that it can't it can generate a table with fully equivalent information that it just has a slightly different form. And we're going to talk a little bit about how it does that actually not that we're going to talk quite a bit about how it does that over the course of these three hours but basically it has a layout engine that allows you to declare what the table structure is going to be and then it will do the sort of computations necessary to generate the cell values and place and organize them into a table for you so it's a it's a it's a system that that has a lot of power in terms of the you give it sort of very expressive instructions and then it will actually go and construct the tables for you. And then yeah it has it supports these multiple rendering outputs. So, go ahead. So that's, I'm going to talk a little bit about the ecosystem or the downstream use of our tables. So, our tables is a fairly low level to create tables. And as you go to production eyes like as if like as an organization that it's most of those tables are standard tables, and some of them are at least repeated patterns and so we don't do any real analysis in our tables it's really only the core components to do what you will learn in this course date like splitting and like creating essentially table and like defining analysis and providing structured objects that it can query and work with. And so the analysis part is in a package called turn that's what you see here so turn essentially uses rights components that can be used together with our tables to make clinical trial tables standard like common clinical trial tables and we have a and we have a project called the TLG catalog that is by now also open source and so maybe it's good to quick open that and spend some time looking at that. And then in the whole context of clinical trial data analysis there's in a downstream part where you would say well those are standard table we don't need to have any parametrication and very few. There's a package called Chevron which I'm not so sure if that's open source yet. But let's quick look at the TLG catalog as I said it's open source to everyone. And here you have the 200 plus tables. Essentially available for everyone. And so they use a mix between turn and R tables. So we can look at adverse events tables. There's a certain naming. I think those are not industry standards but I think I mean that safety summary not that's titled to it. Usually, when you look at an output not so that the 801 it's an adverse events table. It has number of components. There's there is the standard table how it looks like there is. And so each of them like test those are the variations not the first step is how do we get to the data. Then you have the different variation flavors of this table. Then usually in the end you have the teal tab teal is also an S space project to create shiny apps it's also open source. You can look on the insights engineering for teal or you can go and look other presentations on teal in part by me on YouTube. You should find something so very powerful framework to interactively subset data choose then coding it creates the table. That's another table. It's in fact there are table that's used here and there's a show our code so that's reproducibility part of it. And so if you look at those standard like if you look at the table that's how the table looks printed as asking with asking characters. You can look at the codes. Let's me first spend a minute on the data setup view synthetic data. We they're created with a package called random C disk data it's public. Those then once we deploy the packages or once the the current NEST team deploys the package they they put it into an archive the random CD state that's called synthetic data archive. And then the attack you can say I would like to have the latest ADSL data set available in the synthetic DC disk data repository. It gives you two data sets usually to get the outputs that you need because that like it's fairly simple you can look in and see this data. There needs to be some post processing what you see usually in the data set up hot and then if you go to the actual table code not so I would like to create this table you can look. Well, not the session info sorry. The code you can look well those are the variables of interest. And now we are in in our tables territory count values is a is a turn function basic table is not able to function build table is not able to function. And that's essentially how you get the table displayed. And so you can go through all the tables. And that's the pattern to efficacy to lab results to. Yeah, to lots of lots of table not so I encourage you to look into that. But because I think we think most of the table that are required for clinical trial reporting. You should be able to find here it might not match your company's exact light specification. We can talk a little bit like, I think the first question is does it matter. The second question is we can we can discuss that in the like that's different formats. One second. And I just want to address a question that we got. So I'm going to click the answer live. And then. So Karina asked Karina so then asked whether you can reuse this code yes so this all of this code is open source under a commercially permissible open source license. So you can use this code. And to make these tables turn is open source, it is available on GitHub under the insights engineering organization, and it will soon be on CRAN it is not on CRAN yet but it will be soon. Our tables is already on CRAN, in addition to the four matters which is a sort of underlining helper package essentially. The next the answer is you can this code is available and and you can use it. It is it is permissively licensed so that you can so that you can just reuse this this code. Yes. And I think it's fair to say that. I think I mean at least in my view. The first step was see this not standardizing the data. I hope that that work around here that the industry can find a way forward and choose something. I mean, I hope it's this maybe something else not and that would be the next phase of making it easier to review the data and to share the data with health authorities. Essentially, standardize the code not use the same tooling to create outputs. And that's one or at the very least standardize the tables standardize the table. That's a fairly big step here that we create all of those tables and share show look it can be done and that's how it looks and that's a good discussion starting point for the industry to decide. Is this good enough what needs to be changed or is or is it completely in the wrong direction. Yeah, so so I think that's the whole reason to spend that much time on that slide is to show that there's really lots of work that builds on our tables. And also it's fairly exhaustive in in I think in many downstream uses in reporting clinical trial tables. If you see the next slides, I'm going to cover quickly and then I have the ball back to Gabe so as I've mentioned there are six, six categories 250 table 225 table variants from various different domains. And if you can get more into into that topic. I've also briefly mentioned that like our tables is the core general table package for layouting and tabulation and rendering and turn implements essentially analysis statistical calculations. And you can't use turn without our tables. So, you can, I think you can learn turn with beginner understanding of our tables but you don't get get around learning some are tables to use turn efficiently then. Of course, the more you become an advanced user of turn, the more you want to learn about our table. So here's the URL. Again, examples are on the catalog. Yeah, so another another sort of piece of work in that same vein is the our consortium has a working group on our tables for regulatory submission with both which both Adrian and I have been heavily involved in. And that working group unlike the catalog, which is, you know, specific to turn plus our tables and solves a wide, wide variety of tables with just that that one sort of framework with with the extension of turn. This working group represents, you know, multiple table generating packages and multiple pharmaceutical companies are involved in this working group. And, you know, sort of Adrian and I are the representatives for for the art tables approach but it is not specific to the art tables approach. And yeah so you can see one of the things that we, the primary thing that this working group has been doing is writing a book about tables in in our and the structure of this book is that you can see there are, you know, either six or seven tables packages represented. And then there are five, I believe archetypal table example tables that are then worked from raw CDISC data all the way to table. These different in all of these different table engine packages. So, so that is another good resource, and that this book has been, you know, it's a, it's been a long effort by the working group to put all of this together. And you can see as Adrian is showing like this is live just on the GitHub so you can see the draft of the book at any time by going to the to that repository. And in addition to that, we are slated to release the first official edition electronically of the book, the edition one, so to speak, on the 23rd of this month. So that we will probably continue to, you know, add to and refine the book. There's been discussions of adding in, you know, topics like listings and things like that in a future edition but the but the current edition which is basically what I just described the sort of six table engines and five different substantially archetypal tables in each of those systems. That's going to be what's in addition one, and that that is something that we will be sort of officially publishing later this month in addition to, as a, as, as we can see the, you know, whatever this draft version being available right there live on the, on the webpage on the, on the repo. So, so that's another good resource in terms of being aware of what table engines are out there and what they can do and and looking at code that actually generates. So there are each of those each of the tables. There is a turn a pure our table solution and for some of them there is also a turn solution in this book and there are likely turn solutions to all of them in a tlg catalog, as well, plus many many many more tables as we saw. So that's the other sort of larger context piece. And then, after that I think we will get into the specifics of. So I think this one is covered not yes. Yes. So, actually, actually run those two quick. I know it's your name to it but I can create. Yeah you can. Yeah, this is just like, like we mentioned before the. It's important if you want to run all of the code in the in that supplementary code file in the, in the repo that you have the latest GitHub version of our tables. So you can install our tables from crayon which will pull in all of its dependencies, and then you can install the GitHub version of our tables which will then get you that sort of latest bleeding edge version of just just our tables. So, there we go and we can see that's the that's the version that Adrian mentioned the 061 9002. And there you go. So then you will have the version that has their two new functions that we'll be talking about here that are that are new to that version that are sort of developed as we were developing this workshop. And then all the rest of the functions, you know, have been in there for a long time and would even be in the cram version so. Yeah, so this is just we're so we're going to deal all the tables that we use almost all the tables we use in our examples are going to be based on the sort of synthetic ADSL data set that actually ships with our tables called X underscore ADSL. There is our table ships with relatively small synthetic versions of many of the CDISC data set specifications, not every single one but many of them it does have and the one that we'll be using here is ADSL. And you can see here, we've got, you know, standard ADSL variables that we're going to some of which we're going to be using to create the, the tables many of the tables that we're going to go through. So, go ahead, and then this Adrian is going to talk about this is one of the new functions. Go ahead. Let me actually turn off some. Let me see bookmarks. I don't want to view the bookmarks here show all the story to maximize that good. So, we've added a new so for this for this workshop, because we have now run run a couple of trainings on our tables. And, and we thought it would be nice actually to maybe make a little bit of simpler entry into our tables. And so it's heavily leaning on ggplotsq plot quick plot we have now a queue table. And that is part of the current of essentially what's on that branch that's why we, we have installed this from from get up we're going to release it on grand. And so queue table is for creating generalized cross tables it's a little bit more than table and so we're going to go into those details right now. So I think everyone who has used or probably has used a table function and usually it's a factor variable that goes in. If it's string that it also acts of strings but if it's a factor variable, it makes one it gives back and a table object with, if it's one dimensional if it's cross table gives 2d, but with essentially one column here, or at the level and then the counts of how many, how many observation of arm a drug X on that factor variable how many of the and so on. And so if you give a second variable, you get across table with frequency counts with the counts of them. How many are female in a male in B and so on. And so, let me run that. Is that big enough give or should I make it larger my font. I can say it okay audience if you if you would like me to and would like me to increase the font please write I guess in the question or in the chat. I don't see the chat right now. Yes, so that's a that's that's a common very common way if you look at the variable itself. It's a factor level. It's a factor variable. It has the levels and because in cities that is data set labels and the way our deals with the variable labels those data set and variable labels and the way our deals with the labels. It's an attribute label to the variable. And just maybe for those of you who use our studio, our studio has added essentially support for visualizing the labels in the viewer. Anyway, that's besides the but that's what you see here. More important these are the factor levels these are the elements if you're on table or that you get one dimensional two dimensional frequency counts. And so, if you if we do the same thing in queue table, which is part of our tables, quick table. It's a little bit different. The first argument is a data set. And then the second argument is the rose, the variable name that should be used for the rose and then the second to split by the rose and then the set to summarize the rose and then the third argument is the variable name for summary in the column space. And so, if you do that. So, I mean, it's clear from the thing not this is a variable. This here the variable is referred to with the string we don't use non standard evaluation in our table. That's important. And we believe that has some advantages, few advantages, many advantages. So if you do that, what you will see so our table doesn't know about one dimensional table like here so it always goes into the rose so it's essentially if you make. Yeah, always goes into the road the second thing if you have to argument, you get a table object with to them and like with columns and rose to the table. But other than that, pretty much the same. What's to expect it what's to be expected. You see here because often in clinical trials. There is an n equal xx the number of patients in that arm. So that's shown by default. If you look at the queue table. If you look at the queue table function documentation, there should be a argument show call count so if you make this equal falls. It should not show hopefully otherwise it's a brand new feature. Yes, so I think this, it will be you can run those pieces of code as we talk and then ask if you have issues. Is anyone running the code. And maybe that's a that's a poll question. If Rachel if you could write the poll, who's running the code or so that I know how long to wait until somebody maybe ask a question. So we intend to walk through it you can walk with with us and execute the code and look. Why don't we go through the rest of the slides about qtable while we're waiting for that. And then we can come back to people time for to run all the code for qtable if they if they want to do that. Good. And so, I think the main difference as I mentioned already is the main difference is the first argument is always a data frame the second argument interesting is is the second difference it's always a two dimensional table. And by default, we showed an equal xx this this row here the header in the columns. Good. So it's brand new with with with designed it for this for this workshop. And it's primarily intended for exploratory data analysis I think it's more interesting for bias stats and maybe statistical program or quick have to ask a question. If if your job is writing production table that gets sent to the health authority or ends up in any reporting pipeline, but we don't want to use qtable you probably want to learn the larger framework that we're going to discuss in detail. In this in this workshop. And I think this is still me yes. And so, one thing so and so this is essentially equivalent of table to qtable, but because our table is a month is a much more flexible framework for table generation where a couple of quick wins that we got. One is you can give a vector variable names and then you have essentially fast set that frequency table, meaning you say I would like to split in the row space by country by the levels of country and gender, and in the column space by arm. And you get this table, such a nice frequency table. And this is as deep as you want so you can add you can keep adding more variables here you can keep adding more variables in the column space. So it's fully general when you do that it's just a table gets bigger and bigger and less readable. So that's one of the, I think one of the things that table won't provide you that you get fairly easily as an extension of using qtable instead of table. The next extension we have, which you don't get from table to other aggregated other functions or I think if you're in the deep layer space you can group by in summary. As you can see, well, I actually don't want a frequency count I would like to you analyze the variable age, and I would get like to get the mean of the variable age for each group so females from China in our main a, you know, or male USA on B. So we can essentially say for all the, all the, all the, all the subjects that are that are in the cell associate to the cell here. I would like to get the average age and you see that actually shows up here. And just to show you that that works. Here, if you run this. That's the table. It's all fairly quick. That's how it reads. And then maybe two things that there is some labeling that we do automatically so the agent mean show up here so you know it's not a frequency table. And then maybe the last and so it's all general not you can nest as much as you want but I think the tables get less and less readable so you remove some nesting here. We do so. But we do essentially say well we would like to analyze. So five numbers a function you know it returns a vector with five values which is minimum first quanta median third quanta and maximum of the values. So here, we wrote the function to label those because this just returns a vanilla vector. You can give a function that returns a list and each each name the element in the list becomes a row. So that's not a convenience function even like if you look in demographic tables not if you want to have mean standard deviation and range and so on. That's a very powerful feature here in Q table. That's how it would look like so quick executing that here to show how that works. So maybe if you look at five now anything not like five now of any, any numeric variable one to 10 not. We get this. So five now five now to. And so maybe the reason we did we did them. We wrote that function here is we need we need those labels that then show up in the row names, because the five non function itself which is in which is in our just gives a vector back and so. We also needed it to be a list. We need to be a list that's true. Do we do we have an example of multi multi value cell and the slides. But later we have. We've got percentage but but yes, we can also have multi value cells which we show later but there's a reason why it's a list and needs to do like copy line 39 and replace five number range, they'll see why it has to be a list. So range. Range takes in a numeric vector and it gives in a out and a vector numeric vector and so if you do that it's not the least you see it. We're not so yeah yeah we're not generating multiple rows here we're combining multiple values into a single cell for each. Great sort of category right. So that is, that's the reason if you want to declare that it is going to be multiple cell multiple rows. You want the function to return a list. So that's why we both converted to list and give it the names which are then used as the names of the rose. So there's an open question week after this queue table things we address the question and also the police. There's around 41% runs the code so we give a minute or two time to ask if there's any question when you follow with running the code but so the fact back to the five not so it shows up as rose. We do. I think. Yeah, if you have an ace because there's no data associated to here. Then you get an ace. So that's, I think hopefully pretty much what you would expect. Maybe two things. And maybe I go here into slideshow mode. There's a little bit of a little bit of slide transitions. So the first variable, the rose splitting. You see them data labels. And then the five now gives you five rows per group. So you see that in the second orange part. And I think to wrap that part up about q table. If you are using table off and then we hope we have given you an almost drop in replacement you have to change a little bit. But from a you from a complexity of using. I think the complexity is the same, but it provides you further downstream analytical capabilities like nesting like analyzing multiple very like analyzing like. Analyzing the data with analysis functions instead of just counting. You can do that with aggregate and and deeply summarize group by summarize but. I am all table gives you I think a fairly convenient way it also doesn't the rendering and so on in a user friendly way. Again, that looks. I think I'm quite happy that we added that. It's not intended for statistical program. So the reason we added only now and not like three or four years ago is because. Our tables is intended to be a tool to make production ready tables that are fairly complex that are in structure, much more complex than generalized nest across tables. And we're going to show some of that. And that's the third point. And I think with that. Let's give it a minute pass. People can say if they have problems. And we answer some of the questions. So how do you export table as excel or PDF. It's in the documentation of our table. Let me go back to our table documentation. If you look in the reference, there is a function as PDF as HTML as PDF as TS visa to flex table. There's quite a few output things as functions and so it's very well documented. Well, that's around documentation as PDF. Sorry, that was your questions as PDF. So you can give you can give quite a few arguments to say what our font family was the font size and so on. It's also possible to do multiple multiple pages per PDF and then with pagination. It's a very, very flexible tool here is your code. No Excel export. You can export it as You can export it as a CSV, which is load on Excel. So I'm going to answer this here and then and type answer here. One, I guess one and two export. CSV into Excel. So the second can be changed. Yes. Well, so it so in the in the core our tables framework you absolutely can you have full control over how things are rendered in queue table. Currently, no. But that could be added but and again queue table is intended for or like quick EDA like similar to table and so we just wanted something that's going to render nicely. And so it infers how many decimal places do you want so if your function returns an integer object, it will not show any decimal places. But if it shows a numeric, it will do to that's just what we chose as the as the default behavior that seemed reasonable. We can look at adding more control to that although queue table is intended to be sort of really fast with pretty low complexity. Which is why we can't do things that you other other things that you can do in in the core of our tables. So that's one thing. Now, Beth Atkinson asked about why why there why do this when there are already packages that creates on respect well for the first off. We're not creating summaries by group or creating tables. So these are fully fledged our tables objects and you get buys you a lot of things that those other things can't do. Many of them aren't going to be super relevant to sort of exploratory off the cuff. I don't know which is this, but they could be so for example, and we don't talk about this in the slide deck, but you know we can, we can maybe cover it at the end. And it's that was well documented elsewhere, but our table is implemented implements what's called context preserving pagination, which means that when you when you ask an R table to paginate itself. There are plans which rows are summarizing group information and will repeat those after a page break. So that when you read an individual analysis row, it will always have its context. We also support pagination and both vertical and horizontal directions simultaneously, which obviously is necessary because pages have both width and height. So those are a couple of things there's also. Once you have one of these table objects. We support a very robust way of selecting things from them or pathing. This again is something that is not necessarily going to be hard to do for the types of tables that q table makes. Other systems, but some of the more complex tables would actually be quite difficult to do. And so basically this fits into a larger framework that existed already. You can do these capabilities and then it buys you all of the things, all the more powerful features that our tables has that other systems don't have or more complex tables. Yeah, I think maybe a question we should have asked maybe let me write to Rachel. I assume most of the people here from clinical trial analysis, if there are people outside of clinical trial, like I think then their question becomes quite a bit more fair. We do believe that's probably one of the most general table creation framework out there. But in clinical trial, those tables sometimes really are not like a very weird tables. There's more like a report that have like the same columns or shared columns and so. And then the formatting, the formatting, like requirements and the way of dealing with those table and the processing and so on. It really calls for a very flexible and essentially table generation framework. So that's, I think, the reason why we've created our tables. Good. I think in the chat or somewhere if somebody can research that here. I also wonder how can I move this away here. But in the chat, is there any, is there anyone has anyone have an issue right now with running the code? Let us know. Otherwise, we move ahead with Gabe. Yeah. Good. Fabulous. Do you want to share Gabe or should I? Yeah, I can share. Do you guys see the slides? Yeah, okay. Yeah. Alright, so we're down here. 28. So. So, that was Q table. Q table is very new. We think it's a nice utility, but it's not the core of what our tables is. Right. So that's the other answer to the question. We didn't create the R tables package in order to create Q table. The R table is just a utility function that we've written pretty late in our tables lifetime because it seems nice if you're already in the table framework, because you need to do something that those other table frameworks can't easily do. So now let's talk about what the sort of normal traditional R tables framework looks like. This is still our Q tables. So the normal R tables layouting framework goes like this you create a layout which declares the structure of the table, and then you apply that layout to data which will then construct an actual table object. So here we have a very simple example that's similar to what we just did. It's the same table in fact. So we've got our five now to again, but here instead of calling Q table, we're going to call basic table, which says I'm starting a table. Right. And you can, we're not doing so here, but you can say the title of the table, footer materials and stuff and basic table. And then we're going to say, okay, for the arm, I want to split the columns by arm so splitting is faceting so I want faceting in the, in what would be the X axis in a GG plot to be arm, and then I want faceting. In the what what would be the y axis to be sex, and then I want to analyze like within those facets, facet pains, I want to analyze age with this function. And here you can see we're we use the same format, but here we actually do have control over the format with that format option so we could, we could make that one decimal point we can make that three decimal points, etc. So, just to sort of be explicit split calls declares the column faceting right here. The rose declares the row fasting which is just the F and the M the sections, and then to get the actual rows we say analyze. And then once we have our layout, we, I don't I don't have a slide to highlight that but we call build table, and we give the layout and the data, and it constructs the table for us. Now, it's important to note and well actually let's just go ahead. So now let's dig into exactly what's going on there. So, a lot of people sort of view the first step in constructing a table to be calculating the cell values that's a very traditional approach that that other table engines will take. The approach that our tables takes and we can we saw that even in that one small example, we were not calling five num on anything to get a data frame that had those values in it and then rearranging them to make a table. I said, this is the structure of my table within the pain, the facet pains. I want five num, like those five numbers that come out of there to be rows, and then it calculated itself. So that's a, that's a large difference. And the reason that we're doing that is because most people don't think of them this way but tables are actually like tables of the type that we're talking about here like the sort of clinical trial reporting that are faceted data visualizations. And so, here we have a sort of GG plot and with some nested faceting here, and then we have an equivalent table that was generated in our tables. And then we can see that each component in each one of these sides has an exact analog on the other side. Right. So we have, you know, the, the sub plots are, you know, these sections, which is equivalent to what was coming out of five num in our previous example. You know, the individual bars, this is a bar with zero but it's still technically there's a bar there are the, are the cells right here. Rows are actually cut across sub plots, right columns. And then we do the whole sub plot. And so yeah, so that this is the way that our tables thinks about tables. And it sends out being extremely powerful and it lets us do a lot of really cool stuff with those are really succinct set of verbs that are, you know, they're very akin to the types of some of the types of verbs that you would do using more traditional visualization. So, so just just to sort of nail this point home, imagine if you're creating a faceted, you know, complex faceted ggplot visualization, you would never calculate all the values by hand yourself first and then pass that to ggplot like G plot does all the setting for the facets for you, it applies its geomes in order to construct the visual elements, it applies its stats to construct the numerical values that are then like inform the visual elements like ggplot is what does that you don't do that yourself. That's the same in our tables. Our tables is going to act like ggplot much more than it's going to act like deep higher plus GT. It's a very intentional choice that we made, which we think gives a lot of power and a lot of expressivity to the two our tables. So, yeah, so you don't want to you don't want to do that because like it's tedious and error prone and computers are good at that stuff and humans are bad at it so that the computers do the stuff that they're good at. I like to call that the zeroth law of computing. So is this you, Adrian, I don't recall. Yes, that's that's me. I can take over from here. If you can stop the screen. So first, are there any questions about this link between tables and visualizations that I just talked about we will get into more of it in a bit but if you have any questions now. I think it goes while we are switching. So Adrian, you can go ahead and doesn't look like there's any super immediate questions so go ahead and just type the question in the chat if you do have one or in the in the q amp a I meant. So, I think, every time you get lost in our tables. Visualize this, this image to kind of get an understanding where in which phase of table creation, or of the of the journey of creating a table, you know, it's, it's, it's, you can separate it in four pieces. One is you define a layout, the layout describes the faceting, it does describes the analysis, the cell value derivation, and it has formatting instructions. And in the early days of our table, we tried to split that up. We found that not to be to the point it just got over I think overly complicated. So that's what we found to be a good way of grouping those, I think those pieces together. And then you take a data frame and the idea is that art that our table is very powerful in cell value derivation and adding like total column columns not we don't optimally you don't have to do any pre processing of the data frame like you don't derive any cell values you don't stack the data set to get the total column that's all art that's all layout in part. You take the data set and the layout which creates an art table object, which somebody asked about our table object that's a, that's a tree structure implemented with S for you. And it's generally to say, nobody who uses our table ever has to go and understand that that object from a S for design point of the way we've designed it we've provided a lot of tooling around that. So we, we expect from a user to understand how to work with that object as a as an object with the different access and modifier functions. And anyway, and then you can take that object and you can export it to a format that outputs that can be printed or using a web page and so on and so if I ask you have PDF, you have RTF, you have HTML, I think there's latex. There's lots of formats and so I'm now going to show that again in a couple of different ways. And then we go further into the details of layout thing and into the table creation and then into working with those objects. So you define a layout, you take a data set, you call build table with first argument layout, the second of the data frame, and that gives you an art table object. If you make print, you get a format, if you want print on that art table object, it gives you a formatted table in our case or print is what you've seen before when I, when I ran that stuff in our studio. I think that's also one nice difference between our tables and other table frameworks is that our lowest, our most general visualization of a table is ASCII, which makes it very good for interactive analysis on the command line where others, I think start with HTML, which means you have to be on something that can render HTML. It's in a different pane and it's not in the editor and command like in the console. You need something more. Yes, so let's, sorry, yes. So let's look at that in its essentially most simple format. You see library art tables, you create the basic table, basic table, there's lots of arguments, but you can give it no argument. We use the new or pipe. Lots of you know the, the probably the more greater percent, greater percent pipe. We prefer this one. It's part of the both work, the both work here. You call split calls, and you say, well, I would like to split, I would like to create columns for every level of arm. And by that time, it doesn't know what arm is not the only place where arm actually exists as a variable is next ADSL. And that's important, not you declare, you declare a table. It can be complete bogus. We only check certain things as you create the layout. But it will fail in the build. If it doesn't, if those constraints given in the layout are not met, it will fail with build table. It will fail with an informative message that tells you that your variable didn't exist. It's not. So like, it will tell you what happened. And so the next step would then be sort of basic table has one column, one row. And the cell represents all, all, all, all subjects or the entire data set. And then you split, you create a column for each arm level, you analyze the variable age, and you want to get the mean and the format of the cell rendering should be two comma places. So that object, you can go and inspect, and then it can make a build table, give it a data frame and print that. And so I'm going to go through this in, in the. So what, so well, while you're doing that, there's a question, which I have a little so I'm assuming that by output types, this means like output formats like PDF or HTML. HTML or things like that. If not, please re-ask the question with, with what you were, what you were actually trying to understand. But the answer is, it depends. Some, some of them are tables generates itself. So PDF is a, is a native renderer that our tables has. Some of the others do go like we translate into another table type in order to make use of their output capabilities. That's how we support our TF. So HTML and ASCII and PDF currently are native, but it shouldn't. It shouldn't matter too much because, you know, it's still our type of job to make sure that the output looks the way that it's going to watch a look so. So that answers that question. But if not, please, please do answer it again. Good. So I'm going to show now that code step by step and I show you how like, you can follow me. But so you can run each of those aggregated so you can basic table gives you predated table. Split is important. So you can do this, not, then you know, not split column structure uses arm by levels, you can do this. Now we know split by arm and by levels. You can keep adding split rows. And then you can do the analyze not. And so now we know in the end, we analyze age with a certain function. That's not part of the layout. But you can create those layouts. You can inspect those layouts. You can then build the table with a data frame and the layout and you get the actual table. Now, if those constraints by the layout are not met, like for example, if you give it the iris data set, then it essentially says a verbal arm not found not. And I think one way of using layouts. And so there's multiple way of thinking about it. You can create functions that create layout or you can actually store layouts. And then reuse them for, for example, for different subsets of the data set not I can essentially take this and say, well, I'm actually interested in this where all let me see, I take in America, all patients are greater than 80 knots or library deep liar. And so you can subset this on the fly. Let me make subset ADSL. You can say this is equal to x ADSL will filter age greater 22 or 18 knots. If you do that, this will have dimension. We have 400 years to be hired. I think they're all. That's what I just bought. So let's say they're older than 45. Let's see that again. So we have 400. Yeah, so now 35, you run build table on this. We use exactly the same layout and you get lots of missing, but you get the same table for the subsets. Yeah, do I just quickly turn on the column counts so they can see it's an argument of basic table show call counts equals true. So then if we scroll up, we can see these are going to add up to 400. But now when you do it. Now remember this layout is the exact same layout. We're just using different data. So now if you do the second build table. Line 54. Yes, yes. And you scroll up. See now, now these add up to whatever it was 33 or something. Right. So once you have your layout, you can apply. You can apply. You can apply it to any data set that that meets the sort of implicit specification. And then there's a question we answered that later in the formatting part. Yeah, we have we discussed the formatting so we will make sure to answer that then. I'll move forward. Do you have something more capable to know that's yeah. Okay, so. So let's look at that in an animation again. The table here is the table object in our in our cartoon or in our, I guess, I'm drawing. Then the layout is this L, Y, T we created with those functions here. And then when you, this is the data frame, EX, ADSL. And when we print the table, we get the ASCII characters essentially the render table as ASCII. And so there's really big difference between the output of this. This is a string not printed. But this is a tree structure. So we keep all the information of that. There's lots and lots of information in that object down to the unrounded numbers of the cell values. There's pathing information that you can use downstream for lots of different use cases. So I think that's important to consider that this is really something different than this. The table option is different than the printer table. And I think that's fine. We've seen that. I think that I'll pass the ball to Gabe. Yep. Let me just share. So I'm in slide mode. So layout. So now we're going to talk about how these layouts are constructed in a little bit more detail. So as we mentioned, layouts are free data declarations of table structure, which means that they essentially rely on an implicit data specification in terms of the variables being present in whatever data you want to apply them to, but they don't actually access the data. Right. When you're constructing the layout, the typical way to do that does not require the data to be in place. It only requires you to know what variables will be in the data. And it starts with basic table as we saw. So there are a small number of layout ing verbs that when combined can create all of the tables that are supported by our tables. Right. So first, we have a set of verbs that declare fastening. So these are going to declare the structure in both row and column space. Right. So we see seen split rows by and split calls by. Split by cuts of continuous variables. You can also split where each of the facets represents a different variable. That's what the split by multi bar does. So there if you have a table where each of the columns is, you know, a different value. That's one of the ways that you can do that. And you can also, we're not going to talk about it a ton here, but you can actually fully customize the splitting behavior. And we have some links to some advanced training where we discuss that in more detail, but you have full control over the fastening process. But you often won't need to use that control. By default, each of these, if you, if you do it more than once, if you do split split rows by country and then split rows by sex underneath that, that would nest the gender splitting inside each country. So you have, you know, as we saw in Q table. So you'd have, you know, China male female undisclosed or undifferentiated. And then, you know, great bitten male female, etc. Next, we have cell value derivation. This is this, these are what we call the analyze verbs. So these are what are going to generate the actual sort of basic rows of your table. So one of the flavors of this is analyze, which takes a function and then that function is going to receive the data associated with the facet facet pain that's in both the row faceting and column faceting, then analyze call bars, which is basically the way that you analyze when you analyze by multivar so that it knows which it expects you to give it multiple functions, one for each of the columns and then it will pass down the data from the variable that's associated with that column into your into the function. And then finally, we have sort of marginal or group summarization, we have this only currently in row space, but you can generate summary rows which summarize entire facets, regardless of how many. And these can happen at any point in the nesting process. And so those are summarize. And then, Adrian, am I doing this or you doing this? I'm doing this. It's a bit heavy on my part at the beginning, but then Gabe will do the rest. Yeah. So, so are there any any questions on that while we switch sharing doesn't look like it so. First up, continue. Good. I haven't chat yet. Good. So there it is. Can you see my screen analysis? Yep. Yep. Good. Yes, that seems to work. Good. So, and I think one important. I think we've mentioned a couple of time, but I will call it out explicitly. We believe one of the core functionality of our table is the duration of cell values. And that is a little bit con like that other frameworks use start with data frames and then if you start with the data frames, you have a very good tool to derive a data frame, which mainly steeply at these days I think is accepted as the the most useful tool to derive data frame on large probably at least considered by the community. We did we on purpose didn't choose to go that way because if you build fairly complex tables from clinical trials, like using clinical trials with the player, it becomes very, very complex to do that. There's like duplicated data. You have to take account of certain things and then you have to remove it again. And that's not the case with our tables and that's why so much work and and and on the surface complexity not goes into defining the tables framework but I think if you get more used to using our tables and understanding how we think about the cell definition part, which includes the splitting and analysis function definition. I hope you will agree with us that it is a useful addition to or it's a useful direction other than using the player and creating data frames. Good. So and we can talk some more at the end about that if somebody would like to discuss that but but so I've mentioned that I've briefly touched on that before when you when you say basic table and you don't add a split and call and split rows. It's essentially table where all the all the all the patients as objects are associated to the column and all the patient associated to the row. The column and then so if you need what you create the row with analyze so all the patients are associated to the column, you create one row where you analyze age by default analyze takes mean. And so that would be the table. Now, if you want to reply return multiple values, you don't have to call analyze. In in sequence to me could you could you could run analyze for time and say count and or length and or some is not like the sum of the observations that's not missing. You could run analyze for this and this but you can also, because it's in the same splitting world you can or environment you can. You can use that in rose functions and give it what rose would you like to have per cell. So the cell here is in a generalized splitting way so all observation and then and then here because we don't split rows by it's also all observation here. You can have multiple rows returned. So one, one very brief thing here so like just remember that the multiple rows return thing that Adrian was just talking about. That's the same. Exactly the same in fact as a faceted bar plot having more than one bar in each subplot right so that's what's going on, and that should hopefully help that make sense. So, you run that bill table, you get this. And here in rose. So, if you look in turn and that's why we used a font analysis function so in analysis function we mix formatting and and derivation analysis. You can also say well I only want to do the statistical part and I do the formatting later, you can split that into analysis part and and formatting formatted analysis. Here we do it in one step. So in rose and is equal some of non missing of whatever comes in it's a vector here assumed. So if it formats here, so it's an argument to in rose and format for n is essentially xx format for mean standard deviation. Is this one round you could also add another one with xx with x. And we talked about that later media needs this. So this is essentially your function. And so now if you. Keep using the five now because it's a nice, nice example. It's a simple nice example if you run that with any number any any vector not you get that back not it says how does the format sell look this we haven't yet discussed this we have also not discussed. But what's the name. What's the format. Yeah, what's the set not very briefly we distinguish between name for pathing and what's printed. That is useful. If you followed see this standards but you need to change the label for some reason, the pathing into the elements to access the actual end of the of the table object stays the same. I think that's a little bit advanced here we don't have to go much deeper. And if you're on that you get this. So, I think we keep it with that with analysis there is lots to be said about the analysis for the workshop. I think we made. Yeah, I think the most things important things have been covered. There's a summarized row group we showed a little bit like gave shows that in rows if you want to have multiple rows per analysis call. We also run analysis sequentially to create more rows. That's that's pretty much it. Go ahead. There was a question in the chat I was hoping that it was going to get added to the Q&A but it didn't, but the answer is yes. So, but so what's happening in our tables. There you go. Thank you, Caitlin so. So what's happening in our tables as we've talked about like the actual object is like a structured object and so there's a sub table or age that has at each at each level of the tree where that's being analyzed and by default, if you only analyze one variable because you can analyze one one but if you only analyze one variable it doesn't display what variable it is. But there are a couple of different ways. One is show levels equals top left. So do you want to run that quick. I'm doing it right now just want to show you show. It's actually I think it's bar. Okay, yeah. The variable labels. Oh, a moment. School of school. Wall labels. The top left. The top left is the top left is a is a value of show labels. So if you just type top left in there, that should work with an underscore default visible hidden notes not this one. Oh, I see. Okay, so in that case, you can make visible here but you can make visible there's a top left somewhere which Yeah, so then that does that. And then maybe that's Oh, yeah, so I think the top left is for the splitting so it will it will build up the skeleton as you split. The basic table it has a top left not. Yeah, you can specify you can also specify it. You can specify top left is age there. I will change that I think that you should be able to do that. But right now yeah you would add it there but the top left here. Oh no so it's a pen. I think it's a function. Yeah, it's a pen top left. Yeah, I was looking for age. And then and then age goes up there. And you remember that the table does that for you automatically. And it does that using this mechanism. But Yeah, that's just an over so so Adrian, since while we're in there, can you find one that has a couple of levels of role at nesting and do the labels equals top left to show them what I'm talking about there. Yes, I make split rows split split rows by And then here there's an argument called label label pause. Yeah. What does it say here the documentation top left. Yeah. And then we so let's do that. Well, and then if you go up, it will show you there and then you yeah you also have age because you're appending that still, but then if you do like all another one. Yeah, armor. Country or it doesn't really matter. But but Then if you go up. Then you can see sex arm and like it's actually doing a little indenting thing for you so you can build up. You have the option to build up to sort of skeleton because like so like the levels like f and m are levels of sex but they're not labeled as such so you can have that label go up in the top left there. Yeah, so it looks like you currently can't do that for analyze but we can we can add that as well. And then you'll be able to do that in a future version. I think for my answer to the cashiness. Good so format labels. So we have, I believe we introduced that concept. And if you do that, we yet to have not found the, the regular expression to generalize it because we're not completely sure yet. What's the best generalization. We have full requests. We have declined them because they were not general enough for us. So, right now to, to use those format labels, we just keep adding them as people request them and that's what we currently have. There are a couple of so there's a couple of escape files as well so these are the most fully featured things and they are the ones that are most useful and a thing that you're going to see in a second that But to answer the specific question, you can specify any function that will return a single scalar string and our tables will use that as a formatter. You can also specify formats in the form of like the C printf specification. So you do, you do have full control. It's just that some of the features like the one that Adrian's about to show you will sort of have limited utility if you're, if you're specifying functions instead of labels, but you can do it. You can fully control how they're rendered. You can fully control but if you leave the format labels realm, you make your life more difficult. So we, we, if you don't have to probably don't do it, but if you need to, there's multiple ways of doing it. So essentially, it's possible to run a linear regression, save that object into each cell, and then write a formatting instruction that goes and gets out values from the fit object. It's all possible. If you want to do that, that's not a question. But so one of the nice things then and so actually you can test that not like you can you can write our cell. And you can say, well, I want one point one two three four not and I want format is equal access like that. You can say, well, I want format dot x not. You can then do this, you can give it a vector number, it can make something two dimensional like 2.3456789 knots. The format has to be something else. So the format has to be from here. So you can use, for example, any of those here. That's just this one here. So should be fairly accessible to learn about the format labels. The other parts are, if you go and look in the documentation or cell. Or so gave us mentioned should be format and format is a character the format string format. Oh, it's not documented here. No, it is our format or function. This is right there. There's an S print f function. There's also the S print f function is we provide a constructor which produces a formatting function from an S print f spec. So, so that we can we can make that more clear but but that's what's happening like you. So if you do format S print f something like that. I mean, we make it more clear. It's from a desperate effort probably make it probably make a mistake if I do it but I think it's in format. Yeah, so that that's another thing all of this stuff is actually available to other packages all of the actual formatting machinery we pulled it out and put it in for matters and so that's where our tables gets all of its sort of value to string rendering and other other packages could also use that as well if they wanted to. Yes, I'm going to skip that but it's possible it's all in the documentation I think in the vineyards. Now if you have a table and if you use the format format labels, you can run table shell on the table, and it gives you essentially well except for the part that we have currently with the n equal. So we're going to make that also fix that but it gives essentially a table shell and table shells are useful for planning they're useful for defining standards. I think for internal communication. Those are very useful so that's a nice little feature that might be useful in your environment. But it's not it just prints there. It prints the structure, but not the numbers, the values, it just prints the label and format labels instead. And the last part about analysis the two last things about analysis value analysis function sorry is that a felt that's a very sophisticated topic like we provide many arguments if you request them in the in analysis function so you can also like how many how many subjects are associated to the column that you're in how many subjects are total in the data frame. We provide a split context which is even yet more advanced and I think there's even as a point it's a very advanced topic what you can find in a split context because you can split in columns you can split in rows to get to the subset of data that's relevant for you. And you can also ask what is actually the variable that you that like if you say analyze age not if you need that information that you analyze age you can ask for the variable. So that documentation is a fairly long piece of text to read and as you get become an advanced or table programmer you probably spend some time in that documentation. And so I wrapped that discussion up with how do you make account percentage analysis function. And so, essentially do you create a function the first argument is x. That's the, that's whatever vector that's got sent. The second variable is those special variable names that we set that are conditionally provided if you ask for them. And so then you can say well, some of is not missing times one over and call. If you do that, you can then use that function in here. Here we give ourselves with the format so we don't have to specify any formats here and you write build table. Let's run that and then we are back to fast setting. Yes, so that's good. So here, if you run this, if you run this function not 100 and let's say we have 200 not, we will get something like this. And if you use that in the layout, that's what you get not. So that's essentially how you have control over your formatting and multi-valued cells. Yeah. Okay, I'll pass the ball to you. Yeah. I think Stephen has a fairly advanced question here. I think we will address that. I think at the end, I think that might become more clear when you see the first thing a data structure part. Yeah. So declaring facets, we talked a little bit about this already. But just to be completely clear, you'd split rows by and columns by and then there's some other options for each of them as well. These are essentially going to be independent in the same way that what you put in the rows and columns arguments of facet grid are independent and they sort of independent because they're defining the different axes, essentially, and then they come together to define the facet panes. So we're just going to go through some examples and really sort of hammer in the point that this really is just exactly the same as what you probably have already seen in Gigi plot. And that is a useful way of thinking about that here. Right. So we're saying facet columns by arm. And then give me a box plot or facet columns by arm and then give me the range. Right. So these are going to be the same, like you can see the same structure here. The rows, we have again facet grid rows, and then we have split rows by and these are doing the same thing. And then you have grid, which is when you specify both rows and columns. And again, you have, you have a split calls by split rows by. And that's equivalent to these, you know, columns and rows here. And then I don't have the code for this but we do have it I believe in the supplementary materials there just wasn't really room. But you can have nested faceting, right. So here we have nested faceting in the column space, where we first facet by arm and then within each arm we facet by sex. And then, so you can do that in ggplot, and it just sort of it does it repeats the, the labels, which is fine we don't repeat the top labels like the, we just center them so like the a drug axis just once and then it applies to everything that's underneath it. Which is a more traditional sort of table rendering structure for that for that type of information, but it's it's equivalent to just, you know, on the left there you've got a f and then a m. Right. So, so here's the, here's the example that we, we just needed a separate slide for it there. So there's the code with the, with the example for that. And then, am I doing group summaries, because I feel like you were doing group summaries. I do, I do, I do the group summaries and then. So, one second looks like this. Oh, should I go should I continue. Yeah, go ahead. I was answering it. Yeah, yeah. The next split is to discuss this. Yes, right here. Yeah, we've got that. So, oh, so one, one thing I guess I will say is like the one difference between the sort of traditional layout, our tables framework and as you can see that there's separate split rows calls, right, we split rows by sex and then after that we split rows by BHB1HL, which is this like made up high or low for your biomarker variable that we define up above. And for that is what you're about to hear from Adrian. So there is a reason that we don't just say split rows by and then give a vector, and that is what you're about to hear. Yes, so I have a couple of slides that I think we can make a short bio break and then we go to the longer. It's good. So those are all fairly fundamental topics then we get a little bit more advanced in the second. Yes, so. So, so in our tables, we have a concept of group summaries and those group summaries are very general, but also like relevant in the, like if you look for example that was events table not when you say well, in that group like, in that a body system class not I would like to have all patients that also object of at least one address event that's that's considered a summary. And that gets repeated that provides context to whatever follows that like if we didn't analyze the different type of adverse events not that at least one address event is context and so I'm going to show a little bit how that looks here. It has implications of pagination when you split up the table into printable printable sections for for let's say put printing on a page or PDF. Those usually get if you want to repeat it. But so here, no group summaries. That essentially looks like this. It gives you a table when you run this it gives you a table with only the the with all the labels not it gives you the labels there's nothing in those circles it prints essentially the blue cells. So what what what is to be expected here for every level of those you get a cell. Cartesian product, I guess. It's possible as mentioned before that actually you can return multiple rows for this. For this for this blue things for this blue. Because this this is really a fast that it's a subset for this fast that you can make multiple rows. And so three levels of group summaries. Yes, the root, the root is added and then the sex and be a be one HL by marker low high in this case. And so what we've also sneaked in here in that picture is you see the tree now not there's a root. Then you split by arms and arms of level. There's a root in the row space is split by sex and each there's levels for this variable. And then for each level of language gender, we split again. By this variable, which actually read it right here, and then they have high and low so on this path here would be high low. And then we analyze age and then that gives you the raw gives us the row B. And that's then passing that we expect like you have those things accessible, which is quite useful if you are in the city stand if you in any sort of standardized table world. You, those paths become very meaningful to actually access the value and make cross checking, but for here we only look at group summaries. So, if you run this by default you get those, those cells back or rows. So add after the first split call by, if you add before the first split rose by a summarized row groups, you will get those cells here. So you still get the blue light, the squares, but now we've added this and by default it makes account percentage analysis essentially it says for all the data. In the rows like for all the data then split by a B and C not like for essentially all the patients as objects in army. Tell me how many not missing and. And the percentage of all the patients in here are. If you add the second on the off like before BHL or off the gender. It says essentially for, for all sex females, that's how many we have the percentage within here at the top level which is the route so it's the same as arm. And so on so for male and then if you put it in the end. In the lower points, you get those summaries and by default it makes count percentage, but where you put those summarized row groups really matters in your splitting instructions. So you can add a lot of group summaries. And I think if we do that again analyze create create the most specific facet analysis not it can be multiple rows and group summaries summarize marginal facets here. There's many of them the more you split. Anything to add here Gabe. What was that. Yeah, so the. The other thing to keep in mind, which you alluded to very slightly is those circles can actually have multiple rows in them if you want it to as well. There are at least some production tables that are in the catalog where the summary is something along the lines of a number of patients that had at least one adverse events in this class and then a total number of events in the class and that's two rows. And those two rows together make up this group summary. Right and so those two rows together will be repeated after pagination because together they're providing the context so all of the shapes here are going to default to being a single having a multiple rows but they can all have multiple rows and that if you want to. Okay. Yes. Good. And for right now. That is one of the reason why we can't transpose our tables. So you can't take a table object and say T parenthesis, and then the arm shows up on the rows and two roots that's some other reason as well. The design decision we made early on. I think by now, like having been a couple of years in it. I think we have now generalizations that we could think about enabling it but I'm not so sure if that's the future future that we had any time. But the, the flip side of that is that because this framework is so general and powerful. You can't have an existing object and then transpose it, but most tables, you can just directly construct either. Be the transpose of it or something very, very similar. That has the same information in it. So you can still get those, those structures that are sort of transposes of something else. You just have to lay them out and build them directly rather than going from an existing object. Yes. Yeah, so. And so interestingly, that table we've just built here. If you show all the group salaries is very little. Is very simple to create. Now I'm going to spend a couple of minutes walking through that table just to show you how that table would actually look like. It's group salaries. So here's our example. By the way, if you look at this, and if you saw studio, there's that button here. All the titles that have those four dashes in the end will show up here. So we are in the group summary spot. So if we create example ADSL. We create the basic it looks like this. The row that we create. And then I think I don't do all the steps. Actually, I don't do none. I don't know. Yeah. Yeah, I'll do the last one. If we do the full, if we show all row summary, so default wall, it gets a big table. And so, I guess. Well, it's not longer. It's just denser. There's a bunch more. And so to understand what relates to what we hope it's kind of clear. It's clear. I think usually would not show the summary on every level. Probably accepting like special address and table and so on. Those two are probably different. Not it would say number of page with at least one adverse event total number of adverse events and so on. Yeah. So do the queue table show them the queue table. Yes. Yes. It makes it makes it simple that you can just say that that argument summarized groups. It summarizes all the groups. You don't have fine control over which level of splitting you want to summarize. But yeah, I think that's another. I think feature of queue table that the quick that that you get. Yeah. So I suggest we make a five minute. I break and then we go to the slide 6676 is a good. Yeah, sounds good. So we meet in 157 for is a good. Yep. Thank you very much. I think it's 57. Okay, let's go ahead and get started again. I will share my screen. All right, so now we're going to sort of step into a more advanced, a more advanced sort of usage in of the conceptual framework. And we're going to do this in reverse. We're going to look at a table that is complicated. It has multiple parts. And we're going to walk through how to think about it. Like our tables thinks about it. And then we're going to talk about how that translates into how we would construct it in our tables. And so this is a table that's going to summarize is a few different, you know, Cox proportional hazard models, right has an overall one and then it has ones that are by covariate. The first thing that we notice is that this is this has two distinct top level sections right it has a section that's the sort of overall treatment and then it has a section that deals with the covariates. And then for the covariates, that section has two subsections, it has the one that deals with the age covariate and then that has the ones that deals with the race covariate. Right. And then within those, it has a row, a single row in both cases that summarizes the overall covariate. It has effect. Summaries for each value of so for the median of age because ages. Ages numeric, but then for each of the levels of race. Okay, so. Finally, the columns are going to reflect different elements of the same model fit. Right so there's a lot of things that are going on here. But we can. We can construct this table in our tables within the framework that we have talked about. And we're just going to because this is not an advanced training, we're going to just assume the existence of a couple of functions, both code for those functions is provided and we have is available and we have links to that. In this slide right here. So if you go down to, if you follow this link, there will be a section, the beginning of the sections basically the same as what I just walked you through but then later in that section, all of the code that we're using here is available and I'm also it's not currently in the supplemental code dot are but I am going to push that right after the workshop. I'll update that to have that code as well but suppose that we have it's actually two of them we have the Cox model main element direct and we have the Cox model element direct. And so what that is going to do is actually going to do a number of things. This is going to cache the model so that each model is only going to be fit one time. And then the other thing that it's going to do is it's going to extract a particular named aspect of the model, which are these, you know, these things that we have across the top. So either the end or the hazard ratio, conference interval, the, you know, the p value or the interaction p value. Okay. So, then our table ends up being something like that so the block up on the top just has to do with, you know, some very quickly somebody's confused. There's multiple Cox models. So every time you see hazard ratio it's a different Cox model. Yes, you. Yeah, so going back here. So we have the top is a single overall model, I believe, and then that model is fit with an age covariate and then it's fit with a race covariate so there's I believe three models, which are these three here. So overall on at the top that just has the treatment, they all have treatment, but the top one only has the treatment, and then the covariate ones have the treatment and separately, either the age covariate or the race covariate. And then the interaction is between treatments and that correct. I don't I can't speak to the bio statistical decisions that go behind this table. I didn't design it. I just, I just recreated it in the framework, but this was designed by bio statisticians at Roche. So I assumed that whatever they're doing is a smart thing to do. But yeah, so there are multiple models. In fact, there are three models. There's, there's one for each covariate where it has, as you say, both the covariate like the covariate interacted with with the treatment. And then there's the one that just has the treatment, which is the sort of base model. Yeah, so the, the code. Adrian, can you track down the just load up your slides and copy the link the link is in the slide deck. The code to the model. Yeah, this this right here. I guess maybe I can. Well, you have to kind of, you have to search it from there. Yeah, so this is the start of the section that has the code in it. So just, I think the full reproducible code without copy page like with a multiple copy paste game will post it off. Yeah, I will, I have it added to my local. I'll show you guys that in a second, but I have it to my, I have it in my local supplementary code are and I will push that now. One thing to be aware of is it does require turn. Because as we said turn is where the statistical logic is implemented. Our tables does not implement any particular statistical logic besides very basic, you know, count percent type things. But turn is open source so you can install it but you will need to install it to get this code to run. But then what we have, and I'm going to I'll show you guys this code in a second is we have a layout that has split by multi var. And we're just sort of tricking it we're saying the variable is study ID the whole time because we don't really care what the variable is but the labels are these guys here. The end has a ratio, etc. And then the model elements are these here and the reason we're doing this is because these aren't real variables in the data set so it would complain if he's told it. If you tell it to split using a variable that's not in the data set it complains so because that doesn't exist so right so here we have this. And the caching is more related to the advanced training, like you could do this without the caching and it would just fit the model multiple times but I didn't want to do that it's not satisfying to me so I wrote. I run analysis function that actually does caching and it was fairly straightforward to do and in fact it will probably be done automatically in some cases in our tables in the future but it is not right now right now it's something that you will build yourself. And that and then you get the summary of the overall model. This is a summarized row groups which means it would be repeated if we wanted it to not be repeated it could also be an analyze call and otherwise look exactly the same, in which case it did the row which still show up but it would not be repeated. And then we pass it by covariate. Those by multi bar and these are real variables so we don't have to pretend right the variables are age and race. And then we have labels for our variables. We have the split label, which is just covariate here. That's often not displayed but in this case we are displaying it. And the end of mod is just what the end of mod does here is it makes these line up, it makes Asian lineup right under race. So you can control like by default. Asian would be indented from race. And for whatever reason, the bias decisions of the medical writers or what have you didn't want that they wanted that to line up. So, you can. So this is an example of you can fully control the the indenting to move things around. If you don't like where they show up by default. So that's that. And then we summarize that again that's the that's the covariate summary, the level, and then we have analyzed, which the analyze is what gives us those one or multiple rows for the individual effects. Okay, so if you go if you go step back I think the most important piece here is you see the invocation of the modeling not it's in the analyze call vars in summarize and summarize so it's a right. It's a kind of clever use of group summaries and then the right and this this illustrates the fact that like so so there's multiple ways to get columns that behave differently. If you need them to write like by default like the sort of default basic thing is like rows will behave differently but columns will be sort of or sorry yeah different rows will behave differently but within a row, the cells are all sort of have the same meaning that's that's the default if you do split rows by and then analyze mean right like that that's the mean row and then all the cells in the row or means for the different columns. And that's very powerful and I can get you a lot of tables but not all tables are like that some tables as we know have columns that actually have different meanings so this is an example that you can do that as well this is a. Again, this is a very advanced example. Um, so this is also I will say we are like 90% of the way done with transforming this example into a vignette that will ship with our tables so not right now it's so it's I mean you could look in the pool there's a pull request for it already but in the future. There will be a vignette that actually walks through this in full detail. And that's also here to pitch the like that pattern, where you have a simple where you have an analysis function, you can reuse it in various parts and analyze and summarize row groups. That's the same if adverse event you find that pattern often that really all the complexity of applying the same model or same summary at different levels gets quite far. Which I think, if you read the clinical trials tables vignette or table pages you similar pattern. Um, so that. Hey, that may be the case, like I said, like I. I'm not a, you know, practicing clinical bio statistician. So I can't, I can I can tell you that the. The modeling function is being called three times in the construction of this table but I don't. I don't know the details beyond that. I can make sure I will make sure that that's in the vignette though. Once that's merged in. But we're sort of at the limit of my, my knowledge about exactly what this table is doing. Because again, I didn't design the table. I just implemented the table based on this specification that I saw. So with that, let's go ahead and let's go ahead and share my studio. Yes, I want to, yes, I want to reload it. So I'll copy it all this stuff here. This is again, this is complicated. So like, don't be intimidated the vast majority of analysis functions aren't going to look like this and the vast majority of analysis functions that do look like this will be written by bio statisticians and then simply use then it'll put in somewhere like turn, and then they'll simply be used when you're actually constructing tables. So here's the cash model, which just says the formula is the arm times covariate. And the covariate can either be. Yeah, the, and then we and then we'll fit that so this is the cash model. So it receives an environment. And then it checks if that if the model for that covariate exists already if it does grabs it if it doesn't, if it's it again. And then it prints it just so that we know that is being called. And then it puts it in the cash that will, it will get it next time so if we run this. And this the details of exactly what this is doing is are not important the what this is doing is it receives what model element you want so which column you're in essentially. And then it grabs that information from the, from the, from the model object, and then it like figures some stuff out and then it, it calls in rose. So the documentation of what the modeling exactly does isn't the term function if you go up there. Yeah, but this term is not explored so that these are. Again, there's a vignette that doesn't use triple colon and is like explains what's happening here in more detail. This this was this is from a training for the people who write turn, which is why this is so much more complicated and crunchy than the other examples that we have. So it's still valuable to show that you could do it which is why it's in here. But so, if you're a little lost about exactly what this function is don't worry that's not. That's not unexpected and that's not the point of me showing this to you right like the exact details of how this function works are beyond the scope here so we're just going to say that it does work. And then we're going to, then then we're going to use it. So, run that, and then we have, I have turned installed locally. I also have the RCD archives that we need for this particular table to get it out. These are turn functions that do some stuff dealing with how missing this is coded. Then we're going to have filter both ADSL and AD TTE. And this is another function that's for the main effects. And so here, here we're back to the thing. We've got environment so then we build our layout. We can look at our layout here. So these are, you can see bars here, that's the multi bar thing. Again, these are pretend variables, right? Because we can't actually say N and hazard ratio and things like that because those aren't the data. But that's what the labels for these columns will be. And then for split, we have age, race, and then we have call bar analysis, which is like this is an analysis but which variable is being analyzed depends on the column. So that's what that means. And then we do build table. And we can see that this is running that. So that says that I guess it's interacting with itself, which doesn't make a ton of sense, but you can see each of these is one time. And these, this is the table that came out. So we've got our different columns, which have different meanings. And this is, this actually comes from the TLG catalog, I believe, and it recreates the values that are there. So, I don't remember which with the name of table is in the TLG but it's in there. Again, the details of exactly what this function is doing are not really the point here. But you can see that we have a lot of flexibility in terms of columns many different things we have a lot of flexibility in terms of, you know, different levels of summaries. And here it looks like I'm not doing the, the indent mod so you can see actually by the default this this indent mod is right here. I think that sounds the correct indent I would say this is the I like this I like this indent better personally, but you can also see we're using the labels for the variables rather than because this is the sex variable. This is the race variable. Not the sex variable. This is a race variable, but it's labeled ethnicity. This variable name that the variable is all caps AGE, but we have a label that's not fully capitalized here. And then we don't have the treatment thing here we could add that like the treatment colon thing there but this is a very advanced work example. Again, if you're a little bit lost on the details of that. Don't worry about that that's not really the point here the point is that you can do it. And then you can sort of become a power user of our tables and do that or the other point is like this, this is the type of function that would be in turn. And then you would just use it from turn without having to worry about exactly what it's doing. So, so that's another another way in which we don't really have to worry too much. And then I'm going to switch back. Beth, yes, there are. You can add titles and footnotes. And you can also as a completely separate piece of functionality have referential footnotes in either on either columns rows or cells. So all of that stuff is supported. I don't think we have a slide or code for it in this presentation but it is all supported yes. Did we already add the page one out of five. No page numbering is a forthcoming feature that one's not in yet. But I think the cell value referencing and it's not just sell its row column and cell. Row label column label or sell all of them can have footnote references which are different than footer materials, which is like, you know, file analyzed equals whatever, like provenance style stuff. That gets that goes all into pagination and gets repeated. Yeah, and when you paginate the, it understands that only the referential footnotes that are relevant to the page get printed on the page and they take up space and that's that's accounted for in the algorithm. So like all that all that stuff works and is supported. So the final section. Yeah, so the final section here is another one of those these bleeding edge features, which is, you know, this now this concept of an analysis reporting data set. And so as we've talked about and shown in a couple of different cases and a couple of different ways. Our tables objects are not data frames, right. They're just not they're not data frames. They are complex structured objects that have a ton of information in them. You actually, we haven't even really scratched the surface of all the information that they retain and allow you to interact with once you've constructed the table and so we'll probably do a little bit of that with whatever time we have left and including while we're doing Steven's question I think, which I haven't forgotten about, but there are other times when people want a data frame, right. And so, for a long time, so we've had some form of export as a data frame for a long time, but it wasn't really in the form that people really liked so I've now written an ARD. The reporting data set is typically leaves the ARD acronym so an ARD export train like extractor, essentially. So, an ARD is a data set, not a table object, which contains all of the computed values that make up all of the cells. In addition to enough metadata to identify where those go in the table. There's a short description of what an ARD is. And we'll see what I mean by that in just a second. So. So, yeah. And then so ARDs sort of aren't really real yet. So currently there's no specification for the format or contents of ARD I know that that CDISC is working on one, but it is not currently a formal specification. There are pharmaceutical organizations that have some form of these internally. But typically from what I have seen, these often don't have a unified structure, even within an organization. So ARDs for different tables will have sort of different incompatible structures. And they certainly don't agree across organizations, even for the same table, the different organizations will be representing that information in different ways. To what extent, you know, CDISC is going to step in and solve that with a formal specification is unclear yet. Again, they are thinking about one, but there is no, as far as I'm aware, there is no current spec, or even sort of candidate spec, like there's some work towards it, but it's not, it hasn't been fully formalized yet, I don't think. The reason that people want ARD is sometimes, like often people are thinking of ARDs as being a step before the table, so like you create your ARD and then you like feed that into the table engine that obviously from everything that I've just said, over the course of the past two and a half hours is not how our tables is going to view ARDs. There are still some benefits, primarily that you can then do further analysis or visualization of the values that appear in your table. And that you can then you can store those values in a sort of ubiquitously supported format, which is CSV in that case. We can now generate what I am calling ARDs. Again, ARD is not a formal specification that I was able to work against at this point. But I have something that you can generate that has the information I think will go into the ultimate ARDs. It accepts a table object and it returns to sort of what a semi-wide data form. So the columns in the table are different columns in the ARD, but all the row fastening information is sort of built into metadata columns. So this is a very this is this is sort of just a little bit of historical trivia. This is the first really complicated table I ever wrote in our tables as a test for all of the features working well together. So this is probably the oldest non-trivial table that exists in the modern art tables framework. And so it's used in various places in our tests. So what happened, what's happening here is you have, you split by race and then you split by another variable. And then you analyze age and then you analyze age again, just because. And then not nested within those splits, you have another analysis of a variable which has two levels, and then you're going to count those. So that's what's happening here. So you have multiple levels of nesting, you have some analyzer rows that are nested within those levels of fastening, you have other analyzed rows that are not nested within those rows. And you have group summaries going on all over the place, which you can see with all the sense going on there. You have multi-valued cells. So there's a lot of stuff going on. So if you call, so this is just like you call it as ARD. There's a bunch of columns, so I'm trimming a little bit of them out. But basically what ends up happening is it gives you back a data frame where the first, sorry. Yeah, so this is the sort of metadata section of it. And so you've got, you know, split variable one, and then split value one, and split variable two, split value two. So it has detected that there's two level, the maximum level of splitting is two. And then you have the AVAR name, which is the name. It's a little bit different. It's the name of the sub-table. So you can see here it's age, redux instead of age. That may change in the future. The exact specification here is still experimental. But you've got that. And then you've got the row name, which is going to be like the, what's printed for the individual rows. And then you have the row number, which is the absolute position in the overall table. So you can see that three is skipped here because three is a variable label row that doesn't have any cells. So it goes one, two, and then four. Because if you go back here, no, go back. Three, no. Three, the third row here is the row that says age analysis, but it doesn't have any values. And since it doesn't have any values, it doesn't appear in the ARD. So then we have the marginal group summaries, which are marked as such. And you can also tell because the AVAR name is, no, NA, I should say. You can also see that for the higher level summary, the summary of race white, the second split is also NA because it doesn't have a second split, right? Because when we're here, this first row, we don't have a value for the level A, or level B. We don't have a value for that split at all because we're outside of it. We are at a higher level of hierarchy. And that's captured by putting these NA's here, right? And then we have the analyses that are nested within those splitting, right? So this is capturing all of that. We've got race white factor 2A. And we've got two rows for age, which are mean and median. And then age redux is range. And then, you know, these are not group summaries, which is marked there. And then we have the non-nested analyses, right? So these, it is going to correctly say you don't have any value for any of these splits. So you just have your analyzing bar 3, and then you have level 1 and level 2 there. So that is the ARD as it exists. So as I said, there's no formal specification for ARDs right now, which means exactly what exact form that they are in. And whether you know the names of the columns and things like that are ever sort of formalized is something that may happen in the future but is not happening right now. So the way that I've solved that is that as ARD accepts a specification, a spec argument, and right now the only specification that it accepts is V0 underscore experimental. So anything that ends in underscore experimental is going to be subject to change. But once that is crystallized, that would change into just V0 with no experimental. And that will be available forever, right? So if there is a new spec that comes along, that will be V1. And it will start as V1 experimental. And then when we settle on that, then it will sort of be V1. So like you'll always be able to get back to previous versions of what our tables thought in ARD was. And that's the guarantee that we're providing, including any bugs that were in that, right? So even bug fixes are going to bump the version numbers. So you will get exactly the same output for V0 forever. And yeah, so that's the concept of ARDs. I can run you, I can show you guys that in practice. I mean, it's honestly not that exciting. It's exactly what you just saw in the slides. But just for completeness. We have half an hour now for question. I think half of the people here on the call are not. Yeah, well, I mean, it's also very, it's a single function call. So like, I'll just do it very quickly. And then we'll address the questions. So here we have our, this is our big complicated table, which is the reason we're not using X80S cells because these are straight out of our tests. And we got table, there's our table, and there's our ARD. And there you have it. So the other thing to note is that like, these are multi-valued cells right now. And that's a simple how they are a nest longer call to fix that if you want, if you want to fix that. So one thing that I will, that I'll sort of address from, from Steven's question now is. So the, because our tables is not a data frame, our tables object, our tables tables are like complex objects, you can actually ask it what the labels and levels are. So this is, this is our table, right. So we can see that we don't have, you know, we don't, it doesn't tell us that this is race like we can understand because we know what Caucasian is we know that's a level of race. So that if we do row paths. Now suddenly we can see. Oh, okay, this is rates and then white and then content, which is content is what we call the marginal group summaries. They're historical reasons for that. We're doing the game we probably call something else but that's what they're called, and it doesn't really matter because it doesn't make much of any difference. So that you can ask it about what pass and then these paths can actually be used to select elements out of the out of the table. So if we if we do sell values of the table. This is all of the cell values. Right. And that gives you back, it gives them back as the list or nested list. But the values except row path and call path. So let me see if I can do this easily. I'm going to cheat. So that's the values for the first row for each of the columns. You can also select the columns you can ask it about column paths. Okay, here are my column paths and this as you'll notice has that information in it, like what is the name of the split now. Another thing to keep in mind is usually splits will be splits will be simple partitioning based on a categorical variable that's not guaranteed. There are other types of splits. We saw one of them, which was the multivar split. But there are other types of splits. There are, you know, cut splits so you can split by quantiles or other things like that. You can actually fully override what splitting is doing. So the only thing and like that. If you if you search around in that in the link that I gave you for the code. That's a whole that's a that's an advanced training that actually talks quite a bit about customizing splitting behavior. But the bottom line is the facets are not fascinating is generalized in our tables. So they are not. They are not guaranteed to be partitions. In fact, let's just load up the help for So we have 20 minutes to wrap it up and ask questions. Yeah, so if you have while I'm doing this I'm doing this live or so like, if there are any questions please type them in the question and we will answer them I'm just showcasing additional stuff while we while we wait for that. There's a couple of slides more to cover. Oh, I thought that was the end of the. It's not it has some further topics and then. Right. Do you want to do that will I. I'll do that. Can I grab the screen share from you. Yeah, yeah. Good. So. Yes, it's the end of the scripted workshop. I think, as you probably have guessed or gathered from the, from how we talk about those topics there's quite a bit more. There's pagination, which is a fabric topic that is rendering to different formats. There's sorting and pruning sorting meaning if you want to sort within a certain depth of rows like an adverse events table pruning is if you want to remove like zero counts for example. There's also things where you say well, if I have a Metra dictionary not I would like to have all the levels. I would like to display to map. There's path in the gauges showed there's table comparison topics like if you have two tables you want to see what's the difference. We have functionality to compare against baselines, which, which means you can define a reference column that's a title photo that's already mentioned and custom splitting functions quite a bit more. So, I think in this workshop, at least the beginning we tried to be quite a bit more general to make it more accessible. I think the workshop and but there's material in the vignette. You can look at our informal workshop. That's a little bit deeper into our table territory. Yes, and then advanced training that came mentioned. And I think with that. Let's look we have we have now 20 minutes to answer questions. Let me let me let me share my screen and I'll run through the because I think this is going to be useful for people but while I'm doing this. As Adrian said any questions we are happy to answer them we have about 20 minutes so we definitely have time for questions. And I'm just going to like showcase random things that come to mind about our tables while we wait for for those. So, the first thing that I think people probably be interested in is how easy it is to add combination levels. So, here we're going to say, we're going to define our combo levels and what's called a combo DF. Which is, you have the name of the value because it's not one of the values variable takes, and then you have a label, and then you specify which levels of the category for variable you're combining. And then you don't really have to worry about this last one is extra extra args is a whole thing but you typically don't need it at all. And then you say, I'm going to split by arm, but I want to use those combo levels in addition to the combo levels that I did the normal levels. And there you have it that's that's all you need to do. There's no manipulation of the data. There's no duplication of the data. And then we have the additional we have all of these are the same as before and then we have these additional combination columns like combo arm, like virtual combo arm type. And so can you add the total when you want to hear, because you can. Yeah, so total is so you can add so you can add total, like, as a there's a helper function. So I'm going to use this. It adds total, which now I need to add what's it now. And overall, overall, fast level, fast level, fast it is for the when you're building your customs with functions level is for when you do that. Yeah, because that's the most often used, I guess. Yeah, that's the most often used and you can you can do both right like you can add it here. But you can also say, you know, I'm going to bottom here, because I already have an LIT to here. Oh, I already have this. I don't even have to do anything. So, here we are splitting by arm and we have our combo levels, but I'm only adding one just because I only feel like adding one. And then for stacks, we are adding a reading at overall level to add an overall thing. So let's just run that. That's that's within a split. But overall, let's see if you have a simple arm and total. Yeah, so so that's I'll get to I'll get there in just a second. So I guess I'd never actually built this a table to this build table. I'll get to and then small and what it helps if I define the data before I use it. So here we go. So now I have all dinners male and female and then we have a B and a plus B. Now there is also so this is distinct from I want a something all the way off to the right that is just a column with all of the with all of the observations in it. Usually, this is the correct one. So like, you can you can add. You can add, you know, the overall level to arm and so then you can have an all arm, but you can sometimes say, I actually just want this but then I also want to like everybody all the way over on the right there. And so that is a distinct thing, which is like, oh g3b. And it comes right here. Copy this. And then we say yeah, I'm wondering why these are columns and not rose what's going on. Yeah. So I'm actually, yeah, I don't want this one I want this one is that there are some of the splits for rose. We want the one where they're all columns. So then we say, overall call. And what is the name of the column that we want. Awesome. Overall, all subjects. And then you can see, I made it really long, but now we have our awesome overall call that has everybody in it. And then that's all the way off to the right so that's not nested under any of the arms. Which you can see. No, no arm. No arm here. So that's the difference between an overall level and an overall column and our tables gives you easy ways to do both. I think we have a question about exporting a table. Can you show that? Yes. So how do I want to do this? I think we even have a we don't get from the last workshop to export with pagination and everything. So exporting by default involves pagination. That's true. I need a table. So this is probably, well, this, this obviously won't paginate. Vertically, because there's only one row in it. It may or may not paginate horizontally my suspicions that it is going to, but we'll see. And then. So like ready was that and then. No, first we'll do. Let's take, let's take a demographic like table from the vignette. Yeah, so. Well, equals. What? Where am I? Sorry. Yeah. Whatever that's fine. Okay. And then you want a demographics table from the vignettes. So that involves us going over here. I'm not actually in. You can see my super gross lots of files going on here, but let's say. Should I do it? I got it. I just need to. Come on. There we go. I just need to get to where I have our table checked out. I'll send you the code. In chat. Okay. Give me seconds. This is the clinical trial then yet. It's a couple of things you have to copy out. So I've sent it to you in Google chat. Let me just grab that. If you create a new file. Yeah. New file. You're ridiculous. New file. New file. Thank you. All right. So what is renaming this? Because we feel like it. Because we wrote the set, we wrote the code first and then we were like, Oh, that's not what it's named. So. This is a summary function. Here's our table here. Right. So the age and then you've got two variables that are being summarized. Sorry. This is. Gender. Female male. And then within female and male, you've got two. Variables that are being summarized. So this is still a relatively simple table and by our table standards, but you can see that there is quite a bit of structure going on here. And then we can just do. Export as PDF of what do I call it TBL. And then I named the file equals, what is it TMP file? Yeah. Can you open that file then? So we see it. Yeah, I will. Yeah. So. We've got that. And then if we open that, hopefully this will actually. I think you can't open it for us to do. Fine. I will stop sharing my screen. Open it. In a different. Thing. Just give me one moment. And so the, and so the other formats usually you, if you want to import it into PowerPoint or later and so on, you can export it to flex table. You can also, I think, flex table. I think soon we will release the, I think the GT conversion. Corrosion that you can use GTSRTF function, which is essentially. I can't. It's. Let me, let me do it. Yeah, you do it. Okay. So share screen. Yeah. That's the blog of 10 minutes. So. So essentially export as PDF at table. And then you give the file. Give it. Table. Yeah. You can say that you can say that later. You can say it's landscape or not landscape. You can give all this. You can give all this information that will then do this. You can open this. And it will. It didn't actually work for me. So I'm not sure. Yeah. So you've got it there. And so this one fits on one page. If it doesn't fit on one page, it will. It will. It will. It will do the pagination and it would repeat the rows. So, um, I guess. Why don't you paginate, like just call paginate table. Um, I can make it landscape equal true. And then it will paginate it. I assume to. To show that. So there we are. And so the important thing here, actually, if you look at, um, if we zoom in here. So those are. Um, and so if it gets split in between, you can say the number of siblings, not that it doesn't break enough. Like you said needs at least two. I don't want to have an in itself on the previous page. Um, so there's a number of siblings. If you do that, then, um, undifferentiated age gets repeated. If, if you're an undifferentiated, we are not differentiated, but if it would get split between medium and high, which it doesn't, um, it would get split between medium and high. Um, and then it would just repeat you and biomarker. So it keeps its context when it paginated. I'll just set the, set the font size to something bigger so that we can. I don't see it. Yeah. I think it's still not enough. Still not enough. Siblings not. It will never break it here because you don't have enough. So just do, um, men siblings equals zero. And then it will force it to do that. Um, it's, it's, it's a dot dot dot, but it's, um, men siblings equals zero should work. Men underscore siblings. Cause it should just be being passed to, um, there you go. Yeah. So now it gets. Split, um, after N. Um, and so if we go to the thing here, you see. Yeah, it's repeated. You and the HR. You and Asia repeated. So then you know what that, you know, that, that first quote unquote first line, the, the, um, The mean, you know what that's talking about, right? Whereas if you just truncate. Then that you and age wouldn't be there. So you wouldn't, you wouldn't know what the, what the mean. Meant. Um, so that's what we mean by context preserving pagination. Yes. I'll answer this as next question here. I'll answer this here. And so is it possible to super to use up super and superscripts? Not in ASCII. You could do it. Export it to late deck. It's though that won't be a. Yeah. So, so we, we target ASCII. ASCII does not have subscripts and superscripts. It also doesn't have legatures, which is another thing that we have been asked about. Um, And the reason for this is, um, We don't really care what font you use, but our tables needs the font to be mono spaced in order for the pagination that we just saw to the machinery to work. Um, and so superscripts and subscripts. Aren't in mono space. Um, because they don't take up a full character with. Um, So that's also why ligatures aren't, um, aren't in. Um, But yeah, so, so currently no, you would not be able to use subscripts and superscripts. Any, any other questions. Or, or general comments. We have about four minutes left. Any questions or comments? Otherwise, thank you for the attendance. Um, our table is open source. Um, You can go to here. Um, also Gabe is a consultant. Feel free to reach out to him. Um, which actually haven't provided your contact. Um, you can add them to the slides. Yeah. And, um, Yeah, I hope that was useful. Um, If you have feedback for our table, um, if you have technical feedback, um, I guess any feedback, but made it. Yeah. Um, would be good if it can go and rate. Features that features that you think we might not have. Probably a lot of them we do have because again, like our tables is a powerful system. So, um, Yeah. So you can do a lot of things, including stuff that we did not showcase here. Um, but if there are features that you might need, you know, please feel free to open an issue on the. Our table's repo, like we. We address issues. We respond to issues quickly. We address feature requests. That are straightforward. Quickly. Feature requests that are much more complicated. We would then engage with you to see. You know, what we can do that. That would meet your needs. Um, but again, our tables can do. Most of what you see the overwhelming majority of what you see in clinical trial reporting and table in the table context. Uh, there also is just as a very, very, very brief plug with no. Lines. There is also in our listings package, which is much simpler and is designed specifically for listing. Um, so you can look up at that. That's on crayon as well. Um, but we don't have any time to get into that today. Good. I think this is it. Um, thank you very much. Thanks everyone for, uh, for your attendance and your questions. Hopefully this has been useful. Um, and if you have any questions, like we said, please reach out.