 All right, I see there are a good number of people joining today's meeting. Thank you everybody for joining. So we'll get started. This is the our adoption series and today we'll be talking about adoption of our in Japan's farmer industry confirmation. My name is Nina and I'm from road genetic and I will be the facilitator for today's event. So this event is hosted by the archon source of our submissions working group. This is a cross industry collaboration to improve open source language usage in the regulatory setting. And we are a public working group. So anyone is welcome to join our meetings and our meeting agenda meeting materials are available publicly on GitHub. So in today's meeting we will be hearing about the GPMA our task force activities. So we will start with two presentations from Ricky and the UK. The first part of the presentation will cover the GPMA our task force, the past activities and future initiatives. In the in the second half of the presentation, Ricky and UK will share with us a very interesting survey that GPMA conducted on open source software usage. They will share the rationale and motivation behind the survey and they will discuss key findings and their implementations for for the industry. And after that, we will go through several pre collected questions in the Q&A session. And if you have any questions, please feel free to use the chat box. We will collect and curate the questions and then try to follow up with a blog post after this event. The presenters for today's event are Ricky and Yuki. Ricky has worked as a statistical programmer for Jensen Pharmaceuticals since February 2015. He is responsible for statistical analysis in clinical trials and eData submission to PMDA. Before working in the pharma company industry he has experienced developing bank accounting and customer management systems in Japanese technology company. And also he is a startup member of the open source software task force in GPMA. And Yuki has worked as a statistical programmer, a medical scientific expert in medical science lesson for Novartis pharma since April 2017. Recently he works on new drug development and retrospective studies using medical reward data such as electronic health care record and health claims data. Also he is a member of the admiral of development team and a startup member of the open source software task force in Japan Pharmaceutical Manufacturers Association, the GPMA. So with that, I will start, stop sharing and then turn the ball to Ricky to present the GPMA effort and survey result. Thank you for your kind introduction. Can you see my screen? Yes, we can see your screen. First of all, we would like to thank our association for giving us the opportunity to introduce our activity. We are a member of the open source software task force in Japan Pharmaceutical Manufacturers Association, the GPMA. Today we introduce the youth of our in Japan's pharma industry. This is a disclaimer. The views and opinions expressed in this presentation are those of the speakers and do not reflect opinions of any other individual or organization. And we will share the survey results today, but the original version is in Japanese. So please download it from the GPMA homepage and refer to it if needed. This is today's agenda. At first, we will explain about our task force in the GPMA as general background. And next, we will share the open source software usage questionnaire report of the survey we conducted in 2022. And we will have a question and answer session at the end. Okay, and let's move to the general background of the GPMA task force. At first, I will explain about the GPMA. The GPMA is a voluntary association comprising 72 research-oriented pharmaceutical companies. The GPMA established in 1968 with the mission of realizing patient-oriented healthcare has been contributing to global healthcare advocacy through the development of innovative ethical drugs. The GPMA is engaged with various initiatives such as the solution of common issues in the pharmaceutical industry, activities to deepen understanding of pharmaceuticals and international collaboration with concerned parties. And the GPMA and its member companies have established 12 committees comprised of various member companies and six specialized organizations. These companies and specialized organizations carry out activities based on business policies and plans, while building good relationships with a variety of stakeholders in Japan and abroad. Our task force belongs to the drug evaluation committee, which is one of the 12 committees. Next, I will introduce our task force, open-source software. Our task force of open-source software, OSS, is in the Data Science Expert Committee, Drug Evaluation Committee of the GPMA. The purpose of this task force is to investigate the use of OSS, which is being more actively used in the pharmaceutical industry, especially for the annex of clinical trial data and work related to regulatory submission, and to compile and publish a report on the expected benefits and issues when OSS is used. This task force has studied its activities since 2022 and currently consists of 10 members from pharmaceutical companies. As the reason for the task force, many pharmaceutical companies were interested in OSS like R and Python, and some people are considering the use of OSS, for example, about use for regulatory submission, validation and international cooperation, and others. To digress a little, I would like to share with you the requirements regarding PMDA's programming language for analysts aimed at submission for a clinical trial. This screenshot is a part of the technical compromise guide on eData submission, which is one of the documents related to eData submission published by PMDA. You can download this document from PMDA's page. There are also challenges in using R for submission, and PMDA does not require the use of specific software for the data management and analysis of clinical trial data for the purpose of submission. Therefore, the choice of software to be used is left to the applicants, and there is no problem in using R for the purpose of submission. However, it's necessary for the applicant to conduct verification work to ensure the quality of the software, reliability of analysis results, and document the procedure and results. Back to the contents of our task force. This is the content that we have worked on or working on. We have been active since 2022. In 2022, we created two major deliveries. First is about to release the document. The document titled Utilization and Considerations for Open Source Software was described and released by JBMA in 2022. In the document, current activity is related to R in the pharmaceutical industry. The challenges of using R for regulatory submission, package versions and operation management and example of R training are introduced. I will explain this overview on the next slide. Next, we conducted a survey to understand the usage status of open source software. In clinical development work by data science subcommittee member companies and compiled it as the OSS usage questionnaire report. This will be the main focus of today's presentation. And third one about activity from 2023 to this year. The main theme of task force in 2023 is to collect and publish examples of R and Python utilization in the pharmaceutical companies in Japan. The task force is planning to investigate examples of OSS utilization in clinical development and share the introduction procedures, usage environments, software including packages and utilization events. The task force will also continue to consider the use of R for regulatory submission same as 2022. This slide is the overview of the documents utilization and consideration for OSS release 2022. This is a Japanese version only need. You can download this document from JBMA homepage. This report first introduced the definition of OSS and then introduced OSS that has been particularly active in recent years in the pharmaceutical industry. It's advantage point to note when using it. In addition, R is used as an example to introduce the management system and the contents of education and training that are considered to be necessary. When it's assumed that OSS is used for regulatory submission work. I explain each section. Section 1 introduced the types of OSS that are applied at least. An introduction to R and the activities related to R in the pharmaceutical industry. We introduced the R submission working group on R consortium, Pharmabass and R Physician Hub. Section 2 describes the challenges in using OSS. Section 2.1 describes the management of operational tasks. When using R for regulatory submission, it's necessary to prepare SOPs and corresponding work procedure manuals for each of the processes. Also, section 2.2 and 2.3 describe version control of R and R packages. When installing multiple R packages and to ensure that everyone in your organization gets the correct R package type and version. It's important to understand the complex dependencies between R packages and use the correct version. The tracking version and dependencies ensures and reproduces the ability of R program execution results. However, in order to understand and manage the version of individual R packages and the version of dependent R packages or R ones. A suitable management system is required. We would like to share the result of the OSS usage questionnaire report conducted in 2022. The purpose of this questionnaire is to understand the use and condition of OSS in clinical development in pharmaceutical companies. As I mentioned earlier, the reason behind the establishment of this task force was that there are many pharmaceutical companies interested in OSS. And we need to investigate the extent to which each company uses and is interested in OSS. The survey period will be from October to November 2022. This was one year ago. The survey targets from pharmaceutical companies registered with the Data Science Expert Committee in JPMG and includes both domestic and foreign companies. As a result of the survey, we asked 64 companies and received a response from 55 companies. All answers are anonymous. Therefore, we have not been able to identify companies that did not respond and the reasons for non-response are known. From this slide, let's take a look at the answer to each question. The questions are roughly divided into 15 questions. We also have 50 slides available. The first question is, have you initiated or are you considering using OSS for clinical trial related activities? Yes, already used are selected by 30 out of 55 companies. More than half of all companies are already using are. There are also six companies that are currently considering it. And four companies that will be considered it in the future. Additionally, 15 companies have not considered anything yet. Why about 30% of companies that they are not considering using it? The results show a difference in a huge amount of companies that are actively implementing are those that are not. Next question, which steps do you use are multiple choice alone? In previous slide, the 30 companies have already used are. It's often used for sample size estimation, exploratory analysis, modern analysis, and preparation of documents for internal use. There was little use experience of are for preparation and QCIP or clinical study data set and TLX. And preparation of either related materials. It's thought that are is starting being introduced in the process of creating materials that are not directly submitted as regularly submission documents. We have to do companies that chose the last two options and internal documents creation and operational efficiency for more detail. Some companies are using are for project management tool and online manual using are sharing. And there was also the creation of reports in HTML format and presentation material are mapped out. And six companies selected as many respondents as that they use it for population analysis in critical form policy. Next question is, have you experienced or plan to submit documents written in are to regulatory authorities? About half of the companies have submitted documents written in are to the regulatory authorities. On the other hand, about 40% of the companies do not plan to submit in are is already used by many companies. But it's often used for purpose other than preparing preparing documents to be submitted to regulatory authorities. Next question is, have you submitted experiences or plan to submit the program in are to regulatory authorities? Over half of the companies have no experience of submitting the program in are to the regulatory authorities or are not plan to submit in. On the other hand, about 30% of companies have submitted such programs before. Some companies are used are for purpose other than preparing documents to be submitted to regulatory authorities. It's also possible that are is used for documents for which the regulatory authorities does not require program. However, because are is often used in clinical pharmacology studies, it was not possible to confirm the details of whether what was submitted to regulatory authorities was clinical pharmacology study or something else for analysis of clinical trial through this survey. From this slide, I will pass it over to you. Okay. Okay. And this is UK and I will explain from this slide. And next question is what environment use what environment you use are. And this is multiple choice question. Many companies have already started using are in their environment. Over 80% companies answered personally on computers. And also over 30% companies answered shared infrastructure using internal servers. 20% companies answered shared infrastructure with is and no companies use sauce. Next slide please. Okay. Next question is who manage the environment for using are 40% companies answer that it department. Manage the environment for using are on the other hand, in similar proportions about 40% companies answer, they manage and are they manage our environment by individual responsibility. And some companies answered they manage are each department. And next question is, do you manage our libraries 40% company answered. Yes. They manage the library. And about 60% answered no. They do know. In the following questions, many companies mentioned that reliability for libraries as a challenge. Such things that many companies are still considering handling libraries. And next question is how to train and learn are in your company. And this is multiple choice question. 60% company answered self training. It means that they learn are by themselves. And also about 30% answered no training. And they have little experience of receiving training. 25% receiving internal in house training and 10% receiving vendors training. And next question is, have you ever outsourced analysis using all. About 90% companies answered that they have no experience of outsourcing using are. In the response to the following questions. There is an opinion that there are a few CROs can do an analysis using are. So at least in Japan. There may be few opportunity to outsource our analysis. And next question is, what are the expected efforts on introducing are into your company. This is multiple choice question. About 70% companies answered that they expected for modern analysis. And 50% companies answered that they expected for cost reduction. And 30% company answered easy access to information due to the large number of users. On the other hand, few companies about 10% answered that they expected transparency of algorithm. Which is characteristics of open source software. And next question is, what are the concern about are when useful and the a. This is multiple choice question. Regulatory acceptability and library reliability are over 70%. Reliability of the library is the biggest concern because everyone can release a packages. In addition, although there is no message from regulatory authorities such as PMDA to allow specific software. There is there is a concern about the accessibility of are by regulatory authorities, because there is limited experience and information on the use of are. Also, about 60% answered internal environment is their concern. And 40% answered to internal engineers. It means that environment for the use of are is still be still being established for pharmaceutical companies in Japan. And next question is, what are the concern when are is used not only for MBA but also for clinical study related activities. This is multiple choice question. About 60% companies answered rivalry reliability and about 50% companies answered few internal engineers. One third answered internal environment and regulatory acceptability. These answers also shows that many companies have concern about the reliability of rivalry and human resources. And next question is, which department in your company is actually using are. We do not count in detail, but clinical department, the data science, statistical statistical analysis and pharmacology department mainly use are, but other departments also use it. Since the questionnaire was conducted by the data science expert committee of the JPM a. Many responses may come from the clinical department related clinical development related department and division. There is activity that are is already used across the company. And next question is, does your company have a department which is specialized for program. About 40% answers that they have department specialized in programming. Next question is, what do you expect from the JPM task force in the future regarding the use of open source software such as our and Python. We categorize answer into six categories. Use for regulatory submission and the a and case level. OSS reliability difference from sauce and stop up support. And there are particularly high expectation regarding the use of OSS in regulatory submission. Okay, I am now approaching the end of my presentation. In conclusion, I'd like to summarize what we have set during today's pro today's presentation. OSS especially are is already widely used in many Japanese pharmaceutical companies. On the other hand, many concern remain, such as the reliability of packages packages and libraries and limited experience for regular the authority access acceptance. To continue its activity. To ensure that OSS is widely used in the Japanese pharmaceutical companies. Awesome. Any questions you may have. Thank you. Thank you so much Vicky and UK this is really really interesting to see the survey results from the JPM a survey questionnaire. So I have a couple of pre collected questions. So I guess for the Q&A session, I will try to go through all those questions. And for people who have additional questions, please feel free to share that in the chat box and then we will go to that like after the after the event. So my first question is that it is really great to see the wide adoption of our in the Japan pharma industry. And it's really interesting to see like you can you shared a number of use cases over there. I wonder like from your perspective, can you share some specific use cases that you see that the usage of open source language bring a lot of value compared to the traditional ways of doing things. I answered it. The survey results show that all is mainly used for some side estimation, exploratory analysis, model analysis and preparation of documents for internet use. In addition, examples of actual use case in the pharmaceutical and Japan pharmaceutical industry are compiled by the task force this year. And we will show it around this spring. We are still drafting, but for example, constructing table in RTF file to data frame and data visualization by using our training. For example, for sample size information and characteristics comparison of study design and other will be introduced. Well, thank you for sharing. Yeah. And then maybe a follow up question on that one you share a number of very good use cases. Can you share some of the commonly used packages like in your company, such as like packages used for simple size estimation or some or packages used for tlf generation, etc. In this survey, we have not collected information about packages. I'm sorry, I cannot talk about our individual company case study today. But I'm not sure about commonly used packages for sample size calculation, but the tlf generation packages developed by some of us have been introduced in the Japanese community committee and many people interested in tlf. Also, this is just my personal impression. But from the perspective of creating a nice data, I believe it's a player, stringer and lovely date are open use. R is useful and powerful, not only for calculation and data creation, but also for visualization packages such as our training. Yeah. Thank you. I think shiny is definitely very exciting to be adopted by the industry. Yeah. Yeah, and then like another thing I saw in the survey which is very interesting is that you show that like about half of the companies have already submitted materials generated from our two regulatory authorities. I'm curious that do you know whether those are submitted to PMDA or FDA? Unfortunately, such information was not collected in this survey. I think the target of this survey is not divided into Japanese and foreign capital companies. I think that's the number 50% includes both PMDA and FDA submission. We have the opportunity to conduct the survey again in the future. We have to consider this question. Yeah, that would be interesting to learn how many people submitted our generating material to PMDA versus to FDA. Yeah. Yeah. In your presentation, I saw that like you mentioned R and Python specifically, and I'm curious like in Japan, like what is the most commonly used programming language from universities among maybe like master and PhD students. Okay, I answer these questions and I was a PhD student in medical school and in my case, OSS such as R and Python are popular and familiar. The common programming language depends on their department and maybe timing, but I think the younger generation has highly affinity for OSS. And one recent survey shows that Python, Java and C++ are commonly used and run programming language by Japanese university students. In addition, I have an impression that many Japanese pharmaceutical companies programmer do not necessarily have a background of programming in university. And they sometimes run and receive training of joining the company. For example, actually, I learned SAS recently three years. Wow. Thank you. Sorry. I think I lost my question this over here. Give me one second. Yeah. Sorry about that. Yeah, so I think it's very interesting. I think it's pretty similar to what we saw in like universities from other regions. Yeah, I feel like probably people are following similar like a trend in terms of open source software over there. Yeah. I see a follow up question on that. I feel like nowadays everybody's talking about AI and large language model, etc. And then there are a lot of excitement about the open source AI models such as GPT and Lama to etc. So I guess that in Japan, pharma industry, whether this is also a hot topic and whether people start to exploring LLM and what are some potential use cases people may start looking into. The use of large language models on LLMs is often restricted in order to prevent the leakage of privacy and company information. But I believe that we are exploring the potential of LLMs actively. And also, since the current LLMs are developed based on the training data in English, I have that Japanese companies and government aim to develop their original LLMs. In addition, I have that some Japanese company have already started to use LLMs and it is used for creating documents such as a meeting minutes and catching journal article information or collecting real-world data, creating and combating and fixing program calls. I think that's very interesting to learn and also interesting to hear about the concern about the training data has been in English. But the use cases I think are pretty common across different countries. I feel like from our side we are also exploring ways to use LLM to generate code or generate a document, etc. You'll be very interesting to see how far it gets us to. My last question is that I think it's really interesting to see in the survey like you classified like the top areas that the Japan pharma industry want the GPMA task force to look into. And I think those are with very good overlap with some of the global effort, such as like how to make NDA more smoothly, such as package reliability, etc. And then I know that like globally there are a couple of cross industry working groups looking about looking at, for example, software validation recommendation. There is our validation hub under our consortium and like like our team like the our consortium submission working group is looking at the feasibility of using open source software for regulatory submission. And then there are also a lot of cross industry like collaborations on building pharma specific packages such as what you mentioned about mural, etc. Yeah, and I want to ask you that moving forward, how can we collaborate more closely globally, maybe make better connections between GPMA and also some of other nonprofit organization efforts so that we can learn from each other. Thank you. First of all, I believe that it is important to share information with each other. And discussion on the utilization of R has been very active in Japan recent years, but it is really, but it is still limited. And in the regularity submission, there is an opinion that what only R can do is limited and traditional stuff is sufficient. I think it is important for many people to know that unique and flexible function of CR. Actually, our OSS task force has just started in 2022, and we are gathering information. I want to know the global activities related to R in pharmaceutical companies. And besides, I want to I want you all to know about activities in Japan. And there are limited opportunity to discuss R in Japan community. Also in Japanese pharmaceutical pharmaceutical companies. I hope to continue collaborating and discussing with global community such as such as our consortium. Thank you. Yes, that is awesome. I know like Joe Ricker who is the executive director of our consortium is also in the car today. So maybe we can follow up to schedule a another meeting to talk about future collaboration. Okay, thank you. Awesome. Yeah, so I think those are all the questions from my side. And I think this is really, really interesting to see the recent effort from GPMA and also to learn about the survey result. Thank you very much. I think I will close today's session. And thanks everybody for joining. And if you have any questions, please feel free to to share up with us offline and we will try to follow up with a blog post after this event. Thanks everyone. Thanks so much for taking time and joining today.