 Good morning, everyone. So welcome to our last session of this year, CBNDC web minister series. So today we are happy to have Hubertel Ennis, the group by President of the Western Bay to moderate this session. So Hubertel is all yours. And by the way, so we are having a little bit polls during the web minister series. So you can see the chat that we've been learning to do a little bit survey. Thank you, Hubertel is all yours. Sorry, I was trying to unmute. Okay, so welcome, everybody. We have Wakes young today. He's gonna talk about privacy and I guess sort of digital in the digital world. And they were all very familiar with some of these things and I think he's gonna be illuminating us in sort of how to think about these things. He'll have 25 minutes, then there'll be a discussion for 10 minutes, it's another discussion and then we'll have 20 minutes of Q and A. So the panelists can unmute themselves and ask and make comments. They welcome and then other attendees who have to enter questions and comments in the Q and A area. And I'll try to sort of convey those to the speaker and others in the panel. The conference I'm told to remind you that it's being recorded and the video will be posted in the workshop website. So with that, I'll let Wei Gong go. Thanks, it's my great pleasure here to give this seminar. So the paper is Data Privacy Paradox and Digital Demand. It's joined the work with Long Chen and the Yadong Huang at Lohan Academy and Sumiao Yang, a student at Princeton. The motivation is very simple. Data sharing is important. It empowers the boomy fintech industry and the digital economy. I think pretty much everybody in the public already experienced it in some extent. And for some macroeconomists, they even recognize data as a third factor of the macroeconomy at some point, right? Because data sharing has some very nice properties such as non-library and increasing returns to scale. At the same time, the use of data requires data sharing and the data sharing can lead to privacy concerns. And these concerns are becoming more and more important. In fact, the European Union enacted the so-called GDPR back in 2018 and in US, California, as well as several other states had put up similar revelations as well. All of these revelations trying to provide some protection to consumers by giving them the options to opt in or opt out of data sharing with digital platforms. However, despite these massive data privacy revelations, we know actually quite a little about how consumers actually balance their privacy concern and data sharing choices, right? By giving them choice to, you know, choose whether you want to share or not, but do they actually use the choice and how do they use the choices that might have? So these are still open questions. In fact, if you want any answer, the first thing come to mind from the literature is probably the so-called data privacy parallels, right? In fact, there have been extensive studies in the marketing as well as information system literature about how people protect their privacy, right? In fact, there has been a survey by Christy Bredner and the Lawyer's Team showing and collecting a lot for the work, arguing that the consumers often state concerns about data privacy in surveys, yet they share their personal data freely or for various small rewards, such as free pizza, right? So to the extent that most of these studies are down in laboratories and among college students, so there's also legitimate questions. Does this paradox exist in some realistic situations such as data sharing with the digital platforms? If so, and why? So these are the questions we were trying to address in this study. Very quick, let me give you a highlight. So we examine this paradox by combining survey and behavior data of a sample of Alipay users. We confirm the data privacy paradox. Indeed, users with stronger privacy concerns authorize data sharing with roughly the same number of programs in Alipay just as those who say they are not concerned, okay? Then what expands the data privacy paradox? This is not simply due to unreliable survey responses or frustration with protecting one's privacy. In fact, we show that the users with stronger privacy concerns use many programs more intensively and heavy users of many programs also are more likely to cancel data sharing with many programs. So from both sides, there's this policy correlation between privacy concerns and the use of digital applications. So putting this together, we highlight that privacy concerns are likely developed as a byproduct in the process of using digital applications. So this is a highlight. Let me quickly mention the related literature. As I mentioned already, there's an extensive literature on the data privacy paradox. This literature tend to attribute the existence of this paradox to various personal behavioral biases such as ignorance about data sharing, illusion of control, present bias and so on. So here in our study, we will highlight arguably more fundamental issue relating data sharing behavior or privacy concerns to fundamental demand for digital services. And the literature also has highlighted the various reasons for consumers to care about the data privacy, such as price discrimination, data security concerns and the protection of one's personal weakness or vulnerabilities. We are not gonna get into some specific reason for concerns but rather focusing on given the concerns, how concerns are related to observe the data sharing choices. All right, so all studies based on RDP platform, some of you probably familiar, and this is widely used digital payment system in China, but it's more than just a payment system. In fact, actually, he also contains millions of many programs operated by third parties, each requiring authorization of data sharing as an initial entry. So here on the picture, this is a front page of the RDP mobile payment APT on one's cell phone. So the top portion here, this is the payment portion. So if one wants to make a mobile payment, this is this portion. And the middle here, this is the portion for the so-called mini programs. In fact, it holds millions of them. So these are ranked based on the use of the particular user. So some of them are very popular and widely used. So here I give you three examples of the new programs. Whenever you enter a mini program initially, there would be an authorization page opens up, asks you for data authorization before you do anything with it. So the first one here is a part-time job searching a mini program, at the end you ask for your mobile phone, mobile number. The second one here is an application for a friend making. You ask for nickname, profile, gender, and area. And the third one, this is an application for a law consulting lawyer, basically finding a lawyer. You will ask for current location and so on. So basically all study focus on the willingness of different data pay users to authorize data sharing with these mini programs. In July, 2020, we conducted this survey. The survey was sent through the middle on the front page, there's a message box. The link to the survey was sent through the message box to over two million AliPay active users randomly chosen. About 27,000 of them opened the link and the event you're about to have completed the survey and the amount of these people, about 10,000 users say they use mini programs in AliPay. So this becomes eventually our survey sample. The survey is very simple, contains 12 questions about their preferences and concerns about data sharing. Typically the survey is completed in one to three minutes and the respondents are well spread out across China. Let me here quickly summarize some responses to some of the key survey questions. The first, can I interrupt you once instant? It's a clarifying question. In two slides before you were at some point, they'll ask you about the authorization to share? Data sharing organization, yes. It's usually whenever your first time open a mini program. So that you enter your mobile phone, something for the first time, you were like a pop-up window. Yeah, this is a pop-up. If you authorize it, then you can enter, get in and use it. And if you don't authorize, you cannot enter it. You can authorize it, then out, right? So basically... You can't use the app? No, you cannot. Okay, thank you. So basically it's a trade-off, right? You won't use it now. Yeah, got it, got it. Thank you. You have to give it data, basically. Yep. Okay, it's very simple. All right, so the survey question first, we asked them, are you concerned about the privacy issues while using online services? This is a general question. And not surprisingly, 93% say very concerned, 6% concerned, only 1% not concerned. But this is consistent with any survey we have seen. Many of the survey, when you ask them this question, therefore I would ask the majority say they're concerned, right? Okay, the second question, what do you think about the privacy protection audit date? This is specifically about audit date. 48% say very good, 39% are ordinary, and only, you know, 13% say not good or have no idea. And the third question here is the most relevant one for our study. Are you concerned about the necktie impacts caused by information shared to many programs in the audit date? Right? So 46% are very concerned, 39% are concerned, 15% are not concerned. So the ratio of not concerned is not bigger, okay? So based on the answer to these three categories, so this becomes sort of our comparison group. We're going to compare users in each of these three groups who said concerned, very concerned or not concerned, how they share their data, okay? We also asked them the question, what privacy issues are you particularly concerned about when using mini program to audit pay? So this is multiple choice, multiple choice question. So 86% of them click the data leakage and security, 21% click the price discrimination by merchants, 49% are mentioned as seductive advertising and temptation consumption, only 5% click others, okay? So what's nice about our analysis is that we not only have the survey responses, but also we have the background. And in particularly we have the usage of these mini programs. We know a number of mini programs authorized, a number of initial mini program they entered. And we also know after authorization whether a user has canceled the previous authorization or how many of us has been canceled. And also we know for each of the authorized mini program how extensive the number of active days they've been on the program, how many times they use it and how many pages to visit it and so on. So this becomes later, we're gonna use to collaborate with their survey responses and study how their privacy concerns are related to the use of these digital patients. To start our analysis, let's look at simple framework. Consider user is data sharing choice with mini program J. So this is a pair, right? So this is a cost side of the data sharing. The cost can decompose into an individual component, CI. You can think of this as privacy concern, right? A user may be particularly concerned about sharing the data. And then the cost is high. Plus CJ, this is a mini program component. So the particular mini program may ask a lot of sensitive information, then the cost can be high, okay? There was also some a wide noise. Okay, so the benefit side, the BIJ, this user might be particularly like to use digital applications. So the BI can be high or this can be a very powerful mini program or for very useful services. So BJ component can be high and plus some noise. So for the user, whether to authorize data sharing or not basically is the difference in model two, like BIJ minus CIJ. So you can see that if you control everything else for a user who is particularly concerned about the privacy. So we would expect this user to be more reluctant to share data, right? To authorize data sharing. Basically, this is our hypothesis one, basically. Other things being equal, privacy-concerned users are more reluctant to authorize data sharing. I think that this is sort of usually the starting point when we think about this data privacy paradox and this is usually what's understand taken by every study in this literature. So let's look at this. Now let's compare. Based on the survey response to this question, right? Are you concerned about the impacts because of information shared mini program in Alipay? This is a question specifically asked there concerned about sharing data with mini programs, right? So I mentioned that there are three groups, right? 15% of the respondents say not concerned, 39% concerned and the 46% very concerned. That caused the three groups. So let's look at how many data authorizations, data sharing authorizations they have agreed to. So they're very concerned group. On average, each user in this group enter 16.3 mini programs, authorize 11.6. This is for a one-year period before the survey, okay? And for the second group, the concerned group, on average, each user enter 15.5 mini programs and authorize 11.5. And the third group, not concerned group, enter 14.3 mini programs and eventually authorize 11.2. So from this, you can see, well, if you look at the number of authorizations, 11.3, 11.5, they're almost the same, right? There's no difference, okay? It's kind of interesting, right? This is exactly sort of a nicely collaborate to the data privacy paradox, right? Of course, these groups with different degrees of privacy concerns, they end up authorizing on average the same number of data sharing, right? So we don't see the very concerned group share less. But on the other hand, you do see, actually, they rejected that about the 25% of the data authorization initial request, right? So in the sense, this is not sort of a situation, you see, they just enter somewhere and blindly agree to everything, right? So in that sense, they didn't give up their choice. They did actually look at this group, right? So they actually all rejected some fraction, right? So this is not blindly agreeing to all data authorization requests. In that sense, they didn't give up. So you do see the sense of all active choices, but nevertheless, they end up agreeing to the same number roughly, okay? So we can look at the sort of, you know, of course, these new programs are very different and the individuals are different, the sort of, you know, we should put in some controls, right? We can look at the user level, requesting the number of data sharing authorizations with respect to the survey response. So the concern dummy, the very concern dummy as well as their, you know, backgrounds, that digital experience, their age, we can also put into city fix effect as well as gender fix effect, but after all this, it doesn't matter. There's no difference, okay? We can do even nicer. We can look at the user mini program level basically for all the possible pair of user and mini program, there are many pairs, potential pairs, right? So, whether sort of any given pair has been authorized, right? So, and the request this onto the controls. And in particular, at this level, we can put into mini program a fix effect. That means we can compare even a mini program. That means given the kind of information requested, what kind of users are more likely to agree to, to the request, right? Again, there's no difference, right? So, basically, concerned ones, very concerned ones and non-concerned users basically there's no difference, okay? So this is sort of basically the paradox. Why is this the case? I guess sort of immediate question may ask is maybe this just means survey responses are just garbage, right? Because, you know, who cares? You know, answering the surveys, right? Well, it turns out that actually we could validate these survey responses. Individual survey can be noisy, but on average, they're actually reasonable because we can look at the actually certain actions the user take to protect their privacy. Like whether they cancel previous data sharing authorization or whether a user change the default privacy setting on AliPay platform. So, AliPay is not just a payment system. It also has certain social media functions. So, basically there, each user's information can be shared to other users. And there are some defaults. Usually they're very open defaults make each user's personal information transparent to other users, right? So, but each user can also change the privacy, the default setting. And some of them did change. And we see that those who say in the survey are concerned or very concerned about their privacy. You do see they more likely take these active choices to protect their privacy. So that suggests other survey responses are not just garbage, okay? So they're actually certain value behind. But then we go back. Then why do we end up seeing very concerned and concerned and non-concerned user roughly ending up with the same number of data authorizations? So in the survey, we also asked them a bunch of questions whether they agree. I agree to authorize data sharing with meaningful because it's safe in AliPay. This actually has some support. But on the other hand, very concerned with to me seem to agree more to this, right? So this cannot explain the paradox. The second, I agree to authorize data sharing with meaningful because my information is already been shared in many platforms. This clearly do not have much support out there. The third question, I have to share my information in exchange for digital services. And even though I have concerns about my privacy. So this one actually has a fair amount of support and actually are more support and non-concerned and very concerned group than the unconcerned group. So this actually is the one when I actually pointed the right direction is the trade-off, right? And this sort of seems to suggest the concerned one actually somehow agree to the fact that maybe they have some kind of stronger need for these services or something, right? And the fourth question, I authorize data sharing with many programs because the request data are not important, but clearly this one doesn't get much support, okay? Finally, I tend to authorize data sharing with new program that are used by my friends. We know social media is powerful, right? So this social networking effect. But here, you don't see it working to expand the paradox either. All right, so the survey response there point to some kind of trade-off. If we go back to the simple framework I mentioned earlier, right? So it could work. If somehow the privacy concern they use somehow they also have a benefit more from using this digital applications then that may help, right? So if the concern and benefit offset, then that could help. So but can this be right? Usually sort of we would imagine privacy concerns should the deter users from extensive use of digital applications, right? Which typically ask for data sharing, right? So to adjust this issue, let's look at this hypothesis. Other things being equal, privacy concern users use mini-programs less intensively. So this sort of the common sense wisdom, common wisdom, right? So let's look at this. To the extent we actually observe how intensively these users use the authorized mini-program we can just simply look whether the concerned user use more or less, right? And we can come up with three or four different measures. Number of active days, number of APB users, number of launches and number of visit page. And of course all of these are measures. You can see that the concerned users use more than unconcerned and the very concerned users even more. So basically this nice monotonic relationship between the concerns and the intensity of using these mini-programs. So then this is sort of a point to the direction. Somehow there might be a positive correlation between privacy concerns and the demands for digital applications. So this positive correlation is counterintuitive. Why don't privacy concerns deter digital demands, right? So in the end that we come up with this argument, well maybe, you know, because of the whole digital economy is new, right? Users are probably gradually, you know, learning about the demand for these applications as well as understanding their own privacy concerns, right? So in fact, if we plot the survey response privacy concerns with respect to digital experience, this is a very clear monotonic relationship. The longer using the RDP, basically the digital experience here is measured by using RDP, their privacy concerns should be greater as well, right? So then this motivate, but still we, can we be sure the heavy users of digital applications actually concern more about privacy? So to be sure, we look at the third hypothesis, other things being equal, heavy users of mini-programs are more likely to cancel data sharing within mini-programs. So this is a hypothesis that should be taken for granted because heavy user means they benefit more. For those who benefit more, more like to cancel data sharing authorization, get themselves kicked out of these mini-programs. This is not something to be taken for granted, right? Okay, but now let's take a look. We can look at this at both user level as well as user mini-program peer level. Interestingly, we again find this indeed a case if we look at the user level, each user whether a user has canceled, because it canceled data sharing authorization mini-program, we can use different measure of activities on these mini-programs, no matter what we do. So there's a pretty clear significant positive relationship here. We can also look at the user mini-program level, again, whatever we do, this nice positive relationship. I should particularly point out cancellation of something requires some knowledge, right? Maybe one can argue heavy users or active users, they have better knowledge about the RPA application and thus they know how to cancel. Most people actually may not know, right? So this is a natural concern. We can also look at the subsample of the users who actually had previously canceled data authorization before and then look at the sublater period, whether this is the case. Again, we find the same thing. So in a sense, this is not just because of the knowledge about knowing to cancel, but rather this is indeed about concern, okay? So- Wait, three more minutes. Oh, three minutes, I should hurry up, okay? So I should quickly mention, so far the result I presented to you are based on the service sample. You might argue the service sample is biased toward the more active user because more active users are more likely to open up, to notice this survey link and open it up, right? So to adjust this issue, we also look at the random sample of 100,000 users from all active Alipay users. So this sample is larger and more general. So this allows us to robust check. And also we can look at another event study based on user's reaction to a privacy-related instance, okay? So I do the time when we go into the summer statistic, indeed the this random sample users in the sample use Alipay mini-program less often and they authorize less mini-program and so on. They all go the right way, okay? And let me quickly mention this event study. On January 3rd, 2018, Alipay launched a report. And in the report, there's a hidden sort of agreement to allow authorize Alipay's Sarsby credit service agreement. So that means user who failed to notice this checkbox, we end up unintentionally agreed to Alipay's Sarsby credit service. And interestingly, some internet user quickly noticed this misleading design and this incident went viral on Chinese social media and actually this caused a spike, this large spike in the cancellation of the mini-programs on Alipay. Basically this reminded the many users that you need to care about your privacy. And this indeed, so this is what happened. Of course, afterward, Alipay needed to be apologized and canceled everything, but still this happened. So we could use this event study to again highlight, after the incident, who are more likely to cancel data sharing, a heavy user or like the user, right? Not surprisingly, after I show you that you knew the unconditional result from before. So the heavy users highlighted by the red line after the events, their capacity to cancel data sharing authorizations in the program went much higher, larger than a lot of users. And this amount unfiltered the users in the random sample. We can also look at the users who had at least canceled some data authorization before the incidents. We still see the same kind of difference, even though a study more modest now, right? Okay, indicating knowledge still is relevant whether you know how to cancel. All right, so due to the time, I won't go through the robustness. We can use the random sample and we do everything we did using the service sample. The pattern now bears some of that. Okay, due to the time, let me just quickly sum up. We confirm the data privacy paradox. Users with strong privacy concerns authorize data sharing with roughly the same number of many programs on Alipay. Well, it says that data privacy paradox, this is not due to unreliable service responses or frustration with protecting privacy. Rather, we find users with stronger privacy concerns also use mini-program more intensively and happy users of mini-program also more likely to cancel data sharing with mini-programs, both unconditionally and in response to a privacy-related instance. So putting all this together, we believe privacy concerns are likely developed as a product in the process of using digital applications. So that means privacy concerns may intensify with the deepening of the digital economy and thus eventually limiting the increase in returns of data sharing. Thus, it's really important to better protect our data privacy and thus enable more data sharing. Eventually, this is the engine not going to drive the digital economy. Okay, thank you. Great, thank you, Wei. Thank you for being in time and so, I think now it's time for the, yes, discussion or yours. Thank you very much, Roberto and thanks Wei for the interesting presentation and thanks for having me here discussing this really nice paper. So let's see, make sure I can go through, there we go. This Alipay app shows up on different app stores. This is taken from the Apple app store stating that Alipay is a super app designed to offer a bouquet of services trusted by over a billion users, has offerings that span payments, manage finances, choose suitable insurance schemes, hail a cab, we even order in from a favorite restaurant. The Google Play store is not available in China. So this is from the Angie market app store an Android app store has a similar translated description stating Alipay is a fusion of payment, wealth management, life services, government services, insurance, public welfare and so on. In addition to providing convenient basic functions such as payment, transferring collection, it can also quickly complete credit card payment, charge phone bills, pay water, electricity and coal. It can reach hundreds of people in one step through intelligent voice robots, not only discounts on consumption but also manage money easily, accumulate credit and make life easier. Interesting thing from the screenshots for the Android version also shows that there are games, social games perhaps within Alipay, within these many programs. This is relevant because if we look at the categories of many programs and we try to draw parallels to the broader app ecosystem and say iOS or Android, we know that in those ecosystems, games tend to be the most engaging apps. So it's kind of curious whether the same applies to Alipay. Some overview of the setting, Alipay has over two million mini programs that are used by around half or more of the population. Data permissions include gender, phone number, national ID number, credit score and so on. And you cannot use a mini program without granting its requested permission. At least you cannot fully use it. Authorizations are for a finite time, then they need to be reauthorized. They can be withdrawn at any time. It's not clear whether a lack of reauthorization counts the same as a withdrawal. I think that needs to be clarified. Alipay, the platform itself has privacy settings and the default of the platform is low privacy, meaning that users posts, at least some of them are more visible. The users themselves are more easily searchable by others, et cetera. The privacy paradox in this setting is that the survey respondents self-stated privacy concerns are not associated with a lower number of privacy invasive actions, meaning data sharing authorizations with mini programs. And so this is potentially susceptible to the so-called sole of critique that the behavior examined in privacy paradox studies involves people making decisions about risk in a very specific context while their self-reported privacy concerns are much more general in nature. Now this paper addresses that concern by specifically asking about data sharing with mini programs in the survey and the users in the survey then matched with their actual actions regarding the management of access to their data. So that's a really promising setup, really an amazing data set. The surveys conducted in July, 2020 has these 11,000 or so respondents that use mini programs. The survey places respondents into these discrete buckets that's somewhat crude, admittedly. Very concerned, concerned and unconcerned. There is an alternative sample of users that are drawn randomly where their privacy concerns are based on prior AliPay privacy choices. And this kind of random sample allows for more perhaps continuous measure based on the extent to which data is shared with AliPay. This data is then coupled with some time and variant information about the users. Their city, their age, their gender, their experience in AliPay and the user's actual actions over a year or so period of time on AliPay that includes visits to mini programs, data authorizations, revocation or cancellation or withdrawal of authorizations and their engagement measures on these mini programs. The findings, main findings basically that there is a positive correlation between users' privacy concerns and their use of mini programs. This continues to hold when switching to the alternative random sample of users and the authors assert that consumers' privacy concerns may grow with their data accumulated with digital service providers. At the same time, they cannot rule out the so-called present bias where users may overweight present benefits relative to future privacy costs. This is particularly an issue because of this temporal discrepancy. The survey takes place in July, 2020. The data regarding users actions are over a year period of time. In that sense, this present bias can accumulate over time as these users grant more and more authorizations or permissions to mini programs. And at some point, the cost, the future cost can become more salient to these consumers, to these users. And so overall, it's unclear what role this temporal discrepancy between the users actions and when the survey takes place, what role it has in this analysis. So I'm gonna split my comments into four buckets. One is the overall Alipay setting. Another is the model mechanism. Another is some concerns regarding fragmentation. These could be perhaps easily addressed, but I'm not sure if they are. And some potential extensions that seem to be readily possible in the setting and some implications. So this is, again, regarding the overall setting. It's unclear what the outside options to using Alipay and mini programs. And I'll dig more into that in a moment. It's unclear whether all authorizations to mini programs are for the same time period. It could be that more active users bump into more reauthorization requests, for instance, because they use certain mini programs. I know there's some analysis of the user mini program level that might control for some of this, but I'm not sure it fully addresses all of these concerns, so I'll get back to that moment. And as I mentioned before, it's not clear whether a lack of reauthorization is counted the same as a withdrawal or revocation. Can mini programs collect Alipay information in the background or is it just a one-time collection? I mean, clearly they're collecting the actual data from using the mini program, but what about other data? Does revoking authorization delete data or is a stop to future collection? What information do users have about these controls? We kind of need to know that in order to make the linkage to privacy more consistent here. What are the potential privacy harms? I know the authors try to kind of abstract from that, but is it malicious third party that might benefit from leakage, which users seems to care about, or is it the user's fear government access? Which information a user's most concerned about leaking and is that something that mini programs can adjust to? I mean, mini programs are sort of taken as static here, but they might adjust to users' lack of authorization over time and adjust their permissions. I think the authors say that most of the permissions stick to the functionality of the mini programs, but the mini programs can also adjust their functionality over time. What is the distribution of mini programs in terms of their data intensity, say, for active versus less active users, for privacy conscious versus less privacy conscious users? It's unlikely that users use the same distribution of apps. Those can be finance apps, they can be health apps, social apps or gaming apps. The gaming is a dominant engagement-wise in the iOS and Android app store. Does that map to Alipay? It's not clear to me that the user mini app analysis level controls for this completely, especially since these distributions can change over time. What does the Alipay mini program store look like? Do they compete on privacy? Do these mini programs compete on privacy? Are our permissions highlighted? Are they sorted in a certain way by category of function, or is it all search-based? Can they use third-party SDKs and APIs? And does Alipay place any constraints? Do users have any information on that? Can users shop around for a mini program that requires fewer permission within a category? And does Alipay remove any mini programs over time and based on what? Do mini programs' permissions change over time? Do they require authorization when they change those permissions? If they add a permission or if they remove a permission and do users react to that? So the positive correlation kind of switched into the mechanism between user's level of activity and actions restricting data sharing suggests a number of potential mechanisms on the cost side, kind of ignoring the present bias, potential orthogonal mechanism that might be happening as well. And these could include increasing the level of salience for users or sophistication or understanding regarding data flows or data management. It could be concerns about the greater breadth of data shared, for example, more sensitive data, maybe adult content, maybe more data intensive mini programs. It could be concerns about greater accumulation or depth of data shared. It's not clear which mechanism is at play and it could be a combination of them. It's not clear to what extent each one is factoring in here. It also suggests a potential learning by doing model if sophistication is the key here. I'll have some comment on that in a moment. And another concern, another layer of concern is what role income might play here. For example, it might interface with potential harms if the concern is about data leakage and potential fraud or credit theft. It could also interface with a device choice and I'll talk more about fragmentation in a moment, but also interface with a mini program choice. So that's something that doesn't seem to be factored in. Now as far as fragmentation... Yeah, three minutes. Okay, as far as fragmentation, it's not clear what role device and operating system can play here. Tablet versus smartphones. It's not clear that fixed effects or time fixed effects would capture users updating their apps or operating system or their devices. It could be that privacy conscious users are more likely to use certain apps or certain phones, certain devices. This is an example from the Android and iOS screenshots. And this is presumably for the same screenshot. It's a bit hard for me to tell given the language difference, but we can see that it looks quite different and it could be that the interface for revoking authorization looks different as well. What role does that play across operating systems and within an operating system given that there is fragmentation there as well? And we know that iOS and Android update their privacy requirements and privacy interfaces over time. It could be that Alipay does the same. So continuing with fragmentation, it could be that new devices reset permissions. It's not clear. Is it possible to control for app version, operating system version, no mini program version and where the fragmentation is even an issue. It's just not clear here. So this is kind of the final bucket, potential extensions and implications. There is potential for additional analysis here on the panel data. For example, if the first time a user cancels or revokes an authorization or changes an Alipay privacy setting is taken as a treatment, these users could be matched to users who have not done that and that could get at the effect of sophistication on subsequent user behavior. If device data is available, it could look at switchers from Android to iOS or vice versa. If iOS has a better reputation for privacy, for instance. We explore other ways to use panel data. I think there are quite a few opportunities here. And kind of the broader question as far as implications is how do we map privacy concerns within an app, albeit a super app, to privacy concerns within a broader operating system or a device ecosystem? So some other potential extension here that seemed kind of obvious to me is the platform's perspective. And I think this is crucial. What about Alipay here? Some back of the envelope calculations could be what if all users are high concern and have previous experience with cancellation or revocation? What if all users are low concern and have no prior experience with cancellation? How do we compare these extremes to the status quo in terms of engagement on Alipay? It seems to be against the platform's interest here to make the privacy choice or the cancellation choice more salient to users because it might lose an engagement. And so is it even okay to assume that the platform is static over the year period of time from 2019 to 2020 given that it could update its app, it could change its saliency by changing the interface and so on? It seems like the platform would have incentive to interact with that. So I think these are questions that also could be addressed in part or at least could be looked into. And these back of the envelope calculations could shed some light on the platform's incentives. So just to summarize and recap, this is an excellent paper with an amazing dataset. I think my comments can largely be addressed with some more background information, some more controls are available or some more caveats regarding limitations about the data. And there are some really rich opportunities to extend the analysis as far as the model exploiting user or other variations over time, event studies, which the author has recently added and examining platform incentives. Thank you very much. Thank you so much, Lead. That was an amazing discussion. And so I'm thinking that I'm gonna give way sometime to sort of comment on the discussion. Of course, we have the panel, but you've already asked all the possible questions that anybody could possibly think, but so we can see if someone else wants to re-emphasize some of those. But, Weg, do you wanna have some comments about? Thank you, Lead, for the wonderful discussions. They were very thoughtful and provides a lot of suggestions, so very constructive. Certainly, we could spend a lot more time on this. The data is actually really new to us. So ready to understand what's in the data takes a lot of time. And I think we're just getting started with all the data, but certainly we can spend a lot of time. I think as you suggested, this can lead to a lot more understanding about what's behind, right? So I think sort of what you... Let me quickly mention a few things that you mentioned, the easy one, the GES. I think some of these may take us actually many more months to digest. So about the cancellation, the authorization has usually a two-year period that then expires. So this is a voluntary expiration, right? So this is not the countless cancellation in our analysis. We only look at active cancellation before authorization expiration. And you mentioned about this present bias and as well as the other bias. I think that this bias is clearly important for any individual, right? So in making these decisions, it's a decision which you don't really know much about what it has. So clearly, so there are a lot of decisions are made on the instinct, right? So, but at the same time, I think sort of what we find about actually those who end up using more heavily actually end up being more concerned that this relationship couldn't be parallel to all these other potential biases out there. I don't think sort of this can be... It's actually tricky to control for these potential biases, right? But nevertheless, I don't think sort of any of these biases can really extend this fundamental relationship we find either. I think that this is sort of the most important issue. I think I want to clarify. And of course, I will try our best to find a way to address these other biases which are really important. Of course, we are not wrong in sort of controlled experiment that's typical of these other studies. They can actually design a way to tease out these bias, right? We don't. We just, we are using actually administrative data sort of, you know, unless we can convince artifact the wrong sum experiment for us. But that would be a lot more difficult, okay? And in terms of data collection, it's one time or continuous, actually depends on the variable being collected, right? So you see that as for your name, of course that's one time, but only after your location. That is clearly continuous. We never use that. So your new location will be collected by the mini-program, right? So in a sense, I think both can be there. But as you suggested, indeed, this means sort of the kind of data sharing is very rich, right? So some are very intensive and some very intimate, but some others are maybe more innocuous. So in a sense, that's why in our last we're trying to whenever possible put into the mini-program fix effect, which allows to control, you know, given the same mini-program, the same sort of data collection mechanism, right? So how likely a user gonna share the data, or cancel the data sharing later and all that, right? So that's sort of how we're trying to control for it. But we could actually do more systematic work trying to sort out what kind of variable being collected and what kind of data collection is more sensitive. This is a little bit tricky because we don't really know much about, again, right? Consumers, you know, how they rank different variable. This is a challenge by itself, right? Which data is more sensitive, right? Ask for your name, your phone number, or you know, ask for your nickname. Of course, I think ask for your nickname compares. But if I ask for your name, or see your phone number and all these other things, it's very hard to rank them, right? So, you know, in that sense sort of, in doing this exercise, we actually realize a lot of this infrastructure is missing, we lack of some basic understanding about the data. Actually, my co-author, Sumer Ooyang is also here. Sumer, you want to add anything? Okay, so let me just add one thing. So, yeah, firstly, let me thank you for the awesome discussion. And you, one important background information you asked is about like how the operation system like the iPhone or Android phone would change the interpretation of our results. So, since Alipay is a super app and so it's basically all of the interface that you say for different apps can be a little bit different. But for the mini programs that are based on the web services, so no matter what kind of phone you use, it will be just the same. So, that won't affect the interpretation about people using different phones and the choice of the phone would affect their choices of the mini programs. But indeed, I think it's really important to add some discussion on like how we can link the environment of this mini program to the environment of the apps in the app stores. So, I think the settings are quite different because in our case, it's the information that are stored in the Alipay system. So, the platform knows everything but it just the third parties do not know the information. So, you will permit the platform to share some of your information. But for many of the app stores, you call the information. So, you have the right to own the information and you just choose everything. So, that is kind of a little bit different and it does not depend too much on your trust on the platform. So, yeah, that's all I want to add. Thank you. Really, really quickly guys. Can you, maybe you already said this, I apologize, but so the data, how do you get access to the data? Is this something that Alipay share with you? Right. Yeah, I don't really know. We work with two co-authors in the Y Academy, which is a think tank funded by Alibaba Group. So, in a sense, we work with people inside the company. But this is not data that researchers that are listening to this seminar could access. Is that correct? You cannot access. Of course. This data is hard. So, I think assuming I just spend many months in Hanzo just to really to process the data. I see. Any of the panelists have comments, questions? So, can I, maybe I'll, Catherine, you go ahead. I think you're OK. Yeah. Yeah. So, it's a very interesting study and I was wondering, since we are having these CBDC seminars and it's maybe an unfair question, I was wondering what conclusions are you drawing from your results? Is it something that would people drive towards CBDC or is this something? So, often it's said that privacy is something that is a unique selling point of going for CBDC instead of private money. So, what would be your feeling given your results? This is a very important question. Unfortunately, we cannot direct it to us. I guess the answer to your question will depends very much on how the particular CBDC is constructed, right? You know, there are various design out there. We are the central bank actually collect data about the individual making available to others. So, you know, even, you know, the fact that Bitcoin is so popular because of the supposed commitment that not to do anything, right? So, but can, you know, that also reflects so the concern by the public can have on using central bank, right? Can central bank really commit to whatever the, you know, data collection mechanism you put up, right? So, I think for any central bank, you will be tempting to collect the user's data and in fact, actually some of the benefit if I understand correctly required actually collecting consumer data, right? Like using the, you know, digital payment system or digital currency to send the helicopter money to certain people, you know, to implement the individual-based monetary policy would require certainly data collection, right? So, I think this one, I think central bank will have more credibility than any private firm on the privacy issues. But at the same time, I don't think it's perfect, right? So, in that sense, I think whichever sort of privacy concerns should be part of the, you know, the design for CBDC. Yes. Hubert, I'm sorry? Yeah, no, okay. I'll let Shumi out, add to that. You're muted, though. Yeah, yeah. Thank you, Kachun. And so I want to add maybe one other point. So, is that the Alipay is kind of special app. So, it's built on payments. So, in that sense, it's very close to CBDC because they're all kind of digital money. And, but it started as a payment service provider, but then it added a lot of many other layers. So that you see that they use the information, they have a bunch of users, kind of the third party providers come here and build a lot of services to the others. And the trust in Alipay as a platform is like the trust on the central bank. So, and in a lot of the CBDC discussions, we mentioned like the medium of exchange, like the convenience yield or the store of value, like the interest rate. But one important dimension that we want to add here is that the value of the data is crucial here. And now the central bank would have access to a lot of the payment data. And later on, when they build more services, they would have access to even more data. And then the consumer's decisions on this platform will be very crucial. And we need to understand like how they, how the digital demand drives their privacy related decisions and their payment behaviors. So that's all I wanted to add. Thank you. Okay, so it's 11.59. Russell, do you have a question or were you, anybody in the panel has questions or comments we're getting close to time. I mean, we could stay a little longer, but we don't have to. And it's sort of nice to finish on time too. Well, okay, now I has a tape, but sorry, I lost internet for a while. And so this may have already come up, but I wanted to ask way, how should I think about these decisions to cancel data sharing? Should I, I was kind of thinking, is it an individual who's using things more gradually on his or her own decides or is there a big social component? Like someone else said, hey, I'm worried about data sharing. And so do you see clustering of these decisions? And those seem like two very different things, right? So if it's the latter, if there's a social component, then the more someone uses it, the more likely they're to interact with someone else who's saying, hey, I'm worried about privacy, let's cancel. But is there anything in your data that would let you see whether you get these revocations of data sharing being clustered in some way that would indicate it has a social component? I think this is a great question. I certainly believe so that there's a potential role played by social networking, right? So in fact, actually I showed you this reaction to this incidents, right? So when Narikai made this mistake, so you see spikes, right? So in the cancellation, I think part of that probably has something to do with, you know, friends, you know, we chat with each other, and all this weird things happening, we are worried, so. But to do that, we don't have the social network data we need to show that. But I certainly think that this is an important part of it. And of course, in the larger scale of the data, we observe sort of cancellations, the sketchy, right? So, you know, it's still relatively rare. So, you know, individuals cancel, you know, in our survey sample, only 40% of them had cancellation experience, which is sort of already a very active group. And for the random group sample, only about 12% had cancellation. So in that sense, this is still relatively rare. But now there's, you know, this I think will become more common as people become more concerned and also become more used to cans of things, right? And also sort of, I think sort of social networking certainly help people to understand this. I think this probably is the most sort of, you know, people start to understand the privacy better and also understand the fact that you can cancel or cancel this button somewhere. But when you, so if you get concern about privacy, you go in there and you cancel your usage of the app, correct? Yeah, so basically it's that, basically you tell the APP to stop using your data, right? So, I think sort of this- But you can still use the app? No, you cannot. When you start- That's what I'm thinking, that some of these- That you will be kicked out. Right, right, but some of these clustering or whatever could be, you know, this, you know, we're not gonna use that app anymore and you know, my friends are not using it anymore either and we should all cancel it, but not because of data privacy issues, but just because we just, you know- They decide, you know, that their friends decide to go and leave, right? Well, you know, it's sort of like it's you think about it as you find another app that is better or whatever for your group. And so, you know, you all drop out, you know, something like that, right? Yeah, anyway. Yeah, I think that that's clearly it could be a possibility. I think that's probably how- It's tough to differentiate. I was all along, you know, during your talk thinking about the extent to which, for example, when people go in and look at an app, I think you, is it the case? How do you, so the ones that go in and don't accept the privacy authorization are those mixed up with people that just go in, takes a look, read out the description and says, whatever, I don't like this app and leave. You know what I mean? Like those guys don't have privacy concerns. They just took a look and didn't like it or whatever. So are these separate groups or they're together in what you're doing? See, if it's- Okay, go. I think for all day that we stick with one sample. So basically throughout, this is, you know, we just look at what they're doing with this one sample, right? So I don't- No, no, no, but what I'm asking is this. When they go in, what you have some people that did not accept the request for me- The initial authorization. Yeah, but is that distinguishable from people that opened the app, looked at the description and how it looked? And, oh, I see. So before that, you have to download. Is that correct? No, you don't have to download. It's very, very immediate. These are lightweight applications. You can just immediately open up within Alipay. So you don't have to download anything more. But the only cause is that you have to agree to authorize this data- And you can see that that's where they stopped when they got the- Because you see, I'm trying to see- We see actually, this individual opened this mini program, but they decided not to authorize data sharing. And then he gets himself herself out. So then we see this, yeah. So actually, you know, among the very concerned group, actually 25% of the time actually they declined. So that's actually quite interesting, right now. Yeah, yeah, yeah. Oh, okay. So thank you, Roberto. Thank you, Wei. Thank you, Li. Thank you. Thank you. Bye-bye. On the East Coast, so maybe we should officially end this webinar. So it will be the last webinar of 2021. So thank you for being with us. It has been a very long year. So, but we will skip the next one, January, because of the US job market and elsewhere. So, but by the same, in the same time, so we are also interested in about your opinion or idea about organization of our webinar series. So you can find a survey link in the chat box or on our website, cbndc.net. So go to a survey and write something to us. So, happy holiday. So we hope to see you next year. Thank you, everyone. So, if you want to stay at this, get a little bit, so a bit stay behind a bit. Okay, thank you. Bye. Thank you, everybody. Thanks, everybody. Thanks. Thank you. Happy holidays. Thank you, everyone, for organizing.