 I work as a software engineer, especially in IWI's developer, PayPal. Today I'm going to talk about Siri Kit integration, and title life of the talkies, giving every app a voice. The agenda for today is first, I know most of you are experienced developers. I know most of you are experienced IWI's developers, but some of you may be into IWI's development, so I want to give you a little bit background so that everybody will be on the same page. Then I'll talk about Siri Kit and how to integrate. Finally, I'll give you tips that can help you on integrating your apps into Siri Kit. It all started in 2014 when I was in DC. You guys can expect what it is. Apple introduced a new feature that everybody was waiting for, App Extensions. Before App Extensions, there was no way that one app can use your app's features outside your app. So you have to open your app to experience your wonderful features. But if the user is on some other app, he can't use your application's features. So with the App Extensions, Apple introduced a new way of interacting with your apps. Basically, this App Extensions will help you to go beyond your application and expose your particular features so that other applications can also use it. Also, these App Extensions are not a free-fetched app application. They live for a very short time and they are defined for very limited scope of work. Once the scope of work is finished, that extension gets filled. Nowadays, it became mandatory and must ensure for every iOS app to have these App Extensions. We need any sort of App Extensions. So what are the different types of App Extensions that we initially have? Apple started with a small list, like 6 or 7 of them. Today, extension share, action, photo editing, custom keyboard, watch treat, all these. And currently, they have like 19 of them, 1920. So how does the App Extensions works? Imagine, let's say you are browsing through a photo album and you want to send a photo to your WhatsApp friend. So earlier, how you can do this one was only by opening your WhatsApp, clicking on the chat, importing that particular photo and then send. So you have to switch between your photos application and your WhatsApp. But with the App Extensions, you don't have to go out of your photos application to do that. You can use your trade extension there, select WhatsApp, and immediately WhatsApp extension presents inside the context of your photos application. So once the particular task is over, your WhatsApp extension gets dismissed immediately and you still are in your photos application. So once users use your App Extensions, system create, I always ask to launch the App Extensions and App Extensions runs the code. And once that is finished, system creates the App Extensions. So, and why I am talking about App Extensions? Because App will introduce two couple of more App Extensions last year. I mean, for the previous WWDC. They are Watchkit, sorry, iMessage, intense extension, and intense viewer extension. The bottom two are the ones we are using for CDT integration. So remember these two, intense extension and intense viewer extension. So how Siri works? When you talk to Siri, the speech gets converted into an intent. So the intent being like, what is the purpose of the user? What is the task that the user wants to achieve? And then that intent gets converted into an action. And that action, after successful or failure, you get a proper response from Siri. So as an App developer, we have to help Siri in the whole process. So by giving first the vocabulary. So this step is to make Siri understand what the user is intending. Like let's say you have your own messaging app and you have your contact, your spouse's name or somebody's name, you saved as with some unique to that particular app alone. Like let's say you save your spouse's name as sweetheart. And in your contact list, her name is normal, her name itself. But inside your app, it's saved as sweetheart. But if you say Siri that send my sweetheart, so in some message via this particular app, there is no way that Siri knows that sweetheart is your, this particular contact, right? So that specific app contact, app related terms, everything, you have to teach Siri how to understand that. So here is the step. The vocabulary provides you the space to give Siri the value. And then once Siri understands that intent, it will ask you for verification, like what are the different parameters that user wants to do. Let's say you are doing a payment application here. So you want to send some money to somebody and that Siri ask for is the pay correct, is that amount is correct, all these different confirmations that will ask for it. And then once all those required conditions are met, Siri performs that action and finally comes up with the response. And even there, you can provide your own user interface so that Siri can, I mean, you can bring your own custom screens to be in sync with your regular application. So you don't want Siri to give a different UI rather than, I mean, your own app is giving some other UI, if you want to bring the same look and feel into the Siri application, you can do that here. So speaking Siri's language. So how does Siri works is like everything in Siri falls into domains. Domains is a set of group of items that Siri can understand and it knows how to talk about them. So currently Siri has six different domains, it supports messaging, white calling, payments, workouts, write booking and photo search. If your app or some of your app functionality falls into any of this bucket, then you are a good eligible candidate for Siri integration. Inside domains, you can have many actions or they call it as intents. I'll go with a small example here. Let's say I want to send some money to Soop for some reason. So when I say, hey Siri, send $20 to Soop on PayPal for the kids. So with this speech, with this sentence, Siri understands that this particular request belongs to payments domain and it identifies that it has sent payment intent because there is a word send amount is there, right? So based on that, it gets that intent and we have to use PayPal application and pay Soop amount $20 and currency. Here is the interesting thing because you never mentioned that you have to send it in Singapore dollars, right? You said only $20. But based on your location, Siri identifies that you may be asking for Singapore dollars rather than US dollars. So Siri provides that as well and finally Siri also notes the note what is the purpose of this request for the kids. So I'll talk about intent in a little bit because intent is the particular action that you want the Siri to perform. Each intent can have zero to many parameters. In the previous example that we have seen, right, it has like five or six parameters. It depends on your scenario and different domains. You can have up to, I mean, you can have no parameters to some particular many parameters and always intent classifies into a particular domain. So first identify the domain that your particular feature belongs to and then you raise your domains and raise your intents based on that. And then once Siri finds out the intents and asks you for that and then you have to give Siri back the intent response. So how you are handling the intent? You have to give back Siri that information with the format intent response. So in that you give different response codes and you also have to give NSUser activity. How many of you are aware of NSUser activity? So NSUser activity is nowadays a widely used in iOS applications. Handoff, you have no handoff right. So if you are performing some task on some place and then immediately if you go to your own application and you can pick up the same task from there. So NSUser activity can be useful there. So for this sending payments to somebody, let's say if you ask Siri to send money and then for some reason the request failed. You don't want the user to again key in the amount and selecting the pay, all these details right. So you can use NSUser activity to get the details whatever he has earlier. So then pick it from there and you can directly open your app and show that screen so that user can perform the request continuously. Intents life cycle. Basically Intents has three stages in its life cycle. First one is Resolve. So Resolve is a place where Siri asks you to ask for the different, I mean based on the user's request, you have to help Siri to understand those parameters that you have given. So Resolve is a step where Siri asks you for that and you will respond with Resolution Responses. There are different types of Resolution Responses. So let's say in the previous example, Siri identifies that the name, the contact name, the pay name SUB as particular. It is saved as SUB in my contact list. Then there is no question, it is straight away successful. So we'll respond back with success. Let's say my in the contact list, SUB's name is saved with this full name. So Siri asks for a confirmation. Do you meant by this person? So we have to just reply back with the confirmation required for that with this. Then sometimes there may be two or three names matching with that name. So you have to give disambiguation request. And if you are not sure, if there are hundreds of items matching with that particular name, then you cannot provide all the 80 or 100 list of names to user to scroll through, right? So you need some more details, so you just reply back with that. And if any required field is missing from your request, then you can ask for that. It's a needed field. And if it is unsupported, then you can also tell that. And if any field is not required, you can say it is not required. So Siri won't prompt the user to give that particular parameter. For this same payment intent, the protocol that the different resource texts are defined in this instant payment intent handling function. So if you see the functions, different functions available there, are like resolve pay, resolve currency amount, resolve note. So based on your intent, these different methods will be changed. And confirm. The next step is confirm. Confirm is a stage where you do two different types of actions. So you have to tell Siri what is the expected action of the intent. So that Siri will show a confirmation screen to the user with all these details. And this is the place where you check for your requested. Let's say you are sending a payment. Then you have to be logged in, right? So this is the place you check whether the user is logged into your application. Also your backend servers are available for the contact. And also whether he has sufficient balance in his wallet. All these different checks you can make here. And you will reply back with the intent responses. And for certain domains, the confirmation step is not necessary, but for domains like payments and all, this step is mandatory. So you don't want somebody to send money without user confirmation. So here is the protocol that we have to look into in the same payment intent handling. There is a method for confirmation. And based on the intent, we have a different response force available. So there are quite a good number of predefined response force. Some of them are like amount below minimum, amount above maximum. Currency is not supported. And these are some certain number of predefined and most frequently used error force in the response force for this particular domain. So based on the different domains, you will have a different set of error response force. As I spoke earlier, this is the place where you check all your services are available, whether the user is logged in or not. If you want to make sure that the user has to authenticate, you can integrate with your local authentication touch ID framework to prompt for the authentication there. And the final step is to handle. This is the place where Siri actually performs the requested action. That means it actually sends money by calling your network APIs. For the network calls, Siri provides a waiting UI. Like if it is going to take two or three seconds of time, then Siri gives you this type of one waiting UI with a loader and all this. If a particular request is going to take longer time than two or three seconds, you can also send Siri that in progress status code, so that Siri tells the user that even if you close the Siri experience, his request is still in progress. So after he can open his app and see the transaction is completed. This is a particular method for handling in payments. How to add Siri K? So remember, there are these two intense extensions and there are some app components that also you can do while integrating your application. Intense extension. Each app has launched this new intense extension and it can support more than one intense in a single extension. So based on your domain, if you see here, there are intense supported in the playlist there are attributes. So you can have send payment intent and also you can have receive payment intent like that. And if you are developing a wide calling application, then you can have audio calling and video calling intents in the same intense extension rather than having two different intents extensions for each of them. You can put them in a single umbrella. And security. You don't want certain features to be exposed without security. So Apple added that with the security. So the default one they give you is intense restricted by lock so that if you add your particular intent into that group here, then it won't allow the user when the device is locked. And this is the place where we actually implement the result conform and handle steps that we previously discussed. And then we can have our own intense viewer extension, which is an optional one. If you want to bring your own interface or own custom branding experience to Siri, you can use this Siri intense viewer extension. And you can also choose to replace some of the Siri generated UI and you can have your own ones there. Next up is vocabulary. Vocabulary is basically divided into two categories. One is app specific one. Another one is user specific one. If the app specific ones, regardless of the user, all the users of your application knows about those words. So then that is a place for your app specific vocabulary. You have to define them in the main app bundle, not in your extension bundle because Siri gets this vocabulary using this in vocabulary API from your main app so that it updates itself frequently. And there will be cases with user specific vocabulary. So in that we have to provide in an ordered set, let's say in my contacts, I have some favorites and recent contacts. They have to be given higher priority when I talk about, when I request for the particular action, those frequent contacts or favorite items has to be given higher priority over the remaining ones. So we have to provide them in an ordered set of terms. And finally I'll end up with a few tips. In order to work with any extensions, you have to move your core functionality into embedded frameworks rather than having your core duplicating everywhere for each and every extension. It's always better to have an embedded, a separate framework that every extension or every app can reuse there. And also I forgot to add one more here. It's better to have your deep linking capability available for your application so that you can navigate from one place to any other place easily without going through same steps again. So let's say my payment is failed. I can ask user to open the app directly there itself from the Siri experience. Directly it can go to my sending money page, screen. So if a deep linking is not supported then you might be... I mean it's very difficult to go to that screen directly. And the second point is a user must say your app name. So if you have any very fancy app names that user can't even pronounce those, please avoid those because it's very mandatory for the user to pronounce that one. Request authorization. Like any other Apple features or Apple iOS frameworks, you have to get the user's confirmation for this particular feature to be accessed. And the next one is group multiple intents into a single extension where possible. So if you're using a what calling kind of app, then don't put too many extensions, intents extensions into credit cross because whenever Siri is launched your intents extension gets called is in active mode in the background. I mean the process keeps running. So if you have too many intents extensions running behind door that will drain the battery and all other problems will come for the user. So better to put them in a single long rail or wherever it is possible. And the next one is user activity. Use the NS user activity and pass it to the intents response so that user can pick the same request in your main app. And this is one more important thing. If you have a vocabulary added into your app, so you have to update Siri whenever there is a new item added or deleted from that one. Let's say I removed a contact name from my contact book and when I ask Siri, if she still shows that particular contact name it won't give you a proper user experience. So that's why you have to update Siri whenever there is a change in your vocabulary. And also don't forget to reset your vocabulary list when the user is logged out of your application. You don't want to show user specific vocabulary to other users. And also the final thing. You can provide some useful examples and have it in your main app bundle so that whenever user asks Siri what you can do for me then it will show along with all the iOS applications it will also include your particular app also. And also user can see how can we use that one. Otherwise there is no way that user can know that your app is supporting Siri. And what to expect from Apple in future. Maybe currently this domain, I know it's very limited. It's only 5 or 6 of them. But maybe they will increase this to multiple domains. How many of you know about Buzz Uncle, Facebook, chat, messenger, bot? Buzz Uncle. So you can ask, I mean rather than, with the Buzz Uncle you can ask when is the next bus coming, right? So same way you can ask, maybe in future you can ask Siri to directly ask when is the next bus coming. So it will show rather than opening some SDMRP or some apps you can expect those. And maybe they will include some standalone application extensions because we have seen this one in the iMessages extension already. iMessages extensions are, you don't need to have your main application. You can only have your extension app alone. So maybe they can support this one. Because there is one reason I feel for the Apple Watch. Currently you have to have one main app also running on your mobile. So maybe Apple can remove that requirement. Thank you. And if you have any questions? Do you have any framework for the service? Yeah, or SiriKit.framebox. The second question is if you need a payment, is there any security manager that prevents more than an hour person from making a phone name to you? Yeah, you have to, you can add your touch ID there. So you have to authenticate your fingerprint and the touch ID thing. So touch ID is nowadays the most secured way of securing the aspects. How does it work with other languages? It works almost with all the Siri supported languages. And if you have any app specific vocabulary, so you have to put them in the main bundle. And also you can localize them so that Siri can understand with all the other languages also. So you can have different languages to work with. Do you have any experience to share testing with? Yeah, we were testing it manually pretty manually. I'm not sure very anything available. Because even Apple was suggesting to test it very rigorously. They have not mentioned what tools they can do, but they said you have to give it to all most of the people. And it's not like any simple, you have to take care about the user's accent, all this, right? So the many people that you give it to for testing, the better result you will get. And you have to, whatever the people would say, for example, say, can't we be able to allow them their own accent? But let's say I want to give my own thing, you know, to pass it on to them. And how do you use the accent domain, but are you using something like bus type method or something? Can you do something like that? I've got this weird idea, but I've not tried. The intents are tied very tightly bound, linked with the domains. So you can only have particular intents and the request, the resolve step, it will give you only for specific to those particular intents. So it won't give you a general object so that you can use it for anything. So they are very tightly wanted ones. Do you have a staging and production app and you get confused? For this particular one? All our apps are focused on stages. Like staging, like a development app. So you have the official one, you know, and the development app. I mean, it's on the CD, it picks the production one. That depends on your main app, which one it is connected to. So we have a development and release separate branches. Each one points to different stages. Let's say, as you were saying, like the neutralization of the region, you might have access to any type of motion for extension. From extension, I have not tried that, but... This is an off-object or something. This I am sending a message in the background from Siri. So I need to access that off-object. Yeah, I think it's possible, but I have not tried one on my own. We have tried, I mean, for the Siri TV version or any normal extension. Siri extension. I am not sure, but I can check that one. Have you done something like that? Have you done anything like a non-Siri TV extension? From Siri TV, no. We have not tried that. Can I talk to you recently, App Store review, right? Or App Store support, Siri TV? Do you have enough of App Store support to review this Siri TV? Review this is like... To start with, I mean, you can basically integrate almost any particular apps with the termite with a single step, like the result step. So you don't have to provide a vocabulary and also this... Providing you a custom UI. They are not mandatory ones. So... Do you have any specific... products or... No. You said that Siri improves certain information, like single or single. Is there any way I can claim that that if certain words come in, then I expect to use this particular... You can use the result step. So Siri will tell you that a user requested for this particular amount. And also if that particular currency is... This all comes in a... There's a money object, IM money or something. So that will have the amount in decimals and also with the currency code. So if your app doesn't support that particular currency code, you can... You can respond back with the default once you can... Thank you.