 Hi, everyone, and welcome to our webinar on Validating Product Feature Decisions in Mobile Apps. My name is Julia Raffkin, and I am the Director of Product Marketing at UX Chem. I'm originally from New York City, and I've been based in Berlin for the last five years. I've spent most of my career at tech companies in roles focused on innovation, product development, and go-to-market strategy. Like many of you in the audience, I've been a product manager, and today I'm excited to speak about validating product decisions, specifically when it comes to mobile app products. So UX Chem is a collaborative analytics platform for mobile app teams. We enable multiple stakeholders with actionable insights from real user behavior, including qualitative, quantitative, and technical insights across the full product experience. Although I only joined UX Chem recently, the reasoning behind it comes from my experiences as a product manager focused on building nascent products. I believe in what the power of this product can do to enrich not only the life of a product manager, but the entire product development squad, including QA, designers, engineers, et cetera, and enables all of these people to truly be customer centric and to have that user centricity grounded in real data all within one tool. Kind of sounds like a dream. So let's talk about product validation. Validation is the act of confirming something as true or correct. The benefits of validation within the product lifecycle are well known to all of us. We save time and money, we de-risk development and resource allocation, and it helps us achieve our outcomes faster. The reality, though, as every PM out there knows, is that product ideas really do come from everywhere. So everyone has an opinion on what should be built next and how. And it's actually a great problem to have when product ideas are coming from internal employees, direct customer feedback, trends in the market, and even our own executives. The validation process exists to ensure that we're not building in a vacuum or an echo chamber, whatever you prefer, and that what you're building actually delivers value to your end users. So validation is not just the core of product management and of effective product management. It's actually also the core of the scientific method. The basic steps of the scientific method, which is also the basis for really great marketing and many other disciplines, are to make an observation that describes a problem or to have a question and to conduct some research around that. And once you've kind of done those steps, you end up developing a hypothesis which you then go and test. And once you've tested that hypothesis, you draw conclusions and either refine that hypothesis or pivot away to a different hypothesis. And before we really move on to what UXCAM can do, I'd like to talk a little bit about the process of validation and why it's extra important in mobile apps and due to a few different factors. So first of all, the release cycle in mobile apps is a lot more dependent on the end user than in web development. In web, you just refresh the page and you have the latest code out to all of your users, whereas for an app, the user has to take an action. They have to download an update to get the new version of that. This matters because basically fixing issues or getting a feature right becomes a lot harder. Most mobile apps release about every one to two weeks, although we see app development cycles that can be anywhere from two, three days to a full month. But the top 100 grossing apps worldwide release on average of about every 14 days, so two weeks. The second factor to consider is that testing within the mobile app development world is a lot more cumbersome, which makes the cost per every new feature a lot higher. Most developers are developing on PC environments and while mobile device emulators exist, they're obviously not 100%. Furthermore, the QA for mobile apps is inconsistent with actual use because you can't really simulate that environment. So the QA engineers typically lack touch screens. They don't have the same mobile processors. They're also QAing on PCs and they don't get the same real world interference from other apps push notifications. There are also a lot more compatibility permutations to consider, such as device type, manufacturer, screen size, etc. So building a new feature requires much more rigorous testing before release. And finally, the last big factor here is that validation is extra important in mobile apps because mobile users tend to be less patient. So this is partially due to age range factors. Mobile users do tend to be younger. So just an example in the United States, 96% of the 18 to 29 year old demographic has a smartphone versus only 61% of the 65 year old plus demographic that own a smartphone. And obviously you need a smartphone in order to be able to access an app. It's also very easy to uninstall a mobile app if the end user doesn't see the value right away or getting to that aha moment takes too long. So one of the statistics we found is that 29% of smartphone users will immediately switch to another site or app if they can't find information or it's working too slowly. And those are just two reasons why they might navigate away and that's already accounting for almost 30%. And even though mobile makes up for more than 50% of all internet traffic and nowadays all the mobile trends are increasing thanks to both product and device improvements, many users do still prefer to accomplish important online based activities on big screen devices such as making a large home appliance purchase or something like that. So this is why validation in mobile apps is extra important and why we're dedicating a full session to it. So when should we be validating our product and feature decisions? The annoying answer is all the time, but we've split it out into three critical moments when you want to validate a product decision regardless of whether it's a new feature or an optimization or improvement on an existing feature. So the first moment in time is before you make the decision to test whether it actually makes sense and whether the underlying hypothesis is true. The second point in time is when you're building the MVP and releasing the alpha and beta versions to really refine that hypothesis, test usability, track that it's delivering the value that we want to deliver and that is generally directionally aligned to where we want to go before we invest even more resources into it. And finally, the third critical moment is really after it's released and fully rolled out and during that rollout to ensure that we've built the right things for the right business goals and that it's having the impact that we wanted it to have. So before I move on to the next slide, I just want to let you guys know that we're going to dive into some examples of how you can validate at all of these stages with UXCAM. If you're interested in testing out UXCAM for your own app, please contact us. We have a special extended trial available exclusively for product school members. All right. So let's talk about validating before any decisions are made. I will let this play as I talk over it and we'll do that with every video. So as I mentioned before, product ideas can come from anywhere and everywhere, but sometimes you don't necessarily have a concrete idea for where to start. You can use UXCAM to identify existing points of friction by looking into frustration signals that are automatically collected within the app, such as rage taps and UI freezes, or you can identify drop off points via the screen flow to see high level what's happening with a typical user journey and monitor where the quit rates are. These would all be great places to start for optimizing the end user experience and you can obviously filter by a lot of different factors, users, segments, age groups, devices, etc. And you can also drill down into each of the sessions and actually rewatch what's happening in there. So as you can see here, this is the screen flow and you can see what a typical path the user might take would be. Cool. So we're going to move on. In different scenario, let's say an executive says that we should implement cash as a payment option because he or she heard that one of the competitors is offering this as well. Before you send the team into ideation, you want to test whether there's even demand for this as a payment option and whether it will do anything to improve drop off rates on the purchase screen, let alone merit the operational investment of processing cash payments. We can create a session funnel and essentially test whether there's even demand for something like this. So we group it by platform. Right now our demo environment is actually limited to only to Android and with our existing payment methods of credit card and PayPal, we see a drop off of over 83% on the payment page. So there's definitely something going on with payment methods, but we don't necessarily know if cash is the right way to go. It could be. There are a few options here. You can build a new version of the app that also has cash as a payment option and release it to a small subset of users. Or you can also include a simple landing page screen that allows customers to opt in or be approved to use cash as a payment option on their next purchase just to see the demand. We added cash as a payment option and compared the funnels. So in the new funnel, the drop off rate decreased from 83% to 68%, which is a nice jump, but it becomes clear that cash as a payment method is not necessarily the way to fully solve for that drop off. And maybe other options like adding Google Pay or investigating something else should be tested as well. So now we're going to move on to the validation stage during building and what happens there. So is it doing what we had actually hypothesized? In the MVP, the alpha, the beta testing phases, in general validation at the stage is always best to do with your customers. But unlike traditional businesses, mobile product managers only have indirect access to users and customers, which is why it's important to have tools like UXCAM in order to be able to identify which customers to validate with and why. So in the first example that I'll show, UXCAM is utilized to identify power users. So these are people who have completed a purchase and have visited the app more than 10 times, but you can customize this to be whatever you'd like. But it can also be used to identify any user segment of interest. And then with the click of a button, you can send this select group of users a push notification, either offering them a specific deal or the option to become a beta tester of new app features. And then you can actually have this specific subset of users using your new app and you can really test with them and so on and so forth. In the second example, I actually want to talk about how UXCAM can be used to conduct usability testing at scale. And so in our previous example, we send a push notification letting all of our power users opt into the new version of the app, which has our new feature. So now that we have a small group of users using this new feature in a new app version, let's say we made some changes within the checkout process. We can actually observe how the new feature looks and performs across different device types and screen sizes. And we can do this on both a quantitative and a qualitative level. We can also measure engagement time on a particular screen or set up a specific event like a purchase completion and see how often that event is completed and compare it to different app versions. From all of these higher level quantitative and technical indicators, we can also drill down into individual sessions to understand the micro interactions as they actually occur in real time and iterate on the product accordingly, allowing us to truly build with the end user in mind. And finally, we move on to the rollout and release phase. So you've validated the assumptions before pursuing the new feature and you've conducted usability testing with a subset of your users. And now that you've released the new version or new feature to a wider audience, you want to track that the assumptions you made and the early outcomes that you observed are actually moving the needle for business outcomes at scale. This is where UXCAM's customizable dashboard actually comes in. With our customizable dashboard, you can configure any metrics of interest that you would like to track, for example, daily and monthly active users, or the engagement rate on a specific screen or feature. And you can have an at-a-glance view of performance over time, app version, device type, et cetera. And you can essentially have this saved and look at it every day or multiple times a day depending on what kind of product manager you are and really have both a high level of view of the data and also an underlying ability to drill down and really see what the drivers of that data is. So let's say we made an adjustment to the checkout process. So we built a dashboard for this. And you can look at how many checkouts there are per week by country. You can see if there are any changes based on our release cycle. We can also look at how many unique users are checking out per week, how long the checkout process takes to complete by device type, as well as how many clicks it takes users to check out to see if we built something intuitive or if the interactions are too complex. We can also quickly and easily build a new metric, like what I'm doing right now, just like tracking the engagement time on a specific screen by device manufacturer. And all we have to do is configure this widget and you can make it look however is the easiest for you to digest that data. And finally, we get to our last example. So part of the balancing act of making great product decisions is knowing when to be quantitative and when to be qualitative. The great thing about UX cam as a tool is that you can be both and you can rely on both to actually make those decisions. So sometimes trends around user behavior will appear in the data, like a drop off rate on the quick screen or something like that. But sometimes you have to just observe real users in their real environments and using your product to uncover insights about how to improve it or add more value or whether the feature is creating the experience you wanted. For example, we have a grocery delivery app. So we're going to watch a session here. And as you can see in this session, the user will try to click on a photo of the apples to actually enlarge the photo, but the photo of the product itself is not clickable. And that's something that wouldn't really come up in any kind of quantitative data. It might come up in technical data like the rage taps, but it's not really going to come up in any other way other than just watching these sessions and seeing that this is what the user is trying to do. And this can serve as a potential idea for a product improvement if this is where the natural user behavior is headed. So these are only some simple examples of how UX can can be utilized as a tool across the product development cycle to validate product and feature decisions within mobile apps. And the interface is flexible enough that it can be configured to do much more for your app product and it can be used by everyone from product managers to designers, researchers, analysts, QA engineers and even support teams. If you have a mobile app and you're interested in trialing out what UX cam can do for your business's insights, please reach out to us as we have a special extended 30 day free trial offer available for all product school members and webinar attendees. So just to conclude our session. We know that validation is an important and critical part of the product management process, and it's especially critical when it comes to feature development and mobile apps due to factors including the release cycle, imperfect testing capabilities and less patient user demographics. Building products that deliver real value to users can only be accomplished by talking to customers and observing their real behavior in a non controlled environment. Throughout this process, while quantitative and technical insights are valuable, nothing really beats the qualitative behavioral insights of real users in real world settings. And finally, validation is a cross functional process that's at the core of the scientific method, which is why sharing insights from different perspectives and across teams is so vital. UX cam is a flexible tool that can be used throughout all stages of validation to make decisions grounded in a truly 360 view of the product journey and improve customer understanding and the understanding of customer experience across all teams. So thank you for your time. We hope you found this information valuable. And if you have any questions please feel free to reach out to me personally or to our team. Have a great day.