 Thank you so much for keeping the presentations like on time that is so important and thank you again and now we have Praveen Kumar Pulakund from Oracle and he'll be talking about the improved error logging in MySQL 8.0. Thank you. Yes, my name is Praveen Kumar Pulakund. I am working as principal software engineer in MySQL team Oracle. I am with Oracle from more than six years now and I am part of MySQL runtime team mostly contributing to contributed to the transactional data dictionary and to new error logging and other interesting features in runtime as service side statement timeout and today I am talking about improved error logging in MySQL 8.0. Yes, this is our safe harbor statement. As part of this presentation we look at introduction to error log and motivation for error log change in MySQL 8.0 and about improved error log, some of the error log components and new rule based error log filter and at then some of the important changes turn to the error log in MySQL 8.0. The MySQL error log contains required of MySQL startup and shutdown time. Here is an example of MySQL server startup message with the timestamp and the version of the server we are using with the process ID and this is the error message reported for server shutdown time with the timestamp and MySQL error log contains diagnostic messages which are reported during server startup or during server shutdown and while server is running and these messages are labeled with the severity of the error message that is error warning or note. Here are the examples about error messages which are reported while starting the server with the note and warning severity one and on some of the servers on some of the platforms even the call stack is logged into the error message if server is about it exited abnormally and historically we supported logging to only error log stream and to the file this is a system variable which is used to configure the error log output to log error stream out to the file and later we supported logging to the windows event log and to the unique system log and MySQL also supports filtering of the error messages based on the severity of the error message and this is a system variable used to filter the error message and default value for this prior to 8.0 is 3 which means all the error messages with the labels error warning or note are logged to the error file and other valid values for this system variable is one and two if it is set to one only error messages are logged to the error file and if it is set to two error and warning messages are logged to the error destination. So this is a brief introduction about the error log, error log content and the filtering of error messages as I mentioned historically we supported logging to only file and log error stream then we supported logging to the windows event log and then to the unique system log and with the each new logging format the server code is changed and to support new logging format say to log in the JSON format or to the XML format we need to change the server code because of the same reason even customer cannot have their own logging format if customer needs new logging format then the server code needs to be changed and as I mentioned currently log filtering is supported only at the severity level MySQL does not support filtering error log error log messages on the other parameters as error message or a subsystem which is reporting the error and other parameters and we can rate limit some of the error messages that is in some situations same error message is reported in the given or in the particular window time we can rate limit such error messages but to make such changes we have to change the source code which cannot be done by the DB or the user at this point of time to address these issues the error messages or error logging is improved in the MySQL 8.0 to support new logging format without changing the server code the logging is made as loadable in MySQL 8.0 and it is implemented using the MySQL component framework and component can implement services which are defined by the logging subsystem and I can load and log the message in the format required. The service which can be implemented for logging are of two types one is filter and the sync. Sync is a log writer which writes to the error destination and filter is the error component which filters the message. Here is an example of components which are loaded to the server this block is the integral block which is always available this is called as built-in components these are always available with the MySQL server code this can be enabled by using the one of the system variable which I am going to explain in the next slide and if user wants logging in some other format then the component should be loaded and enabled or these are the some of the components which are already there in server or user can write their own logging component loaded and using install component statement and write the error log messages. Error logging subsystem internally builds error log stack that is to pass the logger message to the log components installed and these are enabled by adding to the system variable log error services by default logger service has the these two components that is sync internal and filter internal this writes to the traditional error log file filter internal is used to filter the error message by severity mode which we discussed in which we discussed in the second slide. Logger services the user can add the components to the logger services in the order in the order he wants to do but it is it should be separated by the semicolon as mentioned here the order of components in the logger service is very important by default as I said logger services has these two component which has a filter internal as the first component and the sync internal as second component so in the stack sync internal is loaded first and then the filter internal is loaded since filter internal is loaded so when error message comes it is filtered by filter internal component and then returned to the writer component. This is the overall picture of logging from the MySQL code before we had a very simple error logging statement which is which has the information about only a severity and the error messages. Now new C++ API or C++ class is introduced where more detailed information of error message error message can be logged I will talk about this API in the next slide. Log event class or the object generates a log event line for each of the information supplied to the log event a log item is created that is a key value paired log item which contains a key in this example it is a timestamp and the actual value and the log item priority this is this log event line is example for the shutdown message which also has the log message with the message information and this log event line is passed to the log error stack so depending upon the order or the components registered to the log subsystem the log event line is passed to that component. For example we are suppose we are only using the built-in components then this line is passed to the filter internal first and if we have a rule log error verbose it is set to filter the error message then it is dropped at this component only and the error message does not satisfy the condition specified in the log error verbose it is system variable it is passed to the sync internal which writes to the log error file in the format as timestamp severity and the error message this is the complete APIs of the log event C++ class where we can now specify the error code error code of the error message before we used to mention only the severity and the error message in the server code only now all the error messages are moved to the error message dot text so for each of the error message the error code is generated in the server code and we can specify the server symbol OS error number error message source file where the error message generated source line even the function name subsystem we can specify that the error is generated in the server or in the replication or in the NODB module and we can specify the component if error message is generated at the component in the component code and we can specify other informations as user host thread ID priority and the label message as I mentioned we get it if your error code is mentioned then message is generated from the error message dot text file if not user can pass the error message and verbatim is where we do not use the formator for the text error messages we can just pass the string and the timestamp are the values which we can pass with this function in the code we have wrappers for this class right now that is log error and log plugin error now we are using the log these APIs to log the error message MySQL 8.0 has some of the error log components already in the code as mentioned this is sync internal this logs to this is a traditional error logger which logs to that server log file this is in the built-in components list so this need not have to be loaded and we have log sync json this is a new component introduced in 8.0 this writes log message to the log message to the log destination in the json format and the urn to install the component logs this component is component log sync json and it writes error log messages in the error log file.json with the json extension this component is used to log the error messages to the either windows event log or to the unix system log if user wants to log with the to these windows or unix system windows event log or unix system log then this event this component should be loaded these are the filter components introduce these are filter components introduced in 8.0 this is part of the built-in components which filters error messages on the log error verbosity system variable value and 8.0 introduces new filtering mechanism that is rule-based filtering mechanism we will see about this in the next slide so these are the few components which are implemented in the MySQL.o so this is the new filter that is rule-based filter and it is a if else condition-based language where user can specify the condition on any of the field field is the key in the log item which we saw in the previous slide and he can add a binary comparator he can compare it with the value or the Boolean comparator with then exist or not exist of the key and write a action for the for the condition that is either drop the whole log event or rate limit the log event this called as throttle we have the example in the next slide about it and we can change some of the keys in the log event or you can remove some of the fields if not required from the log event and this is each rule is terminated by full stop in this full stop and the rule should be should be set to the system variable log error filter rules so these are few examples about the filtering the first example is to drop the log event with the priority of information and to remove the source line field from the log event itself so here is a rule set with the log error filter rules if priority is greater than equals to information then drop the whole event log event and in the in the log event if source line is mentioned then answer then that log item is removed from the log event and this rule is this example is to rate limit the events which are of information type and it will it will it will be logged only once in 60 seconds window here is the rule for it if priority is equals to information then throttle and throttle just log only one error message in 60 seconds window and this this example is to change log item value that is error symbol here is a rule for it if error symbol is equals to error startup then change the value to error s so these are some examples about rule based filtering these are some of the changes which are done as part of my SQL 8.0 by default now as I mentioned in the 8.0 by default all the error error message all the all the severity are logged so it is changed to 2 now only only warning and error messages are logged to the error destination and if we see the error log there are many error messages which are not actually error messages are those just information for example server startup message or server shutdown messages which are which were logged as the note or information in the 5.7 so in 8.0 new severity of the label is added to such messages or such system messages now it is so such messages are logged with the system label and these cannot be filtered as they are not the error log messages and all the error messages got the are have the unique error message ID now so if user wants to filter based on the error message ID we can use the error message ID for those I'll so if we see this example this is about the start startup message now this startup message has the new label a system and for this error message this is the new unique error ID and this is logged by the subsystem server and the other for message format remains the same and second example is about that you have any question okay okay and this is the example of another message which is warning actually so it is logged with the warning and the unique error code error ID error message ID and the error message so that's also some reference about the error log and error log components and about the filtering that's it any questions I'm wondering did you get feedback from the field with regards to as a result of the new granularity you get more or less error messages coming out the reason I'm asking that is back in good old days when we had message pages whenever there's an error message your message page will go off if you're a field engineer and the worst part of it is government tenders are not written by human beings what you get is this really barbaric clause that says that if an error goes off you have to resolve it within one hour the thing is by the time you get back to your desk it's one hour already especially if it's on the weekend but this will not change or increase or reduce the frequency of the error messages it is just the the way the error message is logged and the format we want to use it format in which we want to log the message you will get the same security depends on the on the mission on the entry that is reduced yes the application needs to sort of like you are the single only if it is an error because that's the only crucial thing but in case it's just a note that something that it's just more you know make a note of it if you're wanting it something probably you can look at it then sometimes but if it's an error that's probably needs to be alerted I think that's the problem so is there automatic action there right we don't have but the if we the if the user or the developer wants to change the component I mentioned about the component which is either a filter or a writer component log writer component they can write application logic which takes the error messages sees the type of the severity of the error messages and then generate the warning or notification for that it's not part of the product so yeah I am okay any other question okay thank you