 Hi everyone, welcome to this session. I'm sorry to be a little late. And this is built a micro-HET server for embedded systems. And this is online. I'm John Pan. I come from Taiwan. Taiwan is also well-known for most of the mainland area. It's about 36,000 kilometers square. And one of my dreams is having an electric vehicle and a jibbit around and take a picture in Taiwan. And so I want to build an experience platform for studying vehicle controlling. And I wanted the vehicle controlling is in a closed loop model. And in most controlling systems, actually, it's a motor. And the motor always comes with the controller. And it also gets the rotation status of the motor. And the controller can do the controlling algorithm with the feedback and the controlling command. And there's a lot of parameters that will change the motor status. For example, the temperature, humidity, and et cetera. And we can use the encoder to measure the status of a rotation and something else for the senseless method to get the rotation status to. After gathering all of the data, we can do a system identification for the undefined system. For example, we don't have the data sheet of the motor. So in traditional, gathering the data, we use the wireless serial port. And it may be slow. And if we have too many devices have to communicate with, and we only have one central computer, then it will be a problem. And so I think if there is another choice, for example, we can use the TCP-IP-based internet. And it could be faster and convenient for management. The protocol for over TCP-IP, it could be a QTT code or just a REST for way-by-PI on HTTP. There is no best choice for any condition, but always a choice for case-by-case. And in traditional, there is a heterogeneous gateway for internet and traditional communicated protocol. For example, RS232 and Bluetooth DB, or even smart internet, like smart internet or Wi-Fi. Or just we can make a device connect to the internet directly. And for my condition, I want my device can connect to the internet directly. So I want to put an HTTP server on the device. And I can use the browser to send the command to the controller through the HTTP server. And gathering all of the major data from the ADC and from the HTTP server. And I can use the browser to browse the data. And the false state and IoT is fancy for now. However, there's a lot of the limitation. Considering the size and the power restriction, most embedded devices have a limited resource. I mean, it's on the microcontroller unit label, not the big model. For example, Cortex-A or Intel 86 CPU. I mean, it's on the microcontroller unit label. And it has less processors, and less memory, and less storage, and even lower speed. So we may not put a heavy oise on the chip. And so there may not have a process or thread APIs. So we cannot put the Apache or NGIX HTTP server on the platform directly. So I need to have an HTTP server to do the things for that. And before a new HTTP server, we can review the OSI-7 layers that we have run from testbook. The physical layer that define the electrics and the lines, data network and transport layers define how to link with the neighbors, how to route with the IP, and how to pass or build the segment. They are all then controlled by the OS. And therefore, session presentation and application layer are controlled by the application. For example, the HTTP server may have a web API. A client request doesn't have a URL, and then it will have an encoded response within HTML form and a response through the HTTP circuit to the client. And the application and the OS has a yellow line here. The circuit API is important. It split the application and the OS. And later I will tell you why it is important. And the HTTP protocol, the standard IFC 2616, 1.1. HTTP protocol is a request response protocol, a client send request, server respond. And here's an example I see from my browser. It is an HTTP request. And here's for the response. It defines what is the HTTP message. HTTP message consists of request and a response form. And both of them start with the star line and multiple message headers. An empty line, it may be a message body. It is not necessary, but it could be. And HTTP message headers could be a full-type external header, request header, response header, or entity header. And all of the header consists with field name, a column, and a field value. And the single space is preferred between the column and the field value. And the message body, actually, it is an entity body. And we will discuss later. And here is a request sample again. It begins with the star line and also called a request line for HTTP request. And multiple HTTP request message headers, an empty line, and no message body here. It could be, but not for this sample. And the request line begins with a method token and a request UI and a protocol version. And the request method could be the option get-headed-possible-put-delete-transconnect and an extensions method, any method you want. And the request UI is the uniform resource identifier and identifier resource. And here is some of the specific request header samples. And here is the HTTP response example. The star line also calls the status line for HTTP response message. And the response message headers, an empty line, and the HTTP response message body. And it consists with the protocol version and status code and associated textural phrase. The status code could be 500 series is for informational, and 200 is for success, 300 for redirection, 400 is for client-side error, and 500 is for server-side error. And also here is some sample for a specific response header. And the entity body consists of the entity headers, fields, and entity body. And the entity header fields define the metadata information of the entity body. And the entity body is in the format and encodedly defined by the entity header fields. For example, the entity headers may have the content type. And the content type may be the text slash HTML. Then the entity body must be in the HTML format. And here's the little recap. The client-center requests the message in this form. And the HTTP server responds the message in this form. And here is the concurrency and the back-end problem about the HTTP server. A client-center HTTP requests it to the HTTP server. And the HTTP server socket gets a request and rebuilds. We request the message and pass it to the back-end server application. The back-end server application responds to the message and writes through the server socket to the client. However, it's good for this situation. However, if there are multiple clients sent the HTTP request at the same time, then the HTTP server can only handle one of the HTTP requests at the same time. So it choose one of the HTTP requests and pass it to the back-end server application and then write a response to the client. Which one should be processed first? And the yellow block is the IO bound, which means the server reads a request from the socket and writes a response to the socket, which means the CPU has to wait for the IO writing or reading. And so there is a concurrency problem. The server could use the process or thread API for multiple clients. However, the socket works great in the backing mode. It works great for this model. However, the process or thread API must be provided by the OS. And so there is a lot of context switching. Maybe we can use the IO multiplying and non-blocking socket. It could use in a single thread situation. And compared with the process and thread, it is less resource required. And no context switching for this model. And here is some of the IO multiplying and non-blocking APIs, especially the slack function we will use later. And here comes the back-end server applications. The standard RFC 3857, the Common Gateway Interface 1.1. The Common Gateway Interface is a simple interface for running external program software or gateways under an information server. And it's in independent manner. It defines the script. The script is invoked by the server according to this interface. It's not to be a standalone program, but it could be a dynamic loaded or shared library or even a subroutine. And the main data is nanoparameter, which carries information from the server to the script. For example, Abash HTV server receives a request and a policy. And the server puts the request to the header into the environment variables. And then forks to have a child process. And the child process will inherit parents' environment variables. So the child process can get the variables with the, can get the HTTP headers by the environment variables. And then get the request body from the standard input. The Abash HTV server also have the response which is produced and returned from the standard output of the child process. There's also another model, FastCGI, instead of creating a new process for each request. CGI creating each new process for a. Instead of creating a new process for each request, FastCGI uses a persistent process to handle a series of requests. And to service incoming request, the web server sends the environment information and the page request itself to a FastCGI process with the socket or TCP connection. And the connection may be closed at the end of the response, but both of the web server and the FastCGI process persist. They live at the same time. And here's another model, the Nescape server application programming interface, NSAPI. Applications that use NSAPI are referred to as NSAPI plugins. Each plugin implements one or more server application functions. And the NSAPI plugins run inside the server process. And the CGI programs and FastCGI programs run outside of the server process. For my convenience, I hope that micro-HET server could work on limited resource, especially from embedded system. And it could process multiple HTTP clients concurrently. And it also, because it's HTTP server, so it should look like an HTTP server. And here's the project. You can find it on the GitHub. Starline is my ID, and you can find the micro-HET server project with this URL. And I split the backend server application into two parts, the middleware and the server application functions, just like the NSAPI. And I can send a request, and the server use the iomarty-proxy-model, the select function, choose one of the HTTP requests, and then pass the request to the middleware. The middleware do like NSAPI, and dispatch the HTTP request to the corresponding server application functions. And this is called the route. And I will discuss it later. And the server application functions will produce the HTTP response message and the write to the client. And here comes with the server socket. What does it do? And the server socket, just like other server application, it has a listening socket on the sending port. And it uses the selector model to select the ready socket. If the ready socket is a server socket, then it's a separate new client socket. If it is not a server socket, then it must connect communication with the client. So it check a state. This state is ready to be read. If it is yes, then read the socket and build the HTTP request message, and then pass it to the middleware, and then the server application functions will produce the response. And then check the socket is ready to be written, and then write to the client, and then close. And the middleware, register routes before a server starts. And the routes is corresponding to the request line, which means the request method and the request URI. Different request method and the different request URI corresponds to different routes. And we register a route before a server starts, and then middleware get an HTTP request message. It will compare the request method and request URI with the registered route. If it is matched the route, then the middleware will dispatch the HTTP request to match the route. And then it's due to the server application function of the match route. If it is not the match route, then it will check there is any matched state file with the request URI. If so, then read the file and write it into the HTTP request message. If it is not, then the middleware will pass the HTTP request to the not found server application functions, and then it will produce the not found message. All of them answer with return to the middleware, and then middleware write to the pass the response message to the server socket, and server socket write to the client. I have a profile with the Pi version. It works great in Python 3.2. But I have to make sure that encoding during reading and writing socket, because Python 3 is a unique code for string. And the main files are server.py and middleware.py in the library folder. And here is a sample of the Pi version micro-HTB server. First, we have to import the related library and then reduce the route, for example, the game method and the route URI. And then run the HTTP server with the callback for a new request. And the callback for new request is the middleware. The middleware will check a route and dispatch. And then this is one of the applications for the server application functions that will come patch. It will produce the HTML content. And here is a sample that I captured. And I use the browser to request my HTTP server and then it responds to the hello. And I also do the automation test with the Pi unit. And I write it in the client.py in the auto-test folder. And here is some test scenario. I make the request only connect and cross actions. And second, connect with request get method and specific UI and then check the response and then close. And the third one, connect, request the post-met and request a specific UI and then check the response and close. And I make it happen with the multiple client request concurrently. And I check the different UI to make sure different server application functions work correctly. And I also use the Travis CI to do a continuous integration whenever I come into the new code to GitHub. And then Travis CI will do the automation test for my new code, non-new code for whole call. And here is the Travis CI use the Travis YAML file to link with the GitHub. And then we can have some script in the Travis YAML file. And then this is the example that I capture from Travis CI. It is the running capture. And at the end, the Pi unit will tell me it ran nine tests in 0.214 seconds. So I know how many times it takes. And finally, I written in C version for embedded system. Not only for embedded system, it works with the traditional computer, including a container like Docker. And also, the main file are server sources file and header file and the middleware source file and header file in the library folder. And the example for C version micro-HTU server code, just like the Pi version. And also for a server application functions example. And I have put the APIs on the GitHub wiki. And you can find it by this URL. And also works good for the C version micro-HTU server. And I also tried some specific Chinese OAs broke in UTF-8 and Big Five. And it works correct here. And here is the software. And let's go to the hardware. Micro-HTU server on embedded system. First, we have to have OAs which should provide the process, not the process, provide the socket API. The micro-HTU server needs a socket API which provided by the OAs. But put an heavy OAs like Linux on a limited resource board may not be a good idea, because the resource is limited. So we have to choose a lightweight OAs like real-time OAs. And I choose the free autos because it has the clear document and usability. I mean the clear document, which means the document is correct and easy to find. I'll say why the correct is important. Free autos is free, which means freedom. And you can find its introduction from here. But pure free autos does not provide a socket-related API. So we have to do by ourselves. And I use the SCN32F4 discovery board as the main board. But it does not have the Ethernet socket. So I find the ESP01 as a Wi-Fi module. And here is the wiring schema. I use the USART6 connector ESP01 UART. And I use the USART2 as the serial console and the connector computer. And this is the real wiring picture. And here comes the yellow line again. The up three layers are for application layer. And down four layers is for the hardware and OS layers. And the ESP01 does the transport network data link and the physical layer. And I use the serial lines on the SCN32F4 discovery board and the wiring with the ESP01 with the UART. And we still need the socket to UART and the serial device drivers for write and read the socket. So here comes the socket API that we may use. The data types, the constant flags, and some functions, initial function for like for socket and bind and listen, accept. And I offer send and receive, shutdown, close, and maybe manipulate the flags for like set up socket option and function control. And select API, the body script set, and the time interval structure, and select function, and the macros like file descriptor 0 set clear is set. We also need ESP8266 and serial drivers. ESP01 chip is ESP8266. The protocol of the communication between microcontroller and ESP01 is 80 command. And here is some of the command that I have used. And here is the structure of my system. And yellow blocks are we lacked. So we have to do by ourselves. The Chinese means that's DIY. And I implemented a used API as much as possible. And the marking may be used if the function is not important to reduce the complexity. I refer to and copy the Linux Hader files because I have to have the same API for different operating systems. And to make it simple, merge the variety Hader files which include and rewrite them into several files. And I also reference the serial drivers of Linux. And for example, the micro-HGTV server use the send system call and then call a send socket function. And the ESP8266 driver use the USART send function and then call the USART send byte function of the USART. And then it will write the data to the transfer line of the serial line. If there is any data received from the serial line of the microcontroller, then it will trigger an interrupt handler of the hardware. Then it will trigger an interrupt handler. And then the USART function will be called. And then read the byte into a received buffer of the USART. And then the ESP8266 driver or persistent task will check there is any new data in the USART received buffer. If so, then it will get a client request the function to read the data from the received buffer into the corresponding client circuit received queue. And then the micro-HGTV server will use the select function to check there is any ready-to-be-read data in the corresponding socket. If so, then it will call a receive system call and then you call an SQ receive function to read the data in the corresponding client circuit received queue to the application. And the ESP8266 driver provided unusual USART channel. And the max ESP1 is a network interface which translates the system call into 80 comments and manage circuit resources like a fair descriptor of circuits. And then provided a mutex of the USART channel. Because both of the transfer and the receive task of the ESP8266 driver communicated with ESP01 through the same USART channel. And also provided a function to join an SS point. And then the received task is a persistent task process the active request from ESP01. And the transfer task is for dealing the commander going to be sent to the ESP01. And it also provides the circuit, check the circuit is ready to be read or to be written functions. And then here's the flow of the receive task. It enable USART receive pipe and then try to take the mutex. If it does not take the mutex, it will delay and block itself. And then our test code will be in running mode. If it takes the mutex, then it will check the USART receive pipe is readable. If there is more to be read, then you will call get ESP8266 request to call and then read more bytes. There is no more to be read. Then you will give the mutex and then you will delay itself. And then our test code will be in running mode. And then here's for transfer task. It will try to take USART mutex too. And then it takes the mutex. And then it will check the command. If the command from the application is sent, then it will send the circuit. And if it is close, then it will close the circuit. And then give the mutex and then suspend itself. And then our test code will be run until the application ask it to run as it get run. And here's the select function, just like an other operating systems library. And the most important is the file descriptor set. File descriptor set is actually a bit of a write. So I make it as the data type of view on signed 64 base integer. And I also implement the select system call function. It actually goes through each circuit whose file descriptor is less than the file descriptor plus 1. It is interested. And then it goes with the check the read file descriptor set. If the read file descriptor set is not normal and the current file descriptor is interested, which means the flag is set, then it will check the file descriptor is ready to be read. If so, it will increase the count. The count will represent how many file descriptor is interesting and ready to be read or to be written or is exception. If it is not, then it will clear a bit of the corresponding to the file descriptor of the file descriptor set. And then also the same job for the write file descriptor set and the exception file descriptor set. However, the exception file descriptor set is not used for micro GTV server. So I mark it up. And so it's a dummy function. And assemble all parts together. After power on, the software will set double LIDs and the USRT tube peripherals and initial ESP8266 driver. Which will set up the USRT6 and create the ESP8266 receive test and transfer test. And then create the micro GTV server test. And then the Friartos test schedule runs. And it will run the tests. And then for micro GTV server test, it check the ESP8266 state. If it is linked, which means it is connected to the Wi-Fi, and then get the interface IP and add the rods, and then initial the micro GTV server, and then run the micro GTV server. And here should be a real demo here. But the airline lost my luggage. So I can only have this. This is the picture I captured before. This is the board and the after power on. And the no lights have been on. And I use the browser to connect to my board. And here's the serial console from the USRT tube, like this. And here's the wire compass here. If I send the command to the red and blue LID to be light, I set it to one. And then they be light. And here's the sample. And here's a reference. And thank you, and any question? It's similar, but I made it directly on ESP. Because I think it's easier just to use ESP as a micro controller instead of using it as a Wi-Fi. But the main problem I came while doing is that I decided that it would be better to make something like re-entrant web server. So because if you do another tasks on ESP, then you need a web server that doesn't hang on select. You need a polling web server so that you can, in some, let's say, spare time, you can check whether the web server will handle the request. Yeah. So that's why I started it a bit from a different way. Yeah. OK, I started it because I want to control some machines. For example, the vehicles. And ESP-01 can maybe a few PWM channels. And other microcontrollers may have six or eight or even more. Yeah. Actually, it's enough to control a robot like this. Only this. Yeah. For some condition, it could be. But I mean it's, for example, a motor and AC motor, three-phase AC motor, then it will maybe use three PWM channels at the same time because it is three-phase. Yeah. And if I want to make a three axis, then it will multiple by three. So it will be nine. So this is the first why I want to write this. And the second is, oh, this one. I mean, I need the correct document. Correct document. Oh, here. I have read the document of ESP-01. And it tells me the 80 plus CWJP and the column and some string after that. And then ends with carriage return and line feed is a command. However, the correct command is 80 command plus CWJP column, some string and line feed, carriage return and line feed, and a line feed. Yeah. So I need the correct document. So that's why I not use the ESP-01 as the main microcontroller and just for the Wi-Fi module. Do you support just some basic things, or do you support different content and coding, different, let's say, compressions, tank modes, and stuff like that? Or is it just request response in HTML? Actually, it's an HTTP server. And here, pitch up here. The server circuit and the middleware are responsible for the HTTP server. And the back end is for the application. And the application could be the three models, CGFSG or NSAPI alike. So I split here. I'll split here. I mean, this for HTTP server and this is for the real application users want. Yeah. And the users maybe want to command for control the motor, turn right or turn left, or speed up, speed down. And it could be just no response message body, but for response line for status call. And 200 is success, right? And maybe I'll go ahead. Because the problem for me was managing all the cases, like chunk encoding, post responses, and stuff like that. Making just, I already have a code for this simple, let's say, text exchange. But then when you go into the HTTP 1.1 specification, then there's a lot of branching and branching and branching. And then you get to thousands of line of code just to handle, for example, a correct post response and stuff like that. Then if you want add things to add things like web sockets, then it will be a big problem. Yeah. With web sockets, didn't you agree the connection? Yeah. Yeah, you need to handle that. And what I was thinking is just CGI is great, yes? But if you don't have support for the HTTP protocol, the whole thing of it, then sending posts, the browser can send you anything. Yeah. It will send you a post that is zipped in chunk encoding, and then what? So my question is how? Well, actually, you can reject that. Yeah, you can reject that, but you cannot reject everything. So what's the scope of HTTP? I have to say it's part of, not all of, which means general condition it could be used. But for a specific condition like web socket, it will fail. Actually, just the easy request and response will be correct for now. And it could be better. Yeah. For example, I also wanted to work with the HTTP as something like that. And it's in progress and not committed yet. Yeah. Lightweight HTTP. There is a library called lib micro HTTPp. Did you consider to use that? It's a project that is, I believe, from Free Software Foundation. It's used by SystemD GatewayD to show the journal. And we are using that in our Soleta project. It's kind of this one. Last one. Yeah. Is it necessary to use it or to change it? I have considered it. But when I finish it, I find it finally when, after I finished the micro HTTP. And it's bigger than micro HTTP. So it needs more space for the storage. Maybe we can try to reduce it's size. Because it does all that we were saying plus more. Like it does TLS. Yeah. I want to try this. But because this is my study after my job. So this is not my main job. I'm in my daytime. And I write this program after work at home. So I want to try the library micro HTTP because it's the newest project. And it's more complicated. I have to think how to try it. But not yet. Excuse me. How can you set the UR to speed? Oh, 115.2 kilo. And this is the bottleneck of the speed here. ESPO1's control is fast. And PC faster. And it is Cortex-M4 for SCN32F4. And it is faster too. However, this is only 115.2 kilobots. So this is the bottleneck here. And actually, I won't use the peripheral of the SCN32F4 the physical layer. But it has no Ethernet socket or Wi-Fi connected directly to the development board. So I use the ESPO1. No question. And then thank you, everybody.