One of the most popular projects among the ones included in my tutorial about the enc28j60 chip is fore sure WebRelay. This project allows to control an output pin of Arduino using a simple web page, designed to be accessed also using your smartphone. Today I’m going to show you how to implement a similar project with the esp32 chip; it’s also the opportunity to teach how to write a TCP server, especially a web server.
Netconn API
As you know, the esp-idf framework uses the lwip library to manage network communication. This library offers different abstraction levels: the programmer can decide to work on the raw packets or to leverage pre-build components.
To develop my TCP server, I decided to use one of those pre-build components, the Netconn API.
Using Netconn API it’s very easy to implement a server and the required steps are described below:
The netconn_new() method creates a new connection and returns a a pointer to struct netconn which represents the connection:
struct netconn *conn; conn = netconn_new(NETCONN_TCP); |
The parameter defines the connection type… the most common ones are NETCONN_TCP for a connection using the TCP protocol and NETCONN_UDP for a connection using the UDP protocol.
To use the connection in server mode, you have then to associate (bind) it to a specific port… for example a webserver normally listens on port 80 (443 if HTTPS):
netconn_bind(conn, NULL, 80); |
The second parameter (NULL above) allows to bind the connection to a specific IP address and may be useful if the device has more than a network interface. If you pass NULL (or the equivalent IP_ADDR_ANY) you ask the library to bind the connection to any available interfaces.
Now you can start listening on that port with:
netconn_listen(conn); |
Working with a client connection
With the netconn_accept() method, your program can accept a new incoming connection:
struct netconn *newconn; netconn_accept(conn, &newconn); |
The method returns a pointer to a new struct netconn that represents the connection established with the client. This method is blocking: the program is stopped until a client makes a connection request.
Once the connection is established, you can use the netconn_recv() and netconn_write() methods to receive or send data to the client:
netconn_recv(struct netconn* aNetConn, struct netbuf** aNetBuf ); netconn_write(struct netconn* aNetConn, const void* aData, size_t aSize, u8_t aApiFlags); |
To optimize RAM memory usage, the netconn_recv() method handles data using an internal buffer (zero-copy mode). To access the received data you therefore have to:
- declare a new variable as a pointer to struct netbuf
- pass the pointer address as the second parameter of the recv method
- use the netbuf_data() method to obtain a pointer to the data within the netbuffer structure and its length
struct netbuf *inbuf; char *buf; u16_t buflen; netconn_recv(conn, &inbuf); netbuf_data(inbuf, (void**)&buf, &buflen); |
Likewise, the netconn_write() method accepts, as last parameter, a flag to indicate whether or not to copy the buffer content before sending it. To save memory you can therefore, if you’re sure that the buffer won’t be modified by other theads, use NETCONN_NOCOPY as flag:
netconn_write(conn, outbuff, sizeof(outbuff), NETCONN_NOCOPY); |
When the communication with the client is complete, you can close the connection and free the buffer:
netconn_close(conn); netbuf_delete(inbuf); |
HTTP server
What you learned so far can be applied to every TCP server. If you want to communicate with an Internet browser, you have to “talk” the same language, that is the HTTP protocol.
In the example program (available on Github) I implemented a very minimal version of that protocol. When you type an address in your browser (for example www.google.com), the browser connects to Google server and sends a request with the form:
GET <resource> [...] |
The request can have different fields, but the first line always contains the name of the requested resource (html page, image…). In particular if you access the homepage of the website the request will simply be GET /.
The website published by the esp32 chip to control the relay consists of only two pages:
- off.html, displayed when the relay is off
- on.html, displayed when the relay is on
Each page contains a label (“Relay is ON|OFF“) and an image. The image itself is a link to the other page and when you click on it, the relay status is also changed:
The program identifies the resource to be sent analyzing the first line of the request with strstr():
char *first_line = strtok(buf, "\n"); if(strstr(first_line, "GET / ")) [...] else if(strstr(first_line, "GET /on.html ")) [...] else if(strstr(first_line, "GET /on.png ")) [...] |
An HTTP server responds to the browser by first indicating the result of the request. If ok, the return code is 200:
HTTP/1.1 200 OK |
then it returns the media type of the resource and finally it sends the resource file. In this example, the possible media types are:
[checklist]
- text/html for HTML pages
- image/png for images
[/checklist]
Static content
A webserver it normally stores the resouces for the website it publishes on an external device (memory card, hard drive…). For simple projects, you may consider to include all the content inside your program.
For example in my code HTML pages and HTTP protocol messages are defined as static arrays:
const static char http_html_hdr[] = "HTTP/1.1 200 OK\nContent-type: text/html\n\n"; const static char http_png_hdr[] = "HTTP/1.1 200 OK\nContent-type: image/png\n\n"; const static char http_off_hml[] = ""; |
To be able to include also the images, I used a feature of the framework (embedding binary data). You can indeed specify in the component.mk file the binary resources to be included:
In your program you can access the content of the embedded files using the following pointers:
extern const uint8_t on_png_start[] asm("_binary_on_png_start"); extern const uint8_t on_png_end[] asm("_binary_on_png_end"); extern const uint8_t off_png_start[] asm("_binary_off_png_start"); extern const uint8_t off_png_end[] asm("_binary_off_png_end"); |
The syntax is _binary_filename_start|end, replace “.” with “_” in the filename. Having pointers to both the start and the end of the resource content, it’s easy to send it with the netconn_write() method:
netconn_write(conn, on_png_start, on_png_end - on_png_start, NETCONN_NOCOPY); |
Luca,
Great article. Thank you for your time, your generosity and the help that it gives. I have been trying to find an example that shows a web server serving a html file from memory. I would like to be able to serve a large SPA file like ReactJS from a SD memory card. Is that possible?
Hi John, yes it’s possible and I’m working on it! You may also consider (this will be the topic of a future post) to use a partition (FAT or SPIFFS) in the internal flash.
Hi Luca,
You publish really great tutorials! Thanks. Regarding the webserver – is there a simple way how to get user input from the web (e.g. to retrieve content from a web form submitted). I am thinking about something like a “simple admin web console” to set-up / administer SSID, user credentials … where text input is necessary.
Thanks in advance
Marek
Hi Marek, yes it’s quite easy to receive (GET or POST methods) data from the user… you only need to parse the incoming request: GET parameters are added to the URL while POST parameters are added in the request body. One of my future tutorials will be about this!
Hi Luca,
Thanks for the answer. One more thing regarding your demo – the web server is working perfectly fine when connected from android (e.g. from mobile phone), but since I connect from Windows (e.g. notebook running Win10) the pictures downloading is extremely slow (and in some cases even stops the web server on ESP32 to listen to requests and respond) – tested on 2 different Windows devices. Any idea what could be the problem (BTW – without the images it works fine even on Windows)?
Thanks in advance
Marek
It’s working really really well. I could not believe a http server was running on my ESP.
The keyword “static” makes my compile fail though (Cygwin GCC, ESP SDK 2.1) –I am not entirely sure why.
Hi Luca,
Thanks for all your work, this are really great tutorials. I’m trying to do a merge between two of them, the Access Point (18) and Webserver (20). I’ve tried some times, but it didn’t work. Do you have any example or material that may help me to configure it?
Thanks.
Hi Héber, it’s strange… the two elements should work fine: could you share your code and/or any errors?
I’ve already find a way to make it work. thanks a lot.
Hello Luca,
Which relay did you used on this project?
I have 5v jqc-3ff-s-z relay but it did not work.
Thank you.
My mistake was connecting relay from 5v pin-out
Correct way was use relay’s vcc in 3.3v pin-out.
confirm, the esp32 pins run at 3.3V
Luca,
Can you please provide a sample to run this same webserver in only AP mode?
Hi! Yes, it will be the topic of a future tutorial!
Hi Luca,
Great stuff and I applaud your efforts! However neither the standard web server example or SSL examples are working with the current releases of the ESP-IDF. The standard web server does not compile and results in an error:
C:/msys32/home/esp32/luca/14_basic_webserver/main/main.c:232:2: error: unknown type name ‘mdns_server_t’
mdns_server_t* mDNS = NULL;
The SSL Server compiles and I can connect to the SoftAP but trying to bring up the page in a browser results in a connection reset while trying to negotiate TLS. I tried https://192.168.10.1 as well as https://192.168.10.1/off.png just as a test. I tried in Chrome and Firefox and allowed for the invalid certificate in the browser settings.
Hi Erik, unfortunately Espressif changed all the methods for mDNS… I’m updating all my examples
Dear Luca,
Your tutorial is really great.
I have question, after web server connected with WIFI AP , it working well.
But if the AP turned off then turned on again, look like esp web server not working anymore (not reconnect to the AP)
I have to reset the esp32, then it comeback working again.
Can i do for automatic reconnect after AP off and then on?
Thank you in advance.
Woot
Hi, you have to manage the SYSTEM_EVENT_STA_DISCONNECTED event, see this example.
Hi Luca,
Thank you for all the great tutorials. I have a question. How are you able to run 5v relay from 3.3v pin without any additional changes. I am trying to run a 8 channel 5v relay, and though the LED lights up, the relay does not switch on.
Thanks
Raajesh
it probably depends on the sensitivity of the relay… but you should use 3.3V relays to be sure that they switch
Ciao Luca,
complimenti per i tutorials!! Grandissimo!
Hai qualche dritta cu come implementare un client udp?
ciao francesco, è abbastanza semplice, guarda l’esempio fornito dal framework… non ho mai fatto un tutorial perché non ho trovato una applicazione di utilizzo.
Buongiorno Luca e complimenti per tutti i tutorial. Ho iniziato a muovere i primi passi con l’esp32 grazie ai tuoi consigli!
Ora ho un problema, sono partito da questo tuo esempio di web server che ho importato in platformio e sono riuscito a farlo girare bene.
Pero’ le due PNG volevo spostarle, toglierle dal codice buildato, e metterle dentro il file system (SPIFFS).
Tramite platformio, ho insertito le due PNG dentro la cartellatta “data”, buildato e fatto l’upload dell’immagine dello SPIFFS contenente le due PNG.
Creata la partition table ( nvs, data, nvs, 0x9000, 0x6000,
phy_init, data, phy, 0xf000, 0x1000,
factory, app, factory, 0x10000, 1M,
storage, data, spiffs, 0x180000, 1M, )
e fin qui’ tutto ok, operazione che ho gia’ fatto molte altre volte…
Il problema arriva quando ho sostituito la tua send “netconn_write(conn, on_png_start, on_png_end – on_png_start, NETCONN_NOCOPY);” con il seguente codice che va ad aprire il file, manda l’header “HTTP 1.1 200 OK\r\n” e poi con un while faccio le send finche’ il file non e’ finito.
f = fopen(“/spiffs/on.png”, “r”);
if (f == NULL)
{
printf(“Failed to open file for writing”);
while(1);
return;
}
netconn_write(conn, http_png_hdr, sizeof(http_png_hdr) – 1, NETCONN_NOCOPY);
while(fgets(buffer, 500, f))
{
netconn_write(conn, buffer, 500, NETCONN_NOCOPY);
}
fclose (f);
Ora ti volevo chedere, secondo te e’ un problema dello SPIFFS che non riesce a leggere abbastanza velocemente dalla flash i circa 40Kb della png?
Freq dell’ esp32 240MHz
ciao Luca, se guardi il sorgente del mio progetto esp32lights vedrai come ho implementato qualcosa di analogo