In some of the previous tutorials, I explained how to include “external” items (images, SSL certificates…) in your program thanks to the embedding binary data feature of the esp framework. Today I’ll show you how to use part of the flash memory to store data, like it was an hard drive or a memory card.
Partitions and filesystem
You’ve already learned in my fourth tutorial that the flash memory connected to the chip is divided into partitions to store different elements (the bootloader, the main program…) and that you can – via the menuconfig – configure the layout of the partitions using standard templates (for example “single factory app, no OTA”) or o in a totally custom way.
The list of the partitions, their position in the flash memory and their size are defined in the partition table, this too stored in the flash memory at address 0x8000.
We can define a custom layout writing a csv file with the following format:
Each line represents a single partition: you have to specify its type, a subtype, the offset (i.e. the starting address of the partition) and the size. The csv file in the screenshot above corresponds to the following layout:
To verify that the partitions are aligned (= there’s no empty space between them) you have to sum to the offset the size of the partition to find the offset of the next one… for example 0x9000 + 0x6000 = 0xf000.
A partition is just an “area” inside the memory: to be able to store data, you have to “organize” them. For example, I’ve already blogged about the NVS component that store data in the form “key-value” in a dedicated partition. An operating system organizes data on mass storage devices (memory cards, hard drives…) using a filesystem. During the years, many filesystems have been developed, depending on specific requirements or dedicated to specific devices. If you have a Windows PC, your harddrive is probably using the NTFS filesystem, while memory cards usually work with the FAT filesystem. A flash memory has unique features and requirements; for this reason Peter Andersson developed a filesystem optimized for them, named SPIFFS.
Component
To be able to use the SPIFFS filesystem in your program, you have to add to your project’s folder the component developed by Boris Lovosevic (loboris) which acts as a wrapper between the SPIFFS library by Peter Andersson (pellepl) and the Virtual File System (VFS) of the esp-idf framework.
First download from loboris’ Github repository the archive containing all the files:
Now copy the spiffs folder from the zip to the components folder of your project:
The zip contains also the tool (mkspiffs) you can use to “transform” a folder of your PC to a file that represents a SPIFFS partition, ready to be uploaded to the flash memory. You can copy the mkspiffs folder for example in your home directory:
The spiffs component requires some configuration variables, which define the geometry of the partition, i.e. how it is “built”.
You have to add the following parameters to the Kconfig.projbuild file of your project (here I explained the purpose of this file):
[checklist]
- SPIFFS_BASE_ADDR, address of the flash memory the SPIFFS partition starts from
- SPIFFS_SIZE, size – in bytes – of the partition
- SPIFFS_LOG_BLOCK_SIZE, size – in bytes – for each block of memory
- SPIFFS_LOG_PAGE_SIZE, size – in bytes – for each page of memory
[/checklist]
If you want to understand how the SPIFFS partition is divided into blocks and pages, you can read this document.
Base address and size must be the ones you configured in the csv file that defines the layout of the flash memory, while “good” values for block and page size are 8192 e 256.
Prepare and upload the image
If you want to store in the SPIFFS partition some data before the program is executed (for example static images, audio… which will be then used by the program) you can use the mkspiffs tool to prepare a SPIFFS image with the content of a folder of your PC.
First you have to compile mkspiffs:
cd mkspiffs/src make
Now create on your PC a folder and copy in it all the files you want in your SPIFFS partition. You can also create subfolders… for example I created the spiffs_image folder in my home with the following content:
Run the following command to create an .img file which represents an SPIFFS partition with the files above:
./mkspiffs.exe -c /home/luca/spiffs_image/ -b 8192 -p 256 -s 1048576 /home/luca/spiffs_image.img
The -b parameter is the value of LOG_BLOCK_SIZE, -p the value of LOG_PAGE_SIZE and -s the size of the partition. The mkspiffs command shows the content of the image just created:
Now with esptool you can transfer the image into the flash memory:
python $IDF_PATH/components/esptool_py/esptool/esptool.py --chip esp32 --port COM15 --baud 115200 write_flash --flash_size detect 0x180000 /home/luca/spiffs_image.img |
In the command above replace the COM port with the one your devboard is connected to. Pay special attention to the address (0x180000) from which esptool writes the data: it must be the same you specified in the csv with the partition layout.
Use SPIFFS in your program
First include in your program the headers of the SPIFFS wrapper and the VFS component:
// VFS and SPIFFS includes #include "esp_vfs.h" #include "spiffs_vfs.h" |
At the beginning of the program, register the SPIFFS filesystem:
vfs_spiffs_register(); |
This method informs the VFS component about the SPIFFS partition and which functions to call for the different operations (read, write…) on the filesystem. The method also mounts the partition (and formats it if necessary) to make it visible starting from the path /spiffs.
You can verify if the partition was correctly mounted with:
if(spiffs_is_mounted) {...} |
Now, thanks to the VFS, you can use the standard C functions to work with files and folder. For example you can open for reading the “readme.txt” file in the root of the partition with:
FILE *file; file = fopen("/spiffs/readme.txt", "r"); |
and read its content with:
int filechar; while((filechar = fgetc(file)) != EOF) [...] |
In my Github repository you can download a program that allows the user to navigate in a SPIFFS partition with unix-like commands (cd, ls) and to display the content of text files (cat):
Conclusions
The ability to use part of the flash memory to store files and to access them using standard functions makes it easy to develop programs that require external elements (html pages, images, sounds…) without adding memory devices to the esp32 chip.
Ciao Luca.
Se provi a compilare l’esempio di blocca.
Deve esser cambiata la definizione in “esp_vfs.h”
Per risolvere il problema va commentato il campo:
esp_vfs_t vfs = {
// .fd_offset = 0,
Ciao Loris
grazie Loris, in effetti ora il framework ha incluso un driver SPIFFS “ufficiale”, sto rivedendo quindi l’articolo…
Ciao Luca,
è possibile creare con SPIFFS un file locale che contenga un immagine recuperata da una IP cam ?
Un comando del tipo:
> SPIFFS.open(pippo, “w+”);
> File pippo = “http://address_ip:port/snapshot.cgi?user=xxx&pwd=xxx”;
Scusa se ho scritto qualche castroneria, ho questo oggetto da una settimana e sto imparando.
Grazie
Maurizio
Ciao, si non vedo problemi a fare quello che dici… solo va capito nella risposta della IP Cam da che punto “inizia” il bytestream che rappresenta l’immagine.
Ciao Luca, complimenti per i tuoi tuttorial. Veramente ottimi!
Ora che il componente spiffs fa parte dell esp-idf framework, sto provando a caricare una immagine della partizione spiffs creata con il tool mkspffs. Riesco tranquillamente creare l’immagine e caricarla sul’esp con esptool. Tuttavia quando lancio il firmware dell’esp e provo a leggere i file di testo che ho precaricato ( sono dei file on.html e off.html ripresi dal tuo tutoria su ESP32 WEB server), ottengo solo caratteri strani come: ��@ ecc… Ciò non avvinee se il file lo creo direttamente dal firmware lo leggo poi lo leggo e lo ristampo. Sai quale potrebbe essere la causa del problema? Magari qualche problema di compatibilità tra mkspiffs e il componente spiffs del framework? l’ utimo commit del tool mkspiffs risale a 7 mesi fa
Grazie
Diego
ciao Diego! E’ proprio così, anch’io avevo iniziato a fare delle prove ma pare che vi sia un disallineamento tra l’implementazione di SPIFFS del framework e quella del tool… per questo per ora continuo ad utilizzare (vedi il progetto ESP32lights) ancora il componente di loboris che funziona benissimo.
Grazie per la risposta!
Tuttavia volevo condividere con te la soluzione a questo problema. Ho approfondito un po’ la questione e alla fine sono riusito ad utilizzare il componente ufficiale spiffs con il tool mkspiffs di loboris. In 2 parole:
una volta tirato giù il pacchetto del toole mkspiffs, prima di compilarlo va sostituito il file mkspiffs/include/sdkconfig.h con quello del tuo progetto, il quale contiene i corretti parametri ralativi alla partizione spiffs: SPIFFS_BASE_ADDR, SPIFFS_SIZE, SPIFFS_LOG_BLOCK_SIZE e SPIFFS_LOG_PAGE_SIZE.
Poi si può compilare ed utilizzare il tool per creare l’immagine da caricare poi con esptool.py.
N.B. Nel caso vengano cambiati i parametri si deve ricompilare il tool mkspiffs.
Questo è il link al thread dove ho trovato la soluzione https://esp32.com/viewtopic.php?f=2&t=3955&p=18002&hilit=mkspiffs#p18002
Credo che lo stesso valga per le varie verisoni di mkspiffs ( ovvero sia quella di loboris che di igrr).
Io ho tuttavia provato soltato con il pacchetto che è possibile scaricare dal commento del thread (Tue Dec 19, 2017 9:47 am) fornito da loboris.
Ora riesco a leggere correttamente il contenuto dei file caricati usando il componente spiffs del esp-idf framework.
Spero di essereti stato utile,
Diego
ottimo grazie!
Hi Luca
In your flash map / partition the factory 1M section starts at 0x10000 and should be ended at 0x110000
However the spiffs section starts at 0x180000
shouldn’t it starts at 0x110000 ?
Hi… well partitions don’t need to be contiguous but you’re right, in this case I’m wasting some space…