In some of the previous tutorials, I explained how to include “external” items (images, SSL certificates…) in your program thanks to the embedding binary data feature of the esp framework. Today I’ll show you how to use part of the flash memory to store data, like it was an hard drive or a memory card.
Partitions and filesystem
You’ve already learned in my fourth tutorial that the flash memory connected to the chip is divided into partitions to store different elements (the bootloader, the main program…) and that you can – via the menuconfig – configure the layout of the partitions using standard templates (for example “single factory app, no OTA”) or o in a totally custom way.
The list of the partitions, their position in the flash memory and their size are defined in the partition table, this too stored in the flash memory at address 0x8000.
We can define a custom layout writing a csv file with the following format:
Each line represents a single partition: you have to specify its type, a subtype, the offset (i.e. the starting address of the partition) and the size. The csv file in the screenshot above corresponds to the following layout:
To verify that the partitions are aligned (= there’s no empty space between them) you have to sum to the offset the size of the partition to find the offset of the next one… for example 0x9000 + 0x6000 = 0xf000.
A partition is just an “area” inside the memory: to be able to store data, you have to “organize” them. For example, I’ve already blogged about the NVS component that store data in the form “key-value” in a dedicated partition. An operating system organizes data on mass storage devices (memory cards, hard drives…) using a filesystem. During the years, many filesystems have been developed, depending on specific requirements or dedicated to specific devices. If you have a Windows PC, your harddrive is probably using the NTFS filesystem, while memory cards usually work with the FAT filesystem. A flash memory has unique features and requirements; for this reason Peter Andersson developed a filesystem optimized for them, named SPIFFS.
To be able to use the SPIFFS filesystem in your program, you have to add to your project’s folder the component developed by Boris Lovosevic (loboris) which acts as a wrapper between the SPIFFS library by Peter Andersson (pellepl) and the Virtual File System (VFS) of the esp-idf framework.
First download from loboris’ Github repository the archive containing all the files:
Now copy the spiffs folder from the zip to the components folder of your project:
The zip contains also the tool (mkspiffs) you can use to “transform” a folder of your PC to a file that represents a SPIFFS partition, ready to be uploaded to the flash memory. You can copy the mkspiffs folder for example in your home directory:
The spiffs component requires some configuration variables, which define the geometry of the partition, i.e. how it is “built”.
You have to add the following parameters to the Kconfig.projbuild file of your project (here I explained the purpose of this file):
- SPIFFS_BASE_ADDR, address of the flash memory the SPIFFS partition starts from
- SPIFFS_SIZE, size – in bytes – of the partition
- SPIFFS_LOG_BLOCK_SIZE, size – in bytes – for each block of memory
- SPIFFS_LOG_PAGE_SIZE, size – in bytes – for each page of memory
If you want to understand how the SPIFFS partition is divided into blocks and pages, you can read this document.
Base address and size must be the ones you configured in the csv file that defines the layout of the flash memory, while “good” values for block and page size are 8192 e 256.
Prepare and upload the image
If you want to store in the SPIFFS partition some data before the program is executed (for example static images, audio… which will be then used by the program) you can use the mkspiffs tool to prepare a SPIFFS image with the content of a folder of your PC.
First you have to compile mkspiffs:
cd mkspiffs/src make
Now create on your PC a folder and copy in it all the files you want in your SPIFFS partition. You can also create subfolders… for example I created the spiffs_image folder in my home with the following content:
Run the following command to create an .img file which represents an SPIFFS partition with the files above:
./mkspiffs.exe -c /home/luca/spiffs_image/ -b 8192 -p 256 -s 1048576 /home/luca/spiffs_image.img
The -b parameter is the value of LOG_BLOCK_SIZE, -p the value of LOG_PAGE_SIZE and -s the size of the partition. The mkspiffs command shows the content of the image just created:
Now with esptool you can transfer the image into the flash memory:
python $IDF_PATH/components/esptool_py/esptool/esptool.py --chip esp32 --port COM15 --baud 115200 write_flash --flash_size detect 0x180000 /home/luca/spiffs_image.img
In the command above replace the COM port with the one your devboard is connected to. Pay special attention to the address (0x180000) from which esptool writes the data: it must be the same you specified in the csv with the partition layout.
Use SPIFFS in your program
First include in your program the headers of the SPIFFS wrapper and the VFS component:
// VFS and SPIFFS includes #include "esp_vfs.h" #include "spiffs_vfs.h"
At the beginning of the program, register the SPIFFS filesystem:
This method informs the VFS component about the SPIFFS partition and which functions to call for the different operations (read, write…) on the filesystem. The method also mounts the partition (and formats it if necessary) to make it visible starting from the path /spiffs.
You can verify if the partition was correctly mounted with:
Now, thanks to the VFS, you can use the standard C functions to work with files and folder. For example you can open for reading the “readme.txt” file in the root of the partition with:
FILE *file; file = fopen("/spiffs/readme.txt", "r");
and read its content with:
int filechar; while((filechar = fgetc(file)) != EOF) [...]
In my Github repository you can download a program that allows the user to navigate in a SPIFFS partition with unix-like commands (cd, ls) and to display the content of text files (cat):
[youtube id=”hGkbNuskVLw” width=”600″ height=”350″]
The ability to use part of the flash memory to store files and to access them using standard functions makes it easy to develop programs that require external elements (html pages, images, sounds…) without adding memory devices to the esp32 chip.