Change Partition Table through OTA - Over The Air (Arduino - PlatformIO)

[brunofgc]

Board

ESP32-C3

Device Description

Custom Board

Hardware Configuration

It's not relevant, cuz it works for OTA processes that have the same "partitions table scheme".
But, only power components and using BT and WiFi to build a "sensor gateway", no special hardware connected.

Version

v3.0.4

IDE Name

PlatformIO

Operating System

Windows 11

Flash frequency

40MHz

PSRAM enabled

yes

Upload speed

115200

Description

Hi @VojtechBartoska and everyone,

I’m working with an ESP32-C3 using PlatformIO and the Arduino framework, and I’ve run into an issue with updating devices in the field via OTA. The current partition table on the devices limits the maximum firmware size, and my new firmware is larger than what the current partition table allows.

I’ve successfully created a smaller, intermediate firmware that fits within the existing partition table and uses a new partition table configuration. However, even after flashing this intermediate firmware via OTA, it seems that the partition table on the device is not updated. As a result, when I try to upload the larger firmware (which would fit in the new partition layout), I still receive a "too large" error, indicating that the partition table remains unchanged.

I’m looking for a solution to remotely update the partition table via OTA so that I can upload the larger firmware without having to physically access the devices. Is there any way to perform this partition table update via OTA, since I have nowadays many many devices country wide (Brazil)?

Any guidance or suggestions on how to handle this situation would be greatly appreciated.

Thanks in advance for your help!

Sketch

A bigger firmware than it could fit on actual partition table.

Debug Message

[112397][E][Updater.cpp:163] begin(): too large 1351296 > 1310720

Other Steps to Reproduce

Steps:

1. Install my original firmware using "Default 4MB with spiffs (1.2MB APP/1.5MB SPIFFS)" partition_table
2. Send a new firmware with partition_table with "NVS = 20 Kb, OTA = 8Kb, app0 = 1664Kb, app1 = 1664Kb, spiffs = 640Kb, coredump = 64Kb" using Updater.cpp OTA Method
3. See the debug message from serial

I have checked existing issues, online documentation and the Troubleshooting Guide

• I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[lbernstone]

This forum is for bugs in the codebase stored here. Please ask general questions at https://esp32.com
OTA is for replacing non-active code. This is done by keeping 2 partitions for code and switching between them programatically.
There have been discussions about modifying the partition table in place, but bottom line, this will never be supported and is highly dangerous, and will end up bricking some of your devices in the field. There is no mechanism for doing this with the arduino API- this will all be esp-idf functionality.
https://www.esp32.com/viewtopic.php?t=11482
https://www.esp32.com/viewtopic.php?t=39210
https://www.esp32.com/viewtopic.php?t=12771
https://www.google.com/search?q=site%3Aesp32.com+%22partition+table%22

[SuGlider]

I’m looking for a solution to remotely update the partition table via OTA

It is possible, but the firmware must write a new partition table directly to the correct flash area, where the former partition table is located.
This is a risky operation. It can fail and leave the device unreachable and not operational up to someone is able to flash it again using the serial interface. This "updater" must be carefully planned and exhaustively tested within many different scenarios.

Arduino offers no tools for it. It shall be done using ESP-IDF.

[SuGlider]

Converting this issue into a discussion.

[brunofgc]

So, maybe I have a alternative in mind. What if I write an ESP-IDF firmware that fits on current partition_table and current app0/app1 size, just to use this firmware as "intermediate firmware" to re-write this partition_table to the bigger one, and then this ESP-IDF firmware just start the OTA update to my "Arduino final firmware"? Would it be possible?

[brunofgc]

Sorry, I think the answer are on the links provided by @ibernstone. I'll read them and then I'll be back. Sorry and thank you guys for your support! :)