Fix orphan blob accumulation on firmware upgrade #1601
Conversation
weebl2000
force-pushed
the
fix/onetime-blob-cleanup
branch
from
e8d2268 to
5735efd
Compare
Before PR#1495, blobs were written for every advert regardless of whether the advertiser became a contact. Nodes upgrading from older firmware (e.g., v1.11.0) have thousands of orphan blobs consuming storage space. This adds a one-time cleanup on first boot that: - Iterates all blob files in /bl/ - Deletes any blob that doesn't match a current contact's pub_key - Creates a marker file to prevent running again Affects ESP32 and RP2040 platforms (which use /bl/ directory).
weebl2000
force-pushed
the
fix/onetime-blob-cleanup
branch
from
5735efd to
6a9bcbf
Compare
|
Good call, I should have done this so thank you 😁 Couple of things, does this belong in DataStore rather than MyMesh? Then you can just call remove directly. Also it should probably check for the full key to avoid accidentally overwriting blobs with the same prefix. I'm on phone so my apologies if I've misunderstood the code. |
Yeah, DataStore is probably cleaner. Changed that. With regards to checking full key, the filename is only 8-bytes (16 hex chars). If we would want to check full key we would need to read the files for the advert info so that might be overkill. Chance for collision is very low though with 8 bytes. |
|
Ran it on my heltec v4 and all my contacts are still there, but disk usage was reduced by over 1 MB. |
2 participants
Fixes #1519
Related: #1495
What's this about?
PR #1495 fixed the blob leak going forward, but devices upgrading from older firmware (like v1.11.0) still have thousands of orphan blobs sitting in
/bl/eating up storage space.This adds a one-time cleanup that runs on first boot after upgrade - it just walks through
/bl/, checks each blob against the contacts list, and deletes any that don't belong to an actual contact. A marker file prevents it from running again.Only affects ESP32 and RP2040 (the platforms that use the
/bl/directory).This code can probably be removed after a few releases once most users have upgraded past this version.
Note
I also experimented with an LRU approach that caps blob count and evicts oldest files periodically - works well as an ongoing safety net, but this simpler approach should be enough for the upgrade cleanup.
Testing