Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support filtering log messages by module at runtime. #793

Open
t-moe opened this issue Nov 2, 2023 · 4 comments
Open

Support filtering log messages by module at runtime. #793

t-moe opened this issue Nov 2, 2023 · 4 comments

Comments

@t-moe
Copy link
Contributor

t-moe commented Nov 2, 2023

Our usecase:
We develop an embedded iot product where we use defmt to store the logs into the flash of the device. When we have a defective device, the customer sends it back, and we can extract the logs from the flash. Now, it would be nice, to also be able to request the logs from the device over the air. Currently we are only able to filter the stored logs by severity, based on the "msg id". In order to save bandwidth, it would be nice, if the device could also filter the stored logs by module.

To make that work, defmt would need to store a "module id" with each message. (Gated behind a feature?)
Quectel is using a similar approach on their modems (Quectel EPAT Format) and I think it allows for quite flexible usecases....

@BriocheBerlin
Copy link
Contributor

Hi, thanks for your message. What you have going on there sounds neat!
I am not entirely sure I am understanding correctly. Are you offering to do the implementation or would you like us to do it?

@t-moe
Copy link
Contributor Author

t-moe commented Nov 6, 2023

I just wanted to open a tracking issue for the feature for now and to get some feedback on the topic... Would adding such a feature even be in scope for defmt? After all, it changes the binary format...

And yes, I could imagine helping to develop this feature.... But let's first discuss if it's even possible/wanted...

EDIT:
To clarify: We don't have the "defmt store to flash"-feature implemented yet either, but that's definitely something we'll do and I'll contribute to defmt. But storing the logs to flash should be easy and there's no change required in the current API. However, being able to filter the logs by module is a bigger change, that's why I opened this issue to discuss it...

@jonathanpallant
Copy link
Contributor

I like the idea of having both compile-time and run-time log levels, to better suit long-running binaries on embedded devices deployed in the field.

@t-moe
Copy link
Contributor Author

t-moe commented Apr 22, 2024

Am I correct that the format strings indices are no longer sorted by log-level (or never were)?
This would effectively mean that the device has currently no way to filter a log msg at runtime based on its string-index.

(note to myself: Here's how pigweed does it: https://pigweed.dev/pw_log_tokenized/#configuring-metadata-bit-fields )

In the meantime I've implemented a defmt log sink which stores all logs in the flash memory of the device. I've also implemented the on-demand extraction of the stored logs in order to send them to the cloud. I plan to eventually open-source these components. I think with the current data-format I can filter the messages by timestamp at runtime. But I would be nice if I could also filter at least by log-level, better even by modules...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants