fix(core): StreamString performance improvements with large buffers (StreamString::readBytes()) #11229
+21
−0
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
👋 Hello mathieucarbou, we appreciate your contribution to this project! 📘 Please review the project's Contributions Guide for key guidelines on code, documentation, testing, and more. 🖊️ Please also make sure you have read and signed the Contributor License Agreement for this project. Click to see more instructions ...
Review and merge process you can expect ...
|
Test Results 76 files 76 suites 12m 42s ⏱️ Results for commit 436b1ef. ♻️ This comment has been updated with latest results. |
mathieucarbou
force-pushed
the
StreamString
branch
from
a320ffe to
3418c6c
Compare
mathieucarbou
changed the title
mathieucarbou
force-pushed
the
StreamString
branch
from
3418c6c to
436b1ef
Compare
Memory usage test (comparing PR against master branch)The table below shows the summary of memory usage change (decrease - increase) in bytes and percentage for each target.
Click to expand the detailed deltas report [usage change in BYTES]
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
mathieucarbou
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Using
StreamStringon large buffers causes large movements of memory data when reading which can cause the TWDT to trigger, heap fragmentation or crash due to allocation failures.StreamStringis definitely slower than cbuf.h for large buffers.Explanation of the issue with a small MRE: ESP32Async/ESPAsyncWebServer#148 (comment)
This issue was discovered in the OpenDTU project : tbnobody/OpenDTU#2535
Note: the code is not tested yet: this is a proposal meant to open a discussion on a solution.
Sadly it does not solve the issue fully but aims at being backward compatible. It avoids at least to do a memory move for each byte read.
Like explained in the links above, to correcly fix the issue, we would need to make the String immutable when the first read occurs, which implies:
This would allow the StreamString to:
Another implementation version is available here: #11232, which is even faster but not backward compatible if user is accessing the String API after a read.