ESP32 with IDF, crash with event source

[dzungpv]

Platform

ESP32

IDE / Tooling

Arduino (IDE/CLI) as ESP IDF

What happened?

Crash with Event Source:
It is my opensource project here: https://github.com/dzungpv/mitsubishi2MQTT
It build with ESP IDF 4.4.6

Stack Trace

Guru Meditation Error: Core  0 panic'ed (LoadProhibited). Exception was unhandled.

Core  0 register dump:
PC      : 0x400f4715  PS      : 0x00060b30  A0      : 0x800f4751  A1      : 0x3ffd3f60  
0x400f4715: std::__uniq_ptr_impl<AsyncEventSourceClient, std::default_delete<AsyncEventSourceClient> >::_M_ptr() const at /Users/vietdzung/.espressif/tools/xtensa-esp32-elf/esp-2021r2-patch5-8.4.0/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/8.4.0/bits/unique_ptr.h:150
 (inlined by) std::unique_ptr<AsyncEventSourceClient, std::default_delete<AsyncEventSourceClient> >::get() const at /Users/vietdzung/.espressif/tools/xtensa-esp32-elf/esp-2021r2-patch5-8.4.0/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/8.4.0/bits/unique_ptr.h:346
 (inlined by) AsyncEventSource::_handleDisconnect(AsyncEventSourceClient*) at /Users/vietdzung/dev/projects/esp32/mitsubishi2MQTT/components/ESPAsyncWebServer/src/AsyncEventSource.cpp:360

A2      : 0x3ffb59f4  A3      : 0x3ffd5048  A4      : 0xfefefefe  A5      : 0x3ffb5a34  
A6      : 0x3ffd44e8  A7      : 0x00000000  A8      : 0x3ffb5a28  A9      : 0xfefefefe  
A10     : 0x3ffb5a28  A11     : 0x3ffd5b34  A12     : 0x3ffd5048  A13     : 0x00000000  
A14     : 0x00000001  A15     : 0x00000001  SAR     : 0x00000019  EXCCAUSE: 0x0000001c  
EXCVADDR: 0xfefeff06  LBEG    : 0x4000c46c  LEND    : 0x4000c477  LCOUNT  : 0x00000000  


Backtrace: 0x400f4712:0x3ffd3f60 0x400f474e:0x3ffd3f80 0x400f475d:0x3ffd3fa0 0x40100b75:0x3ffd3fc0 0x40100ce2:0x3ffd3ff0 0x40100cf8:0x3ffd4010 0x4010109f:0x3ffd4030 0x4010115f:0x3ffd4050 0x4008defd:0x3ffd4080
0x400f4712: AsyncEventSource::_handleDisconnect(AsyncEventSourceClient*) at /Users/vietdzung/dev/projects/esp32/mitsubishi2MQTT/components/ESPAsyncWebServer/src/AsyncEventSource.cpp:359 (discriminator 1)

0x400f474e: AsyncEventSourceClient::_onDisconnect() at /Users/vietdzung/dev/projects/esp32/mitsubishi2MQTT/components/ESPAsyncWebServer/src/AsyncEventSource.cpp:290

0x400f475d: std::_Function_handler<void (void*, AsyncClient*), AsyncEventSourceClient::AsyncEventSourceClient(AsyncWebServerRequest*, AsyncEventSource*)::{lambda(void*, AsyncClient*)#4}>::_M_invoke(std::_Any_data const&, void*&&, AsyncClient*&&) at /Users/vietdzung/dev/projects/esp32/mitsubishi2MQTT/components/ESPAsyncWebServer/src/AsyncEventSource.cpp:170
 (inlined by) _M_invoke at /Users/vietdzung/.espressif/tools/xtensa-esp32-elf/esp-2021r2-patch5-8.4.0/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/8.4.0/bits/std_function.h:297

0x40100b75: std::function<void (void*, AsyncClient*)>::operator()(void*, AsyncClient*) const at /Users/vietdzung/.espressif/tools/xtensa-esp32-elf/esp-2021r2-patch5-8.4.0/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/8.4.0/bits/std_function.h:687

0x40100ce2: AsyncClient::_fin(tcp_pcb*, signed char) at /Users/vietdzung/dev/projects/esp32/mitsubishi2MQTT/components/AsyncTCP/src/AsyncTCP.cpp:934

0x40100cf8: AsyncClient::_s_fin(void*, tcp_pcb*, signed char) at /Users/vietdzung/dev/projects/esp32/mitsubishi2MQTT/components/AsyncTCP/src/AsyncTCP.cpp:1257

0x4010109f: _handle_async_event(lwip_event_packet_t*) at /Users/vietdzung/dev/projects/esp32/mitsubishi2MQTT/components/AsyncTCP/src/AsyncTCP.cpp:167

0x4010115f: _async_service_task(void*) at /Users/vietdzung/dev/projects/esp32/mitsubishi2MQTT/components/AsyncTCP/src/AsyncTCP.cpp:199

0x4008defd: vPortTaskWrapper at /Users/vietdzung/esp/esp-idf-v4.4.6/components/freertos/port/xtensa/port.c:142

Minimal Reproductible Example (MRE)

It is complex web server but you can try to flash it to an ESP32, then set the wifi, after that open Setup -> Wifi page, waiting for some minutes for scan wifi and update to the list, press back and it crash. https://github.com/dzungpv/mitsubishi2MQTT there is guide to build it with ESP IDF

I confirm that:

• I have read the documentation.
• I have searched for similar discussions.
• I have searched for similar issues.
• I have looked at the examples.
• I have upgraded to the lasted version of ESPAsyncWebServer (and AsyncTCP for ESP32).

[mathieucarbou]

• I have upgraded to the lasted version of ESPAsyncWebServer (and AsyncTCP for ESP32).

Hello,

Your issue report is not clear.
The project and branch linked show that you are using outdated versions of the libs.
Please upgrade first => try directly main branch.

Also did you try use core 1 ?

https://github.com/ESP32Async/ESPAsyncWebServer?tab=readme-ov-file#important-recommendations-for-build-options

[mathieucarbou]

The line numbers of your stack trace do not match latest code: can you please upgrade first, retry and send us an updated stack plus also how ro reproduce. Thanks.

[dzungpv]

The line numbers of your stack trace do not match latest code: can you please upgrade first, retry and send us an updated stack plus also how ro reproduce. Thanks.

I will try to reboot my Mac, it seem cached, but the latest commit using your main repo. Old version do not have this problem. As I remember it is 2.0.x version from me-no-dev

[dzungpv]

I have create a branch to test if you want https://github.com/dzungpv/mitsubishi2MQTT/tree/crash.
Just pull that branch with --recursive, then install ESP IDF 4.4.6, build and flash to any esp32 with idf.py flash monitor.
After it boot you can enter it SSID name HVAC_MAC_ADDRESS and set your wifi, save and reboot. if it not show captivate page, enter it IP: 8.8.8.8 on any browser.

• Step to produce crash: after set wifi you will see it IP show in log, enter that IP, open Setup-> Others page and change TX to 19 and RX to 23, Save and reboot.
• After change TX, RX pin, press Control then change the toggle like below picture, it will crash. This project quite complicate so it a short time i could not make it simpler.

Logs:

Guru Meditation Error: Core  0 panic'ed (LoadProhibited). Exception was unhandled.

Core  0 register dump:
PC      : 0x400f45ed  PS      : 0x00060930  A0      : 0x800f4629  A1      : 0x3ffd1320  
0x400f45ed: std::__uniq_ptr_impl<AsyncEventSourceClient, std::default_delete<AsyncEventSourceClient> >::_M_ptr() const at /Users/vietdzung/.espressif/tools/xtensa-esp32-elf/esp-2021r2-patch5-8.4.0/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/8.4.0/bits/unique_ptr.h:150
 (inlined by) std::unique_ptr<AsyncEventSourceClient, std::default_delete<AsyncEventSourceClient> >::get() const at /Users/vietdzung/.espressif/tools/xtensa-esp32-elf/esp-2021r2-patch5-8.4.0/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/8.4.0/bits/unique_ptr.h:346
 (inlined by) AsyncEventSource::_handleDisconnect(AsyncEventSourceClient*) at /Users/vietdzung/dev/projects/esp32/crash/mitsubishi2MQTT/components/ESPAsyncWebServer/src/AsyncEventSource.cpp:360

A2      : 0x3ffb40ec  A3      : 0x3ffd1ca8  A4      : 0xea26b806  A5      : 0x3ffb412c  
A6      : 0x3ffd1cf8  A7      : 0x00000000  A8      : 0x3ffb4120  A9      : 0xea26b806  
A10     : 0x3ffb4120  A11     : 0x3ffbabf0  A12     : 0x3ffd1ca8  A13     : 0x00000000  
A14     : 0x00000001  A15     : 0x00000001  SAR     : 0x0000001c  EXCCAUSE: 0x0000001c  
EXCVADDR: 0xea26b80e  LBEG    : 0x400014fd  LEND    : 0x4000150d  LCOUNT  : 0xfffffffc  


Backtrace: 0x400f45ea:0x3ffd1320 0x400f4626:0x3ffd1340 0x400f4635:0x3ffd1360 0x40100a4d:0x3ffd1380 0x40100bba:0x3ffd13b0 0x40100bd0:0x3ffd13d0 0x40100f77:0x3ffd13f0 0x40101037:0x3ffd1410 0x4008d7e5:0x3ffd1440
0x400f45ea: AsyncEventSource::_handleDisconnect(AsyncEventSourceClient*) at /Users/vietdzung/dev/projects/esp32/crash/mitsubishi2MQTT/components/ESPAsyncWebServer/src/AsyncEventSource.cpp:359 (discriminator 1)

0x400f4626: AsyncEventSourceClient::_onDisconnect() at /Users/vietdzung/dev/projects/esp32/crash/mitsubishi2MQTT/components/ESPAsyncWebServer/src/AsyncEventSource.cpp:290

0x400f4635: std::_Function_handler<void (void*, AsyncClient*), AsyncEventSourceClient::AsyncEventSourceClient(AsyncWebServerRequest*, AsyncEventSource*)::{lambda(void*, AsyncClient*)#4}>::_M_invoke(std::_Any_data const&, void*&&, AsyncClient*&&) at /Users/vietdzung/dev/projects/esp32/crash/mitsubishi2MQTT/components/ESPAsyncWebServer/src/AsyncEventSource.cpp:170
 (inlined by) _M_invoke at /Users/vietdzung/.espressif/tools/xtensa-esp32-elf/esp-2021r2-patch5-8.4.0/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/8.4.0/bits/std_function.h:297

0x40100a4d: std::function<void (void*, AsyncClient*)>::operator()(void*, AsyncClient*) const at /Users/vietdzung/.espressif/tools/xtensa-esp32-elf/esp-2021r2-patch5-8.4.0/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/8.4.0/bits/std_function.h:687

0x40100bba: AsyncClient::_fin(tcp_pcb*, signed char) at /Users/vietdzung/dev/projects/esp32/crash/mitsubishi2MQTT/components/AsyncTCP/src/AsyncTCP.cpp:934

0x40100bd0: AsyncClient::_s_fin(void*, tcp_pcb*, signed char) at /Users/vietdzung/dev/projects/esp32/crash/mitsubishi2MQTT/components/AsyncTCP/src/AsyncTCP.cpp:1257

0x40100f77: _handle_async_event(lwip_event_packet_t*) at /Users/vietdzung/dev/projects/esp32/crash/mitsubishi2MQTT/components/AsyncTCP/src/AsyncTCP.cpp:167

0x40101037: _async_service_task(void*) at /Users/vietdzung/dev/projects/esp32/crash/mitsubishi2MQTT/components/AsyncTCP/src/AsyncTCP.cpp:199

0x4008d7e5: vPortTaskWrapper at /Users/vietdzung/esp/esp-idf-v4.4.6/components/freertos/port/xtensa/port.c:142

[mathieucarbou]

@willmmiles @vortigont : FYI

[dzungpv]

I revert to ESPAsyncWebServer esphome version [2.1.0 and AsyncTCP version 2.0.1 and it not crash anymore

[mathieucarbou]

I revert to ESPAsyncWebServer esphome version [2.1.0 and AsyncTCP version 2.0.1 and it not crash anymore

That's weird because those are crippled with bugs.

The ESP32Async org are now the official libs maintained by me @willmmiles @vortigont @me-no-dev and myself.

I will have a first look.

[mathieucarbou]

@dzungpv : you crash branch still depends on an old version: ESP32Async/ESPAsyncWebServer @ ^3.7.10

Can you try with our main branch:

[env]
lib_deps_ext = 
	ArduinoJson @6.20.0
	https://github.com/bertmelis/espMqttClient
	ESP32Async/ESPAsyncTCP
	https://github.com/ESP32Async/ESPAsyncWebServer
	https://github.com/dzungpv/HeatPump
lib_deps = 
	bblanchon/ArduinoJson @ ^6.21.3
	https://github.com/bertmelis/espMqttClient
	ESP32Async/ESPAsyncTCP
	https://github.com/ESP32Async/ESPAsyncWebServer
	https://github.com/dzungpv/HeatPump

[dzungpv]

@dzungpv : you crash branch still depends on an old version: ESP32Async/ESPAsyncWebServer @ ^3.7.10

Can you try with our main branch:

[env]
lib_deps_ext = 
	ArduinoJson @6.20.0
	https://github.com/bertmelis/espMqttClient
	ESP32Async/ESPAsyncTCP
	https://github.com/ESP32Async/ESPAsyncWebServer
	https://github.com/dzungpv/HeatPump
lib_deps = 
	bblanchon/ArduinoJson @ ^6.21.3
	https://github.com/bertmelis/espMqttClient
	ESP32Async/ESPAsyncTCP
	https://github.com/ESP32Async/ESPAsyncWebServer
	https://github.com/dzungpv/HeatPump

No, I build with ESP and lib is the latest: https://github.com/dzungpv/mitsubishi2MQTT/blob/crash/.gitmodules

[mathieucarbou]

@dzungpv : you crash branch still depends on an old version: ESP32Async/ESPAsyncWebServer @ ^3.7.10
Can you try with our main branch:

[env]
lib_deps_ext = 
	ArduinoJson @6.20.0
	https://github.com/bertmelis/espMqttClient
	ESP32Async/ESPAsyncTCP
	https://github.com/ESP32Async/ESPAsyncWebServer
	https://github.com/dzungpv/HeatPump
lib_deps = 
	bblanchon/ArduinoJson @ ^6.21.3
	https://github.com/bertmelis/espMqttClient
	ESP32Async/ESPAsyncTCP
	https://github.com/ESP32Async/ESPAsyncWebServer
	https://github.com/dzungpv/HeatPump

No, I build with ESP and lib is the latest: https://github.com/dzungpv/mitsubishi2MQTT/blob/crash/.gitmodules

yeah I misread you setup instructions! So you are not using the pio file at all finally ?

note: you can build with esp-idf with pioarduino: it supports hybrid compilation and you can pass any flag.

Examples:

• https://github.com/ESP32Async/ESPAsyncWebServer/blob/main/pioarduino_examples/IncreaseMaxSockets/platformio.ini
• https://github.com/mathieucarbou/MycilaSafeBoot/blob/de33ca8d685a5b0f3245be8b69b361d384591e04/test/platformio.ini#L54-L71

[dzungpv]

@dzungpv : you crash branch still depends on an old version: ESP32Async/ESPAsyncWebServer @ ^3.7.10
Can you try with our main branch:

[env]
lib_deps_ext = 
	ArduinoJson @6.20.0
	https://github.com/bertmelis/espMqttClient
	ESP32Async/ESPAsyncTCP
	https://github.com/ESP32Async/ESPAsyncWebServer
	https://github.com/dzungpv/HeatPump
lib_deps = 
	bblanchon/ArduinoJson @ ^6.21.3
	https://github.com/bertmelis/espMqttClient
	ESP32Async/ESPAsyncTCP
	https://github.com/ESP32Async/ESPAsyncWebServer
	https://github.com/dzungpv/HeatPump

No, I build with ESP and lib is the latest: https://github.com/dzungpv/mitsubishi2MQTT/blob/crash/.gitmodules

yeah I misread you setup instructions! So you are not using the pio file at all finally ?

That is for platformio, I using it for ESP8266, but crash firmware build with ESP IDF, you can check the CI https://github.com/dzungpv/mitsubishi2MQTT/actions/runs/18914692438 and download the crash binary

[mathieucarbou]

I've looked again:

• the line # for ESpAsyncWebServer in the stack trace do not match - you can verify, just open main branch
• the line # for AsyncTCP in the stack trace do not match the code - you can verify, just open main branch
• looked at your project: your use case is really simple, far more simple than any perf test and examples we have in our examples project, so it should work.
• I saw you are using espMqttClient also: make sure you use the latest version of it (it was also bringing an old AsyncTCP before)
• you also do not use espMqttClient with an AsyncClient: you are using it in a synchronous way, so there cannot be between the 2

I would definitely check again the libs.

If you see the stack traces again please scroll to the source code in our main branches to double check the line numbers to verify they are relevant. Right now, they lead to unmatched places.

[dzungpv]

I've looked again:

* the line # for ESpAsyncWebServer in the stack trace do not match - you can verify, just open main branch

* the line # for AsyncTCP in the stack trace do not match the code - you can verify, just open main branch

* looked at your project: your use case is really simple, far more simple than any perf test and examples we have in our examples project, so it should work.

* I saw you are using espMqttClient also: make sure you use the latest version of it (it was also bringing an old AsyncTCP before)

* you also do not use espMqttClient with an AsyncClient: you are using it in a synchronous way, so there cannot be between the 2

I would definitely check again the libs.

If you see the stack traces again please scroll to the source code in our main branches to double check the line numbers to verify they are relevant. Right now, they lead to unmatched places.

Do you try build and produce crash with ESP IDF? I pull your libs using git submodule, so it is the latest code in the main branch, you say the stack trace not match you code, it seem like it has been striped with gcc, when building using ESP IDF it not auto resolve like with platformio. With espMqttClient i use the latest main code too.

[mathieucarbou]

@dzungpv : please follow this discussion here. Moved here because for now this does look like a lib issue more than a real bug in our libs.

[dzungpv]

@mathieucarbou it work now, it is old version tracking with git. Thank you for support.

[mathieucarbou]

Indeed! You just had to look at the source code of asynctcp and ESPAsyncWS to see that your stack did not match 😉

I hate git sub modules myself because it’s a non re releases dependency over an external repo and this is harder to track current version or switch. I do not like to depend on git for that.

Did you know that esp-idf also has a registry and we deploy our libs inside ? You have an example in the project if i remember. It would be easier for you to grab the dependencies from esp-idf registry, or use pioarduino with hybrid compilation.

I don’t see any reason why you would want to make your simple project compile with esp-idf framework which is far more complex than with pioarduino and pioarduino supports hybrid compilation (you can pass flags and ignore modules and it will recompile Arduino libs).

[dzungpv]

Indeed! You just had to look at the source code of asynctcp and ESPAsyncWS to see that your stack did not match 😉

I hate git sub modules myself because it’s a non re releases dependency over an external repo and this is harder to track current version or switch. I do not like to depend on git for that.

Did you know that esp-idf also has a registry and we deploy our libs inside ? You have an example in the project if i remember. It would be easier for you to grab the dependencies from esp-idf registry, or use pioarduino with hybrid compilation.

I don’t see any reason why you would want to make your simple project compile with esp-idf framework which is far more complex than with pioarduino and pioarduino supports hybrid compilation (you can pass flags and ignore modules and it will recompile Arduino libs).

Yes, next time I will use Esp Idf registry to manage this library, better lib management. I like ESP IDF because I can do much more compare to use only Arduino base