Skip to content

Run one testsuite with observers enabled in CI #13869

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,8 @@ jobs:
-d opcache.enable_cli=1 \
-d opcache.jit_buffer_size=16M \
-d opcache.jit=tracing \
-d zend_test.observer.enabled=1 \
-d zend_test.observer.show_output=0 \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might it make sense to test this on x86_64 in nightly instead? E.g. .github/nightly_matrix.php, under get_matrix_include(), _VARIATION. That one also runs without opcache, with opcache and with JIT(s).

Copy link
Member Author

@bwoebi bwoebi Apr 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I personally would like to see it run in a push-job, with --repeat 2, and not nightly only.
(Because it turns out that's the configuration which catches most errors)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, lgtm then 🙂

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bwoebi Do you really like to run this on ARM only?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dstogov I suppose we can add it on one x86 job as well. Makes sense.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bwoebi You can add it to either the nightly repeat or variation job.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also think it make more sense to setup specialized nightly jobs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dstogov The variation job enables multiple debug checks, I think it's not too out of place there. I'm not sure there's a big benefit in testing each separately. Usually it's pretty clear what is causing the issue, from the error message.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know what is "variation job". Do you mean "Test", "Test Tracing JIT", etc? Of course "Test Observer" and "Tesr Observer + Function JIT" may be add as this kind of sub-jobs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$jobs[] = [
'name' => '_VARIATION',
'branch' => $branch,
'debug' => true,
'zts' => true,
'configuration_parameters' => "CFLAGS='-DZEND_RC_DEBUG=1 -DPROFITABILITY_CHECKS=0 -DZEND_VERIFY_FUNC_INFO=1 -DZEND_VERIFY_TYPE_INFERENCE'",
'timeout_minutes' => 360,
'test_function_jit' => true,
'asan' => false,
];

This job runs tests with additional assertions.

-P -q -x -j2 \
-g FAIL,BORK,LEAK,XLEAK \
--no-progress \
Expand Down
1 change: 1 addition & 0 deletions .github/nightly_matrix.php
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ function get_matrix_include(array $branches) {
'debug' => true,
'zts' => true,
'configuration_parameters' => "CFLAGS='-DZEND_RC_DEBUG=1 -DPROFITABILITY_CHECKS=0 -DZEND_VERIFY_FUNC_INFO=1'",
'run_tests_parameters' => '-d zend_test.observer.enabled=1 -d zend_test.observer.show_output=0',
'timeout_minutes' => 360,
'test_function_jit' => true,
];
Expand Down
1 change: 1 addition & 0 deletions Zend/tests/gh10346.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Florian Sowade
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/ffi/tests/gh12905.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ try {
--INI--
ffi.enable=1
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
zend_test.observer.show_return_value=0
--FILE--
Expand Down
1 change: 1 addition & 0 deletions ext/opcache/tests/gh13712.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ opcache
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
opcache.enable=1
opcache.enable_cli=1
--FILE--
Expand Down
1 change: 1 addition & 0 deletions ext/opcache/tests/jit/ignored_opcodes.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ opcache.jit_buffer_size=1M
opcache.jit=function
;opcache.jit_debug=257
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
zend_test.observer.show_opcode_in_user_handler=ZEND_EXIT, ZEND_BEGIN_SILENCE, ZEND_END_SILENCE
--EXTENSIONS--
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/gh9871.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ zend_test
--INI--
opcache.enable_cli=1
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
zend_test.observer.show_return_value=1
--FILE--
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_backtrace_01.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Show backtrace on init
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
zend_test.observer.show_init_backtrace=1
--FILE--
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_basic_01.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Basic observability of userland functions
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_basic_02.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Basic observability of userland methods
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_basic_03.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Basic observability of includes
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_basic_04.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Basic observability of includes only (no functions)
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_includes=1
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_basic_05.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Basic observability of functions only (no includes)
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_functions=1
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_basic_06.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Basic observability of functions only (with run-time swapping)
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_function_names=foo
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_bug81430_1.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ zend_test
--INI--
memory_limit=20M
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_bug81430_2.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ zend_test
--INI--
memory_limit=20M
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
--SKIPIF--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_bug81435.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ zend_test
--INI--
memory_limit=20M
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_function_names=a,d
opcache.optimization_level=0
--SKIPIF--
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_call_user_func_01.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: call_user_func() from root namespace
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_call_user_func_02.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: call_user_func_array() from root namespace
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_call_user_func_03.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: call_user_func() from namespace
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_call_user_func_04.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: call_user_func_array() from namespace
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_closure_01.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Basic observability of closures
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_closure_02.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Observability of fake closures
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_closure_03.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Observability of closures of builtin functions
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_declarations_01.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Observe function and class declarations
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
zend_test.observer.observe_declaring=1
--FILE--
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_declarations_file_cache.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Observe function and class declarations with file_cache_only
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_declaring=1
opcache.enable_cli=1
opcache.file_cache="{TMP}"
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_error_01.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: End handlers fire after a fatal error
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
zend_test.observer.show_return_value=1
memory_limit=2M
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_error_02.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: End handlers fire after a userland fatal error
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
zend_test.observer.show_return_value=1
--FILE--
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_error_03.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: non-fatal errors do not fire end handlers prematurely
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
zend_test.observer.show_return_value=1
--FILE--
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_error_04.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ zend_test
soap
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
zend_test.observer.show_return_value=1
--FILE--
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_error_05.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: End handlers fire after a userland fatal error
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
zend_test.observer.show_return_value=1
--FILE--
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_eval_01.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Basic eval observability
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_exception_01.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Basic observability of userland functions with uncaught exceptions
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_fiber_01.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Basic fiber switching
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.fiber_init=1
zend_test.observer.fiber_switch=1
zend_test.observer.fiber_destroy=1
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_fiber_02.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Unfinished fiber
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.fiber_switch=1
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_fiber_03.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Nested fibers
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.fiber_switch=1
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_fiber_04.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Nested fibers with unfinished fiber
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.fiber_switch=1
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_fiber_05.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Nested fibers with both unfinished
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.fiber_switch=1
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_fiber_06.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Throwing fiber
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.fiber_switch=1
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_fiber_functions_01.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Basic function observing in fibers
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
zend_test.observer.fiber_init=1
zend_test.observer.fiber_switch=1
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_fiber_functions_02.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Function observing in fibers with unfinished fiber
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
zend_test.observer.fiber_init=1
zend_test.observer.fiber_switch=1
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_fiber_functions_03.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Function observing in fibers with bailout in fiber
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
zend_test.observer.fiber_init=1
zend_test.observer.fiber_switch=1
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_generator_01.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Basic generator observability
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
zend_test.observer.show_return_value=1
--FILE--
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_generator_02.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Generator with explicit return
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
zend_test.observer.show_return_value=1
--FILE--
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_generator_03.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Generator with 'yield from'
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
zend_test.observer.show_return_value=1
--FILE--
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_generator_04.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Generator with manual traversal
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
zend_test.observer.show_return_value=1
--FILE--
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_generator_05.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Generator with uncaught exception
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
zend_test.observer.show_return_value=1
--FILE--
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_magic_01.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Basic magic method observability
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
--FILE--
<?php
Expand Down
1 change: 1 addition & 0 deletions ext/zend_test/tests/observer_opline_01.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Observer: Ensure opline exists on the execute_data
zend_test
--INI--
zend_test.observer.enabled=1
zend_test.observer.show_output=1
zend_test.observer.observe_all=1
zend_test.observer.show_opcode=1
opcache.jit=0
Expand Down
Loading
Loading