Run one testsuite with observers enabled in CI

[bwoebi]

Running one full test suite with an observer doing nothing, to make sure that we do have proper coverage of them.

[iluuu1994]

Might it make sense to test this on x86_64 in nightly instead? E.g. .github/nightly_matrix.php, under get_matrix_include(), _VARIATION. That one also runs without opcache, with opcache and with JIT(s).

[bwoebi]

I personally would like to see it run in a push-job, with --repeat 2, and not nightly only.
(Because it turns out that's the configuration which catches most errors)

[iluuu1994]

Alright, lgtm then 🙂

[dstogov]

@bwoebi Do you really like to run this on ARM only?

[bwoebi]

@dstogov I suppose we can add it on one x86 job as well. Makes sense.

[iluuu1994]

@bwoebi You can add it to either the nightly repeat or variation job.

[dstogov]

I also think it make more sense to setup specialized nightly jobs.

[iluuu1994]

@dstogov The variation job enables multiple debug checks, I think it's not too out of place there. I'm not sure there's a big benefit in testing each separately. Usually it's pretty clear what is causing the issue, from the error message.

[dstogov]

I don't know what is "variation job". Do you mean "Test", "Test Tracing JIT", etc? Of course "Test Observer" and "Tesr Observer + Function JIT" may be add as this kind of sub-jobs.

[iluuu1994]

php-src/.github/nightly_matrix.php

Lines 61 to 70 in dad2d56

	$jobs[] = [
	'name' => '_VARIATION',
	'branch' => $branch,
	'debug' => true,
	'zts' => true,
	'configuration_parameters' => "CFLAGS='-DZEND_RC_DEBUG=1 -DPROFITABILITY_CHECKS=0 -DZEND_VERIFY_FUNC_INFO=1 -DZEND_VERIFY_TYPE_INFERENCE'",
	'timeout_minutes' => 360,
	'test_function_jit' => true,
	'asan' => false,
	];

This job runs tests with additional assertions.

[TimWolla]

see also: #9850 (that one can probably be closed, unless the run-test option is useful?)