[CI] SearchWithMinCompatibleSearchNodeIT testMinVersionAsOldVersion failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-periodic #6692 / 7.1.1_bwc
• elasticsearch-periodic #6600 / 7.4.2_bwc

Reproduction Line:

./gradlew ':qa:mixed-cluster:v7.1.1#mixedClusterTest' -Dtests.class="org.elasticsearch.backwards.SearchWithMinCompatibleSearchNodeIT" -Dtests.method="testMinVersionAsNewVersion" -Dtests.seed=AD85B72E90072786 -Dtests.bwc=true -Dtests.locale=ru -Dtests.timezone=Asia/Chongqing -Druntime.java=22

Applicable branches:
7.17

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0

Issue Reasons:

• [7.17] 2 failures in test testMinVersionAsOldVersion (0.7% fail rate in 268 executions)
• [7.17] 2 failures in pipeline elasticsearch-periodic (13.3% fail rate in 15 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

Pinging @elastic/es-search-foundations (Team:Search Foundations)

[drempapis]

More tests for the SearchWithMinCompatibleSearchNodeIT class: #123707, #123706, and #123705 are failing for the same reason in the same branches. We can safely close those tickets and safely handle all the errors

[drempapis]

The env preparation fails for those tests

java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0 
at __randomizedtesting.SeedInfo.seed([AD85B72E90072786:AC049DD9DEAA5571]:0) 
at jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:100)  
at jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:106)    
at jdk.internal.util.Preconditions.checkIndex(Preconditions.java:302)   
at java.util.Objects.checkIndex(Objects.java:365)   
at java. util.ArrayList.get(ArrayList.java:428)  
at org.elasticsearch.backwards.SearchWithMinCompatibleSearchNodeIT.prepareTestData(SearchWithMinCompatibleSearchNodeIT.java:57)

In the last line, when reading the newVersion

   public void prepareTestData() throws IOException {
        nodes = IndexingIT.buildNodeAndVersions(client());
        numShards = nodes.size();
        numDocs = randomIntBetween(numShards, 16);
        bwcNodes = new ArrayList<>();
        newNodes = new ArrayList<>();
        bwcNodes.addAll(nodes.getBWCNodes());
        newNodes.addAll(nodes.getNewNodes());
        bwcVersion = bwcNodes.get(0).getVersion();
        newVersion = newNodes.get(0).getVersion();
        ...

The same error has occurred in the past: #83903 (comment) and traced here

The tests are not muted in any branch starting from 7.17+
The class has been deleted for 9.0+ branches #114713

In all the cases, this seems to be related to MixedClusterClientYamlTestSuiteIT, e.g., #123704, both classes appear when reporting failures. Christoph had a look in the past

Based on the previous findings, the problem only happens in the 7.17 version for BWC test executions.
The prepareTestData code for the SearchWithMinCompatibleSearchNodeIT class has changed in 8.* versions, and the class has been deleted for 9.* versions.

    public void prepareTestData() throws IOException {
        nodes = MixedClusterTestNodes.buildNodes(client(), BWC_NODES_VERSION);
        numShards = nodes.size();
        numDocs = randomIntBetween(numShards, 16);
        allNodes = new ArrayList<>();
        allNodes.addAll(nodes.getBWCNodes());
        allNodes.addAll(nodes.getNewNodes());

Should we ask for infra support or silently close this issue? According to the release pipeline, branch 7.17 will no longer be maintained starting next month.