jdk_lang_j9 java/lang/invoke/MethodHandles/TestTableSwitch.java Segmentation error vmState=0x00000000

[JasonFengJ9]

Failure link

From https://openj9-jenkins.osuosl.org/job/Test_openjdk24_j9_sanity.openjdk_s390x_linux_Release_testList_0/21/consoleFull (ub20-390-2)

23:53:25  openjdk version "24.0.2-internal" 2025-07-15
23:53:25  OpenJDK Runtime Environment (build 24.0.2-internal-adhoc.****.BuildJDK24s390xlinuxRelease)
23:53:25  Eclipse OpenJ9 VM (build v0.54.0-release-f1f9fee7a8e, JRE 24 Linux s390x-64-Bit Compressed References 20250808_23 (JIT enabled, AOT enabled)
23:53:25  OpenJ9   - f1f9fee7a8e
23:53:25  OMR      - 3f1d7c8b361
23:53:25  JCL      - 905cf12f2c9 based on jdk-24.0.2+12)

Rerun in Grinder - Change TARGET to run only the failed test targets.

Optional info

Failure output (captured from console output)

00:35:15  Running test jdk_lang_j9_0 ...
00:35:15  ===============================================
00:35:15  jdk_lang_j9_0 Start Time: Sat Aug  9 04:35:13 2025 Epoch Time (ms): 1754714113268
00:35:15  variation: -Xdump:system:none -Xdump:heap:none -Xdump:system:events=gpf+abort+traceassert+corruptcache -XX:-JITServerTechPreviewMessage Mode501
00:35:15  JVM_OPTIONS:  -Xdump:system:none -Xdump:heap:none -Xdump:system:events=gpf+abort+traceassert+corruptcache -XX:-JITServerTechPreviewMessage -Xjit -Xgcpolicy:balanced -Xnocompressedrefs -Xverbosegclog 

00:41:12  TEST: java/lang/invoke/MethodHandles/TestTableSwitch.java

00:41:12  Type=Segmentation error vmState=0x00000000
00:41:12  Compiled_method=jdk/internal/classfile/impl/UnboundAttribute$AdHocAttribute.writeTo(Ljdk/internal/classfile/impl/BufWriterImpl;)V
00:41:12  Target=2_90_20250808_23 (Linux 5.4.0-181-generic)
00:41:12  CPU=s390x (4 logical CPUs) (0x1f58f2000 RAM)
00:41:12  ----------- Stack Backtrace -----------
00:41:12   (0x000003FE8D1439C8 [<unknown>+0x0])
00:41:12  ---------------------------------------

50x grinder - https://openj9-jenkins.osuosl.org/job/Grinder/4555/ - the segmentation error wasn't reproduced

[vij-singh]

@r30shah @hzongaro FYI crash on zLinux

[r30shah]

Posting preliminary analysis based on the core-dump here.

0x3fe8d1439ae +682  16:1         e34030680004 lg        %r4, 0x68(%r3)  // getfield 37 jdk/.../DirectCodeBuilder.bytecodesBufWriter (Ljdk/.../BufWriterImpl;)
0x3fe8d1439b4 +688               b9040073     lgr       %r7, %r3
0x3fe8d1439b8 +692               b90400b1     lgr       %r11, %r1
0x3fe8d1439bc +696               a70408e8     brc       0, 0x3fe8d144b8c X>> +5256
0x3fe8d1439c0 +700               b9040073     lgr       %r7, %r3
0x3fe8d1439c4 +704               b90400b1     lgr       %r11, %r1
0x3fe8d1439c8 +708  17:1         bfcf403c     icm       %r12, 0xf, 0x3c(%r4)  // getfield 9 jdk/.../BufWriterImpl.offset (I)

Content in R4 is 0x100000001. Test fails trying to dereference this incorrect address. Looking at the couple of instruction above where R4 is loaded from @(0x68+R3) suggests we are what we think is loading an object field from the reference and try to de-reference that.

(000003FF595B2648h+0x68)/2X
0x000003ff595b26b0: 00000001 00000001

Looking at the object in R3,

> !j9object 0x000003FF595B2648
!J9Object 0x000003FF595B2648 {
	struct J9Class* clazz = !j9class 0x3FF785EE700 // jdk/internal/classfile/impl/StackMapGenerator
	Object flags = 0x00000000;
	J lockword = 0x0000000000000000 (offset = 0) (java/lang/Object) <hidden>
	Ljdk/internal/classfile/impl/StackMapGenerator$Type; thisType = !fj9object 0x3ff595b26f0 (offset = 8) (jdk/internal/classfile/impl/StackMapGenerator)
	Ljava/lang/String; methodName = !fj9object 0x3ff59495750 (offset = 16) (jdk/internal/classfile/impl/StackMapGenerator)
	Ljava/lang/constant/MethodTypeDesc; methodDesc = !fj9object 0x3ff595af7d8 (offset = 24) (jdk/internal/classfile/impl/StackMapGenerator)
	Ljdk/internal/classfile/impl/RawBytecodeHelper$CodeRange; bytecode = !fj9object 0x3ff595b26d0 (offset = 32) (jdk/internal/classfile/impl/StackMapGenerator)
	Ljdk/internal/classfile/impl/SplitConstantPool; cp = !fj9object 0x3ff595a9088 (offset = 40) (jdk/internal/classfile/impl/StackMapGenerator)
	Z isStatic = 0x00000001 (offset = 96) (jdk/internal/classfile/impl/StackMapGenerator) // Offset points to this boolean.
	Ljdk/internal/classfile/impl/LabelContext; labelContext = !fj9object 0x3ff595afdd8 (offset = 48) (jdk/internal/classfile/impl/StackMapGenerator)
	Ljava/util/List; handlers = !fj9object 0x3ff595afeb8 (offset = 56) (jdk/internal/classfile/impl/StackMapGenerator)
	Ljava/util/List; rawHandlers = !fj9object 0x3ff595b2710 (offset = 64) (jdk/internal/classfile/impl/StackMapGenerator)
	Ljdk/internal/classfile/impl/ClassHierarchyImpl; classHierarchy = !fj9object 0x3ff595b2730 (offset = 72) (jdk/internal/classfile/impl/StackMapGenerator)
	Z patchDeadCode = 0x00000001 (offset = 100) (jdk/internal/classfile/impl/StackMapGenerator)
	Z filterDeadLabels = 0x00000000 (offset = 104) (jdk/internal/classfile/impl/StackMapGenerator)
	[Ljdk/internal/classfile/impl/StackMapGenerator$Frame; frames = !fj9object 0x3ff595b2838 (offset = 80) (jdk/internal/classfile/impl/StackMapGenerator)
	I framesCount = 0x00000003 (offset = 108) (jdk/internal/classfile/impl/StackMapGenerator)
	Ljdk/internal/classfile/impl/StackMapGenerator$Frame; currentFrame = !fj9object 0x3ff595b27c0 (offset = 88) (jdk/internal/classfile/impl/StackMapGenerator)
	I maxStack = 0x0000000C (offset = 112) (jdk/internal/classfile/impl/StackMapGenerator)
	I maxLocals = 0x00000014 (offset = 116) (jdk/internal/classfile/impl/StackMapGenerator)
	I exMin = 0x00000078 (offset = 120) (jdk/internal/classfile/impl/StackMapGenerator)
	I exMax = 0xFFFFFFFF (offset = 124) (jdk/internal/classfile/impl/StackMapGenerator)

Offset 0x68 points to a boolean field isStatic [1]. I need to further look into the instructions and bytecode to understand but looking at the bytecodes around the failing instruction, it seems like we are using incorrect object to load a field from.

-1:30    invokevirtual 13 {jdk/internal/classfile/impl/UnboundAttribute$AdHocAttribute.writeBody(Ljdk/internal/classfile/impl/BufWriterImpl;)V}
{  //Inlined #14 0x000003ff785db680 {jdk/internal/classfile/impl/DirectCodeBuilder$4.writeBody}
   14:0     aload0getfield
   14:1     getfield 4 jdk/internal/classfile/impl/DirectCodeBuilder$4.this$0 (Ljdk/internal/classfile/impl/DirectCodeBuilder;)
   14:4     astore2
   14:5     aload1
   14:6     aload2
   14:7     invokevirtual 31 {jdk/internal/classfile/impl/BufWriterImpl.setLabelContext(Ljdk/internal/classfile/impl/LabelContext;)V}
   {  //Inlined #15 0x000003ff785dda88 {jdk/internal/classfile/impl/BufWriterImpl.setLabelContext}
      15:0     aload0
      15:1     aload1
      15:2     putfield 15 jdk/internal/classfile/impl/BufWriterImpl.labelContext (Ljdk/internal/classfile/impl/LabelContext;)
      15:5     return0
   }
   14:10    aload0getfield
   14:11    getfield 4 jdk/internal/classfile/impl/DirectCodeBuilder$4.this$0 (Ljdk/internal/classfile/impl/DirectCodeBuilder;)
   14:14    invokevirtual 32 {jdk/internal/classfile/impl/DirectCodeBuilder.curPc()I}
   {  //Inlined #16 0x000003ff785d5c38 {jdk/internal/classfile/impl/DirectCodeBuilder.curPc}
      16:0     aload0getfield
      16:1     getfield 37 jdk/internal/classfile/impl/DirectCodeBuilder.bytecodesBufWriter (Ljdk/internal/classfile/impl/BufWriterImpl;)
      16:4     invokevirtual 56 {jdk/internal/classfile/impl/BufWriterImpl.size()I}
      {  //Inlined #17 0x000003ff785dde68 {jdk/internal/classfile/impl/BufWriterImpl.size}
         17:0     aload0getfield
         17:1     getfield 9 jdk/internal/classfile/impl/BufWriterImpl.offset (I) <= Failing instruction points to this bytecode offset.
         17:4     return1
      }
      16:7     return1
   }

But from preliminary look, it may not be virtual thread related issue. Tracing back the code to see how we used incorrect object to load from. Also the test has jitdump that I still need to look.

[1]. https://github.com/ibmruntimes/openj9-openjdk-jdk24/blob/905cf12f2c9fa3c4515234d5f07d14d9d4356cfb/src/java.base/share/classes/jdk/internal/classfile/impl/StackMapGenerator.java#L196

[r30shah]

Taking a further look at the disassembly of the failing method (While I am still piecing together how we reached to the failing point),

BEA from the failing job points to the 0x3fe8cf47264 which is exit from method jdk/internal/classfile/impl/SplitConstantPool.canWriteDirect. Looking at the return address stored on the stack before adjustment in the code for canWriteDirect, I can confirm the return address is following,

what (000003FE28007E60h-8)
0x3fe28007e58: 0x000003fe8d143894 {jdk/internal/classfile/impl/UnboundAttribute$AdHocAttribute.writeTo} +400

There are closed to 300 bytes of instructions in between but as BEA points the return to code offset +400, it can be assumed that there was not taken branch in between code from offset +400 to the failing point. So with this information in mind, tracing back the bad value (r3) as described in #22408 (comment).

...
0x3fe8d143894 +400               b9040018     lgr       %r1, %r8 <= Return back to this instruction from call to jdk/internal/classfile/impl/SplitConstantPool.canWriteDirect
0x3fe8d143898 +404               1882         lr        %r8, %r2
0x3fe8d14389a +406               b9040029     lgr       %r2, %r9
...
0x3fe8d1438d8 +468  11:12        e33020300085 lgat      %r3, 0x30(%r2)  // arraylength
...
0x3fe8d143966 +610               e33010180004 lg        %r3, 0x18(%r1) 
...
0x3fe8d14397c +632               e33020200024 stg       %r3, 0x20(%r2)
...
0x3fe8d1439ae +682  16:1         e34030680004 lg        %r4, 0x68(%r3)  // getfield 37 jdk/.../DirectCodeBuilder.bytecodesBufWriter (Ljdk/.../BufWriterImpl;)
...
0x3fe8d1439c8 +708  17:1         bfcf403c     icm       %r12, 0xf, 0x3c(%r4)  // getfield 9 jdk/.../BufWriterImpl.offset (I)

Looking at the dispatch sequence to SplitConstantPool.canWriteDirect to see what we had in GPR8 and GPR9 before the call and what we restored back after returning.

0x3fe8d14378a +134               b9040091     lgr       %r9, %r1
0x3fe8d14378e +138               b9040082     lgr       %r8, %r2
0x3fe8d143792 +142               b9040017     lgr       %r1, %r7
0x3fe8d143796 +146               b9040026     lgr       %r2, %r6
0x3fe8d14379a +150   2:2         c0e5fff92d03 brasl     %r14, 0x3fe8d0691a0 ^{jdk/.../SplitConstantPool.utf8Entry} +12   // invokevirtual 149 {jdk/.../SplitConstantPool.utf8Entry(Ljava/lang/String;)Ljdk/.../AbstractPoolEntry$Utf8EntryImpl;}

From the stack preserved register upon entry to BRASL target, I can find GPR8 and GPR8 to be following values

Please note, inspecting the instructions for the interface dispatch for the two calls after this point, I do not see GPR8 / GPR9 being stored (Or the stack slot being touched).

R8 =
(kca) what 0x000003ff595b2298
tlh(0x3ff78623400) Obj - {jdk/internal/classfile/impl/BufWriterImpl}

R9 =
(kca) what 0x000003ff595b1a48
tlh(0x3ff78623400) Obj - {jdk/internal/classfile/impl/DirectCodeBuilder$4}

0x3fe8d1437a0 +156               b9040019     lgr       %r1, %r9
R1 = Obj - {jdk/internal/classfile/impl/DirectCodeBuilder$4}
0x3fe8d1437a4 +160               b9040072     lgr       %r7, %r2
R2 = Obj - {jdk/internal/classfile/impl/BufWriterImpl}
0x3fe8d1437a8 +164               b9040028     lgr       %r2, %r8
...
R10 = 0x000003ff595a9088 (Obj - {jdk/internal/classfile/impl/SplitConstantPool})
0x3fe8d1437b4 +176               e3a020100004 lg        %r10, 0x10(%r2)
...
0x3fe8d1437be +186               b9040081     lgr       %r8, %r1 <= Preserve GPR1
0x3fe8d1437c2 +190               b9040092     lgr       %r9, %r2 <= Preserve GPR2
0x3fe8d1437c6 +194   5:1         ec6000008070 cgite     %r6, 0 // aload1
0x3fe8d1437cc +200               c478000010da lgrl      %r7, 0x3fe8d145980 J9Class - {jdk/internal/classfile/impl/AbstractPoolEntry$Utf8EntryImpl}  <= GPR7 = 0x000003ff785b5900
0x3fe8d1437d2 +206               b9040016     lgr       %r1, %r6
0x3fe8d1437d6 +210               c040000013ed larl      %r4, 0x3fe8d145fb0 {jdk/.../UnboundAttribute$AdHocAttribute.writeTo} +10412
0x3fe8d1437dc +216               c0e00000002a larl      %r14, 0x3fe8d143830 {jdk/.../UnboundAttribute$AdHocAttribute.writeTo} +300
0x3fe8d1437e2 +222               b90400c1     lgr       %r12, %r1
0x3fe8d1437e6 +226               e37040500021 clg       %r7, 0x50(%r4) @(r4+0x50) = 0x000003ff785b5900
0x3fe8d1437ec +232               e3b040580004 lg        %r11, 0x58(%r4) R11 =  0x000003fe8cf47130 {jdk/internal/classfile/impl/AbstractPoolEntry.constantPool} +6
0x3fe8d1437f2 +238               078b         ber       %r11 <= Call Taken
...
0x3fe8d143830 +300   5:9         e370a000002a lzrg      %r7, 0(%r10) // invokeinterface 15 {java/.../ConstantPoolBuilder.canWriteDirect(Ljava/.../ConstantPool;)Z}   <= R7 = 0x000003ff785be600
0x3fe8d143836 +306               b904001a     lgr       %r1, %r10
0x3fe8d14383a +310               c04000001373 larl      %r4, 0x3fe8d145f20 {jdk/.../UnboundAttribute$AdHocAttribute.writeTo} +10268
0x3fe8d143840 +316               c0e00000002a larl      %r14, 0x3fe8d143894 {jdk/.../UnboundAttribute$AdHocAttribute.writeTo} +400
0x3fe8d143846 +322               b90400c1     lgr       %r12, %r1
0x3fe8d14384a +326               e37040500021 clg       %r7, 0x50(%r4) @(r4+0x50) = 0x000003ff785be600 J9Class - {jdk/internal/classfile/impl/SplitConstantPool}
0x3fe8d143850 +332               e3b040580004 lg        %r11, 0x58(%r4) @(r4+0x58) = 0x000003fe8cf47210
0x3fe8d143856 +338               078b         ber       %r11 // Taken -
...

So bases on the parameters shuffled around and then how BufferWriterImpl object was used after the call at instruction at offset +338, it seems like we register dependency is being swapped hence BufferWriterImpl and this object is being used interchangeably.

Unfortunately jitdump does not resemble same code as the one we fail in. I need to get a compilation log file from the failing run to understand what went wrong (and also this is AOT loaded code so need to get a log file from AOT compilation).

[r30shah]

Launched multiple grinders to get the log file - none failed. Launching another 100x grinder internally.
https://openj9-jenkins.osuosl.org/job/Grinder/4558/ - 50x2
https://openj9-jenkins.osuosl.org/job/Grinder/4557/ - 10x5
https://openj9-jenkins.osuosl.org/job/Grinder/4556/ - 10x1 (Whole jdk_lang test target)

I do not think this should be blocker for JDK24 as I do not see any JDK24 related issue and what I suspect is some RA bug on Z.

[r30shah]

I did launch grinder internally with full test suite for 100x (https://hyc-runtimes-jenkins.swg-devops.com/view/Test_grinder/job/Grinder/53742/), there were other failures in the grinder but I did not get the one reported in this issue. I am going to inspect the core-dump one more time to see if there is a way to easily reproduce this.