CodeCacheReclamation.md

Background

The code generated by the compiler is stored in the Code Cache. It is allocated and managed by the compiler. However, by default, the compiler allocates the total code repository during its initialization, and carves up smaller chunks of memory, namely Code Caches. Each Code Cache holds several methods as well as some space for Trampolines.

Because the code repository is allocated at one time, there is a limited amount of space for methods. Additionally, code in these Code Caches can get stale or invalid for several reasons, such as recompilation, redefinition, class unloading, etc. Under these circumstances, the stale/invalid code takes up memory in the Code Cache that could be used for something else.

Code Cache Reclamation is the process by which unreachable code memory is reclaimed for reuse. The code generated for a method can either be reclaimed in a two-step process, or all at once.

Two-Step Reclamation

Code memory is reclaimed in two steps when the code is still reachable by executing threads, either because it is currently being executed, or because it is part of the callstack(s). Under these circumstances, the code must remain available for all such threads (though it will not be reachable via a new invocation). This reclamation process is typically induced by recompilation.

Once it has been determined that the method body is stale, it is added to the Faint Cache Block list. However, not all of the stale body is slated to get reclaimed; the pre-prologue and some number of instructions are kept intact. This is to ensure that other code that has a call to the stale body can continue execution - as described in the Recompilation doc, the Start PC is patched to jump to a helper.

At the end of a GC cycle, jitReleaseCodeStackWalk is invoked, which first walks over the stacks of all executing Java threads to see if any Faint Cache Blocks are no longer alive, i.e. to see if the code that they are associated with is no longer reachable. jitReleaseCodeCollectMetaData is then called to free (i.e. return to the Code Cache for reuse) the code memory of any of the Faint Cache Blocks that are dead.

Wholesale Reclamation

When a class is unloaded, all of the code memory for the compiled methods within that class is reclaimed. Specifically, the reclamation happens when the jitHookClassLoaderUnload hook is invoked by the VM. The same function jitReleaseCodeCollectMetaData is also used in this form of reclamation. Any code stub that is left behind by the Two-Step Reclamation is also reclaimed at this time.

Fragmentation

One notable problem of Code Cache Reclamation, as with any kind of memory reuse, is fragmentation. This is exacerbated by the fact that Two-Step Reclamation leaves behind a code stub. To tackle this problem, the Code Cache allocation routines use heuristics to determine the best block to use when the compiler requests code memory for a compilation.