Unreachable code produced by pattern matching nested variants

[glennsl]

There seems to be three conditions for this to occur:

1. At least three variants on outer level,
2. At least two variants on inner level (other kinds of "complex" pattern matching logic might be sufficient)
3. No common patterns (for at least one of the outer variants)

Example:

type inner =
| P of int
| S of int
  
type outer =
| A of inner
| B of inner
| C of inner

let f = function
(* These cause unreachable code *)
| A P a -> a + a
| A S a -> a - a
(* These don't, because there's commonality between them *)
| B P a | B S a | C P a | C S a -> a * a

which will produce

function f(param) {
  var exit = 0;
  switch (param.tag | 0) {
    case 0 : 
        var match = param[0];
        if (match.tag) {
          var a = match[0];
          return a - a | 0;
        } else {
          var a$1 = match[0];
          return a$1 + a$1 | 0;
        }
        break;
    case 1 : 
    case 2 : 
        exit = 1;
        break;
    
  }
  if (exit === 1) {
    var a$2 = param[0][0];
    return Caml_int32.imul(a$2, a$2);
  }
  
}

Playground

The break; of case 0 is unreachable since all branches of the preceding if/else returns. This is a problem for two reasons:

1. It's bloat, inflating the size of the code for no purpose
2. It messes with test coverage reports

[bobzhang]

TODO:

• merge (exit 2) (exit 2) in early stage, so that we can have better inlining heuristics
• more precise/complete info about when blocks are marked as "return"