Discussion: Redefine variant bit (111) definition

[kyzer-davis]

Continuation of separate #24 thread

Question:
Should we redefine UUID variant bits 111 (E/F) which are currently "Reserved for future definition." as per the original RFC 4122 Section 4.1.1 source?

---

Proposal:

• In Draft 02 set definition of any UUID Version (UUIDv1/2/3/4/5/6/7/8) + Variant E (111) as a method for signaling an alternative bit layout to any previously defined UUID.
• With that precedent set UUIDv6 could be converted to UUIDv1 (0001) + Variant E/F (111) as a method to signal the alternative encoding for UUIDv1 labeled as UUIDv1ε.

---

Possible text changes depending on feedback from this issue and #24

As such the Draft 01 goes from the current three definitions:

Name	Version	Variant	Description
UUIDv6	0110	10x (8/9/A/B)	UUIDv1 with Re-ordered Gregorian timestamp, explicit start sequence counter, no MAC address
UUIDv7	0111	10x (8/9/A/B)	36-bit Unix epoch timestamp, variable subsecond encoding up to nanoseconds using floating point math and fractions (38 bits allocated to subsecond precision).
UUIDv8	1000	10x (8/9/A/B)	Relaxed implementation, any timestamp goes, future proof the specification, 122 bits to do as you desire with general guidelines of timestamp, sequence, random in that order

To the possible four definitions in Draft 02:

Name	Version	Variant	Description
UUIDv1ε	0001	111 (E/F)	UUIDv1 with Re-ordered Gregorian timestamp, explicit start sequence counter, no MAC address. (What is UUIDv6 in draft 01)
UUIDv6	0110	10x (8/9/A/B)	36-bit Unix epoch timestamp, variable subsecond encoding up to nanoseconds using floating point math and binary fractions (38 bits allocated to subsecond precision). (What is UUIDv7 in draft 01)
UUIDv6ε	0110	111 (E/F)	UUIDv6 with 36-bit Unix epoch timestamp, variable subsecond encoding up to nanoseconds using integers to represent total number of subseconds (30 bits allocated to subsecond precision). (Did not exist in draft 01)
UUIDv7	0111	10x (8/9/A/B)	Relaxed implementation, any timestamp goes, future proof the specification, 122 bits to do as you desire with general guidelines of timestamp, sequence, random in that order. (What was UUIDv8 in draft 01)
UUIDv8	1000	10x (8/9/A/B)	Goes away in draft 02 as it is no longer required.

[nerg4l]

Could you clarify for me what does "variable subsecond encoding up to nanoseconds using floating point math and fractions" and "variable subsecond encoding up to nanoseconds using integers to represent total number of subseconds" mean or what is the difference between them?

[kyzer-davis]

@nerg4l, the topic being discussed in #24 if we kept the current UUIDv7 and just changed it to UUIDv6 (since UUIDv6 becomes UUIDv1ε). Then we can also add an alternative encoding that uses the 30-bit variant without any floating point math and binary fraction encoding as UUIDv6ε. We get the best of both words. UUIDv7 becomes what was UUIDv8 and I drop UUIDv8 from the draft.

I edited the table to add more clarity.

[bradleypeabody]

@kyzer-davis I was thinking, for simplicity, we would only define a meaning for the variants + versions that we actually want. Meaning basically that UUIDv6 I think stays as it is with the old variant, and UUIDv7 and 8 use the new 111 variant. Otherwise I think we have too much variation without any real benefit.

[nerg4l]

Thanks for the clarification.

I don't think it makes sense to create UUIDv6 and UUIDv6ε. Instead of pleasing everyone we should have a clear decision on which one to have. This would simplify implementations by having one less UUID to implement and would help keeping the RFC less complex. Also UUIDv6ε would probably only apply to nanosecond precision.

[nerg4l]

I looked more into this and found two things which should be taken into consideration.

I'm not sure if it is relevant in case of extending the RFC but ITU also has a UUID definition in X.667. Which states the following:

11.2 All UUIDs conforming to this Recommendation | International Standard shall have variant bits with bit 7 of
octet 7 set to 1 and bit 6 of octet 7 set to 0. Bit 5 of octet 7 is the most significant bit of the Clock Sequence and shall be
set in accordance with 12.4.

NOTE – Bit 5 is listed here as a variant bit because its value distinguishes historical formats. Strictly speaking, it is not part of the variant value for this Recommendation | International Standard, which uses only two bits for the variant.

I also checked how variant#0 (NCS) UUID looked like. It seems, it does not have a version bit. https://opensource.apple.com/source/CF/CF-299.35/Base.subproj/uuid.c.auto.html

 * Internal structure of variant #0 UUIDs
 *
 * The first 6 octets are the number of 4 usec units of time that have
 * passed since 1/1/80 0000 GMT.  The next 2 octets are reserved for
 * future use.  The next octet is an address family.  The next 7 octets
 * are a host ID in the form allowed by the specified address family.
 *
 * Note that while the family field (octet 8) was originally conceived
 * of as being able to hold values in the range [0..255], only [0..13]
 * were ever used.  Thus, the 2 MSB of this field are always 0 and are
 * used to distinguish old and current UUID forms.
 *
 * +--------------------------------------------------------------+
 * |                    high 32 bits of time                      |  0-3  .time_high
 * +-------------------------------+-------------------------------
 * |     low 16 bits of time       |  4-5               .time_low
 * +-------+-----------------------+
 * |         reserved              |  6-7               .reserved
 * +---------------+---------------+
 * |    family     |   8                                .family
 * +---------------+----------...-----+
 * |            node ID               |  9-16           .node
 * +--------------------------...-----+

Unfortunately, I could not find anything specification about Microsofts' variant#2.

A lot of people refer to UUDs defined by RFC 4122 as variant#1 version x UUIDs. RFC4122 also states the following about variant and version:

[...] The UUID format is 16 octets; some bits of the eight octet variant field specified below determine finer structure. [...]

[...] As such, it [variant] could more accurately be called a type field; we retain the original term for compatibility. [...]

[...] The version is more accurately a sub-type; again, we retain the term for compatibility. [...]

There for, I assume using variant#3 would allow to redefine the the structure entirely. Moving or removing version from the definition for example. Probably, current implementations of RFC4122 should be ignored because keeping BC looks impossible.

[edo1]

I assume using variant#3 would allow to redefine the the structure entirely. Moving or removing version from the definition for example

It is the last unused variant, so there is no room for error.

[fabiolimace]

I also think the version bits (subtype) are specific to the RFC4122 variant (type), which has many subtypes that must be separated from each other. Variant 3 (111) doesn't even have a structure yet.

This file appears to be the basis for Apple's implementation: https://github.com/BeyondTrust/pbis-open/blob/master/dcerpc/uuid/uuid.c

[broofa]

Using the variant field to signal different bit semantics within RFC 4122 versions is not appropriate. The variant field is the overarching field that dictates layout and semantics of all other bits in a UUID. RFC4122 is very deliberately scoped to just variant == 0b10x. Hell, version isn't even defined outside of that specific variant.

I believe a more correct approach would be to use a different version to distinguish between timestamp encodings, much the way v3 and v5 distinguish between namespace hash algorithms.

And, yes, this would effectively double the number of new UUID versions being proposed, which is not ideal. This is one reason I'd like to see this proposal culled back to just 1 new timestamp version, per #30.

[edo1]

Using the variant field to signal different bit semantics within RFC 4122 versions is not appropriate.

What about using variant=0b111 in a new format (without version)? Expanding the variable part by three bits reduces the probability of collisions by almost an order of magnitude.

[broofa]

What about using variant=0b111 in a new format (without version)? Expanding the variable part by three bits reduces the probability of collisions by almost an order of magnitude.

You mean the version part? I suppose you could do that. But imho, defining a new variant should be a Big Deal™. It should be motivated by the need for a whole new class of UUIDs, or by having exhausted the available version options, which we haven't done yet. E.g. if there was a need to move the version field to the end of the UUID (to improve db-locality?), or it needed to be 6 or 8-bits wide instead of 4.

I don't see such a need at this time.

[edo1]

But imho, defining a new variant should be a Big Deal™

Agree.
There were no sortable UUIDs in the standard. Is this a Big Deal™?
Seriously though, I want the random part to be as large as possible.

[bradleypeabody]

I tried to lay it out here https://github.com/uuid6/uuid6-ietf-draft/blob/master/LATEST.md as best I could. But the idea is if the variant field is set to 0b111, this would mean the version field fits in the bottom (least significant) bits of the 9th byte (so var and ver are in this one byte). This technically loses us only 1 bit - since variant was 3 bits, and version was 4 but now we using 8.

I agree that we should not try to make new variants of v1, v4, etc. or v6 for that matter (since its goal is to be easily adaptable from and as close to v1 as possible). But I think we can use it in v7 and v8 to simplify the bit layout so there's just one byte you have to worry about when determining version info for v7 and v8:

   0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | var |  ver    |                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

I'm hoping this can help move things toward greater simplicity in the spec/proposal.

[broofa]

I'm hoping this can help move things toward greater simplicity in the spec/proposal.

Before	After
RFC4122 is variant 0b10x	RFC4122 is variant 0b10x or 0b111
2122 UUIDs per version	2122 or 2120 UUIDs per version
version in bits 48-51	version in bits 48-51 or bits 67-70

This is not "greater simplicity". And the rift this creates in how versioning works is going to be an ongoing pain in the ass to articulate and rationalize about.

"Look for versions 1-5 here if variant is 0b10x, and versions 6-31 there if variant is 0b111. What's that...? What if variant is 0b111, but the version field < 6? Uh... well... that's not really a thing. I mean, it's technically possible, but we didn't want to confuse people by having different versions with the same name."

Now that I think about it, that last part about 0b111 versions 1-5, is actually pretty awful. That those particular variant-version combinations are possible just seems ripe for confusion and abuse.