[ABI] Rework protocol descriptor metadata.

Reimplement protocol descriptors for Swift protocols as a kind of
context descriptor, dropping the Objective-C protocol compatibility
layout. The new protocol descriptors have several advantages over the
current implementation:

* They drop all of the unused fields required for layout-compatibility
  with Objective-C protocols.
* They encode the full requirement signature of the protocol. This
  maintains more information about the protocol itself, including
  (e.g.) correctly encoding superclass requirements.
* They fit within the general scheme of context descriptors, rather than
  being their own thing, which allows us to share more code with
  nominal type descriptors.
* They only use relative pointers, so they’re smaller and can be placed
  in read-only memory

 Implements rdar://problem/38815359.

Author: DougGregor

Diff for: docs/ABI/TypeMetadata.rst

@@ -410,72 +410,36 @@ See the `protocol descriptor`_ description below.
 Protocol Descriptor
 ~~~~~~~~~~~~~~~~~~~
 
-`Protocol metadata` contains references to zero, one, or more **protocol
-descriptors** that describe the protocols values of the type are required to
-conform to. The protocol descriptor is laid out to be compatible with
-Objective-C ``Protocol`` objects. The layout is as follows:
-
-- An **isa** placeholder is stored at **offset 0**. This field is populated by
-  the Objective-C runtime.
-- The mangled **name** is referenced as a null-terminated C string at
-  **offset 1**.
-- If the protocol inherits one or more other protocols, a pointer to the
-  **inherited protocols list** is stored at **offset 2**. The list starts with
-  the number of inherited protocols as a pointer-sized integer, and is followed
-  by that many protocol descriptor pointers. If the protocol inherits no other
-  protocols, this pointer is null.
-- For an ObjC-compatible protocol, its **required instance methods** are stored
-  at **offset 3** as an ObjC-compatible method list. This is null for native
-  Swift protocols.
-- For an ObjC-compatible protocol, its **required class methods** are stored
-  at **offset 4** as an ObjC-compatible method list. This is null for native
-  Swift protocols.
-- For an ObjC-compatible protocol, its **optional instance methods** are stored
-  at **offset 5** as an ObjC-compatible method list. This is null for native
-  Swift protocols.
-- For an ObjC-compatible protocol, its **optional class methods** are stored
-  at **offset 6** as an ObjC-compatible method list. This is null for native
-  Swift protocols.
-- For an ObjC-compatible protocol, its **instance properties** are stored
-  at **offset 7** as an ObjC-compatible property list. This is null for native
-  Swift protocols.
-- The **size** of the protocol descriptor record is stored as a 32-bit integer
-  at **offset 8**. This is currently 72 on 64-bit platforms and 40 on 32-bit
-  platforms.
-- **Flags** are stored as a 32-bit integer after the size. The following bits
-  are currently used (counting from least significant bit zero):
-
-  * **Bit 0** is the **Swift bit**. It is set for all protocols defined in
-    Swift and unset for protocols defined in Objective-C.
-  * **Bit 1** is the **class constraint bit**. It is set if the protocol is
+Protocol descriptors describe the requirements of a protocol, and act as a
+handle for the protocol itself. The are referenced by `Protocol metadata`_, as
+well as `Protocol Conformance Records`_ and generic requirements. Protocol
+descriptors are only created for non-`@objc` Swift protocols: `@objc` protocols
+are emitted as Objective-C metadata. The layout of Swift protocol descriptors is
+as follows:
+
+- Protocol descriptors are context descriptors, so they are prefixed by context
+  descriptor metadata. (FIXME: these are not yet documented)
+- The 16-bit kind-specific flags of a protocol are defined as follows:
+  * **Bit 0** is the **class constraint bit**. It is set if the protocol is
     **not** class-constrained, meaning that any struct, enum, or class type
     may conform to the protocol. It is unset if only classes can conform to
-    the protocol. (The inverted meaning is for compatibility with Objective-C
-    protocol records, in which the bit is never set. Objective-C protocols can
-    only be conformed to by classes.)
-  * **Bit 2** is the **witness table bit**. It is set if dispatch to the
-    protocol's methods is done through a witness table, which is either passed
-    as an extra parameter to generic functions or included in the existential
-    container layout of protocol types. It is unset if dispatch is done
-    through ``objc_msgSend`` and requires no additional information to accompany
-    a value of conforming type.
-  * **Bit 31** is set by the Objective-C runtime when it has done its
-    initialization of the protocol record. It is unused by the Swift runtime.
-- **Number of mandatory requirements** is stored as a 16-bit integer after
-  the flags. It specifies the number of requirements that do not have default
-  implementations.
-- **Number of requirements** is stored as a 16-bit integer after the flags. It
-  specifies the total number of requirements for the protocol.
-- **Requirements pointer** stored as a 32-bit relative pointer to an array
-  of protocol requirements. The number of elements in the array is specified
-  by the preceding 16-bit integer.
-- **Superclass pointer** stored as a 32-bit relative pointer to class metadata,
-  describing the superclass bound of the protocol.
-- **Associated type names** stored as a 32-bit relative pointer to a
-  null-terminated string. The string contains the names of the associated
-  types, in the order they apparent in the requirements list, separated by
-  spaces.
-
+    the protocol.
+  * **Bit 1** indicates that the protocol is **resilient**.
+  * **Bits 2-7** indicate specify the **special protocol kind**. Only one
+    special protocol kind is defined: the `Error` protocol has value 1.    
+- A pointer to the **name** of the protocol. FIXME: The name is currently
+  mangles, but will change to the (unmangled) name.
+- The number of generic requirements within the **requirement signature** of
+  the protocol. The generic requirements themselves follow the fixed part
+  of the protocol descriptor.
+- The number of **protocol requirements** in the protocol. The protocol
+  requirements follow the generic reuqirements that form the **requirement
+  signature**.
+- A string containing the **associated type names**, a C string comprising the
+  names of all of the associated types in this protocol, separated by spaces,
+  and in the same order as they appear in the protocol requirements.
+- The **generic requirements** that form the **requirement signature**.
+- The **protocol requirements** of the protocol.
 
 Protocol Conformance Records
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Diff for: include/swift/ABI/Metadata.h

@@ -1623,8 +1623,6 @@ class TargetProtocolDescriptorRef {
                         ProtocolDescriptorPointer protocol,
                         ProtocolDispatchStrategy dispatchStrategy) {
 #if SWIFT_OBJC_INTEROP
-    assert(!protocol ||
-           protocol->Flags.getDispatchStrategy() == dispatchStrategy);
     storage = reinterpret_cast<StoredPointer>(protocol)
       | (dispatchStrategy == ProtocolDispatchStrategy::ObjC ? IsObjCBit : 0);
 #else
@@ -1681,7 +1679,8 @@ class TargetProtocolDescriptorRef {
     }
 #endif
 
-    return getSwiftProtocol()->Flags.getClassConstraint();
+    return getSwiftProtocol()->getProtocolContextDescriptorFlags()
+        .getClassConstraint();
   }
 
   /// Determine whether this protocol needs a witness table.
@@ -1702,7 +1701,8 @@ class TargetProtocolDescriptorRef {
     }
 #endif
 
-    return getSwiftProtocol()->Flags.getSpecialProtocol();
+    return getSwiftProtocol()->getProtocolContextDescriptorFlags()
+        .getSpecialProtocol();
   }
 
   /// Retrieve the Swift protocol descriptor.
@@ -1816,76 +1816,7 @@ struct TargetProtocolRequirement {
 
 using ProtocolRequirement = TargetProtocolRequirement<InProcess>;
 
-/// A protocol descriptor. This is not type metadata, but is referenced by
-/// existential type metadata records to describe a protocol constraint.
-/// Its layout is compatible with the Objective-C runtime's 'protocol_t' record
-/// layout.
-template <typename Runtime>
-struct TargetProtocolDescriptor {
-  using StoredPointer = typename Runtime::StoredPointer;
-  /// Unused by the Swift runtime.
-  TargetPointer<Runtime, const void> _ObjC_Isa;
-  
-  /// The mangled name of the protocol.
-  TargetPointer<Runtime, const char> Name;
-  
-  /// The list of protocols this protocol refines.
-  ConstTargetMetadataPointer<Runtime, TargetProtocolDescriptorList>
-    InheritedProtocols;
-  
-  /// Unused by the Swift runtime.
-  TargetPointer<Runtime, const void>
-    _ObjC_InstanceMethods,
-    _ObjC_ClassMethods,
-    _ObjC_OptionalInstanceMethods,
-    _ObjC_OptionalClassMethods,
-    _ObjC_InstanceProperties;
-  
-  /// Size of the descriptor record.
-  uint32_t DescriptorSize;
-  
-  /// Additional flags.
-  ProtocolDescriptorFlags Flags;
-
-  /// The number of requirements described by the Requirements array.
-  /// If any requirements beyond MinimumWitnessTableSizeInWords are present
-  /// in the witness table template, they will be not be overwritten with
-  /// defaults.
-  uint32_t NumRequirements;
-
-  /// Requirement descriptions.
-  RelativeDirectPointer<TargetProtocolRequirement<Runtime>> Requirements;
-
-  /// The superclass of which all conforming types must be a subclass.
-  RelativeDirectPointer<const TargetClassMetadata<Runtime>, /*Nullable=*/true>
-    Superclass;
-
-  /// Associated type names, as a space-separated list in the same order
-  /// as the requirements.
-  RelativeDirectPointer<const char, /*Nullable=*/true> AssociatedTypeNames;
-
-  // This is only used in unittests/Metadata.cpp.
-  constexpr TargetProtocolDescriptor<Runtime>(const char *Name,
-                      const TargetProtocolDescriptorList<Runtime> *Inherited,
-                      ProtocolDescriptorFlags Flags)
-    : _ObjC_Isa(nullptr), Name(Name), InheritedProtocols(Inherited),
-      _ObjC_InstanceMethods(nullptr), _ObjC_ClassMethods(nullptr),
-      _ObjC_OptionalInstanceMethods(nullptr),
-      _ObjC_OptionalClassMethods(nullptr),
-      _ObjC_InstanceProperties(nullptr),
-      DescriptorSize(sizeof(TargetProtocolDescriptor<Runtime>)),
-      Flags(Flags),
-      NumRequirements(0),
-      Requirements(nullptr),
-      Superclass(nullptr),
-      AssociatedTypeNames(nullptr)
-  {}
-
-#ifndef NDEBUG
-  LLVM_ATTRIBUTE_DEPRECATED(void dump() const LLVM_ATTRIBUTE_USED,
-                            "only for use in the debugger");
-#endif
-};
+template<typename Runtime> struct TargetProtocolDescriptor;
 using ProtocolDescriptor = TargetProtocolDescriptor<InProcess>;
 
 /// A witness table for a protocol.
@@ -3001,6 +2932,100 @@ struct TargetAnonymousContextDescriptor final
   }
 };
 
+/// A protocol descriptor.
+///
+/// Protocol descriptors contain information about the contents of a protocol:
+/// it's name, requirements, requirement signature, context, and so on. They
+/// are used both to identify a protocol and to reason about its contents.
+///
+/// Only Swift protocols are defined by a protocol descriptor, whereas
+/// Objective-C (including protocols defined in Swift as @objc) use the
+/// Objective-C protocol layout.
+template<typename Runtime>
+struct TargetProtocolDescriptor final
+    : TargetContextDescriptor<Runtime>,
+      TrailingGenericContextObjects<
+        TargetProtocolDescriptor<Runtime>,
+        TargetGenericContextDescriptorHeader,
+        TargetGenericRequirementDescriptor<Runtime>,
+        TargetProtocolRequirement<Runtime>>
+{
+private:
+  using TrailingObjects
+    = TrailingGenericContextObjects<
+        TargetProtocolDescriptor<Runtime>,
+        TargetGenericContextDescriptorHeader,
+        TargetGenericRequirementDescriptor<Runtime>,
+        TargetProtocolRequirement<Runtime>>;
+
+  friend TrailingObjects;
+
+  template<typename T>
+  using OverloadToken = typename TrailingObjects::template OverloadToken<T>;
+
+public:
+  using TrailingObjects::getGenericContext;
+  using TrailingObjects::numTrailingObjects;
+
+  size_t numTrailingObjects(
+            OverloadToken<TargetGenericRequirementDescriptor<Runtime>>) const {
+    return NumRequirementsInSignature;
+  }
+
+  size_t numTrailingObjects(
+            OverloadToken<TargetProtocolRequirement<Runtime>>) const {
+    return NumRequirements;
+  }
+
+
+  /// The name of the protocol.
+  TargetRelativeDirectPointer<Runtime, const char, /*nullable*/ false> Name;
+
+  /// The number of generic requirements in the requirement signature of the
+  /// protocol.
+  uint32_t NumRequirementsInSignature;
+
+  /// The number of requirements in the protocol.
+  /// If any requirements beyond MinimumWitnessTableSizeInWords are present
+  /// in the witness table template, they will be not be overwritten with
+  /// defaults.
+  uint32_t NumRequirements;
+
+  /// Associated type names, as a space-separated list in the same order
+  /// as the requirements.
+  RelativeDirectPointer<const char, /*Nullable=*/true> AssociatedTypeNames;
+
+  ProtocolContextDescriptorFlags getProtocolContextDescriptorFlags() const {
+    return ProtocolContextDescriptorFlags(this->Flags.getKindSpecificFlags());
+  }
+
+  /// Retrieve the requirements that make up the requirement signature of
+  /// this protocol.
+  llvm::ArrayRef<TargetGenericRequirementDescriptor<Runtime>>
+  getRequirementSignature() const {
+    return {this->template getTrailingObjects<
+                             TargetGenericRequirementDescriptor<Runtime>>(),
+            NumRequirementsInSignature};
+  }
+
+  /// Retrieve the requirements of this protocol.
+  llvm::ArrayRef<TargetProtocolRequirement<Runtime>>
+  getRequirements() const {
+    return {this->template getTrailingObjects<
+                             TargetProtocolRequirement<Runtime>>(),
+            NumRequirements};
+  }
+
+#ifndef NDEBUG
+  LLVM_ATTRIBUTE_DEPRECATED(void dump() const LLVM_ATTRIBUTE_USED,
+                            "only for use in the debugger");
+#endif
+
+  static bool classof(const TargetContextDescriptor<Runtime> *cd) {
+    return cd->getKind() == ContextDescriptorKind::Protocol;
+  }
+};
+
 /// The instantiation cache for generic metadata.  This must be guaranteed
 /// to zero-initialized before it is first accessed.  Its contents are private
 /// to the runtime.

Diff for: include/swift/ABI/MetadataValues.h

@@ -1074,7 +1074,10 @@ enum class ContextDescriptorKind : uint8_t {
   /// This context descriptor represents an anonymous possibly-generic context
   /// such as a function body.
   Anonymous = 2,
-  
+
+  /// This context descriptor represents a protocol context.
+  Protocol = 3,
+
   /// First kind that represents a type of any sort.
   Type_First = 16,
 
@@ -1251,6 +1254,41 @@ class TypeContextDescriptorFlags : public FlagSet<uint16_t> {
                                  class_setSuperclassReferenceKind)
 };
 
+/// Flags for protocol context descriptors. These values are used as the
+/// kindSpecificFlags of the ContextDescriptorFlags for the protocol.
+class ProtocolContextDescriptorFlags : public FlagSet<uint16_t> {
+  enum {
+    /// Whether this protocol is class-constrained.
+    HasClassConstraint = 0,
+    HasClassConstraint_width = 1,
+
+    /// Whether this protocol is resilient.
+    IsResilient = 1,
+
+    /// Special protocol value.
+    SpecialProtocolKind = 2,
+    SpecialProtocolKind_width = 6,
+  };
+
+public:
+  explicit ProtocolContextDescriptorFlags(uint16_t bits) : FlagSet(bits) {}
+  constexpr ProtocolContextDescriptorFlags() {}
+
+  FLAGSET_DEFINE_FLAG_ACCESSORS(IsResilient, isResilient, setIsResilient)
+
+  FLAGSET_DEFINE_FIELD_ACCESSORS(HasClassConstraint,
+                                 HasClassConstraint_width,
+                                 ProtocolClassConstraint,
+                                 getClassConstraint,
+                                 setClassConstraint)
+
+  FLAGSET_DEFINE_FIELD_ACCESSORS(SpecialProtocolKind,
+                                 SpecialProtocolKind_width,
+                                 SpecialProtocol,
+                                 getSpecialProtocol,
+                                 setSpecialProtocol)
+};
+
 enum class GenericParamKind : uint8_t {
   /// A type parameter.
   Type = 0,