Skip to content

Commit a174aa4

Browse files
atrickatrick
committed
Add AST and SILGen support for Builtin.isUnique.
Preparation to fix <rdar://problem/18151694> Add Builtin.checkUnique to avoid lost Array copies. This adds the following new builtins: isUnique : <T> (inout T[?]) -> Int1 isUniqueOrPinned : <T> (inout T[?]) -> Int1 These builtins take an inout object reference and return a boolean. Passing the reference inout forces the optimizer to preserve a retain distinct from what’s required to maintain lifetime for any of the reference's source-level copies, because the called function is allowed to replace the reference, thereby releasing the referent. Before this change, the API entry points for uniqueness checking already took an inout reference. However, after full inlining, it was possible for two source-level variables that reference the same object to appear to be the same variable from the optimizer's perspective because an address to the variable was longer taken at the point of checking uniqueness. Consequently the optimizer could remove "redundant" copies which were actually needed to implement copy-on-write semantics. With a builtin, the variable whose reference is being checked for uniqueness appears mutable at the level of an individual SIL instruction. The kind of reference count checking that Builtin.isUnique performs depends on the argument type: - Native object types are directly checked by reading the strong reference count: (Builtin.NativeObject, known native class reference) - Objective-C object types require an additional check that the dynamic object type uses native swift reference counting: (Builtin.UnknownObject, unknown class reference, class existential) - Bridged object types allow the dymanic object type check to be bypassed based on the pointer encoding: (Builtin.BridgeObject) Any of the above types may also be wrapped in an optional. If the static argument type is optional, then a null check is also performed. Thus, isUnique only returns true for non-null, native swift object references with a strong reference count of one. isUniqueOrPinned has the same semantics as isUnique except that it also returns true if the object is marked pinned regardless of the reference count. This allows for simultaneous non-structural modification of multiple subobjects. In some cases, the standard library can dynamically determine that it has a native reference even though the static type is a bridge or unknown object. Unsafe variants of the builtin are available to allow the additional pointer bit mask and dynamic class lookup to be bypassed in these cases: isUnique_native : <T> (inout T[?]) -> Int1 isUniqueOrPinned_native : <T> (inout T[?]) -> Int1 These builtins perform an implicit cast to NativeObject before checking uniqueness. There’s no way at SIL level to cast the address of a reference, so we need to encapsulate this operation as part of the builtin. Swift SVN r27887
1 parent 152de6b commit a174aa4

25 files changed

+731
-2
lines changed

Diff for: ‎include/swift/AST/Builtins.def

+48
Original file line numberDiff line numberDiff line change
@@ -283,6 +283,54 @@ BUILTIN_SIL_OPERATION(CondFail, "condfail", Special)
283283
/// Fixes the lifetime of any heap references in a value.
284284
BUILTIN_SIL_OPERATION(FixLifetime, "fixLifetime", Special)
285285

286+
/// isUnique : <T> (inout T[?]) -> Int1
287+
///
288+
/// This builtin takes an inout object reference and returns a boolean. Passing
289+
/// the reference inout forces the optimizer to preserve a retain distinct from
290+
/// what’s required to maintain lifetime for any of the reference's source-level
291+
/// copies, because the called function is allowed to replace the reference,
292+
/// thereby releasing the referent.
293+
///
294+
/// The kind of reference count checking that Builtin.isUnique performs depends
295+
/// on the argument type:
296+
///
297+
/// - Native object types are directly checked by reading the
298+
/// strong reference count:
299+
/// (Builtin.NativeObject, known native class reference)
300+
///
301+
/// - Objective-C object types require an additional check that the
302+
/// dynamic object type uses native swift reference counting:
303+
/// (Builtin.UnknownObject, unknown class reference, class existential)
304+
///
305+
/// - Bridged object types allow the dymanic object type check to be
306+
/// passed based on their pointer encoding:
307+
/// (Builtin.BridgeObject)
308+
///
309+
/// Any of the above types may also be wrapped in an optional.
310+
/// If the static argument type is optional, then a null check is also
311+
/// performed.
312+
///
313+
/// Thus, isUnique only returns true for non-null, native swift object
314+
/// references with a strong reference count of one.
315+
BUILTIN_SIL_OPERATION(IsUnique, "isUnique", Special)
316+
317+
/// isUniqueOrPinned : <T> (inout T[?]) -> Int1
318+
///
319+
/// This builtin has the same semantics as isUnique except that it also returns
320+
/// true if the object is marked pinned regardless of the reference count. This
321+
/// allows for simultaneous non-structural modification of multiple subobjects.
322+
BUILTIN_SIL_OPERATION(IsUniqueOrPinned, "isUniqueOrPinned", Special)
323+
324+
/// IsUnique_native : <T> (inout T[?]) -> Int1
325+
///
326+
/// These variants of isUnique implicitly cast to a non-null NativeObject before
327+
/// checking uniqueness. This allows an object reference statically typed as
328+
/// BridgeObject or UnknownObject to be treated as a native object by the
329+
/// runtime.
330+
BUILTIN_SIL_OPERATION(IsUnique_native, "isUnique_native", Special)
331+
BUILTIN_SIL_OPERATION(IsUniqueOrPinned_native, "isUniqueOrPinned_native",
332+
Special)
333+
286334
#undef BUILTIN_SIL_OPERATION
287335

288336
// BUILTIN_RUNTIME_CALL - A call into a runtime function.

Diff for: ‎include/swift/SIL/SILBuilder.h

+9
Original file line numberDiff line numberDiff line change
@@ -935,6 +935,15 @@ class SILBuilder {
935935
SILValue base) {
936936
return insert(new (F.getModule()) MarkDependenceInst(loc, value, base));
937937
}
938+
IsUniqueInst *createIsUnique(SILLocation loc, SILValue operand) {
939+
auto Int1Ty = SILType::getBuiltinIntegerType(1, getASTContext());
940+
return insert(new (F.getModule()) IsUniqueInst(loc, operand, Int1Ty));
941+
}
942+
IsUniqueOrPinnedInst *createIsUniqueOrPinned(SILLocation loc,
943+
SILValue value) {
944+
auto Int1Ty = SILType::getBuiltinIntegerType(1, getASTContext());
945+
return insert(new (F.getModule()) IsUniqueOrPinnedInst(loc, value, Int1Ty));
946+
}
938947

939948
DeallocStackInst *createDeallocStack(SILLocation loc, SILValue operand) {
940949
return insert(new (F.getModule()) DeallocStackInst(loc, operand));

Diff for: ‎include/swift/SIL/SILCloner.h

+14
Original file line numberDiff line numberDiff line change
@@ -1425,6 +1425,20 @@ SILCloner<ImplClass>::visitUnownedReleaseInst(UnownedReleaseInst *Inst) {
14251425
getBuilder().createUnownedRelease(getOpLocation(Inst->getLoc()),
14261426
getOpValue(Inst->getOperand())));
14271427
}
1428+
template<typename ImplClass>
1429+
void SILCloner<ImplClass>::visitIsUniqueInst(IsUniqueInst *Inst) {
1430+
doPostProcess(Inst,
1431+
getBuilder().createIsUnique(getOpLocation(Inst->getLoc()),
1432+
getOpValue(Inst->getOperand())));
1433+
}
1434+
template<typename ImplClass>
1435+
void
1436+
SILCloner<ImplClass>::
1437+
visitIsUniqueOrPinnedInst(IsUniqueOrPinnedInst *Inst) {
1438+
doPostProcess(Inst,
1439+
getBuilder().createIsUniqueOrPinned(getOpLocation(Inst->getLoc()),
1440+
getOpValue(Inst->getOperand())));
1441+
}
14281442

14291443
template<typename ImplClass>
14301444
void

Diff for: ‎include/swift/SIL/SILInstruction.h

+19
Original file line numberDiff line numberDiff line change
@@ -2873,6 +2873,25 @@ class CopyBlockInst :
28732873
: UnaryInstructionBase(loc, operand, operand.getType()) {}
28742874
};
28752875

2876+
/// Given an object reference, return true iff it is non-nil and refers
2877+
/// to a native swift object with strong reference count of 1.
2878+
class IsUniqueInst : public UnaryInstructionBase<ValueKind::IsUniqueInst>
2879+
{
2880+
public:
2881+
IsUniqueInst(SILLocation Loc, SILValue Operand, SILType BoolTy)
2882+
: UnaryInstructionBase(Loc, Operand, BoolTy) {}
2883+
};
2884+
2885+
/// Given an object reference, return true iff it is non-nil and either refers
2886+
/// to a native swift object with strong reference count of 1 or refers to a
2887+
/// pinned object (for simultaneous access to multiple subobjects).
2888+
class IsUniqueOrPinnedInst :
2889+
public UnaryInstructionBase<ValueKind::IsUniqueOrPinnedInst> {
2890+
public:
2891+
IsUniqueOrPinnedInst(SILLocation Loc, SILValue Operand, SILType BoolTy)
2892+
: UnaryInstructionBase(Loc, Operand, BoolTy) {}
2893+
};
2894+
28762895
//===----------------------------------------------------------------------===//
28772896
// DeallocationInsts
28782897
//===----------------------------------------------------------------------===//

Diff for: ‎include/swift/SIL/SILNodes.def

+2
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,8 @@ ABSTRACT_VALUE(SILInstruction, ValueBase)
117117
INST(CopyBlockInst, SILInstruction, MayHaveSideEffects)
118118
INST(StrongPinInst, SILInstruction, MayHaveSideEffects)
119119
INST(StrongUnpinInst, SILInstruction, MayHaveSideEffects)
120+
INST(IsUniqueInst, SILInstruction, MayHaveSideEffects)
121+
INST(IsUniqueOrPinnedInst, SILInstruction, MayHaveSideEffects)
120122

121123
// Literals
122124
ABSTRACT_VALUE(LiteralInst, SILInstruction)

Diff for: ‎lib/AST/Builtins.cpp

+18-1
Original file line numberDiff line numberDiff line change
@@ -659,6 +659,16 @@ static ValueDecl *getTransferArrayOperation(ASTContext &Context, Identifier Id){
659659
return builder.build(Id);
660660
}
661661

662+
static ValueDecl *getIsUniqueOperation(ASTContext &Context, Identifier Id) {
663+
// <T> (@inout T) -> Int1
664+
Type Int1Ty = BuiltinIntegerType::get(1, Context);
665+
666+
GenericSignatureBuilder builder(Context);
667+
builder.addParameter(makeInOut(makeGenericParam()));
668+
builder.setResult(makeConcrete(Int1Ty));
669+
return builder.build(Id);
670+
}
671+
662672
static ValueDecl *getSizeOrAlignOfOperation(ASTContext &Context,
663673
Identifier Id) {
664674
GenericSignatureBuilder builder(Context);
@@ -1445,7 +1455,14 @@ ValueDecl *swift::getBuiltinValueDecl(ASTContext &Context, Identifier Id) {
14451455
case BuiltinValueKind::TakeArrayBackToFront:
14461456
if (!Types.empty()) return nullptr;
14471457
return getTransferArrayOperation(Context, Id);
1448-
1458+
1459+
case BuiltinValueKind::IsUnique:
1460+
case BuiltinValueKind::IsUniqueOrPinned:
1461+
case BuiltinValueKind::IsUnique_native:
1462+
case BuiltinValueKind::IsUniqueOrPinned_native:
1463+
if (!Types.empty()) return nullptr;
1464+
return getIsUniqueOperation(Context, Id);
1465+
14491466
case BuiltinValueKind::Sizeof:
14501467
case BuiltinValueKind::Strideof:
14511468
case BuiltinValueKind::Alignof:

Diff for: ‎lib/IRGen/GenEnum.cpp

+18
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,13 @@ void irgen::EnumImplStrategy::initializeFromParams(IRGenFunction &IGF,
7373
TI->initializeWithTake(IGF, dest, src, T);
7474
}
7575

76+
llvm::Value *irgen::EnumImplStrategy::
77+
loadRefcountedPtr(IRGenFunction &IGF, SourceLoc loc, Address addr) const {
78+
IGF.IGM.error(loc, "Can only load from an address of an optional "
79+
"reference.");
80+
llvm::report_fatal_error("loadRefcountedPtr: Invalid SIL in IRGen");
81+
}
82+
7683
namespace {
7784
/// Implementation strategy for unimplemented enums.
7885
/// Does nothing but produce stub 'undef' values for enum operations.
@@ -2204,6 +2211,13 @@ namespace {
22042211
}
22052212
}
22062213

2214+
llvm::Value *loadRefcountedPtr(IRGenFunction &IGF, SourceLoc loc,
2215+
Address addr) const override {
2216+
// There is no need to bitcast from the enum address. Loading from the
2217+
// reference type emits a bitcast to the proper reference type first.
2218+
return cast<LoadableTypeInfo>(getPayloadTypeInfo()).loadRefcountedPtr(
2219+
IGF, loc, addr).getValue();
2220+
}
22072221
private:
22082222
llvm::ConstantInt *getZeroExtraTagConstant(IRGenModule &IGM) const {
22092223
assert(TIK >= Fixed && "not fixed layout");
@@ -4307,6 +4321,10 @@ namespace {
43074321
const override {
43084322
return Strategy.maskFixedExtraInhabitant(IGF, payload);
43094323
}
4324+
LoadedRef loadRefcountedPtr(IRGenFunction &IGF,
4325+
SourceLoc loc, Address addr) const override {
4326+
return LoadedRef(Strategy.loadRefcountedPtr(IGF, loc, addr), false);
4327+
}
43104328
};
43114329

43124330
/// TypeInfo for dynamically-sized enum types.

Diff for: ‎lib/IRGen/GenEnum.h

+3
Original file line numberDiff line numberDiff line change
@@ -457,6 +457,9 @@ class EnumImplStrategy {
457457

458458
virtual bool needsPayloadSizeInMetadata() const = 0;
459459

460+
virtual llvm::Value *loadRefcountedPtr(IRGenFunction &IGF, SourceLoc loc,
461+
Address addr) const;
462+
460463
private:
461464
EnumImplStrategy(const EnumImplStrategy &) = delete;
462465
EnumImplStrategy &operator=(const EnumImplStrategy &) = delete;

Diff for: ‎lib/IRGen/GenHeap.cpp

+68
Original file line numberDiff line numberDiff line change
@@ -914,6 +914,74 @@ void IRGenFunction::emitUnpin(llvm::Value *value) {
914914
call->setDoesNotThrow();
915915
}
916916

917+
llvm::Value *IRGenFunction::emitLoadNativeRefcountedPtr(Address addr) {
918+
llvm::Value *src =
919+
Builder.CreateBitCast(addr.getAddress(),
920+
IGM.RefCountedPtrTy->getPointerTo());
921+
return Builder.CreateLoad(src);
922+
}
923+
924+
llvm::Value *IRGenFunction::emitLoadUnknownRefcountedPtr(Address addr) {
925+
llvm::Value *src =
926+
Builder.CreateBitCast(addr.getAddress(),
927+
IGM.UnknownRefCountedPtrTy->getPointerTo());
928+
return Builder.CreateLoad(src);
929+
}
930+
931+
llvm::Value *IRGenFunction::emitLoadBridgeRefcountedPtr(Address addr) {
932+
llvm::Value *src =
933+
Builder.CreateBitCast(addr.getAddress(),
934+
IGM.BridgeObjectPtrTy->getPointerTo());
935+
return Builder.CreateLoad(src);
936+
}
937+
938+
llvm::Value *IRGenFunction::
939+
emitIsUniqueCall(llvm::Value *value, SourceLoc loc, bool isNonNull,
940+
bool checkPinned) {
941+
llvm::Constant *fn;
942+
if (value->getType() == IGM.RefCountedPtrTy) {
943+
if (checkPinned) {
944+
if (isNonNull)
945+
fn = IGM.getIsUniquelyReferencedOrPinned_nonNull_nativeFn();
946+
else
947+
fn = IGM.getIsUniquelyReferencedOrPinned_nativeFn();
948+
}
949+
else {
950+
if (isNonNull)
951+
fn = IGM.getIsUniquelyReferenced_nonNull_nativeFn();
952+
else
953+
fn = IGM.getIsUniquelyReferenced_nativeFn();
954+
}
955+
} else if (value->getType() == IGM.UnknownRefCountedPtrTy) {
956+
if (checkPinned) {
957+
if (!isNonNull)
958+
unimplemented(loc, "optional objc ref");
959+
960+
fn = IGM.getIsUniquelyReferencedOrPinnedNonObjC_nonNullFn();
961+
}
962+
else {
963+
if (isNonNull)
964+
fn = IGM.getIsUniquelyReferencedNonObjC_nonNullFn();
965+
else
966+
fn = IGM.getIsUniquelyReferencedNonObjCFn();
967+
}
968+
} else if (value->getType() == IGM.BridgeObjectPtrTy) {
969+
if (!isNonNull)
970+
unimplemented(loc, "optional bridge ref");
971+
972+
if (checkPinned)
973+
fn = IGM.getIsUniquelyReferencedOrPinnedNonObjC_nonNull_bridgeObjectFn();
974+
else
975+
fn = IGM.getIsUniquelyReferencedNonObjC_nonNull_bridgeObjectFn();
976+
} else {
977+
llvm_unreachable("Unexpected LLVM type for a refcounted pointer.");
978+
}
979+
llvm::CallInst *call = Builder.CreateCall(fn, value);
980+
call->setCallingConv(IGM.RuntimeCC);
981+
call->setDoesNotThrow();
982+
return call;
983+
}
984+
917985
#define DEFINE_VALUE_OP(ID) \
918986
void IRGenFunction::emit##ID(llvm::Value *value) { \
919987
if (doesNotRequireRefCounting(value)) return; \

Diff for: ‎lib/IRGen/GenProto.cpp

+5
Original file line numberDiff line numberDiff line change
@@ -1257,6 +1257,11 @@ namespace {
12571257
IGF.emitFixLifetime(value);
12581258
}
12591259

1260+
LoadedRef loadRefcountedPtr(IRGenFunction &IGF, SourceLoc loc,
1261+
Address addr) const override {
1262+
return LoadedRef(IGF.emitLoadUnknownRefcountedPtr(addr), true);
1263+
}
1264+
12601265
const UnownedTypeInfo *
12611266
createUnownedStorageType(TypeConverter &TC) const override {
12621267
// We can just re-use the storage type for the @unowned(safe) type.

Diff for: ‎lib/IRGen/GenType.cpp

+8
Original file line numberDiff line numberDiff line change
@@ -369,6 +369,14 @@ void LoadableTypeInfo::initializeWithCopy(IRGenFunction &IGF,
369369
initialize(IGF, copy, destAddr);
370370
}
371371

372+
LoadedRef LoadableTypeInfo::loadRefcountedPtr(IRGenFunction &IGF,
373+
SourceLoc loc,
374+
Address addr) const {
375+
IGF.IGM.error(loc, "Can only load from an address that holds a reference to "
376+
"a refcounted type or an address of an optional reference.");
377+
llvm::report_fatal_error("loadRefcountedPtr: Invalid SIL in IRGen");
378+
}
379+
372380
static llvm::Constant *asSizeConstant(IRGenModule &IGM, Size size) {
373381
return llvm::ConstantInt::get(IGM.SizeTy, size.getValue());
374382
}

Diff for: ‎lib/IRGen/HeapTypeInfo.h

+16
Original file line numberDiff line numberDiff line change
@@ -253,6 +253,22 @@ class HeapTypeInfo : public SingleScalarTypeInfo<Impl, ReferenceTypeInfo> {
253253
asDerived().emitScalarUnownedRelease(IGF, value);
254254
}
255255

256+
LoadedRef loadRefcountedPtr(IRGenFunction &IGF, SourceLoc loc,
257+
Address addr) const override {
258+
switch (asDerived().getReferenceCounting()) {
259+
case ReferenceCounting::Native:
260+
return LoadedRef(IGF.emitLoadNativeRefcountedPtr(addr), true);;
261+
case ReferenceCounting::ObjC:
262+
case ReferenceCounting::Block:
263+
case ReferenceCounting::Unknown:
264+
return LoadedRef(IGF.emitLoadUnknownRefcountedPtr(addr), true);;
265+
case ReferenceCounting::Bridge:
266+
return LoadedRef(IGF.emitLoadBridgeRefcountedPtr(addr), true);;
267+
case ReferenceCounting::Error:
268+
llvm_unreachable("not supported!");
269+
}
270+
}
271+
256272
const WeakTypeInfo *createWeakStorageType(TypeConverter &TC) const override {
257273
switch (asDerived().getReferenceCounting()) {
258274
case ReferenceCounting::Native:

Diff for: ‎lib/IRGen/IRGenFunction.h

+5
Original file line numberDiff line numberDiff line change
@@ -253,6 +253,11 @@ class IRGenFunction {
253253
void emitUnknownWeakAssign(llvm::Value *value, Address dest);
254254
llvm::Value *emitUnknownWeakLoadStrong(Address src, llvm::Type *type);
255255
llvm::Value *emitUnknownWeakTakeStrong(Address src, llvm::Type *type);
256+
llvm::Value *emitLoadNativeRefcountedPtr(Address addr);
257+
llvm::Value *emitLoadUnknownRefcountedPtr(Address addr);
258+
llvm::Value *emitLoadBridgeRefcountedPtr(Address addr);
259+
llvm::Value *emitIsUniqueCall(llvm::Value *value, SourceLoc loc,
260+
bool isNonNull, bool checkPinned);
256261

257262
//--- Expression emission ------------------------------------------------------
258263
public:

Diff for: ‎lib/IRGen/IRGenSIL.cpp

+29-1
Original file line numberDiff line numberDiff line change
@@ -648,6 +648,8 @@ class IRGenSILFunction :
648648
void visitStrongRetainUnownedInst(StrongRetainUnownedInst *i);
649649
void visitUnownedRetainInst(UnownedRetainInst *i);
650650
void visitUnownedReleaseInst(UnownedReleaseInst *i);
651+
void visitIsUniqueInst(IsUniqueInst *i);
652+
void visitIsUniqueOrPinnedInst(IsUniqueOrPinnedInst *i);
651653
void visitDeallocStackInst(DeallocStackInst *i);
652654
void visitDeallocBoxInst(DeallocBoxInst *i);
653655
void visitDeallocRefInst(DeallocRefInst *i);
@@ -1507,7 +1509,6 @@ void IRGenSILFunction::visitSILBasicBlock(SILBasicBlock *BB) {
15071509
}
15081510
}
15091511
}
1510-
15111512
}
15121513
visit(&I);
15131514
}
@@ -3046,6 +3047,33 @@ void IRGenSILFunction::visitUnownedReleaseInst(swift::UnownedReleaseInst *i) {
30463047
ti.unownedRelease(*this, lowered);
30473048
}
30483049

3050+
static llvm::Value *emitIsUnique(IRGenSILFunction &IGF, SILValue operand,
3051+
SourceLoc loc, bool checkPinned) {
3052+
auto &operTI = cast<LoadableTypeInfo>(IGF.getTypeInfo(operand.getType()));
3053+
LoadedRef ref =
3054+
operTI.loadRefcountedPtr(IGF, loc, IGF.getLoweredAddress(operand));
3055+
3056+
return
3057+
IGF.emitIsUniqueCall(ref.getValue(), loc, ref.isNonNull(), checkPinned);
3058+
}
3059+
3060+
void IRGenSILFunction::visitIsUniqueInst(swift::IsUniqueInst *i) {
3061+
llvm::Value *result = emitIsUnique(*this, i->getOperand(),
3062+
i->getLoc().getSourceLoc(), false);
3063+
Explosion out;
3064+
out.add(result);
3065+
setLoweredExplosion(SILValue(i, 0), out);
3066+
}
3067+
3068+
void IRGenSILFunction::
3069+
visitIsUniqueOrPinnedInst(swift::IsUniqueOrPinnedInst *i) {
3070+
llvm::Value *result = emitIsUnique(*this, i->getOperand(),
3071+
i->getLoc().getSourceLoc(), true);
3072+
Explosion out;
3073+
out.add(result);
3074+
setLoweredExplosion(SILValue(i, 0), out);
3075+
}
3076+
30493077
void IRGenSILFunction::visitAllocStackInst(swift::AllocStackInst *i) {
30503078
const TypeInfo &type = getTypeInfo(i->getElementType());
30513079

0 commit comments

Comments
 (0)