llvm.org GIT mirror llvm / f9e7219
Introduce @llvm.experimental.deoptimize Summary: This intrinsic, together with deoptimization operand bundles, allow frontends to express transfer of control and frame-local state from one (typically more specialized, hence faster) version of a function into another (typically more generic, hence slower) version. In languages with a fully integrated managed runtime this intrinsic can be used to implement "uncommon trap" like functionality. In unmanaged languages like C and C++, this intrinsic can be used to represent the slow paths of specialized functions. Note: this change does not address how `@llvm.experimental_deoptimize` is lowered. That will be done in a later change. Reviewers: chandlerc, rnk, atrick, reames Subscribers: llvm-commits, kmod, mjacob, maksfb, mcrosier, JosephTremoulet Differential Revision: http://reviews.llvm.org/D17732 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@263281 91177308-0d34-0410-b5e6-96231b3b80d8 Sanjoy Das 3 years ago
10 changed file(s) with 327 addition(s) and 1 deletion(s). Raw diff Collapse all Expand all
15321532 usual as long as they take into account the first two properties.
15331533
15341534 More specific types of operand bundles are described below.
1535
1536 .. _deopt_opbundles:
15351537
15361538 Deoptimization Operand Bundles
15371539 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1210112103 This intrinsic does nothing, and it's removed by optimizers and ignored
1210212104 by codegen.
1210312105
12106 '``llvm.experimental.deoptimize``' Intrinsic
12107 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
12108
12109 Syntax:
12110 """""""
12111
12112 ::
12113
12114 declare type @llvm.experimental.deoptimize(...) [ "deopt"(...) ]
12115
12116 Overview:
12117 """""""""
12118
12119 This intrinsic, together with :ref:`deoptimization operand bundles
12120 `, allow frontends to express transfer of control and
12121 frame-local state from the currently executing (typically more specialized,
12122 hence faster) version of a function into another (typically more generic, hence
12123 slower) version.
12124
12125 In languages with a fully integrated managed runtime like Java and JavaScript
12126 this intrinsic can be used to implement "uncommon trap" or "side exit" like
12127 functionality. In unmanaged languages like C and C++, this intrinsic can be
12128 used to represent the slow paths of specialized functions.
12129
12130
12131 Arguments:
12132 """"""""""
12133
12134 The intrinsic takes an arbitrary number of arguments, whose meaning is
12135 decided by the :ref:`lowering strategy`.
12136
12137 Semantics:
12138 """"""""""
12139
12140 The ``@llvm.experimental.deoptimize`` intrinsic executes an attached
12141 deoptimization continuation (denoted using a :ref:`deoptimization
12142 operand bundle `) and returns the value returned by
12143 the deoptimization continuation. Defining the semantic properties of
12144 the continuation itself is out of scope of the language reference --
12145 as far as LLVM is concerned, the deoptimization continuation can
12146 invoke arbitrary side effects, including reading from and writing to
12147 the entire heap.
12148
12149 Deoptimization continuations expressed using ``"deopt"`` operand bundles always
12150 continue execution to the end of the physical frame containing them, so all
12151 calls to ``@llvm.experimental.deoptimize`` must be in "tail position":
12152
12153 - ``@llvm.experimental.deoptimize`` cannot be invoked.
12154 - The call must immediately precede a :ref:`ret ` instruction.
12155 - The ``ret`` instruction must return the value produced by the
12156 ``@llvm.experimental.deoptimize`` call if there is one, or void.
12157
12158 Note that the above restrictions imply that the return type for a call to
12159 ``@llvm.experimental.deoptimize`` will match the return type of its immediate
12160 caller.
12161
12162 The inliner composes the ``"deopt"`` continuations of the caller into the
12163 ``"deopt"`` continuations present in the inlinee, and also updates calls to this
12164 intrinsic to return directly from the frame of the function it inlined into.
12165
12166 .. _deoptimize_lowering:
12167
12168 Lowering:
12169 """""""""
12170
12171 Lowering for ``@llvm.experimental.deoptimize`` is not yet implemented,
12172 and is a work in progress.
12173
1210412174 Stack Map Intrinsics
1210512175 --------------------
1210612176
392392 return std::find_if(Range.begin(), Range.end(), Pred);
393393 }
394394
395 /// Provide wrappers to std::remove_if which take ranges instead of having to
396 /// pass begin/end explicitly.
397 template
398 auto remove_if(R &&Range, UnaryPredicate &&P) -> decltype(Range.begin()) {
399 return std::remove_if(Range.begin(), Range.end(), P);
400 }
401
395402 //===----------------------------------------------------------------------===//
396403 // Extra additions to
397404 //===----------------------------------------------------------------------===//
110110 TerminatorInst *getTerminator();
111111 const TerminatorInst *getTerminator() const;
112112
113 /// \brief Returns the call instruction calling @llvm.experimental.deoptimize
114 /// prior to the terminating return instruction of this basic block, if such a
115 /// call is present. Otherwise, returns null.
116 CallInst *getTerminatingDeoptimizeCall();
117 const CallInst *getTerminatingDeoptimizeCall() const {
118 return const_cast(this)->getTerminatingDeoptimizeCall();
119 }
120
113121 /// \brief Returns the call instruction marked 'musttail' prior to the
114122 /// terminating return instruction of this basic block, if such a call is
115123 /// present. Otherwise, returns null.
452452 CALLSITE_DELEGATE_GETTER(getOperandBundle(ID));
453453 }
454454
455 unsigned countOperandBundlesOfType(uint32_t ID) const {
456 CALLSITE_DELEGATE_GETTER(countOperandBundlesOfType(ID));
457 }
458
455459 IterTy arg_begin() const {
456460 CALLSITE_DELEGATE_GETTER(arg_begin());
457461 }
592592 def int_debugtrap : Intrinsic<[]>,
593593 GCCBuiltin<"__builtin_debugtrap">;
594594
595 // Support for dynamic deoptimization (or de-specialization)
596 def int_experimental_deoptimize : Intrinsic<[llvm_any_ty], [llvm_vararg_ty],
597 [Throws]>;
598
595599 // NOP: calls/invokes to this intrinsic are removed by codegen
596600 def int_donothing : Intrinsic<[], [], [IntrNoMem]>;
597601
158158 if (CI->isMustTailCall())
159159 return CI;
160160 }
161 return nullptr;
162 }
163
164 CallInst *BasicBlock::getTerminatingDeoptimizeCall() {
165 if (InstList.empty())
166 return nullptr;
167 auto *RI = dyn_cast(&InstList.back());
168 if (!RI || RI == &InstList.front())
169 return nullptr;
170
171 if (auto *CI = dyn_cast_or_null(RI->getPrevNode()))
172 if (Function *F = CI->getCalledFunction())
173 if (F->getIntrinsicID() == Intrinsic::experimental_deoptimize)
174 return CI;
175
161176 return nullptr;
162177 }
163178
40814081 "masked_store: vector mask must be same length as data", CS);
40824082 break;
40834083 }
4084
4085 case Intrinsic::experimental_deoptimize: {
4086 Assert(CS.isCall(), "experimental_deoptimize cannot be invoked", CS);
4087 Assert(CS.countOperandBundlesOfType(LLVMContext::OB_deopt) == 1,
4088 "experimental_deoptimize must have exactly one "
4089 "\"deopt\" operand bundle");
4090 Assert(CS.getType() == CS.getInstruction()->getFunction()->getReturnType(),
4091 "experimental_deoptimize return type must match caller return type");
4092
4093 if (CS.isCall()) {
4094 auto *DeoptCI = CS.getInstruction();
4095 auto *RI = dyn_cast(DeoptCI->getNextNode());
4096 Assert(RI,
4097 "calls to experimental_deoptimize must be followed by a return");
4098
4099 if (!CS.getType()->isVoidTy() && RI)
4100 Assert(RI->getReturnValue() == DeoptCI,
4101 "calls to experimental_deoptimize must be followed by a return "
4102 "of the value computed by experimental_deoptimize");
4103 }
4104
4105 break;
4106 }
40844107 };
40854108 }
40864109
425425
426426 if (!CI || CI->doesNotThrow() || isa(CI->getCalledValue()))
427427 continue;
428
429 // We do not need to (and in fact, cannot) convert possibly throwing calls
430 // to @llvm.experimental_deoptimize into invokes. The caller's "segment" of
431 // the deoptimization continuation attached to the newly inlined
432 // @llvm.experimental_deoptimize call should contain the exception handling
433 // logic, if any.
434 if (auto *F = CI->getCalledFunction())
435 if (F->getIntrinsicID() == Intrinsic::experimental_deoptimize)
436 continue;
428437
429438 if (auto FuncletBundle = CI->getOperandBundle(LLVMContext::OB_funclet)) {
430439 // This call is nested inside a funclet. If that funclet has an unwind
16121621 replaceDbgDeclareForAlloca(AI, AI, DIB, /*Deref=*/false);
16131622 }
16141623
1615 bool InlinedMustTailCalls = false;
1624 bool InlinedMustTailCalls = false, InlinedDeoptimizeCalls = false;
16161625 if (InlinedFunctionInfo.ContainsCalls) {
16171626 CallInst::TailCallKind CallSiteTailKind = CallInst::TCK_None;
16181627 if (CallInst *CI = dyn_cast(TheCall))
16241633 CallInst *CI = dyn_cast(&I);
16251634 if (!CI)
16261635 continue;
1636
1637 if (Function *F = CI->getCalledFunction())
1638 InlinedDeoptimizeCalls |=
1639 F->getIntrinsicID() == Intrinsic::experimental_deoptimize;
16271640
16281641 // We need to reduce the strength of any inlined tail calls. For
16291642 // musttail, we have to avoid introducing potential unbounded stack
17981811 }
17991812 }
18001813
1814 if (InlinedDeoptimizeCalls) {
1815 // We need to at least remove the deoptimizing returns from the Return set,
1816 // so that the control flow from those returns does not get merged into the
1817 // caller (but terminate it instead). If the caller's return type does not
1818 // match the callee's return type, we also need to change the return type of
1819 // the intrinsic.
1820 if (Caller->getReturnType() == TheCall->getType()) {
1821 auto NewEnd = remove_if(Returns, [](ReturnInst *RI) {
1822 return RI->getParent()->getTerminatingDeoptimizeCall() != nullptr;
1823 });
1824 Returns.erase(NewEnd, Returns.end());
1825 } else {
1826 SmallVector NormalReturns;
1827 Function *NewDeoptIntrinsic = Intrinsic::getDeclaration(
1828 Caller->getParent(), Intrinsic::experimental_deoptimize,
1829 {Caller->getReturnType()});
1830
1831 for (ReturnInst *RI : Returns) {
1832 CallInst *DeoptCall = RI->getParent()->getTerminatingDeoptimizeCall();
1833 if (!DeoptCall) {
1834 NormalReturns.push_back(RI);
1835 continue;
1836 }
1837
1838 auto *CurBB = RI->getParent();
1839 RI->eraseFromParent();
1840
1841 SmallVector CallArgs(DeoptCall->arg_begin(),
1842 DeoptCall->arg_end());
1843
1844 SmallVector OpBundles;
1845 DeoptCall->getOperandBundlesAsDefs(OpBundles);
1846 DeoptCall->eraseFromParent();
1847 assert(!OpBundles.empty() &&
1848 "Expected at least the deopt operand bundle");
1849
1850 IRBuilder<> Builder(CurBB);
1851 Value *NewDeoptCall =
1852 Builder.CreateCall(NewDeoptIntrinsic, CallArgs, OpBundles);
1853 if (NewDeoptCall->getType()->isVoidTy())
1854 Builder.CreateRetVoid();
1855 else
1856 Builder.CreateRet(NewDeoptCall);
1857 }
1858
1859 // Leave behind the normal returns so we can merge control flow.
1860 std::swap(Returns, NormalReturns);
1861 }
1862 }
1863
18011864 // Handle any inlined musttail call sites. In order for a new call site to be
18021865 // musttail, the source of the clone and the inlined call site must have been
18031866 // musttail. Therefore it's safe to return without merging control into the
0 ; RUN: opt -S -always-inline < %s | FileCheck %s
1
2 declare i8 @llvm.experimental.deoptimize.i8(...)
3
4 define i8 @callee(i1* %c) alwaysinline {
5 %c0 = load volatile i1, i1* %c
6 br i1 %c0, label %left, label %right
7
8 left:
9 %c1 = load volatile i1, i1* %c
10 br i1 %c1, label %lleft, label %lright
11
12 lleft:
13 %v0 = call i8(...) @llvm.experimental.deoptimize.i8(i32 1) [ "deopt"(i32 1) ]
14 ret i8 %v0
15
16 lright:
17 ret i8 10
18
19 right:
20 %c2 = load volatile i1, i1* %c
21 br i1 %c2, label %rleft, label %rright
22
23 rleft:
24 %v1 = call i8(...) @llvm.experimental.deoptimize.i8(i32 1, i32 300, float 500.0, <2 x i32*> undef) [ "deopt"(i32 1) ]
25 ret i8 %v1
26
27 rright:
28 %v2 = call i8(...) @llvm.experimental.deoptimize.i8() [ "deopt"(i32 1) ]
29 ret i8 %v2
30 }
31
32 define void @caller_0(i1* %c, i8* %ptr) {
33 ; CHECK-LABEL: @caller_0(
34 entry:
35 %v = call i8 @callee(i1* %c) [ "deopt"(i32 2) ]
36 store i8 %v, i8* %ptr
37 ret void
38
39 ; CHECK: lleft.i:
40 ; CHECK-NEXT: call void (...) @llvm.experimental.deoptimize.isVoid(i32 1) [ "deopt"(i32 2, i32 1) ]
41 ; CHECK-NEXT: ret void
42
43 ; CHECK: rleft.i:
44 ; CHECK-NEXT: call void (...) @llvm.experimental.deoptimize.isVoid(i32 1, i32 300, float 5.000000e+02, <2 x i32*> undef) [ "deopt"(i32 2, i32 1) ]
45 ; CHECK-NEXT: ret void
46
47 ; CHECK: rright.i:
48 ; CHECK-NEXT: call void (...) @llvm.experimental.deoptimize.isVoid() [ "deopt"(i32 2, i32 1) ]
49 ; CHECK-NEXT: ret void
50
51 ; CHECK: callee.exit:
52 ; CHECK-NEXT: store i8 10, i8* %ptr
53 ; CHECK-NEXT: ret void
54
55 }
56
57 define i32 @caller_1(i1* %c, i8* %ptr) personality i8 3 {
58 ; CHECK-LABEL: @caller_1(
59 entry:
60 %v = invoke i8 @callee(i1* %c) [ "deopt"(i32 3) ] to label %normal
61 unwind label %unwind
62
63 ; CHECK: lleft.i:
64 ; CHECK-NEXT: %0 = call i32 (...) @llvm.experimental.deoptimize.i32(i32 1) [ "deopt"(i32 3, i32 1) ]
65 ; CHECK-NEXT: ret i32 %0
66
67 ; CHECK: rleft.i:
68 ; CHECK-NEXT: %1 = call i32 (...) @llvm.experimental.deoptimize.i32(i32 1, i32 300, float 5.000000e+02, <2 x i32*> undef) [ "deopt"(i32 3, i32 1) ]
69 ; CHECK-NEXT: ret i32 %1
70
71 ; CHECK: rright.i:
72 ; CHECK-NEXT: %2 = call i32 (...) @llvm.experimental.deoptimize.i32() [ "deopt"(i32 3, i32 1) ]
73 ; CHECK-NEXT: ret i32 %2
74
75 ; CHECK: callee.exit:
76 ; CHECK-NEXT: br label %normal
77
78 ; CHECK: normal:
79 ; CHECK-NEXT: store i8 10, i8* %ptr
80 ; CHECK-NEXT: ret i32 42
81
82 unwind:
83 %lp = landingpad i32 cleanup
84 ret i32 43
85
86 normal:
87 store i8 %v, i8* %ptr
88 ret i32 42
89 }
0 ; RUN: not opt -verify < %s 2>&1 | FileCheck %s
1
2 declare i8 @llvm.experimental.deoptimize.i8(...)
3 declare void @llvm.experimental.deoptimize.isVoid(...)
4
5 declare void @unknown()
6
7 define void @f_notail() {
8 entry:
9 call void(...) @llvm.experimental.deoptimize.isVoid(i32 0) [ "deopt"() ]
10 ; CHECK: calls to experimental_deoptimize must be followed by a return
11 call void @unknown()
12 ret void
13 }
14
15 define void @f_nodeopt() {
16 entry:
17 call void(...) @llvm.experimental.deoptimize.isVoid()
18 ; CHECK: experimental_deoptimize must have exactly one "deopt" operand bundle
19 ret void
20 }
21
22 define void @f_invoke() personality i8 3 {
23 entry:
24 invoke void(...) @llvm.experimental.deoptimize.isVoid(i32 0, float 0.0) to label %ok unwind label %not_ok
25 ; CHECK: experimental_deoptimize cannot be invoked
26
27 ok:
28 ret void
29
30 not_ok:
31 %0 = landingpad { i8*, i32 }
32 filter [0 x i8*] zeroinitializer
33 ret void
34 }
35
36 define i8 @f_incorrect_return() {
37 entry:
38 %val = call i8(...) @llvm.experimental.deoptimize.i8() [ "deopt"() ]
39 ; CHECK: calls to experimental_deoptimize must be followed by a return of the value computed by experimental_deoptimize
40 ret i8 0
41 }