llvm.org GIT mirror llvm / 8e1c20f
[Kaleidoscope][BuildingAJIT] Start filling in text for chapter 3. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275518 91177308-0d34-0410-b5e6-96231b3b80d8 Lang Hames 3 years ago
4 changed file(s) with 155 addition(s) and 31 deletion(s). Raw diff Collapse all Expand all
1515 chapter discusses lazy JITing and shows you how to enable it by adding an ORC
1616 CompileOnDemand layer the JIT from `Chapter 2 `_.
1717
18 Lazy Compilation
19 ================
20
21 When we add a module to the KaleidoscopeJIT class described in Chapter 2 it is
22 immediately optimized, compiled and linked for us by the IRTransformLayer,
23 IRCompileLayer and ObjectLinkingLayer respectively. This scheme, where all the
24 work to make a Module executable is done up front, is relatively simple to
25 understand its performance characteristics are easy to reason about. However,
26 it will lead to very high startup times if the amount of code to be compiled is
27 large, and may also do a lot of unnecessary compilation if only a few compiled
28 functions are ever called at runtime. A truly "just-in-time" compiler should
29 allow us to defer the compilation of any given function until the moment that
30 function is first called, improving launch times and eliminating redundant work.
31 In fact, the ORC APIs provide us with a layer to lazily compile LLVM IR:
32 *CompileOnDemandLayer*.
33
34 The CompileOnDemandLayer conforms to the layer interface described in Chapter 2,
35 but the addModuleSet method behaves quite differently from the layers we have
36 seen so far: rather than doing any work up front, it just constructs a *stub*
37 for each function in the module and arranges for the stub to trigger compilation
38 of the actual function the first time it is called. Because stub functions are
39 very cheap to produce CompileOnDemand's addModuleSet method runs very quickly,
40 reducing the time required to launch the first function to be executed, and
41 saving us from doing any redundant compilation. By conforming to the layer
42 interface, CompileOnDemand can be easily added on top of our existing JIT class.
43 We just need a few changes:
44
45 .. code-block:: c++
46
47 ...
48 #include "llvm/ExecutionEngine/SectionMemoryManager.h"
49 #include "llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h"
50 #include "llvm/ExecutionEngine/Orc/CompileUtils.h"
51 ...
52
53 ...
54 class KaleidoscopeJIT {
55 private:
56 std::unique_ptr TM;
57 const DataLayout DL;
58 std::unique_ptr CompileCallbackManager;
59 ObjectLinkingLayer<> ObjectLayer;
60 IRCompileLayer CompileLayer;
61
62 typedef std::function(std::unique_ptr)>
63 OptimizeFunction;
64
65 IRTransformLayer OptimizeLayer;
66 CompileOnDemandLayer CODLayer;
67
68 public:
69 typedef decltype(CODLayer)::ModuleSetHandleT ModuleHandle;
70
71 First we need to include the CompileOnDemandLayer.h header, then add two new
72 members: a std::unique_ptr and a CompileOnDemandLayer,
73 to our class. The CompileCallbackManager is a utility that enables us to
74 create re-entry points into the compiler for functions that we want to lazily
75 compile. In the next chapter we'll be looking at this class in detail, but for
76 now we'll be treating it as an opaque utility: We just need to pass a reference
77 to it into our new CompileOnDemandLayer, and the layer will do all the work of
78 setting up the callbacks using the callback manager we gave it.
79
80
81 KaleidoscopeJIT()
82 : TM(EngineBuilder().selectTarget()), DL(TM->createDataLayout()),
83 CompileLayer(ObjectLayer, SimpleCompiler(*TM)),
84 OptimizeLayer(CompileLayer,
85 [this](std::unique_ptr M) {
86 return optimizeModule(std::move(M));
87 }),
88 CompileCallbackManager(
89 orc::createLocalCompileCallbackManager(TM->getTargetTriple(), 0)),
90 CODLayer(OptimizeLayer,
91 [this](Function &F) { return std::set({&F}); },
92 *CompileCallbackManager,
93 orc::createLocalIndirectStubsManagerBuilder(
94 TM->getTargetTriple())) {
95 llvm::sys::DynamicLibrary::LoadLibraryPermanently(nullptr);
96 }
97
98 Next we have to update our constructor to initialize the new members. To create
99 an appropriate compile callback manager we use the
100 createLocalCompileCallbackManager function, which takes a TargetMachine and a
101 TargetAddress to call if it receives a request to compile an unknown function.
102 In our simple JIT this situation is unlikely to come up, so we'll cheat and
103 just pass '0' here. In a production quality JIT you could give the address of a
104 function that throws an exception in order to unwind the JIT'd code stack.
105
106 Now we can construct our CompileOnDemandLayer. Following the pattern from
107 previous layers we start by passing a reference to the next layer down in our
108 stack -- the OptimizeLayer. Next we need to supply a 'partitioning function':
109 when a not-yet-compiled function is called, the CompileOnDemandLayer will call
110 this function to ask us what we would like to compile. At a minimum we need to
111 compile the function being called (given by the argument to the partitioning
112 function), but we could also request that the CompileOnDemandLayer compile other
113 functions that are unconditionally called (or highly likely to be called) from
114 the function being called. For KaleidoscopeJIT we'll keep it simple and just
115 request compilation of the function that was called. Next we pass a reference to
116 our CompileCallbackManager. Finally, we need to supply an "indirect stubs
117 manager builder". This is a function that constructs IndirectStubManagers, which
118 are in turn used to build the stubs for each module. The CompileOnDemandLayer
119 will call the indirect stub manager builder once for each call to addModuleSet,
120 and use the resulting indirect stubs manager to create stubs for all functions
121 in all modules added. If/when the module set is removed from the JIT the
122 indirect stubs manager will be deleted, freeing any memory allocated to the
123 stubs. We supply this function by using the
124 createLocalIndirectStubsManagerBuilder utility.
125
126 // ...
127 if (auto Sym = CODLayer.findSymbol(Name, false))
128 // ...
129 return CODLayer.addModuleSet(std::move(Ms),
130 make_unique(),
131 std::move(Resolver));
132 // ...
133
134 // ...
135 return CODLayer.findSymbol(MangledNameStream.str(), true);
136 // ...
137
138 // ...
139 CODLayer.removeModuleSet(H);
140 // ...
141
142 Finally, we need to replace the references to OptimizeLayer in our addModule,
143 findSymbol, and removeModule methods. With that, we're up and running.
144
18145 **To be done:**
19146
20 **(1) Describe lazy function-at-a-time JITing and how it differs from the kind
21 of eager module-at-a-time JITing that we've been doing so far.**
22
23 **(2) Discuss CompileCallbackManagers and IndirectStubManagers.**
24
25 **(3) Describe CompileOnDemandLayer (automates these components and builds stubs
26 and lazy compilation callbacks for IR) and how to add it to the JIT.**
147 ** Discuss CompileCallbackManagers and IndirectStubManagers in more detail.**
27148
28149 Full Code Listing
29150 =================
4141 private:
4242 std::unique_ptr TM;
4343 const DataLayout DL;
44 std::unique_ptr CompileCallbackManager;
4544 ObjectLinkingLayer<> ObjectLayer;
4645 IRCompileLayer CompileLayer;
4746
4948 OptimizeFunction;
5049
5150 IRTransformLayer OptimizeLayer;
51
52 std::unique_ptr CompileCallbackManager;
5253 CompileOnDemandLayer CODLayer;
5354
5455 public:
5657
5758 KaleidoscopeJIT()
5859 : TM(EngineBuilder().selectTarget()), DL(TM->createDataLayout()),
59 CompileCallbackManager(
60 orc::createLocalCompileCallbackManager(TM->getTargetTriple(), 0)),
6160 CompileLayer(ObjectLayer, SimpleCompiler(*TM)),
6261 OptimizeLayer(CompileLayer,
6362 [this](std::unique_ptr M) {
6463 return optimizeModule(std::move(M));
6564 }),
65 CompileCallbackManager(
66 orc::createLocalCompileCallbackManager(TM->getTargetTriple(), 0)),
6667 CODLayer(OptimizeLayer,
6768 [this](Function &F) { return std::set({&F}); },
6869 *CompileCallbackManager,
6565 private:
6666 std::unique_ptr TM;
6767 const DataLayout DL;
68 ObjectLinkingLayer<> ObjectLayer;
69 IRCompileLayer CompileLayer;
70
71 typedef std::function(std::unique_ptr)>
72 OptimizeFunction;
73
74 IRTransformLayer OptimizeLayer;
75
6876 std::unique_ptr CompileCallbackMgr;
6977 std::unique_ptr IndirectStubsMgr;
70 ObjectLinkingLayer<> ObjectLayer;
71 IRCompileLayer CompileLayer;
72
73 typedef std::function(std::unique_ptr)>
74 OptimizeFunction;
75
76 IRTransformLayer OptimizeLayer;
7778
7879 public:
7980 typedef decltype(OptimizeLayer)::ModuleSetHandleT ModuleHandle;
8182 KaleidoscopeJIT()
8283 : TM(EngineBuilder().selectTarget()),
8384 DL(TM->createDataLayout()),
84 CompileCallbackMgr(
85 orc::createLocalCompileCallbackManager(TM->getTargetTriple(), 0)),
8685 CompileLayer(ObjectLayer, SimpleCompiler(*TM)),
8786 OptimizeLayer(CompileLayer,
8887 [this](std::unique_ptr M) {
8988 return optimizeModule(std::move(M));
90 }) {
89 }),
90 CompileCallbackMgr(
91 orc::createLocalCompileCallbackManager(TM->getTargetTriple(), 0)) {
9192 auto IndirectStubsMgrBuilder =
9293 orc::createLocalIndirectStubsManagerBuilder(TM->getTargetTriple());
9394 IndirectStubsMgr = IndirectStubsMgrBuilder();
6868
6969 class KaleidoscopeJIT {
7070 private:
71 MyRemote &Remote;
7271 std::unique_ptr TM;
7372 const DataLayout DL;
73 ObjectLinkingLayer<> ObjectLayer;
74 IRCompileLayer CompileLayer;
75
76 typedef std::function(std::unique_ptr)>
77 OptimizeFunction;
78
79 IRTransformLayer OptimizeLayer;
80
7481 JITCompileCallbackManager *CompileCallbackMgr;
7582 std::unique_ptr IndirectStubsMgr;
76 ObjectLinkingLayer<> ObjectLayer;
77 IRCompileLayer CompileLayer;
78
79 typedef std::function(std::unique_ptr)>
80 OptimizeFunction;
81
82 IRTransformLayer OptimizeLayer;
83 MyRemote &Remote;
8384
8485 public:
8586 typedef decltype(OptimizeLayer)::ModuleSetHandleT ModuleHandle;
8687
8788 KaleidoscopeJIT(MyRemote &Remote)
88 : Remote(Remote),
89 TM(EngineBuilder().selectTarget()),
89 : TM(EngineBuilder().selectTarget()),
9090 DL(TM->createDataLayout()),
9191 CompileLayer(ObjectLayer, SimpleCompiler(*TM)),
9292 OptimizeLayer(CompileLayer,
9393 [this](std::unique_ptr M) {
9494 return optimizeModule(std::move(M));
95 }) {
95 }),
96 Remote(Remote) {
9697 auto CCMgrOrErr = Remote.enableCompileCallbacks(0);
9798 if (!CCMgrOrErr) {
9899 logAllUnhandledErrors(CCMgrOrErr.takeError(), errs(),