llvm.org GIT mirror llvm / cf3bf4f
[docs][ORC] Start work on an ORC design doc. Very much a work in progress. This initial version describes some of the high level design goals and basic symbol lookup rules. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@361089 91177308-0d34-0410-b5e6-96231b3b80d8 Lang Hames 5 months ago
2 changed file(s) with 253 addition(s) and 0 deletion(s). Raw diff Collapse all Expand all
0 ===============================
1 ORC Design and Implementation
2 ===============================
4 Introduction
5 ============
7 This document aims to provide a high-level overview of the ORC APIs and
8 implementation. Except where otherwise stated, all discussion applies to
9 the design of the APIs as of LLVM verison 9 (ORCv2).
11 Use-cases
12 =========
14 ORC aims to provide a modular API for building in-memory compilers,
15 including JIT compilers. There are a wide range of use cases for such
16 in-memory compilers. For example:
18 1. The LLVM tutorials use an in-memory compiler to execute expressions
19 compiled from a toy languge: Kaleidoscope.
21 2. The LLVM debugger, LLDB, uses a cross-compiling in-memory compiler for
22 expression evaluation within the debugger. Here, cross compilation is used
23 to allow expressions compiled within the debugger session to be executed on
24 the debug target, which may be a different device/architecture.
26 3. In high-performance JITs (e.g. JVMs, Julia) that want to make use of LLVM's
27 optimizations within an existing JIT infrastructure.
29 4. In interpreters and REPLs, e.g. Cling (C++) and the Swift interpreter.
31 By adoping a modular, library based design we aim to make ORC useful in as many
32 of these contexts as possible.
34 Features
35 ========
37 ORC provides the following features:
39 - JIT-linking: Allows relocatable object files (COFF, ELF, MachO)[1]_ to be
40 added to a JIT session. The objects will be loaded, linked, and made executable
41 in a target process, which may be the same process that contains the JIT session
42 and linker, or may be another process (even one running on a different machine or
43 architecture) that communicates with the JIT via RPC.
45 - LLVM IR compilation: Off the shelf components (IRCompileLayer,
46 SimpleCompiler, ConcurrentIRCompiler) allow LLVM IR to be added to a JIT session
47 and made executable.
49 - Lazy compilation: ORC provides lazy-compilation stubs that can be used to
50 defer compilation of functions until they are called at runtime.
52 - Custom compilers: Clients can supply custom compilers for each symbol that
53 they define in their JIT session. ORC will run the user-supplied compiler when
54 the a definition of a symbol is needed.
56 - Concurrent JIT'd code and concurrent compilation: Since most compilers are
57 embarrassingly parallel ORC provides off-the-shelf infrastructure for running
58 compilers concurrently and ensures that their work is done before allowing
59 dependent threads of JIT'd code to proceed.
61 - Orthogonality and composability: Each of the features above can be used (or
62 not) independently. It is possible to put ORC components together to make a
63 non-lazy, in-process, single threaded JIT or a lazy, out-of-process, concurrent
64 JIT, or anything in between.
66 LLJIT and LLLazyJIT
67 ===================
69 While ORC is a library for building JITs it also provides two basic JIT
70 implementations off-the-shelf. These are useful both as replacements for
71 earlier LLVM JIT APIs (e.g. MCJIT), and as examples of how to build a JIT
72 class out of ORC components.
74 The LLJIT class supports compilation of LLVM IR and linking of relocatable
75 object files. All operations are performed eagerly on symbol lookup (i.e. a
76 symbol's definition is compiled as soon as you attempt to look up its address).
78 The LLLazyJIT extends LLJIT to add lazy compilation of LLVM IR. When an LLVM
79 IR module is added via the addLazyIRModule method, function bodies in that
80 module will not be compiled until they are first called.
82 Design Overview
83 ===============
85 ORC's JIT'd program model aims to emulate the linking and symbol resolution
86 rules used by the static and dynamic linkers. This allows ORC to JIT LLVM
87 IR (which was designed for static compilation) naturally, including support
88 for linker-specific constructs like weak symbols, symbol linkage, and
89 visibility. To see how this works, imagine a program ``foo`` which links
90 against a pair of dynamic libraries: ``libA`` and ``libB``. On the command
91 line building this system might look like:
94 .. code-block:: bash
96 $ clang++ -shared -o libA.dylib a1.cpp a2.cpp
97 $ clang++ -shared -o libB.dylib b1.cpp b2.cpp
98 $ clang++ -o myapp myapp.cpp -L. -lA -lB
99 $ ./myapp
101 This would translate into ORC API calls on a "CXXCompilingLayer"
102 (with error-check omitted for brevity) as:
104 .. code-block:: c++
106 ExecutionSession ES;
107 RTDyldObjectLinkingLayer ObjLinkingLayer(
108 ES, []() { return llvm::make_unique(); });
109 CXXCompileLayer CXXLayer(ES, ObjLinkingLayer);
111 // Create JITDylib "A" and add code to it using the CXX layer.
112 auto &LibA = ES.createJITDylib("A");
113 CXXLayer.add(LibA, MemoryBuffer::getFile("a1.cpp"));
114 CXXLayer.add(LibA, MemoryBuffer::getFile("a2.cpp"));
116 // Create JITDylib "B" and add code to it using the CXX layer.
117 auto &LibB = ES.createJITDylib("B");
118 CXXLayer.add(LibB, MemoryBuffer::getFile("b1.cpp"));
119 CXXLayer.add(LibB, MemoryBuffer::getFile("b2.cpp"));
121 // Specify the search order for the main JITDylib. This is equivalent to a
122 // "links against" relationship in a command-line link.
123 ES.getMainJITDylib().setSearchOrder({{&LibA, false}, {&LibB, false}});
124 CXXLayer.add(ES.getMainJITDylib(), MemoryBuffer::getFile("main.cpp"));
126 // Look up the JIT'd main, cast it to a function pointer, then call it.
127 auto MainSym = ExitOnErr(ES.lookup({&ES.getMainJITDylib()}, "main"));
128 auto *Main = (int(*)(int, char*[]))MainSym.getAddress();
130 int Result = Main(...);
133 How and when the JIT compilation in this example occurs would depend on the
134 implementation of the hypothetical CXXCompilingLayer, but the linking rules
135 should be the same regardless. For example, if a1.cpp and a2.cpp both define a
136 function "foo" the API should generate a duplicate definition error. On the
137 other hand, if a1.cpp and b1.cpp both define "foo" there is no error (different
138 dynamic libraries may define the same symbol). If main.cpp refers to "foo", it
139 should bind to the definition in LibA rather than the one in LibB, since
140 main.cpp is part of the "main" dylib, and the main dylib links against LibA
141 before LibB.
143 Many JIT clients will have no need for this strict adherence to the usual
144 ahead-of-time linking rules and should be able to get by just fine by putting
145 all of their code in a single JITDylib. However, clients who want to JIT code
146 for languages/projects that traditionally rely on ahead-of-time linking (e.g.
147 C++) will find that this feature makes life much easier.
149 Symbol lookup in ORC serves two other important functions which we discuss in
150 more detail below: (1) It triggers compilation of the symbol(s) searched for,
151 and (2) it provides the synchronization mechanism for concurrent compilation.
153 When a lookup call is made, it searches for a *set* of requested symbols
154 (single symbol lookup is implemented as a convenience function on top of the
155 bulk-lookup APIs). The *materializers* for these symbols (usually compilers,
156 but in general anything that ultimately writes a usable definition into
157 memory) are collected and passed to the ExecutionSession's
158 dispatchMaterialization method. By performing lookups on multiple symbols at
159 once we ensure that the JIT knows about all required work for that query
160 up-front. By making the dispatchMaterialization function client configurable
161 we make it possible to execute the materializers on multiple threads
162 concurrently.
164 Under the hood, lookup operations are implemented in terms of query objects.
165 The first search for any given symbol triggers *materialization* of that symbol
166 and appends the query to the symbol table entry. Any subsequent lookup for that
167 symbol (lookups can be made from any thread at any time after the JIT is set up)
168 will simply append its query object to the list of queries waiting on that
169 symbol's definition. Once a definition has been materialized ORC will notify all
170 queries that are waiting on it, and once all symbols for a query have been
171 materialized the caller is notified (via a callback) that the query completed
172 successfully (the successful result is a map of symbol names to addresses). If
173 any symbol fails to materialize then all pending queries for that symbol are
174 notified of the failure.
176 Top Level APIs
177 ==============
179 Many of ORC's top-level APIs are visible in the example above:
181 - *ExecutionSession* represents the JIT'd program and provides context for the
182 JIT: It contains the JITDylibs, error reporting mechanisms, and dispatches the
183 materializers.
185 - *JITDylibs* provide the symbol tables.
187 - *Layers* (ObjLinkingLayer and CXXLayer) are wrappers around compilers and
188 allow clients to add uncompiled program representations supported by those
189 compilers to JITDylibs.
191 Several other important APIs are used explicitly. JIT clients need not be aware
192 of them, but Layer authors will use them:
194 - *MaterializationUnit* - When XXXLayer::add is invoked it wraps the given
195 program representation (in this example, C++ source) in a MaterializationUnit,
196 which is then stored in the JITDylib. MaterializationUnits are responsible for
197 describing the definitions they provide, and for unwrapping the program
198 representation and passing it back to the layer when compilation is required
199 (this ownership shuffle makes writing thread-safe layers easier, since the
200 ownership of the program representation will be passed back on the stack,
201 rather than having to be fished out of a Layer member, which would require
202 synchronization).
204 - *MaterializationResponsibility* - When a MaterializationUnit hands a program
205 representation back to the layer it comes with an associated
206 MaterializationResponsibility object. This object tracks the definitions
207 that must be materialized and provides a way to notify the JITDylib once they
208 are either successfully materialized or a failure occurs.
210 Handy utilities
211 ===============
213 TBD: absolute symbols, aliases, off-the-shelf layers.
215 Laziness
216 ========
218 Laziness in ORC is provided by a utility called "lazy-reexports". The aim of
219 this utility is to re-use the synchronization provided by the symbol lookup
220 mechanism to make it safe to lazily compile functions, even if calls to the
221 stub occur simultaneously on multiple threads of JIT'd code. It does this by
222 reducing lazy compilation to symbol lookup: The lazy stub performs a lookup of
223 its underlying definition on first call, updating the function body pointer
224 once the definition is available. If additional calls arrive on other threads
225 while compilation is ongoing they will be safely blocked by the normal lookup
226 synchronization guarantee (no result until the result is safe) and can also
227 proceed as soon as compilation completes.
229 TBD: Usage example.
231 Supporting Custom Compilers
232 ===========================
234 TBD.
236 Low Level (MCJIT style) Use
237 ===========================
239 TBD.
241 Future Features
242 ===============
244 TBD: Speculative compilation. Object Caches.
246 .. [1] Formats/architectures vary in terms of supported features. MachO and
247 ELF tend to have better support than COFF. Patches very welcome!
8888 GetElementPtr
8989 Frontend/PerformanceTips
9090 MCJITDesignAndImplementation
91 ORCv2DesignAndImplementation
9192 CodeOfConduct
9293 CompileCudaWithLLVM
9394 ReportingGuide
373374 :doc:`MCJITDesignAndImplementation`
374375 Describes the inner workings of MCJIT execution engine.
377 :doc:`ORCv2DesignAndImplementation`
378 Describes the design and implementation of the ORC APIs, including some
379 usage examples.
376381 :doc:`BranchWeightMetadata`
377382 Provides information about Branch Prediction Information.