llvm.org GIT mirror llvm / b5fc9f5
Fix several grammaros and a few HTML usage items. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@29665 91177308-0d34-0410-b5e6-96231b3b80d8 Reid Spencer 13 years ago
1 changed file(s) with 228 addition(s) and 235 deletion(s). Raw diff Collapse all Expand all
11 "http://www.w3.org/TR/html4/strict.dtd">
22
33
4 LLVM Link Time Optimization: <del>design and i</del>mplementation
4 LLVM Link Time Optimization: <ins>Design and I</ins>mplementation
55
66
77
88
9 LLVM Link Time Optimization: design and implementation
9 LLVM Link Time Optimization: Design and Implementation
1010
1111
1212
4444
4545
4646

47 LLVM features powerful intermodular optimization which can be used at link time.
48 Link Time Optimization is another name of intermodular optimization when it
49 is done during link stage. This document describes the interface between LLVM
50 intermodular optimizer and the linker and its design.
47 LLVM features powerful intermodular optimizations which can be used at link
48 time. Link Time Optimization is another name for intermodular optimization
49 when performed during the link stage. This document describes the interface
50 and design between the LLVM intermodular optimizer and the linker.

51
52
53
54
55 Design Philosophy
56
57
58
59
60

61 The LLVM Link Time Optimizer provides complete transparency, while doing
62 intermodular optimization, in the compiler tool chain. Its main goal is to let
63 the developer take advantage of intermodular optimizations without making any
64 significant changes to the developer's makefiles or build system. This is
65 achieved through tight integration with the linker. In this model, the linker
66 treates LLVM bytecode files like native object files and allows mixing and
67 matching among them. The linker uses LLVMlto, a dynamically
68 loaded library, to handle LLVM bytecode files. This tight integration between
69 the linker and LLVM optimizer helps to do optimizations that are not possible
70 in other models. The linker input allows the optimizer to avoid relying on
71 conservative escape analysis.
5172

5273
5374
54
55
56 Design Philosophy
57
58
59
60
61

62 The LLVM Link Time Optimizer seeks complete transparency, while doing intermodular
63 optimization, in compiler tool chain. Its main goal is to let developer take
64 advantage of intermodular optimizer without making any significant changes to
65 their makefiles or build system. This is achieved through tight integration with
66 linker. In this model, linker treates LLVM bytecode files like native objects
67 file and allows mixing and matching among them. The linker uses
68 LLVMlto, a dynamically loaded library, to handle LLVM bytecode
69 files. This tight integration between the linker and LLVM optimizer helps to do
70 optimizations that are not possible in other models. The linker input allows
71 optimizer to avoid relying on conservative escape analysis.
75
76
77 Example of link time optimization
78
79
80
81

The following example illustrates the advantages of LTO's integrated

82 approach and clean interface.

83
84
  • Input source file a.c is compiled into LLVM byte code form.
  • 85
  • Input source file main.c is compiled into native object code.
  • 86
    87
    
                      
                    
    88 --- a.h ---
    89 extern int foo1(void);
    90 extern void foo2(void);
    91 extern void foo4(void);
    92 --- a.c ---
    93 #include "a.h"
    94
    95 static signed int i = 0;
    96
    97 void foo2(void) {
    98 i = -1;
    99 }
    100
    101 static int foo3() {
    102 foo4();
    103 return 10;
    104 }
    105
    106 int foo1(void) {
    107 int data = 0;
    108
    109 if (i < 0) { data = foo3(); }
    110
    111 data = data + 42;
    112 return data;
    113 }
    114
    115 --- main.c ---
    116 #include <stdio.h>
    117 #include "a.h"
    118
    119 void foo4(void) {
    120 printf ("Hi\n");
    121 }
    122
    123 int main() {
    124 return foo1();
    125 }
    126
    127 --- command lines ---
    128 $ llvm-gcc4 --emit-llvm -c a.c -o a.o # <-- a.o is LLVM bytecode file
    129 $ llvm-gcc4 -c main.c -o main.o # <-- main.o is native object file
    130 $ llvm-gcc4 a.o main.o -o main # <-- standard link command without any modifications
    131
    132

    In this example, the linker recognizes that foo2() is an

    133 externally visible symbol defined in LLVM byte code file. This information
    134 is collected using readLLVMObjectFile().
    135 Based on this information, the linker completes its usual symbol resolution
    136 pass and finds that foo2() is not used anywhere. This information
    137 is used by the LLVM optimizer and it removes foo2(). As soon as
    138 foo2() is removed, the optimizer recognizes that condition
    139 i < 0 is always false, which means foo3() is never
    140 used. Hence, the optimizer removes foo3(), also. And this in turn,
    141 enables linker to remove foo4(). This example illustrates the
    142 advantage of tight integration with the linker. Here, the optimizer can not
    143 remove foo3() without the linker's input.
    144

    145
    146
    147
    148
    149 Alternative Approaches
    150
    151
    152
    153
    154
    Compiler driver invokes link time optimizer separately.
    155
    In this model the link time optimizer is not able to take advantage of
    156 information collected during the linker's normal symbol resolution phase.
    157 In the above example, the optimizer can not remove foo2() without
    158 the linker's input because it is externally visible. This in turn prohibits
    159 the optimizer from removing foo3().
    160
    Use separate tool to collect symbol information from all object
    161 files.
    162
    In this model, a new, separate, tool or library replicates the linker's
    163 capability to collect information for link time optimization. Not only is
    164 this code duplication difficult to justify, but it also has several other
    165 disadvantages. For example, the linking semantics and the features
    166 provided by the linker on various platform are not unique. This means,
    167 this new tool needs to support all such features and platforms in one
    168 super tool or a separate tool per platform is required. This increases
    169 maintance cost for link time optimizer significantly, which is not
    170 necessary. This approach also requires staying synchronized with linker
    171 developements on various platforms, which is not the main focus of the link
    172 time optimizer. Finally, this approach increases end user's build time due
    173 to the duplication of work done by this separate tool and the linker itself.
    174
    175
    176
    177
    178
    179
    180 Multi-phase communication between LLVM and linker
    181
    182
    183
    184

    The linker collects information about symbol defininitions and uses in

    185 various link objects which is more accurate than any information collected
    186 by other tools during typical build cycles. The linker collects this
    187 information by looking at the definitions and uses of symbols in native .o
    188 files and using symbol visibility information. The linker also uses
    189 user-supplied information, such as a list of exported symbols. LLVM
    190 optimizer collects control flow information, data flow information and knows
    191 much more about program structure from the optimizer's point of view.
    192 Our goal is to take advantage of tight intergration between the linker and
    193 the optimizer by sharing this information during various linking phases.
    72194

    73195
    74196
    75197
    76198
    77 Example of link time optimization
    78
    79
    80
    81
    82

    Following example illustrates advantage of integrated approach that uses

    83 clean interface.
    84
    85
  • Input source file a.c is compiled into LLVM byte code form.
  • 86
  • Input source file main.c is compiled into native object code.
  • 87
    88
    89 --- a.h ---
    90
    extern int foo1(void);
    91
    extern void foo2(void);
    92
    extern void foo4(void);
    93
    --- a.c ---
    94
    #include "a.h"
    95
    96
    static signed int i = 0;
    97
    98
    void foo2(void) {
    99
    i = -1;
    100
    }
    101
    102
    static int foo3() {
    103
    foo4();
    104
    return 10;
    105
    }
    106
    107
    int foo1(void) {
    108
    int data = 0;
    109
    110
    if (i < 0) { data = foo3(); }
    111
    112
    data = data + 42;
    113
    return data;
    114
    }
    115
    116
    --- main.c ---
    117
    #include <stdio.h>
    118
    #include "a.h"
    119
    120
    void foo4(void) {
    121
    printf ("Hi\n");
    122
    }
    123
    124
    int main() {
    125
    return foo1();
    126
    }
    127
    128
    --- command lines ---
    129
    $ llvm-gcc4 --emit-llvm -c a.c -o a.o # <-- a.o is LLVM bytecode file
    130
    $ llvm-gcc4 -c main.c -o main.o # <-- main.o is native object file
    131
    $ llvm-gcc4 a.o main.o -o main # <-- standard link command without any modifications
    132
    133
    134

    135 In this example, the linker recognizes that foo2() is a externally visible
    136 symbol defined in LLVM byte code file. This information is collected using
    137 readLLVMObjectFile() . Based on this
    138 information, linker completes its usual symbol resolution pass and finds that
    139 foo2() is not used anywhere. This information is used by LLVM optimizer
    140 and it removes foo2(). As soon as foo2() is removed, optimizer
    141 recognizes that condition i < 0 is always false, which means
    142 foo3() is never used. Hence, optimizer removes foo3() also.
    143 And this in turn, enables linker to remove foo4().
    144 This example illustrates advantage of tight integration with linker. Here,
    145 optimizer can not remove foo3() without the linker's input.
    199 Phase 1 : Read LLVM Bytecode Files
    200
    201
    202
    203

    The linker first reads all object files in natural order and collects

    204 symbol information. This includes native object files as well as LLVM byte
    205 code files. In this phase, the linker uses
    206 readLLVMObjectFile() to collect symbol
    207 information from each LLVM bytecode files and updates its internal global
    208 symbol table accordingly. The intent of this interface is to avoid overhead
    209 in the non LLVM case, where all input object files are native object files,
    210 by putting this code in the error path of the linker. When the linker sees
    211 the first llvm .o file, it dlopen()s the dynamic library. This is
    212 to allow changes to the LLVM LTO code without relinking the linker.
    146213

    147214
    148215
    149216
    150217
    151 Alternative Approaches
    152
    153
    154
    155

    156
    157
  • Compiler driver invokes link time optimizer separately.
  • 158

    In this model link time optimizer is not able to take advantage of information
    159 collected during normal linker's symbol resolution phase. In above example,
    160 optimizer can not remove foo2() without linker's input because it is
    161 externally visible. And this in turn prohibits optimizer from removing foo3().
    162

    163
  • Use separate tool to collect symbol information from all object file.
  • 164

    In this model, this new separate tool or library replicates linker's
    165 capabilities to collect information for link time optimizer. Not only such code
    166 duplication is difficult to justify but it also has several other disadvantages.
    167 For example, the linking semantics and the features provided by linker on
    168 various platform are not unique. This means, this new tool needs to support all
    169 such features and platforms in one super tool or one new separate tool per
    170 platform is required. This increases maintance cost for link time optimizer
    171 significantly, which is not necessary. Plus, this approach requires staying
    172 synchronized with linker developements on various platforms, which is not the
    173 main focus of link time optimizer. Finally, this approach increases end user's build
    174 time due to duplicate work done by this separate tool and linker itself.
    175
    176
    177
    178
    179
    180 Multi-phase communication between LLVM and linker
    181
    182
    183
    184

    185 The linker collects information about symbol defininitions and uses in various
    186 link objects which is more accurate than any information collected by other tools
    187 during typical build cycle.
    188 The linker collects this information by looking at definitions and uses of
    189 symbols in native .o files and using symbol visibility information. The linker
    190 also uses user supplied information, such as list of exported symbol.
    191 LLVM optimizer collects control flow information, data flow information and
    192 knows much more about program structure from optimizer's point of view. Our
    193 goal is to take advantage of tight intergration between the linker and
    194 optimizer by sharing this information during various linking phases.
    218 Phase 2 : Symbol Resolution
    219
    220
    221
    222

    In this stage, the linker resolves symbols using global symbol table

    223 information to report undefined symbol errors, read archive members, resolve
    224 weak symbols, etc. The linker is able to do this seamlessly even though it
    225 does not know the exact content of input LLVM bytecode files because it uses
    226 symbol information provided by
    227 readLLVMObjectFile(). If dead code
    228 stripping is enabled then the linker collects the list of live symbols.
    229

    230
    231
    232
    233
    234 Phase 3 : Optimize Bytecode Files
    235
    236
    237

    After symbol resolution, the linker updates symbol information supplied

    238 by LLVM bytecode files appropriately. For example, whether certain LLVM
    239 bytecode supplied symbols are used or not. In the example above, the linker
    240 reports that foo2() is not used anywhere in the program, including
    241 native .o files. This information is used by the LLVM interprocedural
    242 optimizer. The linker uses optimizeModules()
    243 and requests an optimized native object file of the LLVM portion of the
    244 program.
    195245

    196246
    197247
    198248
    199249
    200 Phase 1 : Read LLVM Bytecode Files
    201
    202
    203
    204

    205 The linker first reads all object files in natural order and collects symbol
    206 information. This includes native object files as well as LLVM byte code files.
    207 In this phase, the linker uses readLLVMObjectFile()
    208 to collect symbol information from each LLVM bytecode files and updates its
    209 internal global symbol table accordingly. The intent of this interface is to
    210 avoid overhead in the non LLVM case, where all input object files are native
    211 object files, by putting this code in the error path of the linker. When the
    212 linker sees the first llvm .o file, it dlopen()s the dynamic library. This is
    213 to allow changes to LLVM part without relinking the linker.
    214

    215
    216
    217
    218
    219 Phase 2 : Symbol Resolution
    220
    221
    222
    223

    224 In this stage, the linker resolves symbols using global symbol table information
    225 to report undefined symbol errors, read archive members, resolve weak
    226 symbols etc... The linker is able to do this seamlessly even though it does not
    227 know exact content of input LLVM bytecode files because it uses symbol information
    228 provided by readLLVMObjectFile() .
    229 If dead code stripping is enabled then linker collects list of live symbols.
    230

    231
    232
    233
    234
    235 Phase 3 : Optimize Bytecode Files
    236
    237
    238

    239 After symbol resolution, the linker updates symbol information supplied by LLVM
    240 bytecode files appropriately. For example, whether certain LLVM bytecode
    241 supplied symbols are used or not. In the example above, the linker reports
    242 that foo2() is not used anywhere in the program, including native .o
    243 files. This information is used by LLVM interprocedural optimizer. The
    244 linker uses optimizeModules() and requests
    245 optimized native object file of the LLVM portion of the program.
    246

    247
    248
    249
    250
    251250 Phase 4 : Symbol Resolution after optimization
    252251
    253252
    254253
    255

    256 In this phase, the linker reads optimized native object file and updates internal
    257 global symbol table to reflect any changes. Linker also collects information
    258 about any change in use of external symbols by LLVM bytecode files. In the examle
    259 above, the linker notes that foo4() is not used any more. If dead code
    260 striping is enabled then linker refreshes live symbol information appropriately
    261 and performs dead code stripping.
    262
    263 After this phase, the linker continues linking as if it never saw LLVM bytecode
    264 files.
    265

    254

    In this phase, the linker reads optimized a native object file and

    255 updates the internal global symbol table to reflect any changes. The linker
    256 also collects information about any changes in use of external symbols by
    257 LLVM bytecode files. In the examle above, the linker notes that
    258 foo4() is not used any more. If dead code stripping is enabled then
    259 the linker refreshes the live symbol information appropriately and performs
    260 dead code stripping.

    261

    After this phase, the linker continues linking as if it never saw LLVM

    262 bytecode files.

    266263
    267264
    268265
    271268
    272269
    273270
    274

    275 LLVMlto is a dynamic library that is part of the LLVM tools, and is
    276 intended for use by a linker. LLVMlto provides an abstract C++ interface
    277 to use the LLVM interprocedural optimizer without exposing details of LLVM
    278 internals. The intention is to keep the interface as stable as possible even
    279 when the LLVM optimizer continues to evolve.
    280

    271

    LLVMlto is a dynamic library that is part of the LLVM tools, and

    272 is intended for use by a linker. LLVMlto provides an abstract C++
    273 interface to use the LLVM interprocedural optimizer without exposing details
    274 of LLVM's internals. The intention is to keep the interface as stable as
    275 possible even when the LLVM optimizer continues to evolve.

    281276
    282277
    283278
    286281
    287282
    288283
    289

    290 LLVMSymbol class is used to describe the externally visible functions
    291 and global variables, tdefined in LLVM bytecode files, to linker.
    292 This includes symbol visibility information. This information is used by linker
    293 to do symbol resolution. For example : function foo2() is defined inside
    294 a LLVM bytecode module and it is externally visible symbol.
    295 This helps linker connect use of foo2() in native object file with
    296 future definition of symbol foo2(). The linker will see actual definition
    297 of foo2() when it receives optimized native object file in
    298 Symbol Resolution after optimization phase. If the linker does not find any
    299 use of foo2(), it updates LLVMSymbol visibility information to notify
    300 LLVM intermodular optimizer that it is dead. The LLVM intermodular optimizer
    301 takes advantage of such information to generate better code.
    302

    284

    The LLVMSymbol class is used to describe the externally visible

    285 functions and global variables, defined in LLVM bytecode files, to the linker.
    286 This includes symbol visibility information. This information is used by
    287 the linker to do symbol resolution. For example: function foo2() is
    288 defined inside an LLVM bytecode module and it is an externally visible symbol.
    289 This helps the linker connect the use of foo2() in native object
    290 files with a future definition of the symbol foo2(). The linker
    291 will see the actual definition of foo2() when it receives the
    292 optimized native object file in
    293 Symbol Resolution after optimization phase. If the
    294 linker does not find any uses of foo2(), it updates LLVMSymbol
    295 visibility information to notify LLVM intermodular optimizer that it is dead.
    296 The LLVM intermodular optimizer takes advantage of such information to
    297 generate better code.

    303298
    304299
    305300
    308303
    309304
    310305
    311

    312 readLLVMObjectFile() is used by the linker to read LLVM bytecode files
    313 and collect LLVMSymbol nformation. This routine also
    314 supplies list of externally defined symbols that are used by LLVM bytecode
    315 files. Linker uses this symbol information to do symbol resolution. Internally,
    316 LLVMlto maintains LLVM bytecode modules in memory. This
    317 function also provides list of external references used by bytecode file.
    318

    306

    The readLLVMObjectFile() function is used by the linker to read

    307 LLVM bytecode files and collect LLVMSymbol nformation. This routine also
    308 supplies a list of externally defined symbols that are used by LLVM bytecode
    309 files. The linker uses this symbol information to do symbol resolution.
    310 Internally, LLVMlto maintains LLVM bytecode modules in
    311 memory. This function also provides a list of external references used by
    312 bytecode files.

    319313
    320314
    321315
    324318
    325319
    326320
    327

    328 The linker invokes optimizeModules to optimize already read LLVM
    329 bytecode files by applying LLVM intermodular optimization techniques. This
    330 function runs LLVM intermodular optimizer and generates native object code
    331 as .o file at name and location provided by the linker.
    332

    321

    The linker invokes optimizeModules to optimize already read

    322 LLVM bytecode files by applying LLVM intermodular optimization techniques.
    323 This function runs the LLVM intermodular optimizer and generates native
    324 object code as .o files at the name and location provided by the
    325 linker.

    333326
    334327
    335328
    340333
    341334
    342335
    343

    ... incomplete ...

    336

    ... To be completed ...

    344337
    345338
    346339