llvm.org GIT mirror llvm / c86ea9f
[docs] Add new document on building distributions Summary: This document is an attempt to provide a guide for best practices for using the LLVM build system to generate distributable LLVM-based tools. Most of the document is geared toward distributions of LLVM-based toolchains, but much of it also applies to distributing other LLVM-based tools and libraries. Reviewers: tstellar, phosek, jroelofs, hans, sylvestre.ledru Reviewed By: tstellar Subscribers: smeenai, dschuff, arphaman, winksaville, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D62040 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@361272 91177308-0d34-0410-b5e6-96231b3b80d8 Chris Bieneman 3 months ago
2 changed file(s) with 212 addition(s) and 0 deletion(s). Raw diff Collapse all Expand all
0 ===============================
1 Building a Distribution of LLVM
2 ===============================
3
4 .. contents::
5 :local:
6
7 Introduction
8 ============
9
10 This document is geared toward people who want to build and package LLVM and any
11 combination of LLVM sub-project tools for distribution. This document covers
12 useful features of the LLVM build system as well as best practices and general
13 information about packaging LLVM.
14
15 If you are new to CMake you may find the :doc:`CMake` or :doc:`CMakePrimer`
16 documentation useful. Some of the things covered in this document are the inner
17 workings of the builds described in the :doc:`AdvancedBuilds` document.
18
19 General Distribution Guidance
20 =============================
21
22 When building a distribution of a compiler it is generally advised to perform a
23 bootstrap build of the compiler. That means building a "stage 1" compiler with
24 your host toolchain, then building the "stage 2" compiler using the "stage 1"
25 compiler. This is done so that the compiler you distribute benefits from all the
26 bug fixes, performance optimizations and general improvements provided by the
27 new compiler.
28
29 In deciding how to build your distribution there are a few trade-offs that you
30 will need to evaluate. The big two are:
31
32 #. Compile time of the distribution against performance of the built compiler
33
34 #. Binary size of the distribution against performance of the built compiler
35
36 The guidance for maximizing performance of the generated compiler is to use LTO,
37 PGO, and statically link everything. This will result in an overall larger
38 distribution, and it will take longer to generate, but it provides the most
39 opportunity for the compiler to optimize.
40
41 The guidance for minimizing distribution size is to dynamically link LLVM and
42 Clang libraries into the tools to reduce code duplication. This will come at a
43 substantial performance penalty to the generated binary both because it reduces
44 optimization opportunity, and because dynamic linking requires resolving symbols
45 at process launch time, which can be very slow for C++ code.
46
47 .. _shared_libs:
48
49 .. warning::
50 One very important note: Distributions should never be built using the
51 *BUILD_SHARED_LIBS* CMake option. That option exists for optimizing developer
52 workflow only. Due to design and implementation decisions, LLVM relies on
53 global data which can end up being duplicated across shared libraries
54 resulting in bugs. As such this is not a safe way to distribute LLVM or
55 LLVM-based tools.
56
57 The simplest example of building a distribution with reasonable performance is
58 captured in the DistributionExample CMake cache file located at
59 clang/cmake/caches/DistributionExample.cmake. The following command will perform
60 and install the distribution build:
61
62 .. code-block:: console
63
64 $ cmake -G Ninja -C /cmake/caches/DistributionExample.cmake
65 $ ninja stage2-distribution
66 $ ninja stage2-install-distribution
67
68 Difference between ``install`` and ``install-distribution``
69 -----------------------------------------------------------
70
71 One subtle but important thing to note is the difference between the ``install``
72 and ``install-distribution`` targets. The ``install`` target is expected to
73 install every part of LLVM that your build is configured to generate except the
74 LLVM testing tools. Alternatively the ``install-distribution`` target, which is
75 recommended for building distributions, only installs specific parts of LLVM as
76 specified at configuration time by *LLVM_DISTRIBUTION_COMPONENTS*.
77
78 Additionally by default the ``install`` target will install the LLVM testing
79 tools as the public tools. This can be changed well by setting
80 *LLVM_INSTALL_TOOLCHAIN_ONLY* to ``On``. The LLVM tools are intended for
81 development and testing of LLVM, and should only be included in distributions
82 that support LLVM development.
83
84 When building with *LLVM_DISTRIBUTION_COMPONENTS* the build system also
85 generates a ``distribution`` target which builds all the components specified in
86 the list. This is a convenience build target to allow building just the
87 distributed pieces without needing to build all configured targets.
88
89 Special Notes for Library-only Distributions
90 --------------------------------------------
91
92 One of the most powerful features of LLVM is its library-first design mentality
93 and the way you can compose a wide variety of tools using different portions of
94 LLVM. Even in this situation using *BUILD_SHARED_LIBS* is not supported. If you
95 want to distribute LLVM as a shared library for use in a tool, the recommended
96 method is using *LLVM_BUILD_LLVM_DYLIB*, and you can use *LLVM_DYLIB_COMPONENTS*
97 to configure which LLVM components are part of libLLVM.
98
99 Options for Optimizing LLVM
100 ===========================
101
102 There are four main build optimizations that our CMake build system supports.
103 When performing a bootstrap build it is not beneficial to do anything other than
104 setting *CMAKE_BUILD_TYPE* to ``Release`` for the stage-1 compiler. This is
105 because the more intensive optimizations are expensive to perform and the
106 stage-1 compiler is thrown away. All of the further options described should be
107 set on the stage-2 compiler either using a CMake cache file, or by prefixing the
108 option with *BOOTSTRAP_*.
109
110 The first and simplest to use is the compiler optimization level by setting the
111 *CMAKE_BUILD_TYPE* option. The main values of interest are ``Release`` or
112 ``RelWithDebInfo``. By default the ``Release`` option uses the ``-O3``
113 optimization level, and ``RelWithDebInfo`` uses ``-O2``. If you want to generate
114 debug information and use ``-O3`` you can override the
115 *CMAKE__FLAGS_RELWITHDEBINFO* option for C and CXX.
116 DistributionExample.cmake does this.
117
118 Another easy to use option is Link-Time-Optimization. You can set the
119 *LLVM_ENABLE_LTO* option on your stage-2 build to ``Thin`` or ``Full`` to enable
120 building LLVM with LTO. These options will significantly increase link time of
121 the binaries in the distribution, but it will create much faster binaries. This
122 option should not be used if your distribution includes static archives, as the
123 objects inside the archive will be LLVM bitcode, which is not portable.
124
125 The :doc:`AdvancedBuilds` documentation describes the built-in tooling for
126 generating LLVM profiling information to drive Profile-Guided-Optimization. The
127 in-tree profiling tests are very limited, and generating the profile takes a
128 significant amount of time, but it can result in a significant improvement in
129 the performance of the generated binaries.
130
131 In addition to PGO profiling we also have limited support in-tree for generating
132 linker order files. These files provide the linker with a suggested ordering for
133 functions in the final binary layout. This can measurably speed up clang by
134 physically grouping functions that are called temporally close to eachother. The
135 current tooling is only available on Darwin systems with ``dtrace(1)``. It is
136 worth noting that dtrace is non-deterministic, and so the order file generation
137 using dtrace is also non-deterministic.
138
139 Options for Reducing Size
140 =========================
141
142 .. warning::
143 Any steps taken to reduce the binary size will come at a cost of runtime
144 performance in the generated binaries.
145
146 The simplest and least significant way to reduce binary size is to set the
147 *CMAKE_BUILD_TYPE* variable to ``MinSizeRel``, which will set the compiler
148 optimization level to ``-Os`` which optimizes for binary size. This will have
149 both the least benefit to size and the least impact on performance.
150
151 The most impactful way to reduce binary size is to dynamically link LLVM into
152 all the tools. This reduces code size by decreasing duplication of common code
153 between the LLVM-based tools. This can be done by setting the following two
154 CMake options to ``On``: *LLVM_BUILD_LLVM_DYLIB* and *LLVM_LINK_LLVM_DYLIB*.
155
156 .. warning::
157 Distributions should never be built using the *BUILD_SHARED_LIBS* CMake
158 option. (:ref:`See the warning above for more explanation `.).
159
160 Relevant CMake Options
161 ======================
162
163 This section provides documentation of the CMake options that are intended to
164 help construct distributions. This is not an exhaustive list, and many
165 additional options are documented in the :doc:`CMake` page. Some key options
166 that are already documented include: *LLVM_TARGETS_TO_BUILD*,
167 *LLVM_ENABLE_PROJECTS*, *LLVM_BUILD_LLVM_DYLIB*, and *LLVM_LINK_LLVM_DYLIB*.
168
169 **LLVM_ENABLE_RUNTIMES**:STRING
170 When building a distribution that includes LLVM runtime projects (i.e. libcxx,
171 compiler-rt, libcxxabi, libunwind...), it is important to build those projects
172 with the just-built compiler.
173
174 **LLVM_DISTRIBUTION_COMPONENTS**:STRING
175 This variable can be set to a semi-colon separated list of LLVM build system
176 components to install. All LLVM-based tools are components, as well as most
177 of the libraries and runtimes. Component names match the names of the build
178 system targets.
179
180 **LLVM_RUNTIME_DISTRIBUTION_COMPONENTS**:STRING
181 This variable can be set to a semi-colon separated list of runtime library
182 components. This is used in conjunction with *LLVM_ENABLE_RUNTIMES* to specify
183 components of runtime libraries that you want to include in your distribution.
184 Just like with *LLVM_DISTRIBUTION_COMPONENTS*, component names match the names
185 of the build system targets.
186
187 **LLVM_DYLIB_COMPONENTS**:STRING
188 This variable can be set to a semi-colon separated name of LLVM library
189 components. LLVM library components are either library names with the LLVM
190 prefix removed (i.e. Support, Demangle...), LLVM target names, or special
191 purpose component names. The special purpose component names are:
192
193 #. ``all`` - All LLVM available component libraries
194 #. ``Native`` - The LLVM target for the Native system
195 #. ``AllTargetsAsmPrinters`` - All the included target ASM printers libraries
196 #. ``AllTargetsAsmParsers`` - All the included target ASM parsers libraries
197 #. ``AllTargetsDescs`` - All the included target descriptions libraries
198 #. ``AllTargetsDisassemblers`` - All the included target dissassemblers libraries
199 #. ``AllTargetsInfos`` - All the included target info libraries
200
201 **LLVM_INSTALL_TOOLCHAIN_ONLY**:BOOL
202 This option defaults to ``Off``: when set to ``On`` it removes many of the
203 LLVM development and testing tools as well as component libraries from the
204 default ``install`` target. Including the development tools is not recommended
205 for distributions as many of the LLVM tools are only intended for development
206 and testing use.
9494 ReportingGuide
9595 Benchmarking
9696 Docker
97 BuildingADistribution
9798
9899 :doc:`GettingStarted`
99100 Discusses how to get up and running quickly with the LLVM infrastructure.
175176
176177 :doc:`Docker`
177178 A reference for using Dockerfiles provided with LLVM.
179
180 :doc:`BuildingADistribution`
181 A best-practices guide for using LLVM's CMake build system to package and
182 distribute LLVM-based tools.
178183
179184
180185 Programming Documentation