llvm.org GIT mirror llvm / 1517625
Add docs+a script for building clang/LLVM with PGO Depending on who you ask, PGO grants a 15%-25% improvement in build times when using clang. Sadly, hooking everything up properly to generate a profile and apply it to clang isn't always straightforward. This script (and the accompanying docs) aim to make this process easier; ideally, a single invocation of the given script. In terms of testing, I've got a cronjob on my Debian box that's meant to run this a few times per week, and I tried manually running it on a puny Gentoo box I have (four whole Atom cores!). Nothing obviously broke. ¯\_(ツ)_/¯ I don't know if we have a Python style guide, so I just shoved this through yapf with all the defaults on. Finally, though the focus is clang at the moment, the hope is that this is easily applicable to other LLVM-y tools with minimal effort (e.g. lld, opt, ...). Hence, this lives in llvm/utils and tries to be somewhat ambiguous about naming. Differential Revision: https://reviews.llvm.org/D53598 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@345427 91177308-0d34-0410-b5e6-96231b3b80d8 George Burgess IV 11 months ago
3 changed file(s) with 654 addition(s) and 0 deletion(s). Raw diff Collapse all Expand all
0 =============================================================
1 How To Build Clang and LLVM with Profile-Guided Optimizations
2 =============================================================
3
4 Introduction
5 ============
6
7 PGO (Profile-Guided Optimization) allows your compiler to better optimize code
8 for how it actually runs. Users report that applying this to Clang and LLVM can
9 decrease overall compile time by 20%.
10
11 This guide walks you through how to build Clang with PGO, though it also applies
12 to other subprojects, such as LLD.
13
14
15 Using the script
16 ================
17
18 We have a script at ``utils/collect_and_build_with_pgo.py``. This script is
19 tested on a few Linux flavors, and requires a checkout of LLVM, Clang, and
20 compiler-rt. Despite the the name, it performs four clean builds of Clang, so it
21 can take a while to run to completion. Please see the script's ``--help`` for
22 more information on how to run it, and the different options available to you.
23 If you want to get the most out of PGO for a particular use-case (e.g. compiling
24 a specific large piece of software), please do read the section below on
25 'benchmark' selection.
26
27 Please note that this script is only tested on a few Linux distros. Patches to
28 add support for other platforms, as always, are highly appreciated. :)
29
30 This script also supports a ``--dry-run`` option, which causes it to print
31 important commands instead of running them.
32
33
34 Selecting 'benchmarks'
35 ======================
36
37 PGO does best when the profiles gathered represent how the user plans to use the
38 compiler. Notably, highly accurate profiles of llc building x86_64 code aren't
39 incredibly helpful if you're going to be targeting ARM.
40
41 By default, the script above does two things to get solid coverage. It:
42
43 - runs all of Clang and LLVM's lit tests, and
44 - uses the instrumented Clang to build Clang, LLVM, and all of the other
45 LLVM subprojects available to it.
46
47 Together, these should give you:
48
49 - solid coverage of building C++,
50 - good coverage of building C,
51 - great coverage of running optimizations,
52 - great coverage of the backend for your host's architecture, and
53 - some coverage of other architectures (if other arches are supported backends).
54
55 Altogether, this should cover a diverse set of uses for Clang and LLVM. If you
56 have very specific needs (e.g. your compiler is meant to compile a large browser
57 for four different platforms, or similar), you may want to do something else.
58 This is configurable in the script itself.
59
60
61 Building Clang with PGO
62 =======================
63
64 If you prefer to not use the script, this briefly goes over how to build
65 Clang/LLVM with PGO.
66
67 First, you should have at least LLVM, Clang, and compiler-rt checked out
68 locally.
69
70 Next, at a high level, you're going to need to do the following:
71
72 1. Build a standard Release Clang and the relevant libclang_rt.profile library
73 2. Build Clang using the Clang you built above, but with instrumentation
74 3. Use the instrumented Clang to generate profiles, which consists of two steps:
75
76 - Running the instrumented Clang/LLVM/lld/etc. on tasks that represent how
77 users will use said tools.
78 - Using a tool to convert the "raw" profiles generated above into a single,
79 final PGO profile.
80
81 4. Build a final release Clang (along with whatever other binaries you need)
82 using the profile collected from your benchmark
83
84 In more detailed steps:
85
86 1. Configure a Clang build as you normally would. It's highly recommended that
87 you use the Release configuration for this, since it will be used to build
88 another Clang. Because you need Clang and supporting libraries, you'll want
89 to build the ``all`` target (e.g. ``ninja all`` or ``make -j4 all``).
90
91 2. Configure a Clang build as above, but add the following CMake args:
92
93 - ``-DLLVM_BUILD_INSTRUMENTED=IR`` -- This causes us to build everything
94 with instrumentation.
95 - ``-DLLVM_BUILD_RUNTIME=No`` -- A few projects have bad interactions when
96 built with profiling, and aren't necessary to build. This flag turns them
97 off.
98 - ``-DCMAKE_C_COMPILER=/path/to/stage1/clang`` - Use the Clang we built in
99 step 1.
100 - ``-DCMAKE_CXX_COMPILER=/path/to/stage1/clang++`` - Same as above.
101
102 In this build directory, you simply need to build the ``clang`` target (and
103 whatever supporting tooling your benchmark requires).
104
105 3. As mentioned above, this has two steps: gathering profile data, and then
106 massaging it into a useful form:
107
108 a. Build your benchmark using the Clang generated in step 2. The 'standard'
109 benchmark recommended is to run ``check-clang`` and ``check-llvm`` in your
110 instrumented Clang's build directory, and to do a full build of Clang/LLVM
111 using your instrumented Clang. So, create yet another build directory,
112 with the following CMake arguments:
113
114 - ``-DCMAKE_C_COMPILER=/path/to/stage2/clang`` - Use the Clang we built in
115 step 2.
116 - ``-DCMAKE_CXX_COMPILER=/path/to/stage2/clang++`` - Same as above.
117
118 If your users are fans of debug info, you may want to consider using
119 ``-DCMAKE_BUILD_TYPE=RelWithDebInfo`` instead of
120 ``-DCMAKE_BUILD_TYPE=Release``. This will grant better coverage of
121 debug info pieces of clang, but will take longer to complete and will
122 result in a much larger build directory.
123
124 It's recommended to build the ``all`` target with your instrumented Clang,
125 since more coverage is often better.
126
127 b. You should now have a few ``*.profdata`` files in
128 ``path/to/stage2/profiles/``. You need to merge these using
129 ``llvm-profdata`` (even if you only have one! The profile merge transforms
130 profraw into actual profile data, as well). This can be done with
131 ``/path/to/stage1/llvm-profdata -merge
132 -output=/path/to/output/profdata.prof path/to/stage2/profiles/*.profdata``.
133
134 4. Now, build your final, PGO-optimized Clang. To do this, you'll want to pass
135 the following additional arguments to CMake.
136
137 - ``-DLLVM_PROFDATA_FILE=/path/to/output/profdata.prof`` - Use the PGO
138 profile from the previous step.
139 - ``-DCMAKE_C_COMPILER=/path/to/stage1/clang`` - Use the Clang we built in
140 step 1.
141 - ``-DCMAKE_CXX_COMPILER=/path/to/stage1/clang++`` - Same as above.
142
143 From here, you can build whatever targets you need.
144
145 .. note::
146 You may see warnings about a mismatched profile in the build output. These
147 are generally harmless. To silence them, you can add
148 ``-DCMAKE_C_FLAGS='-Wno-backend-plugin'
149 -DCMAKE_CXX_FLAGS='-Wno-backend-plugin'`` to your CMake invocation.
150
151
152 Congrats! You now have a Clang built with profile-guided optimizations, and you
153 can delete all but the final build directory if you'd like.
154
155 If this worked well for you and you plan on doing it often, there's a slight
156 optimization that can be made: LLVM and Clang have a tool called tblgen that's
157 built and run during the build process. While it's potentially nice to build
158 this for coverage as part of step 3, none of your other builds should benefit
159 from building it. You can pass the CMake options
160 ``-DCLANG_TABLEGEN=/path/to/stage1/bin/clang-tblgen
161 -DLLVM_TABLEGEN=/path/to/stage1/bin/llvm-tblgen`` to steps 2 and onward to avoid
162 these useless rebuilds.
6767 CMakePrimer
6868 AdvancedBuilds
6969 HowToBuildOnARM
70 HowToBuildWithPGO
7071 HowToCrossCompileBuiltinsOnArm
7172 HowToCrossCompileLLVM
7273 CommandGuide/index
105106
106107 :doc:`HowToBuildOnARM`
107108 Notes on building and testing LLVM/Clang on ARM.
109
110 :doc:`HowToBuildWithPGO`
111 Notes on building LLVM/Clang with PGO.
108112
109113 :doc:`HowToCrossCompileBuiltinsOnArm`
110114 Notes on cross-building and testing the compiler-rt builtins for Arm.
0 #!/usr/bin/env python3
1 """
2 This script:
3 - Builds clang with user-defined flags
4 - Uses that clang to build an instrumented clang, which can be used to collect
5 PGO samples
6 - Builds a user-defined set of sources (default: clang) to act as a
7 "benchmark" to generate a PGO profile
8 - Builds clang once more with the PGO profile generated above
9
10 This is a total of four clean builds of clang (by default). This may take a
11 while. :)
12 """
13
14 import argparse
15 import collections
16 import multiprocessing
17 import os
18 import shlex
19 import shutil
20 import subprocess
21 import sys
22
23 ### User configuration
24
25
26 # If you want to use a different 'benchmark' than building clang, make this
27 # function do what you want. out_dir is the build directory for clang, so all
28 # of the clang binaries will live under "${out_dir}/bin/". Using clang in
29 # ${out_dir} will magically have the profiles go to the right place.
30 #
31 # You may assume that out_dir is a freshly-built directory that you can reach
32 # in to build more things, if you'd like.
33 def _run_benchmark(env, out_dir, include_debug_info):
34 """The 'benchmark' we run to generate profile data."""
35 target_dir = env.output_subdir('instrumentation_run')
36
37 # `check-llvm` and `check-clang` are cheap ways to increase coverage. The
38 # former lets us touch on the non-x86 backends a bit if configured, and the
39 # latter gives us more C to chew on (and will send us through diagnostic
40 # paths a fair amount, though the `if (stuff_is_broken) { diag() ... }`
41 # branches should still heavily be weighted in the not-taken direction,
42 # since we built all of LLVM/etc).
43 _build_things_in(env, target_dir, what=['check-llvm', 'check-clang'])
44
45 # Building tblgen gets us coverage; don't skip it. (out_dir may also not
46 # have them anyway, but that's less of an issue)
47 cmake = _get_cmake_invocation_for_bootstrap_from(
48 env, out_dir, skip_tablegens=False)
49
50 if include_debug_info:
51 cmake.add_flag('CMAKE_BUILD_TYPE', 'RelWithDebInfo')
52
53 _run_fresh_cmake(env, cmake, target_dir)
54
55 # Just build all the things. The more data we have, the better.
56 _build_things_in(env, target_dir, what=['all'])
57
58
59 ### Script
60
61
62 class CmakeInvocation:
63 _cflags = ['CMAKE_C_FLAGS', 'CMAKE_CXX_FLAGS']
64 _ldflags = [
65 'CMAKE_EXE_LINKER_FLAGS',
66 'CMAKE_MODULE_LINKER_FLAGS',
67 'CMAKE_SHARED_LINKER_FLAGS',
68 ]
69
70 def __init__(self, cmake, maker, cmake_dir):
71 self._prefix = [cmake, '-G', maker, cmake_dir]
72
73 # Map of str -> (list|str).
74 self._flags = {}
75 for flag in CmakeInvocation._cflags + CmakeInvocation._ldflags:
76 self._flags[flag] = []
77
78 def add_new_flag(self, key, value):
79 self.add_flag(key, value, allow_overwrites=False)
80
81 def add_flag(self, key, value, allow_overwrites=True):
82 if key not in self._flags:
83 self._flags[key] = value
84 return
85
86 existing_value = self._flags[key]
87 if isinstance(existing_value, list):
88 existing_value.append(value)
89 return
90
91 if not allow_overwrites:
92 raise ValueError('Invalid overwrite of %s requested' % key)
93
94 self._flags[key] = value
95
96 def add_cflags(self, flags):
97 # No, I didn't intend to append ['-', 'O', '2'] to my flags, thanks :)
98 assert not isinstance(flags, str)
99 for f in CmakeInvocation._cflags:
100 self._flags[f].extend(flags)
101
102 def add_ldflags(self, flags):
103 assert not isinstance(flags, str)
104 for f in CmakeInvocation._ldflags:
105 self._flags[f].extend(flags)
106
107 def to_args(self):
108 args = self._prefix.copy()
109 for key, value in sorted(self._flags.items()):
110 if isinstance(value, list):
111 # We preload all of the list-y values (cflags, ...). If we've
112 # nothing to add, don't.
113 if not value:
114 continue
115 value = ' '.join(value)
116
117 arg = '-D' + key
118 if value != '':
119 arg += '=' + value
120 args.append(arg)
121 return args
122
123
124 class Env:
125 def __init__(self, llvm_dir, use_make, output_dir, default_cmake_args,
126 dry_run):
127 self.llvm_dir = llvm_dir
128 self.use_make = use_make
129 self.output_dir = output_dir
130 self.default_cmake_args = default_cmake_args.copy()
131 self.dry_run = dry_run
132
133 def get_default_cmake_args_kv(self):
134 return self.default_cmake_args.items()
135
136 def get_cmake_maker(self):
137 return 'Ninja' if not self.use_make else 'Unix Makefiles'
138
139 def get_make_command(self):
140 if self.use_make:
141 return ['make', '-j{}'.format(multiprocessing.cpu_count())]
142 return ['ninja']
143
144 def output_subdir(self, name):
145 return os.path.join(self.output_dir, name)
146
147 def has_llvm_subproject(self, name):
148 if name == 'compiler-rt':
149 subdir = 'projects/compiler-rt'
150 elif name == 'clang':
151 subdir = 'tools/clang'
152 else:
153 raise ValueError('Unknown subproject: %s' % name)
154
155 return os.path.isdir(os.path.join(self.llvm_dir, subdir))
156
157 # Note that we don't allow capturing stdout/stderr. This works quite nicely
158 # with dry_run.
159 def run_command(self,
160 cmd,
161 cwd=None,
162 check=False,
163 silent_unless_error=False):
164 cmd_str = ' '.join(shlex.quote(s) for s in cmd)
165 print(
166 'Running `%s` in %s' % (cmd_str, shlex.quote(cwd or os.getcwd())))
167
168 if self.dry_run:
169 return
170
171 if silent_unless_error:
172 stdout, stderr = subprocess.PIPE, subprocess.STDOUT
173 else:
174 stdout, stderr = None, None
175
176 # Don't use subprocess.run because it's >= py3.5 only, and it's not too
177 # much extra effort to get what it gives us anyway.
178 popen = subprocess.Popen(
179 cmd,
180 stdin=subprocess.DEVNULL,
181 stdout=stdout,
182 stderr=stderr,
183 cwd=cwd)
184 stdout, _ = popen.communicate()
185 return_code = popen.wait(timeout=0)
186
187 if not return_code:
188 return
189
190 if silent_unless_error:
191 print(stdout.decode('utf-8', 'ignore'))
192
193 if check:
194 raise subprocess.CalledProcessError(
195 returncode=return_code, cmd=cmd, output=stdout, stderr=None)
196
197
198 def _get_default_cmake_invocation(env):
199 inv = CmakeInvocation(
200 cmake='cmake', maker=env.get_cmake_maker(), cmake_dir=env.llvm_dir)
201 for key, value in env.get_default_cmake_args_kv():
202 inv.add_new_flag(key, value)
203 return inv
204
205
206 def _get_cmake_invocation_for_bootstrap_from(env, out_dir,
207 skip_tablegens=True):
208 clang = os.path.join(out_dir, 'bin', 'clang')
209 cmake = _get_default_cmake_invocation(env)
210 cmake.add_new_flag('CMAKE_C_COMPILER', clang)
211 cmake.add_new_flag('CMAKE_CXX_COMPILER', clang + '++')
212
213 # We often get no value out of building new tblgens; the previous build
214 # should have them. It's still correct to build them, just slower.
215 def add_tablegen(key, binary):
216 path = os.path.join(out_dir, 'bin', binary)
217
218 # Check that this exists, since the user's allowed to specify their own
219 # stage1 directory (which is generally where we'll source everything
220 # from). Dry runs should hope for the best from our user, as well.
221 if env.dry_run or os.path.exists(path):
222 cmake.add_new_flag(key, path)
223
224 if skip_tablegens:
225 add_tablegen('LLVM_TABLEGEN', 'llvm-tblgen')
226 add_tablegen('CLANG_TABLEGEN', 'clang-tblgen')
227
228 return cmake
229
230
231 def _build_things_in(env, target_dir, what):
232 cmd = env.get_make_command() + what
233 env.run_command(cmd, cwd=target_dir, check=True)
234
235
236 def _run_fresh_cmake(env, cmake, target_dir):
237 if not env.dry_run:
238 try:
239 shutil.rmtree(target_dir)
240 except FileNotFoundError:
241 pass
242
243 os.makedirs(target_dir, mode=0o755)
244
245 cmake_args = cmake.to_args()
246 env.run_command(
247 cmake_args, cwd=target_dir, check=True, silent_unless_error=True)
248
249
250 def _build_stage1_clang(env):
251 target_dir = env.output_subdir('stage1')
252 cmake = _get_default_cmake_invocation(env)
253 _run_fresh_cmake(env, cmake, target_dir)
254
255 # FIXME: The full build here is somewhat unfortunate. It's primarily
256 # because I don't know what to call libclang_rt.profile for arches that
257 # aren't x86_64 (and even then, it's in a subdir that contains clang's
258 # current version). It would be nice to figure out what target I can
259 # request to magically have libclang_rt.profile built for ${host}
260 _build_things_in(env, target_dir, what=['all'])
261 return target_dir
262
263
264 def _generate_instrumented_clang_profile(env, stage1_dir, profile_dir,
265 output_file):
266 llvm_profdata = os.path.join(stage1_dir, 'bin', 'llvm-profdata')
267 if env.dry_run:
268 profiles = [os.path.join(profile_dir, '*.profraw')]
269 else:
270 profiles = [
271 os.path.join(profile_dir, f) for f in os.listdir(profile_dir)
272 if f.endswith('.profraw')
273 ]
274 cmd = [llvm_profdata, 'merge', '-output=' + output_file] + profiles
275 env.run_command(cmd, check=True)
276
277
278 def _build_instrumented_clang(env, stage1_dir):
279 assert os.path.isabs(stage1_dir)
280
281 target_dir = os.path.join(env.output_dir, 'instrumented')
282 cmake = _get_cmake_invocation_for_bootstrap_from(env, stage1_dir)
283 cmake.add_new_flag('LLVM_BUILD_INSTRUMENTED', 'IR')
284
285 # libcxx's configure step messes with our link order: we'll link
286 # libclang_rt.profile after libgcc, and the former requires atexit from the
287 # latter. So, configure checks fail.
288 #
289 # Since we don't need libcxx or compiler-rt anyway, just disable them.
290 cmake.add_new_flag('LLVM_BUILD_RUNTIME', 'No')
291
292 _run_fresh_cmake(env, cmake, target_dir)
293 _build_things_in(env, target_dir, what=['clang', 'lld'])
294
295 profiles_dir = os.path.join(target_dir, 'profiles')
296 return target_dir, profiles_dir
297
298
299 def _build_optimized_clang(env, stage1_dir, profdata_file):
300 if not env.dry_run and not os.path.exists(profdata_file):
301 raise ValueError('Looks like the profdata file at %s doesn\'t exist' %
302 profdata_file)
303
304 target_dir = os.path.join(env.output_dir, 'optimized')
305 cmake = _get_cmake_invocation_for_bootstrap_from(env, stage1_dir)
306 cmake.add_new_flag('LLVM_PROFDATA_FILE', os.path.abspath(profdata_file))
307
308 # We'll get complaints about hash mismatches in `main` in tools/etc. Ignore
309 # it.
310 cmake.add_cflags(['-Wno-backend-plugin'])
311 _run_fresh_cmake(env, cmake, target_dir)
312 _build_things_in(env, target_dir, what=['clang'])
313 return target_dir
314
315
316 Args = collections.namedtuple('Args', [
317 'do_optimized_build',
318 'include_debug_info',
319 'profile_location',
320 'stage1_dir',
321 ])
322
323
324 def _parse_args():
325 parser = argparse.ArgumentParser(
326 description='Builds LLVM and Clang with instrumentation, collects '
327 'instrumentation profiles for them, and (optionally) builds things'
328 'with these PGO profiles. By default, it\'s assumed that you\'re '
329 'running this from your LLVM root, and all build artifacts will be '
330 'saved to $PWD/out.')
331 parser.add_argument(
332 '--cmake-extra-arg',
333 action='append',
334 default=[],
335 help='an extra arg to pass to all cmake invocations. Note that this '
336 'is interpreted as a -D argument, e.g. --cmake-extra-arg FOO=BAR will '
337 'be passed as -DFOO=BAR. This may be specified multiple times.')
338 parser.add_argument(
339 '--dry-run',
340 action='store_true',
341 help='print commands instead of running them')
342 parser.add_argument(
343 '--llvm-dir',
344 default='.',
345 help='directory containing an LLVM checkout (default: $PWD)')
346 parser.add_argument(
347 '--no-optimized-build',
348 action='store_true',
349 help='disable the final, PGO-optimized build')
350 parser.add_argument(
351 '--out-dir',
352 help='directory to write artifacts to (default: $llvm_dir/out)')
353 parser.add_argument(
354 '--profile-output',
355 help='where to output the profile (default is $out/pgo_profile.prof)')
356 parser.add_argument(
357 '--stage1-dir',
358 help='instead of having an initial build of everything, use the given '
359 'directory. It is expected that this directory will have clang, '
360 'llvm-profdata, and the appropriate libclang_rt.profile already built')
361 parser.add_argument(
362 '--use-debug-info-in-benchmark',
363 action='store_true',
364 help='use a regular build instead of RelWithDebInfo in the benchmark. '
365 'This increases benchmark execution time and disk space requirements, '
366 'but gives more coverage over debuginfo bits in LLVM and clang.')
367 parser.add_argument(
368 '--use-make',
369 action='store_true',
370 default=shutil.which('ninja') is None,
371 help='use Makefiles instead of ninja')
372
373 args = parser.parse_args()
374
375 llvm_dir = os.path.abspath(args.llvm_dir)
376 if args.out_dir is None:
377 output_dir = os.path.join(llvm_dir, 'out')
378 else:
379 output_dir = os.path.abspath(args.out_dir)
380
381 extra_args = {'CMAKE_BUILD_TYPE': 'Release'}
382 for arg in args.cmake_extra_arg:
383 if arg.startswith('-D'):
384 arg = arg[2:]
385 elif arg.startswith('-'):
386 raise ValueError('Unknown not- -D arg encountered; you may need '
387 'to tweak the source...')
388 split = arg.split('=', 1)
389 if len(split) == 1:
390 key, val = split[0], ''
391 else:
392 key, val = split
393 extra_args[key] = val
394
395 env = Env(
396 default_cmake_args=extra_args,
397 dry_run=args.dry_run,
398 llvm_dir=llvm_dir,
399 output_dir=output_dir,
400 use_make=args.use_make,
401 )
402
403 if args.profile_output is not None:
404 profile_location = args.profile_output
405 else:
406 profile_location = os.path.join(env.output_dir, 'pgo_profile.prof')
407
408 result_args = Args(
409 do_optimized_build=not args.no_optimized_build,
410 include_debug_info=args.use_debug_info_in_benchmark,
411 profile_location=profile_location,
412 stage1_dir=args.stage1_dir,
413 )
414
415 return env, result_args
416
417
418 def _looks_like_llvm_dir(directory):
419 """Arbitrary set of heuristics to determine if `directory` is an llvm dir.
420
421 Errs on the side of false-positives."""
422
423 contents = set(os.listdir(directory))
424 expected_contents = [
425 'CODE_OWNERS.TXT',
426 'cmake',
427 'docs',
428 'include',
429 'utils',
430 ]
431
432 if not all(c in contents for c in expected_contents):
433 return False
434
435 try:
436 include_listing = os.listdir(os.path.join(directory, 'include'))
437 except NotADirectoryError:
438 return False
439
440 return 'llvm' in include_listing
441
442
443 def _die(*args, **kwargs):
444 kwargs['file'] = sys.stderr
445 print(*args, **kwargs)
446 sys.exit(1)
447
448
449 def _main():
450 env, args = _parse_args()
451
452 if not _looks_like_llvm_dir(env.llvm_dir):
453 _die('Looks like %s isn\'t an LLVM directory; please see --help' %
454 env.llvm_dir)
455 if not env.has_llvm_subproject('clang'):
456 _die('Need a clang checkout at tools/clang')
457 if not env.has_llvm_subproject('compiler-rt'):
458 _die('Need a compiler-rt checkout at projects/compiler-rt')
459
460 def status(*args):
461 print(*args, file=sys.stderr)
462
463 if args.stage1_dir is None:
464 status('*** Building stage1 clang...')
465 stage1_out = _build_stage1_clang(env)
466 else:
467 stage1_out = args.stage1_dir
468
469 status('*** Building instrumented clang...')
470 instrumented_out, profile_dir = _build_instrumented_clang(env, stage1_out)
471 status('*** Running profdata benchmarks...')
472 _run_benchmark(env, instrumented_out, args.include_debug_info)
473 status('*** Generating profile...')
474 _generate_instrumented_clang_profile(env, stage1_out, profile_dir,
475 args.profile_location)
476
477 print('Final profile:', args.profile_location)
478 if args.do_optimized_build:
479 status('*** Building PGO-optimized binaries...')
480 optimized_out = _build_optimized_clang(env, stage1_out,
481 args.profile_location)
482 print('Final build directory:', optimized_out)
483
484
485 if __name__ == '__main__':
486 _main()