llvm.org GIT mirror llvm / 7398f4a
[ELF] Implement Dependent Libraries Feature This patch implements a limited form of autolinking primarily designed to allow either the --dependent-library compiler option, or "comment lib" pragmas ( https://docs.microsoft.com/en-us/cpp/preprocessor/comment-c-cpp?view=vs-2017) in C/C++ e.g. #pragma comment(lib, "foo"), to cause an ELF linker to automatically add the specified library to the link when processing the input file generated by the compiler. Currently this extension is unique to LLVM and LLD. However, care has been taken to design this feature so that it could be supported by other ELF linkers. The design goals were to provide: - A simple linking model for developers to reason about. - The ability to to override autolinking from the linker command line. - Source code compatibility, where possible, with "comment lib" pragmas in other environments (MSVC in particular). Dependent library support is implemented differently for ELF platforms than on the other platforms. Primarily this difference is that on ELF we pass the dependent library specifiers directly to the linker without manipulating them. This is in contrast to other platforms where they are mapped to a specific linker option by the compiler. This difference is a result of the greater variety of ELF linkers and the fact that ELF linkers tend to handle libraries in a more complicated fashion than on other platforms. This forces us to defer handling the specifiers to the linker. In order to achieve a level of source code compatibility with other platforms we have restricted this feature to work with libraries that meet the following "reasonable" requirements: 1. There are no competing defined symbols in a given set of libraries, or if they exist, the program owner doesn't care which is linked to their program. 2. There may be circular dependencies between libraries. The binary representation is a mergeable string section (SHF_MERGE, SHF_STRINGS), called .deplibs, with custom type SHT_LLVM_DEPENDENT_LIBRARIES (0x6fff4c04). The compiler forms this section by concatenating the arguments of the "comment lib" pragmas and --dependent-library options in the order they are encountered. Partial (-r, -Ur) links are handled by concatenating .deplibs sections with the normal mergeable string section rules. As an example, #pragma comment(lib, "foo") would result in: .section ".deplibs","MS",@llvm_dependent_libraries,1 .asciz "foo" For LTO, equivalent information to the contents of a the .deplibs section can be retrieved by the LLD for bitcode input files. LLD processes the dependent library specifiers in the following way: 1. Dependent libraries which are found from the specifiers in .deplibs sections of relocatable object files are added when the linker decides to include that file (which could itself be in a library) in the link. Dependent libraries behave as if they were appended to the command line after all other options. As a consequence the set of dependent libraries are searched last to resolve symbols. 2. It is an error if a file cannot be found for a given specifier. 3. Any command line options in effect at the end of the command line parsing apply to the dependent libraries, e.g. --whole-archive. 4. The linker tries to add a library or relocatable object file from each of the strings in a .deplibs section by; first, handling the string as if it was specified on the command line; second, by looking for the string in each of the library search paths in turn; third, by looking for a lib<string>.a or lib<string>.so (depending on the current mode of the linker) in each of the library search paths. 5. A new command line option --no-dependent-libraries tells LLD to ignore the dependent libraries. Rationale for the above points: 1. Adding the dependent libraries last makes the process simple to understand from a developers perspective. All linkers are able to implement this scheme. 2. Error-ing for libraries that are not found seems like better behavior than failing the link during symbol resolution. 3. It seems useful for the user to be able to apply command line options which will affect all of the dependent libraries. There is a potential problem of surprise for developers, who might not realize that these options would apply to these "invisible" input files; however, despite the potential for surprise, this is easy for developers to reason about and gives developers the control that they may require. 4. This algorithm takes into account all of the different ways that ELF linkers find input files. The different search methods are tried by the linker in most obvious to least obvious order. 5. I considered adding finer grained control over which dependent libraries were ignored (e.g. MSVC has /nodefaultlib:<library>); however, I concluded that this is not necessary: if finer control is required developers can fall back to using the command line directly. RFC thread: http://lists.llvm.org/pipermail/llvm-dev/2019-March/131004.html. Differential Revision: https://reviews.llvm.org/D60274 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@360984 91177308-0d34-0410-b5e6-96231b3b80d8 Ben Dunbobbin 3 months ago
22 changed file(s) with 180 addition(s) and 12 deletion(s). Raw diff Collapse all Expand all
284284 The paramter identifies an additional library search path to be considered
285285 when looking up libraries after the inclusion of this option.
286286
287 ``SHT_LLVM_DEPENDENT_LIBRARIES`` Section (Dependent Libraries)
288 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
289
290 This section contains strings specifying libraries to be added to the link by
291 the linker.
292
293 The section should be consumed by the linker and not written to the output.
294
295 The strings are encoded as standard null-terminated UTF-8 strings.
296
297 For example:
298
299 .. code-block:: gas
300
301 .section ".deplibs","MS",@llvm_dependent_libraries,1
302 .asciz "library specifier 1"
303 .asciz "library specifier 2"
304
305 The interpretation of the library specifiers is defined by the consuming linker.
306
287307 ``SHT_LLVM_CALL_GRAPH_PROFILE`` Section (Call Graph Profile)
288308 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
289309
60806080 Automatic Linker Flags Named Metadata
60816081 =====================================
60826082
6083 Some targets support embedding flags to the linker inside individual object
6083 Some targets support embedding of flags to the linker inside individual object
60846084 files. Typically this is used in conjunction with language extensions which
6085 allow source files to explicitly declare the libraries they depend on, and have
6086 these automatically be transmitted to the linker via object files.
6085 allow source files to contain linker command line options, and have these
6086 automatically be transmitted to the linker via object files.
60876087
60886088 These flags are encoded in the IR using named metadata with the name
60896089 ``!llvm.linker.options``. Each operand is expected to be a metadata node
60946094 linker options, presumably to link against ``libz`` and the ``Cocoa``
60956095 framework::
60966096
6097 !0 = !{ !"-lz" },
6098 !1 = !{ !"-framework", !"Cocoa" } } }
6097 !0 = !{ !"-lz" }
6098 !1 = !{ !"-framework", !"Cocoa" }
60996099 !llvm.linker.options = !{ !0, !1 }
61006100
61016101 The metadata encoding as lists of lists of options, as opposed to a collapsed
61076107 Each individual option is required to be either a valid option for the target's
61086108 linker, or an option that is reserved by the target specific assembly writer or
61096109 object file emitter. No other aspect of these options is defined by the IR.
6110
6111 Dependent Libs Named Metadata
6112 =============================
6113
6114 Some targets support embedding of strings into object files to indicate
6115 a set of libraries to add to the link. Typically this is used in conjunction
6116 with language extensions which allow source files to explicitly declare the
6117 libraries they depend on, and have these automatically be transmitted to the
6118 linker via object files.
6119
6120 The list is encoded in the IR using named metadata with the name
6121 ``!llvm.dependent-libraries``. Each operand is expected to be a metadata node
6122 which should contain a single string operand.
6123
6124 For example, the following metadata section contains two library specfiers::
6125
6126 !0 = !{!"a library specifier"}
6127 !1 = !{!"another library specifier"}
6128 !llvm.dependent-libraries = !{ !0, !1 }
6129
6130 Each library specifier will be handled independently by the consuming linker.
6131 The effect of the library specifiers are defined by the consuming linker.
61106132
61116133 .. _summary:
61126134
840840 SHT_LLVM_CALL_GRAPH_PROFILE = 0x6fff4c02, // LLVM Call Graph Profile.
841841 SHT_LLVM_ADDRSIG = 0x6fff4c03, // List of address-significant symbols
842842 // for safe ICF.
843 SHT_LLVM_DEPENDENT_LIBRARIES = 0x6fff4c04, // LLVM Dependent Library Specifiers.
843844 // Android's experimental support for SHT_RELR sections.
844845 // https://android.googlesource.com/platform/bionic/+/b7feec74547f84559a1467aca02708ff61346d2a/libc/include/elf.h#512
845846 SHT_ANDROID_RELR = 0x6fffff00, // Relocation entries; only offsets.
114114 std::vector> ModuleSymIndices;
115115
116116 StringRef TargetTriple, SourceFileName, COFFLinkerOpts;
117 std::vector DependentLibraries;
117118 std::vector ComdatTable;
118119
119120 public:
153154
154155 /// Returns linker options specified in the input file.
155156 StringRef getCOFFLinkerOpts() const { return COFFLinkerOpts; }
157
158 /// Returns dependent library specifiers from the input file.
159 ArrayRef getDependentLibraries() const { return DependentLibraries; }
156160
157161 /// Returns the path to the InputFile.
158162 StringRef getName() const;
4646
4747 std::string LinkerOpts;
4848
49 std::string DependentLibraries;
50
4951 std::unique_ptr Mod;
5052 MemoryBufferRef MBRef;
5153 ModuleSymbolTable SymTab;
152154
153155 StringRef getLinkerOpts() { return LinkerOpts; }
154156
157 StringRef getDependentLibraries() { return DependentLibraries; }
158
155159 const std::vector &getAsmUndefinedRefs() { return _asm_undefines; }
156160
157161 private:
124124 Str SectionName;
125125 };
126126
127
127128 struct Header {
128129 /// Version number of the symtab format. This number should be incremented
129130 /// when the format changes, but it does not need to be incremented if a
130131 /// change to LLVM would cause it to create a different symbol table.
131132 Word Version;
132 enum { kCurrentVersion = 1 };
133 enum { kCurrentVersion = 2 };
133134
134135 /// The producer's version string (LLVM_VERSION_STRING " " LLVM_REVISION).
135136 /// Consumers should rebuild the symbol table from IR if the producer's
146147
147148 /// COFF-specific: linker directives.
148149 Str COFFLinkerOpts;
150
151 /// Dependent Library Specifiers
152 Range DependentLibraries;
149153 };
150154
151155 } // end namespace storage
230234 ArrayRef Comdats;
231235 ArrayRef Symbols;
232236 ArrayRef Uncommons;
237 ArrayRef DependentLibraries;
233238
234239 StringRef str(storage::Str S) const { return S.get(Strtab); }
235240
250255 Comdats = range(header().Comdats);
251256 Symbols = range(header().Symbols);
252257 Uncommons = range(header().Uncommons);
258 DependentLibraries = range(header().DependentLibraries);
253259 }
254260
255261 using symbol_range = iterator_range>;
282288
283289 /// COFF-specific: returns linker options specified in the input file.
284290 StringRef getCOFFLinkerOpts() const { return str(header().COFFLinkerOpts); }
291
292 /// Returns dependent library specifiers
293 std::vector getDependentLibraries() const {
294 std::vector Specifiers;
295 Specifiers.reserve(DependentLibraries.size());
296 for (auto S : DependentLibraries) {
297 Specifiers.push_back(str(S));
298 }
299 return Specifiers;
300 }
285301 };
286302
287303 /// Ephemeral symbols produced by Reader::symbols() and
4343 * @{
4444 */
4545
46 #define LTO_API_VERSION 23
46 #define LTO_API_VERSION 24
4747
4848 /**
4949 * \since prior to LTO_API_VERSION=3
297297 lto_module_get_linkeropts(lto_module_t mod);
298298
299299 /**
300 * Returns the module's dependent library specifiers.
301 *
302 * \since LTO_API_VERSION=24
303 */
304 extern const char*
305 lto_module_get_dependent_libraries(lto_module_t mod);
306
307 /**
300308 * Diagnostic severity.
301309 *
302310 * \since LTO_API_VERSION=7
270270 }
271271 }
272272
273 if (NamedMDNode *DependentLibraries = M.getNamedMetadata("llvm.dependent-libraries")) {
274 auto *S = C.getELFSection(".deplibs", ELF::SHT_LLVM_DEPENDENT_LIBRARIES,
275 ELF::SHF_MERGE | ELF::SHF_STRINGS, 1, "");
276
277 Streamer.SwitchSection(S);
278
279 for (const auto &Operand : DependentLibraries->operands()) {
280 Streamer.EmitBytes(
281 cast(cast(Operand)->getOperand(0))->getString());
282 Streamer.EmitIntValue(0, 1);
283 }
284 }
285
273286 unsigned Version = 0;
274287 unsigned Flags = 0;
275288 StringRef Section;
424424 File->TargetTriple = FOrErr->TheReader.getTargetTriple();
425425 File->SourceFileName = FOrErr->TheReader.getSourceFileName();
426426 File->COFFLinkerOpts = FOrErr->TheReader.getCOFFLinkerOpts();
427 File->DependentLibraries = FOrErr->TheReader.getDependentLibraries();
427428 File->ComdatTable = FOrErr->TheReader.getComdatTable();
428429
429430 for (unsigned I = 0; I != FOrErr->Mods.size(); ++I) {
645645 emitLinkerFlagsForGlobalCOFF(OS, Sym.symbol, TT, M);
646646 }
647647
648 // Add other interesting metadata here.
649 }
648 // Dependent Libraries
649 raw_string_ostream OSD(DependentLibraries);
650 if (NamedMDNode *DependentLibraries = getModule().getNamedMetadata("llvm.dependent-libraries"))
651 for (MDNode *N : DependentLibraries->operands())
652 OSD << " " << cast(N->getOperand(0))->getString();
653 }
614614 Type = ELF::SHT_LLVM_LINKER_OPTIONS;
615615 else if (TypeName == "llvm_call_graph_profile")
616616 Type = ELF::SHT_LLVM_CALL_GRAPH_PROFILE;
617 else if (TypeName == "llvm_dependent_libraries")
618 Type = ELF::SHT_LLVM_DEPENDENT_LIBRARIES;
617619 else if (TypeName.getAsInteger(0, Type))
618620 return TokError("unknown section type");
619621 }
151151 OS << "llvm_linker_options";
152152 else if (Type == ELF::SHT_LLVM_CALL_GRAPH_PROFILE)
153153 OS << "llvm_call_graph_profile";
154 else if (Type == ELF::SHT_LLVM_DEPENDENT_LIBRARIES)
155 OS << "llvm_dependent_libraries";
154156 else
155157 report_fatal_error("unsupported type 0x" + Twine::utohexstr(Type) +
156158 " for section " + getSectionName());
252252 STRINGIFY_ENUM_CASE(ELF, SHT_LLVM_LINKER_OPTIONS);
253253 STRINGIFY_ENUM_CASE(ELF, SHT_LLVM_CALL_GRAPH_PROFILE);
254254 STRINGIFY_ENUM_CASE(ELF, SHT_LLVM_ADDRSIG);
255 STRINGIFY_ENUM_CASE(ELF, SHT_LLVM_DEPENDENT_LIBRARIES);
255256 STRINGIFY_ENUM_CASE(ELF, SHT_GNU_ATTRIBUTES);
256257 STRINGIFY_ENUM_CASE(ELF, SHT_GNU_HASH);
257258 STRINGIFY_ENUM_CASE(ELF, SHT_GNU_verdef);
8888 std::string COFFLinkerOpts;
8989 raw_string_ostream COFFLinkerOptsOS{COFFLinkerOpts};
9090
91 std::vector DependentLibraries;
92
9193 void setStr(storage::Str &S, StringRef Value) {
9294 S.Offset = StrtabBuilder.add(Value);
9395 S.Size = Value.size();
136138 for (MDNode *MDOptions : LinkerOptions->operands())
137139 for (const MDOperand &MDOption : cast(MDOptions)->operands())
138140 COFFLinkerOptsOS << " " << cast(MDOption)->getString();
141 }
142 }
143
144 if (TT.isOSBinFormatELF()) {
145 if (auto E = M->materializeMetadata())
146 return E;
147 if (NamedMDNode *N = M->getNamedMetadata("llvm.dependent-libraries")) {
148 for (MDNode *MDOptions : N->operands()) {
149 const auto OperandStr =
150 cast(cast(MDOptions)->getOperand(0))->getString();
151 storage::Str Specifier;
152 setStr(Specifier, OperandStr);
153 DependentLibraries.emplace_back(Specifier);
154 }
139155 }
140156 }
141157
311327 writeRange(Hdr.Comdats, Comdats);
312328 writeRange(Hdr.Symbols, Syms);
313329 writeRange(Hdr.Uncommons, Uncommons);
314
330 writeRange(Hdr.DependentLibraries, DependentLibraries);
315331 *reinterpret_cast(Symtab.data()) = Hdr;
316332 return Error::success();
317333 }
454454 ECase(SHT_LLVM_LINKER_OPTIONS);
455455 ECase(SHT_LLVM_CALL_GRAPH_PROFILE);
456456 ECase(SHT_LLVM_ADDRSIG);
457 ECase(SHT_LLVM_DEPENDENT_LIBRARIES);
457458 ECase(SHT_GNU_ATTRIBUTES);
458459 ECase(SHT_GNU_HASH);
459460 ECase(SHT_GNU_verdef);
0 ; RUN: llc -mtriple x86_64-elf -filetype asm -o - %s | FileCheck %s
1 ; REQUIRES: x86-registered-target
2
3 !llvm.dependent-libraries = !{!0, !1, !0}
4
5 !0 = !{!"foo"}
6 !1 = !{!"b a r"}
7
8 ; CHECK: .section .deplibs,"MS",@llvm_dependent_libraries,1
9 ; CHECK-NEXT: .ascii "foo"
10 ; CHECK-NEXT: .byte 0
11 ; CHECK-NEXT: .ascii "b a r"
12 ; CHECK-NEXT: .byte 0
13 ; CHECK-NEXT: .ascii "foo"
14 ; CHECK-NEXT: .byte 0
88 !0 = !{!"/include:foo"}
99 !llvm.linker.options = !{ !0 }
1010
11 ; CHECK: {{^dependent libraries: \"foo\" \"b a r\" \"baz\"$}}
12 !1 = !{!"foo"}
13 !2 = !{!"b a r"}
14 !3 = !{!"baz"}
15 !llvm.dependent-libraries = !{!1, !2, !3}
16
1117 @g1 = global i32 0
1218
1319 ; CHECK-NOT: fallback g1
292292 // CHECK-NEXT: ]
293293 // CHECK: }
294294
295 // Test SHT_LLVM_DEPENDENT_LIBRARIES
296
297 .section .deplibs,"MS",@llvm_dependent_libraries,1
298 // ASM: .section .deplibs,"MS",@llvm_dependent_libraries,1
299
300 // CHECK: Section {
301 // CHECK: Name: .deplibs
302 // CHECK-NEXT: Type: SHT_LLVM_DEPENDENT_LIBRARIES
303 // CHECK-NEXT: Flags [
304 // CHECK-NEXT: SHF_MERGE
305 // CHECK-NEXT: SHF_STRINGS
306 // CHECK-NEXT: ]
307 // CHECK: }
88
99 ; BCA:
1010 ; Version stored at offset 0.
11 ; BCA-NEXT: blob data = '\x01\x00\x00\x00\x06\x00\x00\x00\x08\x00\x00\x00D\x00\x00\x00\x01\x00\x00\x00P\x00\x00\x00\x00\x00\x00\x00P\x00\x00\x00\x02\x00\x00\x00\x80\x00\x00\x00\x00\x00\x00\x00\x0E\x00\x00\x00\x18\x00\x00\x00&\x00\x00\x00\x0B\x00\x00\x001\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\xFF\xFF\xFF\xFF\x00$\x00\x00\x03\x00\x00\x00\x03\x00\x00\x00\x03\x00\x00\x00\x03\x00\x00\x00\xFF\xFF\xFF\xFF\x08$\x00\x00'
11 ; BCA-NEXT: blob data = '\x02\x00\x00\x00\x06\x00\x00\x00\x08\x00\x00\x00L\x00\x00\x00\x01\x00\x00\x00X\x00\x00\x00\x00\x00\x00\x00X\x00\x00\x00\x02\x00\x00\x00\x88\x00\x00\x00\x00\x00\x00\x00\x0E\x00\x00\x00\x18\x00\x00\x00&\x00\x00\x00\x0B\x00\x00\x001\x00\x00\x00\x00\x00\x00\x00\x88\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\xFF\xFF\xFF\xFF\x00$\x00\x00\x03\x00\x00\x00\x03\x00\x00\x00\x03\x00\x00\x00\x03\x00\x00\x00\xFF\xFF\xFF\xFF\x08$\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x03\x00\x00\x00\x03\x00\x00\x00'
1212 ; BCA-NEXT:
1313 ; BCA-NEXT:
1414 ; BCA-NEXT: blob data = 'foobarproducerx86_64-unknown-linux-gnuirsymtab.ll'
1515 ; BCA-NEXT:
1616
17 ; SYMTAB: version: 1
17 ; SYMTAB: version: 2
1818 ; SYMTAB-NEXT: producer: producer
1919 ; SYMTAB-NEXT: target triple: x86_64-unknown-linux-gnu
2020 ; SYMTAB-NEXT: source filename: irsymtab.ll
21 ; SYMTAB-NEXT: {{^dependent libraries: \"foo\" \"bar\"$}}
2122 ; SYMTAB-NEXT: D------X foo
2223 ; SYMTAB-NEXT: DU-----X bar
2324
3031 }
3132
3233 declare void @bar()
34
35 !llvm.dependent-libraries = !{!0, !1}
36
37 !0 = !{!"foo"}
38 !1 = !{!"bar"}
359359 if (TT.isOSBinFormatCOFF())
360360 outs() << "linker opts: " << Input->getCOFFLinkerOpts() << '\n';
361361
362 if (TT.isOSBinFormatELF()) {
363 outs() << "dependent libraries:";
364 for (auto L : Input->getDependentLibraries())
365 outs() << " \"" << L << "\"";
366 outs() << '\n';
367 }
368
362369 std::vector ComdatTable = Input->getComdatTable();
363370 for (const InputFile::Symbol &Sym : Input->symbols()) {
364371 switch (Sym.getVisibility()) {
28562856 return "LLVM_CALL_GRAPH_PROFILE";
28572857 case SHT_LLVM_ADDRSIG:
28582858 return "LLVM_ADDRSIG";
2859 case SHT_LLVM_DEPENDENT_LIBRARIES:
2860 return "LLVM_DEPENDENT_LIBRARIES";
28592861 // FIXME: Parse processor specific GNU attributes
28602862 case SHT_GNU_ATTRIBUTES:
28612863 return "ATTRIBUTES";
322322
323323 const char* lto_module_get_linkeropts(lto_module_t mod) {
324324 return unwrap(mod)->getLinkerOpts().data();
325 }
326
327 const char* lto_module_get_dependent_libraries(lto_module_t mod) {
328 return unwrap(mod)->getDependentLibraries().data();
325329 }
326330
327331 void lto_codegen_set_diagnostic_handler(lto_code_gen_t cg,