llvm.org GIT mirror llvm / 1123a81
[PowerPC] Yet another approach to __tls_get_addr This patch is a third attempt to properly handle the local-dynamic and global-dynamic TLS models. In my original implementation, calls to __tls_get_addr were hidden from view until the asm-printer phase, at which point the underlying branch-and-link instruction was created with proper relocations. This mostly worked well, but I used some repellent techniques to ensure that the TLS_GET_ADDR nodes at the SD and MI levels correctly received input from GPR3 and produced output into GPR3. This proved to work badly in the presence of multiple TLS variable accesses, with the copies to and from GPR3 being scheduled incorrectly and generally creating havoc. In r221703, I addressed that problem by representing the calls to __tls_get_addr as true calls during instruction lowering. This had the advantage of removing all of the bad hacks and relying on the existing call machinery to properly glue the copies in place. It looked like this was going to be the right way to go. However, as a side effect of the recent discovery of problems with linker optimizations for TLS, we discovered cases of suboptimal code generation with this strategy. The problem comes when tls_get_addr is called for the same address, and there is a resulting CSE opportunity. It turns out that in such cases MachineCSE will common the addis/addi instructions that set up the input value to tls_get_addr, but will not common the calls themselves. MachineCSE does not have any machinery to common idempotent calls. This is perfectly sensible, since presumably this would be done at the IR level, and introducing calls in the back end isn't commonplace. In any case, we end up with two calls to __tls_get_addr when one would suffice, and that isn't good. I presumed that the original design would have allowed commoning of the machine-specific nodes that hid the __tls_get_addr calls, so as suggested by Ulrich Weigand, I went back to that design and cleaned it up so that the copies were properly held together by glue nodes. However, it turned out that this didn't work either...the presence of copies to physical registers kept the machine-specific nodes from being commoned also. All of which leads to the design presented here. This is a return to the original design, except that no attempt is made to introduce copies to and from GPR3 during instruction lowering. Virtual registers are used until prior to register allocation. At that point, a special pass is run that identifies the machine-specific nodes that hide the tls_get_addr calls and introduces the copies to and from GPR3 around them. The register allocator then coalesces these copies away. With this design, MachineCSE succeeds in commoning tls_get_addr calls where possible, and we get nice optimal code generation (better than GCC at the moment, which does not common these calls). One additional problem must be dealt with: After introducing the mentions of the physical register GPR3, the aggressive anti-dependence breaker sees opportunities to improve scheduling by selecting a different register instead. Flags must be used on the instruction descriptions to tell the anti-dependence breaker to keep its hands in its pockets. One thing missing from the original design was recording a definition of the link register on the GET_TLS_ADDR nodes. Doing this was found to be insufficient to force a stack frame to be created, which led to looping behavior because two different LR values were stored at the same address. This appears to have been an oversight in PPCFrameLowering::determineFrameLayout(), which is repaired here. Because MustSaveLR() returns true for calls to builtin_return_address, this changed the expected behavior of test/CodeGen/PowerPC/retaddr2.ll, which now stacks a frame but formerly did not. I've fixed the test case to reflect this. There are existing TLS tests to catch regressions; the checks in test/CodeGen/PowerPC/tls-store2.ll proved to be too restrictive in the face of instruction scheduling with these changes, so I fixed that up. I've added a new test case based on the PrettyStackTrace module that demonstrated the original problem. This checks that we get correct code generation and that CSE of the calls to __get_tls_addr has taken place. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227976 91177308-0d34-0410-b5e6-96231b3b80d8 Bill Schmidt 5 years ago
14 changed file(s) with 287 addition(s) and 114 deletion(s). Raw diff Collapse all Expand all
3030 PPCTargetObjectFile.cpp
3131 PPCTargetTransformInfo.cpp
3232 PPCSelectionDAGInfo.cpp
33 PPCTLSDynamicCall.cpp
3334 PPCVSXCopy.cpp
3435 PPCVSXFMAMutate.cpp
3536 )
3838 FunctionPass *createPPCVSXFMAMutatePass();
3939 FunctionPass *createPPCBranchSelectionPass();
4040 FunctionPass *createPPCISelDag(PPCTargetMachine &TM);
41 FunctionPass *createPPCTLSDynamicCallPass();
4142 void LowerPPCMachineInstrToMCInst(const MachineInstr *MI, MCInst &OutMI,
4243 AsmPrinter &AP, bool isDarwin);
4344
8889 MO_TOC_LO = 7 << 4,
8990
9091 // Symbol for VK_PPC_TLS fixup attached to an ADD instruction
91 MO_TLS = 8 << 4,
92
93 // Symbols for VK_PPC_TLSGD and VK_PPC_TLSLD in __tls_get_addr
94 // call sequences.
95 MO_TLSLD = 9 << 4,
96 MO_TLSGD = 10 << 4
92 MO_TLS = 8 << 4
9793 };
9894 } // end namespace PPCII
9995
100100 const MachineInstr &MI);
101101 void LowerPATCHPOINT(MCStreamer &OutStreamer, StackMaps &SM,
102102 const MachineInstr &MI);
103 void EmitTlsCall(const MachineInstr *MI, MCSymbolRefExpr::VariantKind VK);
103104 };
104105
105106 /// PPCLinuxAsmPrinter - PowerPC assembly printer, customized for Linux
403404 EmitToStreamer(OutStreamer, MCInstBuilder(PPC::NOP));
404405 }
405406
407 /// EmitTlsCall -- Given a GETtls[ld]ADDR[32] instruction, print a
408 /// call to __tls_get_addr to the current output stream.
409 void PPCAsmPrinter::EmitTlsCall(const MachineInstr *MI,
410 MCSymbolRefExpr::VariantKind VK) {
411 StringRef Name = "__tls_get_addr";
412 MCSymbol *TlsGetAddr = OutContext.GetOrCreateSymbol(Name);
413 MCSymbolRefExpr::VariantKind Kind = MCSymbolRefExpr::VK_None;
414
415 assert(MI->getOperand(0).isReg() &&
416 ((Subtarget.isPPC64() && MI->getOperand(0).getReg() == PPC::X3) ||
417 (!Subtarget.isPPC64() && MI->getOperand(0).getReg() == PPC::R3)) &&
418 "GETtls[ld]ADDR[32] must define GPR3");
419 assert(MI->getOperand(1).isReg() &&
420 ((Subtarget.isPPC64() && MI->getOperand(1).getReg() == PPC::X3) ||
421 (!Subtarget.isPPC64() && MI->getOperand(1).getReg() == PPC::R3)) &&
422 "GETtls[ld]ADDR[32] must read GPR3");
423
424 if (!Subtarget.isPPC64() && !Subtarget.isDarwin() &&
425 TM.getRelocationModel() == Reloc::PIC_)
426 Kind = MCSymbolRefExpr::VK_PLT;
427 const MCSymbolRefExpr *TlsRef =
428 MCSymbolRefExpr::Create(TlsGetAddr, Kind, OutContext);
429 const MachineOperand &MO = MI->getOperand(2);
430 const GlobalValue *GValue = MO.getGlobal();
431 MCSymbol *MOSymbol = getSymbol(GValue);
432 const MCExpr *SymVar = MCSymbolRefExpr::Create(MOSymbol, VK, OutContext);
433 EmitToStreamer(OutStreamer,
434 MCInstBuilder(Subtarget.isPPC64() ?
435 PPC::BL8_NOP_TLS : PPC::BL_TLS)
436 .addExpr(TlsRef)
437 .addExpr(SymVar));
438 }
439
406440 /// EmitInstruction -- Print out a single PowerPC MI in Darwin syntax to
407441 /// the current output stream.
408442 ///
806840 .addExpr(SymGotTlsGD));
807841 return;
808842 }
843 case PPC::GETtlsADDR:
844 // Transform: %X3 = GETtlsADDR %X3,
845 // Into: BL8_NOP_TLS __tls_get_addr(sym@tlsgd)
846 case PPC::GETtlsADDR32: {
847 // Transform: %R3 = GETtlsADDR32 %R3,
848 // Into: BL_TLS __tls_get_addr(sym@tlsgd)@PLT
849 EmitTlsCall(MI, MCSymbolRefExpr::VK_PPC_TLSGD);
850 return;
851 }
809852 case PPC::ADDIStlsldHA: {
810853 // Transform: %Xd = ADDIStlsldHA %X2,
811854 // Into: %Xd = ADDIS8 %X2, sym@got@tlsld@ha
841884 .addReg(MI->getOperand(0).getReg())
842885 .addReg(MI->getOperand(1).getReg())
843886 .addExpr(SymGotTlsLD));
887 return;
888 }
889 case PPC::GETtlsldADDR:
890 // Transform: %X3 = GETtlsldADDR %X3,
891 // Into: BL8_NOP_TLS __tls_get_addr(sym@tlsld)
892 case PPC::GETtlsldADDR32: {
893 // Transform: %R3 = GETtlsldADDR32 %R3,
894 // Into: BL_TLS __tls_get_addr(sym@tlsld)@PLT
895 EmitTlsCall(MI, MCSymbolRefExpr::VK_PPC_TLSLD);
844896 return;
845897 }
846898 case PPC::ADDISdtprelHA:
354354 return FuncInfo->hasNonRISpills();
355355 }
356356
357 /// MustSaveLR - Return true if this function requires that we save the LR
358 /// register onto the stack in the prolog and restore it in the epilog of the
359 /// function.
360 static bool MustSaveLR(const MachineFunction &MF, unsigned LR) {
361 const PPCFunctionInfo *MFI = MF.getInfo();
362
363 // We need a save/restore of LR if there is any def of LR (which is
364 // defined by calls, including the PIC setup sequence), or if there is
365 // some use of the LR stack slot (e.g. for builtin_return_address).
366 // (LR comes in 32 and 64 bit versions.)
367 MachineRegisterInfo::def_iterator RI = MF.getRegInfo().def_begin(LR);
368 return RI !=MF.getRegInfo().def_end() || MFI->isLRStoreRequired();
369 }
370
357371 /// determineFrameLayout - Determine the size of the frame and maximum call
358372 /// frame size.
359373 unsigned PPCFrameLowering::determineFrameLayout(MachineFunction &MF,
380394 // stackless code if all local vars are reg-allocated.
381395 bool DisableRedZone = MF.getFunction()->getAttributes().
382396 hasAttribute(AttributeSet::FunctionIndex, Attribute::NoRedZone);
397 unsigned LR = RegInfo->getRARegister();
383398 if (!DisableRedZone &&
384399 (Subtarget.isPPC64() || // 32-bit SVR4, no stack-
385400 !Subtarget.isSVR4ABI() || // allocated locals.
387402 FrameSize <= 224 && // Fits in red zone.
388403 !MFI->hasVarSizedObjects() && // No dynamic alloca.
389404 !MFI->adjustsStack() && // No calls.
405 !MustSaveLR(MF, LR) &&
390406 !RegInfo->hasBasePointer(MF)) { // No special alignment.
391407 // No need for frame
392408 if (UpdateMF)
11071123 }
11081124 }
11091125
1110 /// MustSaveLR - Return true if this function requires that we save the LR
1111 /// register onto the stack in the prolog and restore it in the epilog of the
1112 /// function.
1113 static bool MustSaveLR(const MachineFunction &MF, unsigned LR) {
1114 const PPCFunctionInfo *MFI = MF.getInfo();
1115
1116 // We need a save/restore of LR if there is any def of LR (which is
1117 // defined by calls, including the PIC setup sequence), or if there is
1118 // some use of the LR stack slot (e.g. for builtin_return_address).
1119 // (LR comes in 32 and 64 bit versions.)
1120 MachineRegisterInfo::def_iterator RI = MF.getRegInfo().def_begin(LR);
1121 return RI !=MF.getRegInfo().def_end() || MFI->isLRStoreRequired();
1122 }
1123
11241126 void
11251127 PPCFrameLowering::processFunctionBeforeCalleeSavedScan(MachineFunction &MF,
11261128 RegScavenger *) const {
790790 case PPCISD::SHL: return "PPCISD::SHL";
791791 case PPCISD::CALL: return "PPCISD::CALL";
792792 case PPCISD::CALL_NOP: return "PPCISD::CALL_NOP";
793 case PPCISD::CALL_TLS: return "PPCISD::CALL_TLS";
794 case PPCISD::CALL_NOP_TLS: return "PPCISD::CALL_NOP_TLS";
795793 case PPCISD::MTCTR: return "PPCISD::MTCTR";
796794 case PPCISD::BCTRL: return "PPCISD::BCTRL";
797795 case PPCISD::BCTRL_LOAD_TOC: return "PPCISD::BCTRL_LOAD_TOC";
825823 case PPCISD::ADD_TLS: return "PPCISD::ADD_TLS";
826824 case PPCISD::ADDIS_TLSGD_HA: return "PPCISD::ADDIS_TLSGD_HA";
827825 case PPCISD::ADDI_TLSGD_L: return "PPCISD::ADDI_TLSGD_L";
826 case PPCISD::GET_TLS_ADDR: return "PPCISD::GET_TLS_ADDR";
828827 case PPCISD::ADDIS_TLSLD_HA: return "PPCISD::ADDIS_TLSLD_HA";
829828 case PPCISD::ADDI_TLSLD_L: return "PPCISD::ADDI_TLSLD_L";
829 case PPCISD::GET_TLSLD_ADDR: return "PPCISD::GET_TLSLD_ADDR";
830830 case PPCISD::ADDIS_DTPREL_HA: return "PPCISD::ADDIS_DTPREL_HA";
831831 case PPCISD::ADDI_DTPREL_L: return "PPCISD::ADDI_DTPREL_L";
832832 case PPCISD::VADD_SPLAT: return "PPCISD::VADD_SPLAT";
16851685 return LowerLabelRef(TgtBAHi, TgtBALo, isPIC, DAG);
16861686 }
16871687
1688 // Generate a call to __tls_get_addr for the given GOT entry Op.
1689 std::pair
1690 PPCTargetLowering::lowerTLSCall(SDValue Op, SDLoc dl,
1691 SelectionDAG &DAG) const {
1692
1693 Type *IntPtrTy = getDataLayout()->getIntPtrType(*DAG.getContext());
1694 TargetLowering::ArgListTy Args;
1695 TargetLowering::ArgListEntry Entry;
1696 Entry.Node = Op;
1697 Entry.Ty = IntPtrTy;
1698 Args.push_back(Entry);
1699
1700 TargetLowering::CallLoweringInfo CLI(DAG);
1701 CLI.setDebugLoc(dl).setChain(DAG.getEntryNode())
1702 .setCallee(CallingConv::C, IntPtrTy,
1703 DAG.getTargetExternalSymbol("__tls_get_addr", getPointerTy()),
1704 std::move(Args), 0);
1705
1706 return LowerCallTo(CLI);
1707 }
1708
17091688 SDValue PPCTargetLowering::LowerGlobalTLSAddress(SDValue Op,
17101689 SelectionDAG &DAG) const {
17111690
17521731 }
17531732
17541733 if (Model == TLSModel::GeneralDynamic) {
1755 SDValue TGA = DAG.getTargetGlobalAddress(GV, dl, PtrVT, 0,
1756 PPCII::MO_TLSGD);
1734 SDValue TGA = DAG.getTargetGlobalAddress(GV, dl, PtrVT, 0, 0);
17571735 SDValue GOTPtr;
17581736 if (is64bit) {
17591737 setUsesTOCBasePtr(DAG);
17661744 else
17671745 GOTPtr = DAG.getNode(PPCISD::PPC32_PICGOT, dl, PtrVT);
17681746 }
1769 SDValue GOTEntry = DAG.getNode(PPCISD::ADDI_TLSGD_L, dl, PtrVT,
1770 GOTPtr, TGA);
1771 std::pair CallResult = lowerTLSCall(GOTEntry, dl, DAG);
1772 return CallResult.first;
1747 SDValue GOTEntry = DAG.getNode(PPCISD::ADDI_TLSGD_L, dl,
1748 PtrVT, GOTPtr, TGA);
1749 return DAG.getNode(PPCISD::GET_TLS_ADDR, dl, PtrVT, GOTEntry, TGA);
17731750 }
17741751
17751752 if (Model == TLSModel::LocalDynamic) {
1776 SDValue TGA = DAG.getTargetGlobalAddress(GV, dl, PtrVT, 0,
1777 PPCII::MO_TLSLD);
1753 SDValue TGA = DAG.getTargetGlobalAddress(GV, dl, PtrVT, 0, 0);
17781754 SDValue GOTPtr;
17791755 if (is64bit) {
17801756 setUsesTOCBasePtr(DAG);
17891765 }
17901766 SDValue GOTEntry = DAG.getNode(PPCISD::ADDI_TLSLD_L, dl, PtrVT,
17911767 GOTPtr, TGA);
1792 std::pair CallResult = lowerTLSCall(GOTEntry, dl, DAG);
1793 SDValue TLSAddr = CallResult.first;
1794 SDValue Chain = CallResult.second;
1795 SDValue DtvOffsetHi = DAG.getNode(PPCISD::ADDIS_DTPREL_HA, dl, PtrVT,
1796 Chain, TLSAddr, TGA);
1768 SDValue TLSAddr = DAG.getNode(PPCISD::GET_TLSLD_ADDR, dl,
1769 PtrVT, GOTEntry, TGA);
1770 SDValue DtvOffsetHi = DAG.getNode(PPCISD::ADDIS_DTPREL_HA, dl,
1771 PtrVT, TLSAddr, TGA);
17971772 return DAG.getNode(PPCISD::ADDI_DTPREL_L, dl, PtrVT, DtvOffsetHi, TGA);
17981773 }
17991774
38173792 if (Callee.getNode()) {
38183793 Ops.push_back(Chain);
38193794 Ops.push_back(Callee);
3820
3821 // If this is a call to __tls_get_addr, find the symbol whose address
3822 // is to be taken and add it to the list. This will be used to
3823 // generate __tls_get_addr(@tlsgd) or __tls_get_addr(@tlsld).
3824 // We find the symbol by walking the chain to the CopyFromReg, walking
3825 // back from the CopyFromReg to the ADDI_TLSGD_L or ADDI_TLSLD_L, and
3826 // pulling the symbol from that node.
3827 if (ExternalSymbolSDNode *S = dyn_cast(Callee))
3828 if (!strcmp(S->getSymbol(), "__tls_get_addr")) {
3829 assert(!needIndirectCall && "Indirect call to __tls_get_addr???");
3830 SDNode *AddI = Chain.getNode()->getOperand(2).getNode();
3831 SDValue TGTAddr = AddI->getOperand(1);
3832 assert(TGTAddr.getNode()->getOpcode() == ISD::TargetGlobalTLSAddress &&
3833 "Didn't find target global TLS address where we expected one");
3834 Ops.push_back(TGTAddr);
3835 CallOpc = PPCISD::CALL_TLS;
3836 }
38373795 }
38383796 // If this is a tail call add stack pointer delta.
38393797 if (isTailCall)
39963954 Ops.insert(std::next(Ops.begin()), AddTOC);
39973955 } else if ((CallOpc == PPCISD::CALL) &&
39983956 (!isLocalCall(Callee) ||
3999 DAG.getTarget().getRelocationModel() == Reloc::PIC_)) {
3957 DAG.getTarget().getRelocationModel() == Reloc::PIC_))
40003958 // Otherwise insert NOP for non-local calls.
40013959 CallOpc = PPCISD::CALL_NOP;
4002 } else if (CallOpc == PPCISD::CALL_TLS)
4003 // For 64-bit SVR4, TLS calls are always non-local.
4004 CallOpc = PPCISD::CALL_NOP_TLS;
40053960 }
40063961
40073962 Chain = DAG.getNode(CallOpc, dl, NodeTys, Ops);
100100 /// SVR4 calls.
101101 CALL, CALL_NOP,
102102
103 /// CALL_TLS and CALL_NOP_TLS - Versions of CALL and CALL_NOP used
104 /// to access TLS variables.
105 CALL_TLS, CALL_NOP_TLS,
106
107103 /// CHAIN,FLAG = MTCTR(VAL, CHAIN[, INFLAG]) - Directly corresponds to a
108104 /// MTCTR instruction.
109105 MTCTR,
227223 /// sym\@got\@tlsgd\@l.
228224 ADDI_TLSGD_L,
229225
226 /// G8RC = GET_TLS_ADDR %X3, Symbol - For the general-dynamic TLS
227 /// model, produces a call to __tls_get_addr(sym\@tlsgd).
228 GET_TLS_ADDR,
229
230230 /// G8RC = ADDIS_TLSLD_HA %X2, Symbol - For the local-dynamic TLS
231231 /// model, produces an ADDIS8 instruction that adds the GOT base
232232 /// register to sym\@got\@tlsld\@ha.
237237 /// sym\@got\@tlsld\@l.
238238 ADDI_TLSLD_L,
239239
240 /// G8RC = ADDIS_DTPREL_HA %X3, Symbol, Chain - For the
241 /// local-dynamic TLS model, produces an ADDIS8 instruction
242 /// that adds X3 to sym\@dtprel\@ha. The Chain operand is needed
243 /// to tie this in place following a copy to %X3 from the result
244 /// of a GET_TLSLD_ADDR.
240 /// G8RC = GET_TLSLD_ADDR %X3, Symbol - For the local-dynamic TLS
241 /// model, produces a call to __tls_get_addr(sym\@tlsld).
242 GET_TLSLD_ADDR,
243
244 /// G8RC = ADDIS_DTPREL_HA %X3, Symbol - For the local-dynamic TLS
245 /// model, produces an ADDIS8 instruction that adds X3 to
246 /// sym\@dtprel\@ha.
245247 ADDIS_DTPREL_HA,
246248
247249 /// G8RC = ADDI_DTPREL_L G8RReg, Symbol - For the local-dynamic TLS
634636 SDValue LowerFRAMEADDR(SDValue Op, SelectionDAG &DAG) const;
635637 SDValue LowerConstantPool(SDValue Op, SelectionDAG &DAG) const;
636638 SDValue LowerBlockAddress(SDValue Op, SelectionDAG &DAG) const;
637 std::pair lowerTLSCall(SDValue Op, SDLoc dl,
638 SelectionDAG &DAG) const;
639639 SDValue LowerGlobalTLSAddress(SDValue Op, SelectionDAG &DAG) const;
640640 SDValue LowerGlobalAddress(SDValue Op, SelectionDAG &DAG) const;
641641 SDValue LowerJumpTable(SDValue Op, SelectionDAG &DAG) const;
200200 (BL8 texternalsym:$dst)>;
201201 def : Pat<(PPCcall_nop (i64 texternalsym:$dst)),
202202 (BL8_NOP texternalsym:$dst)>;
203
204 def : Pat<(PPCcall_nop_tls texternalsym:$func, tglobaltlsaddr:$sym),
205 (BL8_NOP_TLS texternalsym:$func, tglobaltlsaddr:$sym)>;
206203
207204 // Atomic operations
208205 let usesCustomInserter = 1 in {
903900 [(set i64:$rD,
904901 (PPCaddiTlsgdL i64:$reg, tglobaltlsaddr:$disp))]>,
905902 isPPC64;
903 let hasExtraSrcRegAllocReq = 1, hasExtraDefRegAllocReq = 1, Defs = [LR8] in
904 def GETtlsADDR : Pseudo<(outs g8rc:$rD), (ins g8rc:$reg, tlsgd:$sym),
905 "#GETtlsADDR",
906 [(set i64:$rD,
907 (PPCgetTlsAddr i64:$reg, tglobaltlsaddr:$sym))]>,
908 isPPC64;
906909 def ADDIStlsldHA: Pseudo<(outs g8rc:$rD), (ins g8rc_nox0:$reg, s16imm64:$disp),
907910 "#ADDIStlsldHA",
908911 [(set i64:$rD,
913916 [(set i64:$rD,
914917 (PPCaddiTlsldL i64:$reg, tglobaltlsaddr:$disp))]>,
915918 isPPC64;
919 let hasExtraSrcRegAllocReq = 1, hasExtraDefRegAllocReq = 1, Defs = [LR8] in
920 def GETtlsldADDR : Pseudo<(outs g8rc:$rD), (ins g8rc:$reg, tlsgd:$sym),
921 "#GETtlsldADDR",
922 [(set i64:$rD,
923 (PPCgetTlsldAddr i64:$reg, tglobaltlsaddr:$sym))]>,
924 isPPC64;
916925 def ADDISdtprelHA: Pseudo<(outs g8rc:$rD), (ins g8rc_nox0:$reg, s16imm64:$disp),
917926 "#ADDISdtprelHA",
918927 [(set i64:$rD,
109109 def PPCaddTls : SDNode<"PPCISD::ADD_TLS", SDTIntBinOp, []>;
110110 def PPCaddisTlsgdHA : SDNode<"PPCISD::ADDIS_TLSGD_HA", SDTIntBinOp>;
111111 def PPCaddiTlsgdL : SDNode<"PPCISD::ADDI_TLSGD_L", SDTIntBinOp>;
112 def PPCgetTlsAddr : SDNode<"PPCISD::GET_TLS_ADDR", SDTIntBinOp>;
112113 def PPCaddisTlsldHA : SDNode<"PPCISD::ADDIS_TLSLD_HA", SDTIntBinOp>;
113114 def PPCaddiTlsldL : SDNode<"PPCISD::ADDI_TLSLD_L", SDTIntBinOp>;
114 def PPCaddisDtprelHA : SDNode<"PPCISD::ADDIS_DTPREL_HA", SDTIntBinOp,
115 [SDNPHasChain]>;
115 def PPCgetTlsldAddr : SDNode<"PPCISD::GET_TLSLD_ADDR", SDTIntBinOp>;
116 def PPCaddisDtprelHA : SDNode<"PPCISD::ADDIS_DTPREL_HA", SDTIntBinOp>;
116117 def PPCaddiDtprelL : SDNode<"PPCISD::ADDI_DTPREL_L", SDTIntBinOp>;
117118
118119 def PPCvperm : SDNode<"PPCISD::VPERM", SDT_PPCvperm, []>;
135136 def PPCcall : SDNode<"PPCISD::CALL", SDT_PPCCall,
136137 [SDNPHasChain, SDNPOptInGlue, SDNPOutGlue,
137138 SDNPVariadic]>;
138 def PPCcall_tls : SDNode<"PPCISD::CALL_TLS", SDT_PPCCall,
139 [SDNPHasChain, SDNPOptInGlue, SDNPOutGlue,
140 SDNPVariadic]>;
141139 def PPCcall_nop : SDNode<"PPCISD::CALL_NOP", SDT_PPCCall,
142140 [SDNPHasChain, SDNPOptInGlue, SDNPOutGlue,
143141 SDNPVariadic]>;
144 def PPCcall_nop_tls : SDNode<"PPCISD::CALL_NOP_TLS", SDT_PPCCall,
145 [SDNPHasChain, SDNPOptInGlue, SDNPOutGlue,
146 SDNPVariadic]>;
147142 def PPCmtctr : SDNode<"PPCISD::MTCTR", SDT_PPCCall,
148143 [SDNPHasChain, SDNPOptInGlue, SDNPOutGlue]>;
149144 def PPCbctrl : SDNode<"PPCISD::BCTRL", SDTNone,
24582453 def : Pat<(PPCcall (i32 texternalsym:$dst)),
24592454 (BL texternalsym:$dst)>;
24602455
2461 def : Pat<(PPCcall_tls texternalsym:$func, tglobaltlsaddr:$sym),
2462 (BL_TLS texternalsym:$func, tglobaltlsaddr:$sym)>;
2463
24642456 def : Pat<(PPCtc_return (i32 tglobaladdr:$dst), imm:$imm),
24652457 (TCRETURNdi tglobaladdr:$dst, imm:$imm)>;
24662458
25152507 "#ADDItlsgdL32",
25162508 [(set i32:$rD,
25172509 (PPCaddiTlsgdL i32:$reg, tglobaltlsaddr:$disp))]>;
2510 let hasExtraSrcRegAllocReq = 1, hasExtraDefRegAllocReq = 1, Defs = [LR] in
2511 def GETtlsADDR32 : Pseudo<(outs gprc:$rD), (ins gprc:$reg, tlsgd32:$sym),
2512 "GETtlsADDR32",
2513 [(set i32:$rD,
2514 (PPCgetTlsAddr i32:$reg, tglobaltlsaddr:$sym))]>;
25182515 def ADDItlsldL32 : Pseudo<(outs gprc:$rD), (ins gprc_nor0:$reg, s16imm:$disp),
25192516 "#ADDItlsldL32",
25202517 [(set i32:$rD,
25212518 (PPCaddiTlsldL i32:$reg, tglobaltlsaddr:$disp))]>;
2519 let hasExtraSrcRegAllocReq = 1, hasExtraDefRegAllocReq = 1, Defs = [LR] in
2520 def GETtlsldADDR32 : Pseudo<(outs gprc:$rD), (ins gprc:$reg, tlsgd32:$sym),
2521 "GETtlsldADDR32",
2522 [(set i32:$rD,
2523 (PPCgetTlsldAddr i32:$reg,
2524 tglobaltlsaddr:$sym))]>;
25222525 def ADDIdtprelL32 : Pseudo<(outs gprc:$rD), (ins gprc_nor0:$reg, s16imm:$disp),
25232526 "#ADDIdtprelL32",
25242527 [(set i32:$rD,
136136 case PPCII::MO_TLS:
137137 RefKind = MCSymbolRefExpr::VK_PPC_TLS;
138138 break;
139 case PPCII::MO_TLSGD:
140 RefKind = MCSymbolRefExpr::VK_PPC_TLSGD;
141 break;
142 case PPCII::MO_TLSLD:
143 RefKind = MCSymbolRefExpr::VK_PPC_TLSLD;
144 break;
145139 }
146140
147141 if (MO.getTargetFlags() == PPCII::MO_PLT_OR_STUB && !isDarwin)
0 //===---------- PPCTLSDynamicCall.cpp - TLS Dynamic Call Fixup ------------===//
1 //
2 // The LLVM Compiler Infrastructure
3 //
4 // This file is distributed under the University of Illinois Open Source
5 // License. See LICENSE.TXT for details.
6 //
7 //===----------------------------------------------------------------------===//
8 //
9 // This pass fixes up GETtls[ld]ADDR[32] machine instructions so that
10 // they read and write GPR3. These are really call instructions, so
11 // must use the calling convention registers. This is done in a late
12 // pass so that TLS variable accesses can be fully commoned.
13 //
14 //===----------------------------------------------------------------------===//
15
16 #include "PPCInstrInfo.h"
17 #include "PPC.h"
18 #include "PPCInstrBuilder.h"
19 #include "PPCTargetMachine.h"
20 #include "llvm/CodeGen/MachineFunctionPass.h"
21 #include "llvm/CodeGen/MachineInstrBuilder.h"
22 #include "llvm/Support/Debug.h"
23 #include "llvm/Support/raw_ostream.h"
24
25 using namespace llvm;
26
27 #define DEBUG_TYPE "ppc-tls-dynamic-call"
28
29 namespace llvm {
30 void initializePPCTLSDynamicCallPass(PassRegistry&);
31 }
32
33 namespace {
34 // PPCTLSDynamicCall pass - Add copies to and from GPR3 around
35 // GETtls[ld]ADDR[32] machine instructions. These instructions
36 // are actually call instructions, so the register choice is
37 // constrained. We delay introducing these copies as late as
38 // possible so that TLS variable accesses can be fully commoned.
39 struct PPCTLSDynamicCall : public MachineFunctionPass {
40 static char ID;
41 PPCTLSDynamicCall() : MachineFunctionPass(ID) {
42 initializePPCTLSDynamicCallPass(*PassRegistry::getPassRegistry());
43 }
44
45 const PPCTargetMachine *TM;
46 const PPCInstrInfo *TII;
47
48 protected:
49 bool processBlock(MachineBasicBlock &MBB) {
50 bool Changed = false;
51 bool Is64Bit = TM->getSubtargetImpl()->isPPC64();
52
53 for (MachineBasicBlock::iterator I = MBB.begin(), IE = MBB.end();
54 I != IE; ++I) {
55 MachineInstr *MI = I;
56
57 if (MI->getOpcode() != PPC::GETtlsADDR &&
58 MI->getOpcode() != PPC::GETtlsldADDR)
59 continue;
60
61 DEBUG(dbgs() << "TLS Dynamic Call Fixup:\n " << *MI;);
62
63 unsigned OutReg = MI->getOperand(0).getReg();
64 unsigned InReg = MI->getOperand(1).getReg();
65 DebugLoc DL = MI->getDebugLoc();
66 unsigned GPR3 = Is64Bit ? PPC::X3 : PPC::R3;
67
68 BuildMI(MBB, I, DL, TII->get(TargetOpcode::COPY), GPR3)
69 .addReg(InReg);
70 MI->getOperand(0).setReg(GPR3);
71 MI->getOperand(1).setReg(GPR3);
72 BuildMI(MBB, ++I, DL, TII->get(TargetOpcode::COPY), OutReg)
73 .addReg(GPR3);
74
75 Changed = true;
76 }
77
78 return Changed;
79 }
80
81 public:
82 bool runOnMachineFunction(MachineFunction &MF) override {
83 TM = static_cast(&MF.getTarget());
84 TII = TM->getSubtargetImpl()->getInstrInfo();
85
86 bool Changed = false;
87
88 for (MachineFunction::iterator I = MF.begin(); I != MF.end();) {
89 MachineBasicBlock &B = *I++;
90 if (processBlock(B))
91 Changed = true;
92 }
93
94 return Changed;
95 }
96
97 void getAnalysisUsage(AnalysisUsage &AU) const override {
98 MachineFunctionPass::getAnalysisUsage(AU);
99 }
100 };
101 }
102
103 INITIALIZE_PASS_BEGIN(PPCTLSDynamicCall, DEBUG_TYPE,
104 "PowerPC TLS Dynamic Call Fixup", false, false)
105 INITIALIZE_PASS_END(PPCTLSDynamicCall, DEBUG_TYPE,
106 "PowerPC TLS Dynamic Call Fixup", false, false)
107
108 char PPCTLSDynamicCall::ID = 0;
109 FunctionPass*
110 llvm::createPPCTLSDynamicCallPass() { return new PPCTLSDynamicCall(); }
258258 initializePPCVSXFMAMutatePass(*PassRegistry::getPassRegistry());
259259 insertPass(VSXFMAMutateEarly ? &RegisterCoalescerID : &MachineSchedulerID,
260260 &PPCVSXFMAMutateID);
261 addPass(createPPCTLSDynamicCallPass());
261262 }
262263
263264 void PPCPassConfig::addPreSched2() {
1111 ; CHECK-LABEL: @test1
1212 ; CHECK: mflr 0
1313 ; CHECK: std 0, 16(1)
14 ; FIXME: These next two lines don't both need to load the same value.
15 ; CHECK-DAG: ld 3, 16(1)
14 ; CHECK-DAG: ld 3, 64(1)
1615 ; CHECK-DAG: ld 0, 16(1)
1716 ; CHECK: mtlr 0
1817 ; CHECK: blr
0 ; RUN: llc -march=ppc64 -mcpu=pwr7 -O2 -relocation-model=pic < %s | FileCheck %s
1 ; RUN: llc -march=ppc64 -mcpu=pwr7 -O2 -relocation-model=pic < %s | grep "__tls_get_addr" | count 1
2
3 ; This test was derived from LLVM's own
4 ; PrettyStackTraceEntry::~PrettyStackTraceEntry(). It demonstrates an
5 ; opportunity for CSE of calls to __tls_get_addr().
6
7 target datalayout = "e-m:e-i64:64-n32:64"
8 target triple = "powerpc64le-unknown-linux-gnu"
9
10 %"class.llvm::PrettyStackTraceEntry" = type { i32 (...)**, %"class.llvm::PrettyStackTraceEntry"* }
11
12 @_ZTVN4llvm21PrettyStackTraceEntryE = unnamed_addr constant [5 x i8*] [i8* null, i8* null, i8* bitcast (void (%"class.llvm::PrettyStackTraceEntry"*)* @_ZN4llvm21PrettyStackTraceEntryD2Ev to i8*), i8* bitcast (void (%"class.llvm::PrettyStackTraceEntry"*)* @_ZN4llvm21PrettyStackTraceEntryD0Ev to i8*), i8* bitcast (void ()* @__cxa_pure_virtual to i8*)], align 8
13 @_ZL20PrettyStackTraceHead = internal thread_local unnamed_addr global %"class.llvm::PrettyStackTraceEntry"* null, align 8
14 @.str = private unnamed_addr constant [87 x i8] c"PrettyStackTraceHead == this && \22Pretty stack trace entry destruction is out of order\22\00", align 1
15 @.str1 = private unnamed_addr constant [64 x i8] c"/home/wschmidt/llvm/llvm-test2/lib/Support/PrettyStackTrace.cpp\00", align 1
16 @__PRETTY_FUNCTION__._ZN4llvm21PrettyStackTraceEntryD2Ev = private unnamed_addr constant [62 x i8] c"virtual llvm::PrettyStackTraceEntry::~PrettyStackTraceEntry()\00", align 1
17
18 declare void @_ZN4llvm21PrettyStackTraceEntryD2Ev(%"class.llvm::PrettyStackTraceEntry"* %this) unnamed_addr
19 declare void @__cxa_pure_virtual()
20 declare void @__assert_fail(i8*, i8*, i32 zeroext, i8*)
21 declare void @_ZdlPv(i8*)
22
23 define void @_ZN4llvm21PrettyStackTraceEntryD0Ev(%"class.llvm::PrettyStackTraceEntry"* %this) unnamed_addr align 2 {
24 entry:
25 %0 = getelementptr inbounds %"class.llvm::PrettyStackTraceEntry"* %this, i64 0, i32 0
26 store i32 (...)** bitcast (i8** getelementptr inbounds ([5 x i8*]* @_ZTVN4llvm21PrettyStackTraceEntryE, i64 0, i64 2) to i32 (...)**), i32 (...)*** %0, align 8
27 %1 = load %"class.llvm::PrettyStackTraceEntry"** @_ZL20PrettyStackTraceHead, align 8
28 %cmp.i = icmp eq %"class.llvm::PrettyStackTraceEntry"* %1, %this
29 br i1 %cmp.i, label %_ZN4llvm21PrettyStackTraceEntryD2Ev.exit, label %cond.false.i
30
31 cond.false.i: ; preds = %entry
32 tail call void @__assert_fail(i8* getelementptr inbounds ([87 x i8]* @.str, i64 0, i64 0), i8* getelementptr inbounds ([64 x i8]* @.str1, i64 0, i64 0), i32 zeroext 119, i8* getelementptr inbounds ([62 x i8]* @__PRETTY_FUNCTION__._ZN4llvm21PrettyStackTraceEntryD2Ev, i64 0, i64 0))
33 unreachable
34
35 _ZN4llvm21PrettyStackTraceEntryD2Ev.exit: ; preds = %entry
36 %NextEntry.i.i = getelementptr inbounds %"class.llvm::PrettyStackTraceEntry"* %this, i64 0, i32 1
37 %2 = bitcast %"class.llvm::PrettyStackTraceEntry"** %NextEntry.i.i to i64*
38 %3 = load i64* %2, align 8
39 store i64 %3, i64* bitcast (%"class.llvm::PrettyStackTraceEntry"** @_ZL20PrettyStackTraceHead to i64*), align 8
40 %4 = bitcast %"class.llvm::PrettyStackTraceEntry"* %this to i8*
41 tail call void @_ZdlPv(i8* %4)
42 ret void
43 }
44
45 ; CHECK-LABEL: _ZN4llvm21PrettyStackTraceEntryD0Ev:
46 ; CHECK: addis [[REG1:[0-9]+]], 2, _ZL20PrettyStackTraceHead@got@tlsld@ha
47 ; CHECK: addi 3, [[REG1]], _ZL20PrettyStackTraceHead@got@tlsld@l
48 ; CHECK: bl __tls_get_addr(_ZL20PrettyStackTraceHead@tlsld)
49 ; CHECK: addis 3, 3, _ZL20PrettyStackTraceHead@dtprel@ha
50 ; CHECK: ld {{[0-9]+}}, _ZL20PrettyStackTraceHead@dtprel@l(3)
51 ; CHECK: std {{[0-9]+}}, _ZL20PrettyStackTraceHead@dtprel@l(3)
1818 }
1919
2020 ; CHECK-LABEL: call_once:
21 ; CHECK: addis 3, 2, __once_callable@got@tlsgd@ha
22 ; CHECK: addi 3, 3, __once_callable@got@tlsgd@l
21 ; CHECK: addi 3, {{[0-9]+}}, __once_callable@got@tlsgd@l
2322 ; CHECK: bl __tls_get_addr(__once_callable@tlsgd)
2423 ; CHECK-NEXT: nop
2524 ; CHECK: std {{[0-9]+}}, 0(3)
26 ; CHECK: addis 3, 2, __once_call@got@tlsgd@ha
27 ; CHECK: addi 3, 3, __once_call@got@tlsgd@l
25 ; CHECK: addi 3, {{[0-9]+}}, __once_call@got@tlsgd@l
2826 ; CHECK: bl __tls_get_addr(__once_call@tlsgd)
2927 ; CHECK-NEXT: nop
3028 ; CHECK: std {{[0-9]+}}, 0(3)