llvm.org GIT mirror llvm / b79469c
[GlobalISel] Enable legalizing non-power-of-2 sized types. This changes the interface of how targets describe how to legalize, see the below description. 1. Interface for targets to describe how to legalize. In GlobalISel, the API in the LegalizerInfo class is the main interface for targets to specify which types are legal for which operations, and what to do to turn illegal type/operation combinations into legal ones. For each operation the type sizes that can be legalized without having to change the size of the type are specified with a call to setAction. This isn't different to how GlobalISel worked before. For example, for a target that supports 32 and 64 bit adds natively: for (auto Ty : {s32, s64}) setAction({G_ADD, 0, s32}, Legal); or for a target that needs a library call for a 32 bit division: setAction({G_SDIV, s32}, Libcall); The main conceptual change to the LegalizerInfo API, is in specifying how to legalize the type sizes for which a change of size is needed. For example, in the above example, how to specify how all types from i1 to i8388607 (apart from s32 and s64 which are legal) need to be legalized and expressed in terms of operations on the available legal sizes (again, i32 and i64 in this case). Before, the implementation only allowed specifying power-of-2-sized types (e.g. setAction({G_ADD, 0, s128}, NarrowScalar). A worse limitation was that if you'd wanted to specify how to legalize all the sized types as allowed by the LLVM-IR LangRef, i1 to i8388607, you'd have to call setAction 8388607-3 times and probably would need a lot of memory to store all of these specifications. Instead, the legalization actions that need to change the size of the type are specified now using a "SizeChangeStrategy". For example: setLegalizeScalarToDifferentSizeStrategy( G_ADD, 0, widenToLargerAndNarrowToLargest); This example indicates that for type sizes for which there is a larger size that can be legalized towards, do it by Widening the size. For example, G_ADD on s17 will be legalized by first doing WidenScalar to make it s32, after which it's legal. The "NarrowToLargest" indicates what to do if there is no larger size that can be legalized towards. E.g. G_ADD on s92 will be legalized by doing NarrowScalar to s64. Another example, taken from the ARM backend is: for (unsigned Op : {G_SDIV, G_UDIV}) { setLegalizeScalarToDifferentSizeStrategy(Op, 0, widenToLargerTypesUnsupportedOtherwise); if (ST.hasDivideInARMMode()) setAction({Op, s32}, Legal); else setAction({Op, s32}, Libcall); } For this example, G_SDIV on s8, on a target without a divide instruction, would be legalized by first doing action (WidenScalar, s32), followed by (Libcall, s32). The same principle is also followed for when the number of vector lanes on vector data types need to be changed, e.g.: setAction({G_ADD, LLT::vector(8, 8)}, LegalizerInfo::Legal); setAction({G_ADD, LLT::vector(16, 8)}, LegalizerInfo::Legal); setAction({G_ADD, LLT::vector(4, 16)}, LegalizerInfo::Legal); setAction({G_ADD, LLT::vector(8, 16)}, LegalizerInfo::Legal); setAction({G_ADD, LLT::vector(2, 32)}, LegalizerInfo::Legal); setAction({G_ADD, LLT::vector(4, 32)}, LegalizerInfo::Legal); setLegalizeVectorElementToDifferentSizeStrategy( G_ADD, 0, widenToLargerTypesUnsupportedOtherwise); As currently implemented here, vector types are legalized by first making the vector element size legal, followed by then making the number of lanes legal. The strategy to follow in the first step is set by a call to setLegalizeVectorElementToDifferentSizeStrategy, see example above. The strategy followed in the second step "moreToWiderTypesAndLessToWidest" (see code for its definition), indicating that vectors are widened to more elements so they map to natively supported vector widths, or when there isn't a legal wider vector, split the vector to map it to the widest vector supported. Therefore, for the above specification, some example legalizations are: * getAction({G_ADD, LLT::vector(3, 3)}) returns {WidenScalar, LLT::vector(3, 8)} * getAction({G_ADD, LLT::vector(3, 8)}) then returns {MoreElements, LLT::vector(8, 8)} * getAction({G_ADD, LLT::vector(20, 8)}) returns {FewerElements, LLT::vector(16, 8)} 2. Key implementation aspects. How to legalize a specific (operation, type index, size) tuple is represented by mapping intervals of integers representing a range of size types to an action to take, e.g.: setScalarAction({G_ADD, LLT:scalar(1)}, {{1, WidenScalar}, // bit sizes [ 1, 31[ {32, Legal}, // bit sizes [32, 33[ {33, WidenScalar}, // bit sizes [33, 64[ {64, Legal}, // bit sizes [64, 65[ {65, NarrowScalar} // bit sizes [65, +inf[ }); Please note that most of the code to do the actual lowering of non-power-of-2 sized types is currently missing, this is just trying to make it possible for targets to specify what is legal, and how non-legal types should be legalized. Probably quite a bit of further work is needed in the actual legalizing and the other passes in GlobalISel to support non-power-of-2 sized types. I hope the documentation in LegalizerInfo.h and the examples provided in the various {Target}LegalizerInfo.cpp and LegalizerInfoTest.cpp explains well enough how this is meant to be used. This drops the need for LLT::{half,double}...Size(). Differential Revision: https://reviews.llvm.org/D30529 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317560 91177308-0d34-0410-b5e6-96231b3b80d8 Kristof Beyls 1 year, 9 months ago
14 changed file(s) with 1157 addition(s) and 452 deletion(s). Raw diff Collapse all Expand all
2525 #include
2626 #include
2727 #include
28 #include
2829
2930 namespace llvm {
3031
119120 }
120121 }
121122
123 typedef std::pair SizeAndAction;
124 typedef std::vector SizeAndActionsVec;
125 using SizeChangeStrategy =
126 std::function;
127
122128 /// More friendly way to set an action for common types that have an LLT
123129 /// representation.
130 /// The LegalizeAction must be one for which NeedsLegalizingToDifferentSize
131 /// returns false.
124132 void setAction(const InstrAspect &Aspect, LegalizeAction Action) {
133 assert(!needsLegalizingToDifferentSize(Action));
125134 TablesInitialized = false;
126 unsigned Opcode = Aspect.Opcode - FirstOp;
127 if (Actions[Opcode].size() <= Aspect.Idx)
128 Actions[Opcode].resize(Aspect.Idx + 1);
129 Actions[Aspect.Opcode - FirstOp][Aspect.Idx][Aspect.Type] = Action;
130 }
131
132 /// If an operation on a given vector type (say ) isn't explicitly
133 /// specified, we proceed in 2 stages. First we legalize the underlying scalar
134 /// (so that there's at least one legal vector with that scalar), then we
135 /// adjust the number of elements in the vector so that it is legal. The
136 /// desired action in the first step is controlled by this function.
137 void setScalarInVectorAction(unsigned Opcode, LLT ScalarTy,
138 LegalizeAction Action) {
139 assert(!ScalarTy.isVector());
140 ScalarInVectorActions[std::make_pair(Opcode, ScalarTy)] = Action;
141 }
135 const unsigned OpcodeIdx = Aspect.Opcode - FirstOp;
136 if (SpecifiedActions[OpcodeIdx].size() <= Aspect.Idx)
137 SpecifiedActions[OpcodeIdx].resize(Aspect.Idx + 1);
138 SpecifiedActions[OpcodeIdx][Aspect.Idx][Aspect.Type] = Action;
139 }
140
141 /// The setAction calls record the non-size-changing legalization actions
142 /// to take on specificly-sized types. The SizeChangeStrategy defines what
143 /// to do when the size of the type needs to be changed to reach a legally
144 /// sized type (i.e., one that was defined through a setAction call).
145 /// e.g.
146 /// setAction ({G_ADD, 0, LLT::scalar(32)}, Legal);
147 /// setLegalizeScalarToDifferentSizeStrategy(
148 /// G_ADD, 0, widenToLargerTypesAndNarrowToLargest);
149 /// will end up defining getAction({G_ADD, 0, T}) to return the following
150 /// actions for different scalar types T:
151 /// LLT::scalar(1)..LLT::scalar(31): {WidenScalar, 0, LLT::scalar(32)}
152 /// LLT::scalar(32): {Legal, 0, LLT::scalar(32)}
153 /// LLT::scalar(33)..: {NarrowScalar, 0, LLT::scalar(32)}
154 ///
155 /// If no SizeChangeAction gets defined, through this function,
156 /// the default is unsupportedForDifferentSizes.
157 void setLegalizeScalarToDifferentSizeStrategy(const unsigned Opcode,
158 const unsigned TypeIdx,
159 SizeChangeStrategy S) {
160 const unsigned OpcodeIdx = Opcode - FirstOp;
161 if (ScalarSizeChangeStrategies[OpcodeIdx].size() <= TypeIdx)
162 ScalarSizeChangeStrategies[OpcodeIdx].resize(TypeIdx + 1);
163 ScalarSizeChangeStrategies[OpcodeIdx][TypeIdx] = S;
164 }
165
166 /// See also setLegalizeScalarToDifferentSizeStrategy.
167 /// This function allows to set the SizeChangeStrategy for vector elements.
168 void setLegalizeVectorElementToDifferentSizeStrategy(const unsigned Opcode,
169 const unsigned TypeIdx,
170 SizeChangeStrategy S) {
171 const unsigned OpcodeIdx = Opcode - FirstOp;
172 if (VectorElementSizeChangeStrategies[OpcodeIdx].size() <= TypeIdx)
173 VectorElementSizeChangeStrategies[OpcodeIdx].resize(TypeIdx + 1);
174 VectorElementSizeChangeStrategies[OpcodeIdx][TypeIdx] = S;
175 }
176
177 /// A SizeChangeStrategy for the common case where legalization for a
178 /// particular operation consists of only supporting a specific set of type
179 /// sizes. E.g.
180 /// setAction ({G_DIV, 0, LLT::scalar(32)}, Legal);
181 /// setAction ({G_DIV, 0, LLT::scalar(64)}, Legal);
182 /// setLegalizeScalarToDifferentSizeStrategy(
183 /// G_DIV, 0, unsupportedForDifferentSizes);
184 /// will result in getAction({G_DIV, 0, T}) to return Legal for s32 and s64,
185 /// and Unsupported for all other scalar types T.
186 static SizeAndActionsVec
187 unsupportedForDifferentSizes(const SizeAndActionsVec &v) {
188 return increaseToLargerTypesAndDecreaseToLargest(v, Unsupported,
189 Unsupported);
190 }
191
192 /// A SizeChangeStrategy for the common case where legalization for a
193 /// particular operation consists of widening the type to a large legal type,
194 /// unless there is no such type and then instead it should be narrowed to the
195 /// largest legal type.
196 static SizeAndActionsVec
197 widenToLargerTypesAndNarrowToLargest(const SizeAndActionsVec &v) {
198 assert(v.size() > 0 &&
199 "At least one size that can be legalized towards is needed"
200 " for this SizeChangeStrategy");
201 return increaseToLargerTypesAndDecreaseToLargest(v, WidenScalar,
202 NarrowScalar);
203 }
204
205 static SizeAndActionsVec
206 widenToLargerTypesUnsupportedOtherwise(const SizeAndActionsVec &v) {
207 return increaseToLargerTypesAndDecreaseToLargest(v, WidenScalar,
208 Unsupported);
209 }
210
211 static SizeAndActionsVec
212 narrowToSmallerAndUnsupportedIfTooSmall(const SizeAndActionsVec &v) {
213 return decreaseToSmallerTypesAndIncreaseToSmallest(v, NarrowScalar,
214 Unsupported);
215 }
216
217 static SizeAndActionsVec
218 narrowToSmallerAndWidenToSmallest(const SizeAndActionsVec &v) {
219 assert(v.size() > 0 &&
220 "At least one size that can be legalized towards is needed"
221 " for this SizeChangeStrategy");
222 return decreaseToSmallerTypesAndIncreaseToSmallest(v, NarrowScalar,
223 WidenScalar);
224 }
225
226 /// A SizeChangeStrategy for the common case where legalization for a
227 /// particular vector operation consists of having more elements in the
228 /// vector, to a type that is legal. Unless there is no such type and then
229 /// instead it should be legalized towards the widest vector that's still
230 /// legal. E.g.
231 /// setAction({G_ADD, LLT::vector(8, 8)}, Legal);
232 /// setAction({G_ADD, LLT::vector(16, 8)}, Legal);
233 /// setAction({G_ADD, LLT::vector(2, 32)}, Legal);
234 /// setAction({G_ADD, LLT::vector(4, 32)}, Legal);
235 /// setLegalizeVectorElementToDifferentSizeStrategy(
236 /// G_ADD, 0, moreToWiderTypesAndLessToWidest);
237 /// will result in the following getAction results:
238 /// * getAction({G_ADD, LLT::vector(8,8)}) returns
239 /// (Legal, vector(8,8)).
240 /// * getAction({G_ADD, LLT::vector(9,8)}) returns
241 /// (MoreElements, vector(16,8)).
242 /// * getAction({G_ADD, LLT::vector(8,32)}) returns
243 /// (FewerElements, vector(4,32)).
244 static SizeAndActionsVec
245 moreToWiderTypesAndLessToWidest(const SizeAndActionsVec &v) {
246 return increaseToLargerTypesAndDecreaseToLargest(v, MoreElements,
247 FewerElements);
248 }
249
250 /// Helper function to implement many typical SizeChangeStrategy functions.
251 static SizeAndActionsVec
252 increaseToLargerTypesAndDecreaseToLargest(const SizeAndActionsVec &v,
253 LegalizeAction IncreaseAction,
254 LegalizeAction DecreaseAction);
255 /// Helper function to implement many typical SizeChangeStrategy functions.
256 static SizeAndActionsVec
257 decreaseToSmallerTypesAndIncreaseToSmallest(const SizeAndActionsVec &v,
258 LegalizeAction DecreaseAction,
259 LegalizeAction IncreaseAction);
142260
143261 /// Determine what action should be taken to legalize the given generic
144262 /// instruction opcode, type-index and type. Requires computeTables to have
157275 std::tuple
158276 getAction(const MachineInstr &MI, const MachineRegisterInfo &MRI) const;
159277
160 /// Iterate the given function (typically something like doubling the width)
161 /// on Ty until we find a legal type for this operation.
162 Optional findLegalizableSize(const InstrAspect &Aspect,
163 function_ref NextType) const {
164 if (Aspect.Idx >= Actions[Aspect.Opcode - FirstOp].size())
165 return None;
166
167 LegalizeAction Action;
168 const TypeMap &Map = Actions[Aspect.Opcode - FirstOp][Aspect.Idx];
169 LLT Ty = Aspect.Type;
170 do {
171 Ty = NextType(Ty);
172 auto ActionIt = Map.find(Ty);
173 if (ActionIt == Map.end()) {
174 auto DefaultIt = DefaultActions.find(Aspect.Opcode);
175 if (DefaultIt == DefaultActions.end())
176 return None;
177 Action = DefaultIt->second;
178 } else
179 Action = ActionIt->second;
180 } while (needsLegalizingToDifferentSize(Action));
181 return Ty;
182 }
183
184 /// Find what type it's actually OK to perform the given operation on, given
185 /// the general approach we've decided to take.
186 Optional findLegalType(const InstrAspect &Aspect, LegalizeAction Action) const;
187
188 std::pair findLegalAction(const InstrAspect &Aspect,
189 LegalizeAction Action) const {
190 auto LegalType = findLegalType(Aspect, Action);
191 if (!LegalType)
192 return std::make_pair(LegalizeAction::Unsupported, LLT());
193 return std::make_pair(Action, *LegalType);
194 }
195
196 /// Find the specified \p Aspect in the primary (explicitly set) Actions
197 /// table. Returns either the action the target requested or NotFound if there
198 /// was no setAction call.
199 LegalizeAction findInActions(const InstrAspect &Aspect) const {
200 if (Aspect.Opcode < FirstOp || Aspect.Opcode > LastOp)
201 return NotFound;
202 if (Aspect.Idx >= Actions[Aspect.Opcode - FirstOp].size())
203 return NotFound;
204 const TypeMap &Map = Actions[Aspect.Opcode - FirstOp][Aspect.Idx];
205 auto ActionIt = Map.find(Aspect.Type);
206 if (ActionIt == Map.end())
207 return NotFound;
208
209 return ActionIt->second;
210 }
211
212278 bool isLegal(const MachineInstr &MI, const MachineRegisterInfo &MRI) const;
213279
214280 virtual bool legalizeCustom(MachineInstr &MI,
216282 MachineIRBuilder &MIRBuilder) const;
217283
218284 private:
285 /// The SizeAndActionsVec is a representation mapping between all natural
286 /// numbers and an Action. The natural number represents the bit size of
287 /// the InstrAspect. For example, for a target with native support for 32-bit
288 /// and 64-bit additions, you'd express that as:
289 /// setScalarAction(G_ADD, 0,
290 /// {{1, WidenScalar}, // bit sizes [ 1, 31[
291 /// {32, Legal}, // bit sizes [32, 33[
292 /// {33, WidenScalar}, // bit sizes [33, 64[
293 /// {64, Legal}, // bit sizes [64, 65[
294 /// {65, NarrowScalar} // bit sizes [65, +inf[
295 /// });
296 /// It may be that only 64-bit pointers are supported on your target:
297 /// setPointerAction(G_GEP, 0, LLT:pointer(1),
298 /// {{1, Unsupported}, // bit sizes [ 1, 63[
299 /// {64, Legal}, // bit sizes [64, 65[
300 /// {65, Unsupported}, // bit sizes [65, +inf[
301 /// });
302 void setScalarAction(const unsigned Opcode, const unsigned TypeIndex,
303 const SizeAndActionsVec &SizeAndActions) {
304 const unsigned OpcodeIdx = Opcode - FirstOp;
305 SmallVector &Actions = ScalarActions[OpcodeIdx];
306 setActions(TypeIndex, Actions, SizeAndActions);
307 }
308 void setPointerAction(const unsigned Opcode, const unsigned TypeIndex,
309 const unsigned AddressSpace,
310 const SizeAndActionsVec &SizeAndActions) {
311 const unsigned OpcodeIdx = Opcode - FirstOp;
312 if (AddrSpace2PointerActions[OpcodeIdx].find(AddressSpace) ==
313 AddrSpace2PointerActions[OpcodeIdx].end())
314 AddrSpace2PointerActions[OpcodeIdx][AddressSpace] = {{}};
315 SmallVector &Actions =
316 AddrSpace2PointerActions[OpcodeIdx].find(AddressSpace)->second;
317 setActions(TypeIndex, Actions, SizeAndActions);
318 }
319
320 /// If an operation on a given vector type (say ) isn't explicitly
321 /// specified, we proceed in 2 stages. First we legalize the underlying scalar
322 /// (so that there's at least one legal vector with that scalar), then we
323 /// adjust the number of elements in the vector so that it is legal. The
324 /// desired action in the first step is controlled by this function.
325 void setScalarInVectorAction(const unsigned Opcode, const unsigned TypeIndex,
326 const SizeAndActionsVec &SizeAndActions) {
327 unsigned OpcodeIdx = Opcode - FirstOp;
328 SmallVector &Actions =
329 ScalarInVectorActions[OpcodeIdx];
330 setActions(TypeIndex, Actions, SizeAndActions);
331 }
332
333 /// See also setScalarInVectorAction.
334 /// This function let's you specify the number of elements in a vector that
335 /// are legal for a legal element size.
336 void setVectorNumElementAction(const unsigned Opcode,
337 const unsigned TypeIndex,
338 const unsigned ElementSize,
339 const SizeAndActionsVec &SizeAndActions) {
340 const unsigned OpcodeIdx = Opcode - FirstOp;
341 if (NumElements2Actions[OpcodeIdx].find(ElementSize) ==
342 NumElements2Actions[OpcodeIdx].end())
343 NumElements2Actions[OpcodeIdx][ElementSize] = {{}};
344 SmallVector &Actions =
345 NumElements2Actions[OpcodeIdx].find(ElementSize)->second;
346 setActions(TypeIndex, Actions, SizeAndActions);
347 }
348
349 /// A partial SizeAndActionsVec potentially doesn't cover all bit sizes,
350 /// i.e. it's OK if it doesn't start from size 1.
351 static void checkPartialSizeAndActionsVector(const SizeAndActionsVec& v) {
352 #ifndef NDEBUG
353 // The sizes should be in increasing order
354 int prev_size = -1;
355 for(auto SizeAndAction: v) {
356 assert(SizeAndAction.first > prev_size);
357 prev_size = SizeAndAction.first;
358 }
359 // - for every Widen action, there should be a larger bitsize that
360 // can be legalized towards (e.g. Legal, Lower, Libcall or Custom
361 // action).
362 // - for every Narrow action, there should be a smaller bitsize that
363 // can be legalized towards.
364 int SmallestNarrowIdx = -1;
365 int LargestWidenIdx = -1;
366 int SmallestLegalizableToSameSizeIdx = -1;
367 int LargestLegalizableToSameSizeIdx = -1;
368 for(size_t i=0; i
369 switch (v[i].second) {
370 case FewerElements:
371 case NarrowScalar:
372 if (SmallestNarrowIdx == -1)
373 SmallestNarrowIdx = i;
374 break;
375 case WidenScalar:
376 case MoreElements:
377 LargestWidenIdx = i;
378 break;
379 case Unsupported:
380 break;
381 default:
382 if (SmallestLegalizableToSameSizeIdx == -1)
383 SmallestLegalizableToSameSizeIdx = i;
384 LargestLegalizableToSameSizeIdx = i;
385 }
386 }
387 if (SmallestNarrowIdx != -1) {
388 assert(SmallestLegalizableToSameSizeIdx != -1);
389 assert(SmallestNarrowIdx > SmallestLegalizableToSameSizeIdx);
390 }
391 if (LargestWidenIdx != -1)
392 assert(LargestWidenIdx < LargestLegalizableToSameSizeIdx);
393 #endif
394 }
395
396 /// A full SizeAndActionsVec must cover all bit sizes, i.e. must start with
397 /// from size 1.
398 static void checkFullSizeAndActionsVector(const SizeAndActionsVec& v) {
399 #ifndef NDEBUG
400 // Data structure invariant: The first bit size must be size 1.
401 assert(v.size() >= 1);
402 assert(v[0].first == 1);
403 checkPartialSizeAndActionsVector(v);
404 #endif
405 }
406
407 /// Sets actions for all bit sizes on a particular generic opcode, type
408 /// index and scalar or pointer type.
409 void setActions(unsigned TypeIndex,
410 SmallVector &Actions,
411 const SizeAndActionsVec &SizeAndActions) {
412 checkFullSizeAndActionsVector(SizeAndActions);
413 if (Actions.size() <= TypeIndex)
414 Actions.resize(TypeIndex + 1);
415 Actions[TypeIndex] = SizeAndActions;
416 }
417
418 static SizeAndAction findAction(const SizeAndActionsVec &Vec,
419 const uint32_t Size);
420
421 /// Returns the next action needed to get the scalar or pointer type closer
422 /// to being legal
423 /// E.g. findLegalAction({G_REM, 13}) should return
424 /// (WidenScalar, 32). After that, findLegalAction({G_REM, 32}) will
425 /// probably be called, which should return (Lower, 32).
426 /// This is assuming the setScalarAction on G_REM was something like:
427 /// setScalarAction(G_REM, 0,
428 /// {{1, WidenScalar}, // bit sizes [ 1, 31[
429 /// {32, Lower}, // bit sizes [32, 33[
430 /// {33, NarrowScalar} // bit sizes [65, +inf[
431 /// });
432 std::pair
433 findScalarLegalAction(const InstrAspect &Aspect) const;
434
435 /// Returns the next action needed towards legalizing the vector type.
436 std::pair
437 findVectorLegalAction(const InstrAspect &Aspect) const;
438
219439 static const int FirstOp = TargetOpcode::PRE_ISEL_GENERIC_OPCODE_START;
220440 static const int LastOp = TargetOpcode::PRE_ISEL_GENERIC_OPCODE_END;
221441
222 using TypeMap = DenseMap;
223 using SIVActionMap = DenseMap, LegalizeAction>;
224
225 SmallVector Actions[LastOp - FirstOp + 1];
226 SIVActionMap ScalarInVectorActions;
227 DenseMap, uint16_t> MaxLegalVectorElts;
228 DenseMap DefaultActions;
229
230 bool TablesInitialized = false;
442 // Data structures used temporarily during construction of legality data:
443 typedef DenseMap TypeMap;
444 SmallVector SpecifiedActions[LastOp - FirstOp + 1];
445 SmallVector
446 ScalarSizeChangeStrategies[LastOp - FirstOp + 1];
447 SmallVector
448 VectorElementSizeChangeStrategies[LastOp - FirstOp + 1];
449 bool TablesInitialized;
450
451 // Data structures used by getAction:
452 SmallVector ScalarActions[LastOp - FirstOp + 1];
453 SmallVector ScalarInVectorActions[LastOp - FirstOp + 1];
454 std::unordered_map>
455 AddrSpace2PointerActions[LastOp - FirstOp + 1];
456 std::unordered_map>
457 NumElements2Actions[LastOp - FirstOp + 1];
231458 };
232459
233 } // end namespace llvm
460 } // end namespace llvm.
234461
235462 #endif // LLVM_CODEGEN_GLOBALISEL_LEGALIZERINFO_H
134134 return pointer(getAddressSpace(), getScalarSizeInBits());
135135 else
136136 return scalar(getScalarSizeInBits());
137 }
138
139 /// Get a low-level type with half the size of the original, by halving the
140 /// size of the scalar type involved. For example `s32` will become `s16`,
141 /// `<2 x s32>` will become `<2 x s16>`.
142 LLT halfScalarSize() const {
143 assert(!IsPointer && getScalarSizeInBits() > 1 &&
144 getScalarSizeInBits() % 2 == 0 && "cannot half size of this type");
145 return LLT{/*isPointer=*/false, IsVector ? true : false,
146 IsVector ? getNumElements() : (uint16_t)0,
147 getScalarSizeInBits() / 2, /*AddressSpace=*/0};
148 }
149
150 /// Get a low-level type with twice the size of the original, by doubling the
151 /// size of the scalar type involved. For example `s32` will become `s64`,
152 /// `<2 x s32>` will become `<2 x s64>`.
153 LLT doubleScalarSize() const {
154 assert(!IsPointer && "cannot change size of this type");
155 return LLT{/*isPointer=*/false, IsVector ? true : false,
156 IsVector ? getNumElements() : (uint16_t)0,
157 getScalarSizeInBits() * 2, /*AddressSpace=*/0};
158 }
159
160 /// Get a low-level type with half the size of the original, by halving the
161 /// number of vector elements of the scalar type involved. The source must be
162 /// a vector type with an even number of elements. For example `<4 x s32>`
163 /// will become `<2 x s32>`, `<2 x s32>` will become `s32`.
164 LLT halfElements() const {
165 assert(isVector() && getNumElements() % 2 == 0 && "cannot half odd vector");
166 if (getNumElements() == 2)
167 return scalar(getScalarSizeInBits());
168
169 return LLT{/*isPointer=*/false, /*isVector=*/true,
170 (uint16_t)(getNumElements() / 2), getScalarSizeInBits(),
171 /*AddressSpace=*/0};
172 }
173
174 /// Get a low-level type with twice the size of the original, by doubling the
175 /// number of vector elements of the scalar type involved. The source must be
176 /// a vector type. For example `<2 x s32>` will become `<4 x s32>`. Doubling
177 /// the number of elements in sN produces <2 x sN>.
178 LLT doubleElements() const {
179 return LLT{IsPointer ? true : false, /*isVector=*/true,
180 (uint16_t)(getNumElements() * 2), getScalarSizeInBits(),
181 IsPointer ? getAddressSpace() : 0};
182137 }
183138
184139 void print(raw_ostream &OS) const;
172172
173173 MIRBuilder.setInstr(MI);
174174
175 int64_t SizeOp0 = MRI.getType(MI.getOperand(0).getReg()).getSizeInBits();
176 int64_t NarrowSize = NarrowTy.getSizeInBits();
177
175178 switch (MI.getOpcode()) {
176179 default:
177180 return UnableToLegalize;
178181 case TargetOpcode::G_IMPLICIT_DEF: {
179 int NumParts = MRI.getType(MI.getOperand(0).getReg()).getSizeInBits() /
180 NarrowTy.getSizeInBits();
182 // FIXME: add support for when SizeOp0 isn't an exact multiple of
183 // NarrowSize.
184 if (SizeOp0 % NarrowSize != 0)
185 return UnableToLegalize;
186 int NumParts = SizeOp0 / NarrowSize;
181187
182188 SmallVector DstRegs;
183189 for (int i = 0; i < NumParts; ++i) {
190196 return Legalized;
191197 }
192198 case TargetOpcode::G_ADD: {
199 // FIXME: add support for when SizeOp0 isn't an exact multiple of
200 // NarrowSize.
201 if (SizeOp0 % NarrowSize != 0)
202 return UnableToLegalize;
193203 // Expand in terms of carry-setting/consuming G_ADDE instructions.
194 int NumParts = MRI.getType(MI.getOperand(0).getReg()).getSizeInBits() /
195 NarrowTy.getSizeInBits();
204 int NumParts = SizeOp0 / NarrowTy.getSizeInBits();
196205
197206 SmallVector Src1Regs, Src2Regs, DstRegs;
198207 extractParts(MI.getOperand(1).getReg(), NarrowTy, NumParts, Src1Regs);
220229 if (TypeIdx != 1)
221230 return UnableToLegalize;
222231
223 int64_t NarrowSize = NarrowTy.getSizeInBits();
224 int NumParts =
225 MRI.getType(MI.getOperand(1).getReg()).getSizeInBits() / NarrowSize;
232 int64_t SizeOp1 = MRI.getType(MI.getOperand(1).getReg()).getSizeInBits();
233 // FIXME: add support for when SizeOp1 isn't an exact multiple of
234 // NarrowSize.
235 if (SizeOp1 % NarrowSize != 0)
236 return UnableToLegalize;
237 int NumParts = SizeOp1 / NarrowSize;
226238
227239 SmallVector SrcRegs, DstRegs;
228240 SmallVector Indexes;
269281 return Legalized;
270282 }
271283 case TargetOpcode::G_INSERT: {
272 if (TypeIdx != 0)
273 return UnableToLegalize;
274
275 int64_t NarrowSize = NarrowTy.getSizeInBits();
276 int NumParts =
277 MRI.getType(MI.getOperand(0).getReg()).getSizeInBits() / NarrowSize;
284 // FIXME: add support for when SizeOp0 isn't an exact multiple of
285 // NarrowSize.
286 if (SizeOp0 % NarrowSize != 0)
287 return UnableToLegalize;
288
289 int NumParts = SizeOp0 / NarrowSize;
278290
279291 SmallVector SrcRegs, DstRegs;
280292 SmallVector Indexes;
329341 return Legalized;
330342 }
331343 case TargetOpcode::G_LOAD: {
332 unsigned NarrowSize = NarrowTy.getSizeInBits();
333 int NumParts =
334 MRI.getType(MI.getOperand(0).getReg()).getSizeInBits() / NarrowSize;
344 // FIXME: add support for when SizeOp0 isn't an exact multiple of
345 // NarrowSize.
346 if (SizeOp0 % NarrowSize != 0)
347 return UnableToLegalize;
348 int NumParts = SizeOp0 / NarrowSize;
335349 LLT OffsetTy = LLT::scalar(
336350 MRI.getType(MI.getOperand(1).getReg()).getScalarSizeInBits());
337351
356370 return Legalized;
357371 }
358372 case TargetOpcode::G_STORE: {
359 unsigned NarrowSize = NarrowTy.getSizeInBits();
360 int NumParts =
361 MRI.getType(MI.getOperand(0).getReg()).getSizeInBits() / NarrowSize;
373 // FIXME: add support for when SizeOp0 isn't an exact multiple of
374 // NarrowSize.
375 if (SizeOp0 % NarrowSize != 0)
376 return UnableToLegalize;
377 int NumParts = SizeOp0 / NarrowSize;
362378 LLT OffsetTy = LLT::scalar(
363379 MRI.getType(MI.getOperand(1).getReg()).getScalarSizeInBits());
364380
380396 return Legalized;
381397 }
382398 case TargetOpcode::G_CONSTANT: {
383 unsigned NarrowSize = NarrowTy.getSizeInBits();
384 int NumParts =
385 MRI.getType(MI.getOperand(0).getReg()).getSizeInBits() / NarrowSize;
399 // FIXME: add support for when SizeOp0 isn't an exact multiple of
400 // NarrowSize.
401 if (SizeOp0 % NarrowSize != 0)
402 return UnableToLegalize;
403 int NumParts = SizeOp0 / NarrowSize;
386404 const APInt &Cst = MI.getOperand(1).getCImm()->getValue();
387405 LLVMContext &Ctx = MIRBuilder.getMF().getFunction()->getContext();
388406
409427 // ...
410428 // AN = BinOp BN, CN
411429 // A = G_MERGE_VALUES A1, ..., AN
412 unsigned NarrowSize = NarrowTy.getSizeInBits();
413 int NumParts =
414 MRI.getType(MI.getOperand(0).getReg()).getSizeInBits() / NarrowSize;
430
431 // FIXME: add support for when SizeOp0 isn't an exact multiple of
432 // NarrowSize.
433 if (SizeOp0 % NarrowSize != 0)
434 return UnableToLegalize;
435 int NumParts = SizeOp0 / NarrowSize;
415436
416437 // List the registers where the destination will be scattered.
417438 SmallVector DstRegs;
853874 case TargetOpcode::G_ADD: {
854875 unsigned NarrowSize = NarrowTy.getSizeInBits();
855876 unsigned DstReg = MI.getOperand(0).getReg();
856 int NumParts = MRI.getType(DstReg).getSizeInBits() / NarrowSize;
877 unsigned Size = MRI.getType(DstReg).getSizeInBits();
878 int NumParts = Size / NarrowSize;
879 // FIXME: Don't know how to handle the situation where the small vectors
880 // aren't all the same size yet.
881 if (Size % NarrowSize != 0)
882 return UnableToLegalize;
857883
858884 MIRBuilder.setInstr(MI);
859885
2727 #include "llvm/Support/MathExtras.h"
2828 #include "llvm/Target/TargetOpcodes.h"
2929 #include
30 #include
31 #include
32 #include
33
30 #include
3431 using namespace llvm;
3532
36 LegalizerInfo::LegalizerInfo() {
37 DefaultActions[TargetOpcode::G_IMPLICIT_DEF] = NarrowScalar;
38
39 // FIXME: these two can be legalized to the fundamental load/store Jakob
40 // proposed. Once loads & stores are supported.
41 DefaultActions[TargetOpcode::G_ANYEXT] = Legal;
42 DefaultActions[TargetOpcode::G_TRUNC] = Legal;
43
44 DefaultActions[TargetOpcode::G_INTRINSIC] = Legal;
45 DefaultActions[TargetOpcode::G_INTRINSIC_W_SIDE_EFFECTS] = Legal;
46
47 DefaultActions[TargetOpcode::G_ADD] = NarrowScalar;
48 DefaultActions[TargetOpcode::G_LOAD] = NarrowScalar;
49 DefaultActions[TargetOpcode::G_STORE] = NarrowScalar;
50 DefaultActions[TargetOpcode::G_OR] = NarrowScalar;
51
52 DefaultActions[TargetOpcode::G_BRCOND] = WidenScalar;
53 DefaultActions[TargetOpcode::G_INSERT] = NarrowScalar;
54 DefaultActions[TargetOpcode::G_EXTRACT] = NarrowScalar;
55 DefaultActions[TargetOpcode::G_FNEG] = Lower;
33 LegalizerInfo::LegalizerInfo() : TablesInitialized(false) {
34 // Set defaults.
35 // FIXME: these two (G_ANYEXT and G_TRUNC?) can be legalized to the
36 // fundamental load/store Jakob proposed. Once loads & stores are supported.
37 setScalarAction(TargetOpcode::G_ANYEXT, 1, {{1, Legal}});
38 setScalarAction(TargetOpcode::G_ZEXT, 1, {{1, Legal}});
39 setScalarAction(TargetOpcode::G_SEXT, 1, {{1, Legal}});
40 setScalarAction(TargetOpcode::G_TRUNC, 0, {{1, Legal}});
41 setScalarAction(TargetOpcode::G_TRUNC, 1, {{1, Legal}});
42
43 setScalarAction(TargetOpcode::G_INTRINSIC, 0, {{1, Legal}});
44 setScalarAction(TargetOpcode::G_INTRINSIC_W_SIDE_EFFECTS, 0, {{1, Legal}});
45
46 setLegalizeScalarToDifferentSizeStrategy(
47 TargetOpcode::G_IMPLICIT_DEF, 0, narrowToSmallerAndUnsupportedIfTooSmall);
48 setLegalizeScalarToDifferentSizeStrategy(
49 TargetOpcode::G_ADD, 0, widenToLargerTypesAndNarrowToLargest);
50 setLegalizeScalarToDifferentSizeStrategy(
51 TargetOpcode::G_OR, 0, widenToLargerTypesAndNarrowToLargest);
52 setLegalizeScalarToDifferentSizeStrategy(
53 TargetOpcode::G_LOAD, 0, narrowToSmallerAndUnsupportedIfTooSmall);
54 setLegalizeScalarToDifferentSizeStrategy(
55 TargetOpcode::G_STORE, 0, narrowToSmallerAndUnsupportedIfTooSmall);
56
57 setLegalizeScalarToDifferentSizeStrategy(
58 TargetOpcode::G_BRCOND, 0, widenToLargerTypesUnsupportedOtherwise);
59 setLegalizeScalarToDifferentSizeStrategy(
60 TargetOpcode::G_INSERT, 0, narrowToSmallerAndUnsupportedIfTooSmall);
61 setLegalizeScalarToDifferentSizeStrategy(
62 TargetOpcode::G_EXTRACT, 0, narrowToSmallerAndUnsupportedIfTooSmall);
63 setLegalizeScalarToDifferentSizeStrategy(
64 TargetOpcode::G_EXTRACT, 1, narrowToSmallerAndUnsupportedIfTooSmall);
65 setScalarAction(TargetOpcode::G_FNEG, 0, {{1, Lower}});
5666 }
5767
5868 void LegalizerInfo::computeTables() {
59 for (unsigned Opcode = 0; Opcode <= LastOp - FirstOp; ++Opcode) {
60 for (unsigned Idx = 0, End = Actions[Opcode].size(); Idx != End; ++Idx) {
61 for (auto &Action : Actions[Opcode][Idx]) {
62 LLT Ty = Action.first;
63 if (!Ty.isVector())
64 continue;
65
66 auto &Entry = MaxLegalVectorElts[std::make_pair(Opcode + FirstOp,
67 Ty.getElementType())];
68 Entry = std::max(Entry, Ty.getNumElements());
69 assert(TablesInitialized == false);
70
71 for (unsigned OpcodeIdx = 0; OpcodeIdx <= LastOp - FirstOp; ++OpcodeIdx) {
72 const unsigned Opcode = FirstOp + OpcodeIdx;
73 for (unsigned TypeIdx = 0; TypeIdx != SpecifiedActions[OpcodeIdx].size();
74 ++TypeIdx) {
75 // 0. Collect information specified through the setAction API, i.e.
76 // for specific bit sizes.
77 // For scalar types:
78 SizeAndActionsVec ScalarSpecifiedActions;
79 // For pointer types:
80 std::map AddressSpace2SpecifiedActions;
81 // For vector types:
82 std::map ElemSize2SpecifiedActions;
83 for (auto LLT2Action : SpecifiedActions[OpcodeIdx][TypeIdx]) {
84 const LLT Type = LLT2Action.first;
85 const LegalizeAction Action = LLT2Action.second;
86
87 auto SizeAction = std::make_pair(Type.getSizeInBits(), Action);
88 if (Type.isPointer())
89 AddressSpace2SpecifiedActions[Type.getAddressSpace()].push_back(
90 SizeAction);
91 else if (Type.isVector())
92 ElemSize2SpecifiedActions[Type.getElementType().getSizeInBits()]
93 .push_back(SizeAction);
94 else
95 ScalarSpecifiedActions.push_back(SizeAction);
6996 }
97
98 // 1. Handle scalar types
99 {
100 // Decide how to handle bit sizes for which no explicit specification
101 // was given.
102 SizeChangeStrategy S = &unsupportedForDifferentSizes;
103 if (TypeIdx < ScalarSizeChangeStrategies[OpcodeIdx].size() &&
104 ScalarSizeChangeStrategies[OpcodeIdx][TypeIdx] != nullptr)
105 S = ScalarSizeChangeStrategies[OpcodeIdx][TypeIdx];
106 std::sort(ScalarSpecifiedActions.begin(), ScalarSpecifiedActions.end());
107 checkPartialSizeAndActionsVector(ScalarSpecifiedActions);
108 setScalarAction(Opcode, TypeIdx, S(ScalarSpecifiedActions));
109 }
110
111 // 2. Handle pointer types
112 for (auto PointerSpecifiedActions : AddressSpace2SpecifiedActions) {
113 std::sort(PointerSpecifiedActions.second.begin(),
114 PointerSpecifiedActions.second.end());
115 checkPartialSizeAndActionsVector(PointerSpecifiedActions.second);
116 // For pointer types, we assume that there isn't a meaningfull way
117 // to change the number of bits used in the pointer.
118 setPointerAction(
119 Opcode, TypeIdx, PointerSpecifiedActions.first,
120 unsupportedForDifferentSizes(PointerSpecifiedActions.second));
121 }
122
123 // 3. Handle vector types
124 SizeAndActionsVec ElementSizesSeen;
125 for (auto VectorSpecifiedActions : ElemSize2SpecifiedActions) {
126 std::sort(VectorSpecifiedActions.second.begin(),
127 VectorSpecifiedActions.second.end());
128 const uint16_t ElementSize = VectorSpecifiedActions.first;
129 ElementSizesSeen.push_back({ElementSize, Legal});
130 checkPartialSizeAndActionsVector(VectorSpecifiedActions.second);
131 // For vector types, we assume that the best way to adapt the number
132 // of elements is to the next larger number of elements type for which
133 // the vector type is legal, unless there is no such type. In that case,
134 // legalize towards a vector type with a smaller number of elements.
135 SizeAndActionsVec NumElementsActions;
136 for (SizeAndAction BitsizeAndAction : VectorSpecifiedActions.second) {
137 assert(BitsizeAndAction.first % ElementSize == 0);
138 const uint16_t NumElements = BitsizeAndAction.first / ElementSize;
139 NumElementsActions.push_back({NumElements, BitsizeAndAction.second});
140 }
141 setVectorNumElementAction(
142 Opcode, TypeIdx, ElementSize,
143 moreToWiderTypesAndLessToWidest(NumElementsActions));
144 }
145 std::sort(ElementSizesSeen.begin(), ElementSizesSeen.end());
146 SizeChangeStrategy VectorElementSizeChangeStrategy =
147 &unsupportedForDifferentSizes;
148 if (TypeIdx < VectorElementSizeChangeStrategies[OpcodeIdx].size() &&
149 VectorElementSizeChangeStrategies[OpcodeIdx][TypeIdx] != nullptr)
150 VectorElementSizeChangeStrategy =
151 VectorElementSizeChangeStrategies[OpcodeIdx][TypeIdx];
152 setScalarInVectorAction(
153 Opcode, TypeIdx, VectorElementSizeChangeStrategy(ElementSizesSeen));
70154 }
71155 }
72156
89173 Aspect.Opcode == TargetOpcode::G_UNMERGE_VALUES)
90174 return std::make_pair(Legal, Aspect.Type);
91175
92 LLT Ty = Aspect.Type;
93 LegalizeAction Action = findInActions(Aspect);
94 // LegalizerHelper is not able to handle non-power-of-2 types right now, so do
95 // not try to legalize them unless they are marked as Legal or Custom.
96 // FIXME: This is a temporary hack until the general non-power-of-2
97 // legalization works.
98 if (!isPowerOf2_64(Ty.getSizeInBits()) &&
99 !(Action == Legal || Action == Custom))
100 return std::make_pair(Unsupported, LLT());
101
102 if (Action != NotFound)
103 return findLegalAction(Aspect, Action);
104
105 unsigned Opcode = Aspect.Opcode;
106 if (!Ty.isVector()) {
107 auto DefaultAction = DefaultActions.find(Aspect.Opcode);
108 if (DefaultAction != DefaultActions.end() && DefaultAction->second == Legal)
109 return std::make_pair(Legal, Ty);
110
111 if (DefaultAction != DefaultActions.end() && DefaultAction->second == Lower)
112 return std::make_pair(Lower, Ty);
113
114 if (DefaultAction == DefaultActions.end() ||
115 DefaultAction->second != NarrowScalar)
116 return std::make_pair(Unsupported, LLT());
117 return findLegalAction(Aspect, NarrowScalar);
118 }
119
120 LLT EltTy = Ty.getElementType();
121 int NumElts = Ty.getNumElements();
122
123 auto ScalarAction = ScalarInVectorActions.find(std::make_pair(Opcode, EltTy));
124 if (ScalarAction != ScalarInVectorActions.end() &&
125 ScalarAction->second != Legal)
126 return findLegalAction(Aspect, ScalarAction->second);
127
128 // The element type is legal in principle, but the number of elements is
129 // wrong.
130 auto MaxLegalElts = MaxLegalVectorElts.lookup(std::make_pair(Opcode, EltTy));
131 if (MaxLegalElts > NumElts)
132 return findLegalAction(Aspect, MoreElements);
133
134 if (MaxLegalElts == 0) {
135 // Scalarize if there's no legal vector type, which is just a special case
136 // of FewerElements.
137 return std::make_pair(FewerElements, EltTy);
138 }
139
140 return findLegalAction(Aspect, FewerElements);
176 if (Aspect.Type.isScalar() || Aspect.Type.isPointer())
177 return findScalarLegalAction(Aspect);
178 assert(Aspect.Type.isVector());
179 return findVectorLegalAction(Aspect);
141180 }
142181
143182 std::tuple
144183 LegalizerInfo::getAction(const MachineInstr &MI,
145184 const MachineRegisterInfo &MRI) const {
146185 SmallBitVector SeenTypes(8);
147 const MCInstrDesc &MCID = MI.getDesc();
148 const MCOperandInfo *OpInfo = MCID.OpInfo;
149 for (unsigned i = 0, e = MCID.getNumOperands(); i != e; ++i) {
186 const MCOperandInfo *OpInfo = MI.getDesc().OpInfo;
187 // FIXME: probably we'll need to cache the results here somehow?
188 for (unsigned i = 0; i < MI.getDesc().getNumOperands(); ++i) {
150189 if (!OpInfo[i].isGenericType())
151190 continue;
152191
153 // We don't want to repeatedly check the same operand index, that
154 // could get expensive.
192 // We must only record actions once for each TypeIdx; otherwise we'd
193 // try to legalize operands multiple times down the line.
155194 unsigned TypeIdx = OpInfo[i].getGenericTypeIndex();
156195 if (SeenTypes[TypeIdx])
157196 continue;
171210 return std::get<0>(getAction(MI, MRI)) == Legal;
172211 }
173212
174 Optional LegalizerInfo::findLegalType(const InstrAspect &Aspect,
175 LegalizeAction Action) const {
176 switch(Action) {
177 default:
178 llvm_unreachable("Cannot find legal type");
213 bool LegalizerInfo::legalizeCustom(MachineInstr &MI, MachineRegisterInfo &MRI,
214 MachineIRBuilder &MIRBuilder) const {
215 return false;
216 }
217
218 LegalizerInfo::SizeAndActionsVec
219 LegalizerInfo::increaseToLargerTypesAndDecreaseToLargest(
220 const SizeAndActionsVec &v, LegalizeAction IncreaseAction,
221 LegalizeAction DecreaseAction) {
222 SizeAndActionsVec result;
223 unsigned LargestSizeSoFar = 0;
224 if (v.size() >= 1 && v[0].first != 1)
225 result.push_back({1, IncreaseAction});
226 for (size_t i = 0; i < v.size(); ++i) {
227 result.push_back(v[i]);
228 LargestSizeSoFar = v[i].first;
229 if (i + 1 < v.size() && v[i + 1].first != v[i].first + 1) {
230 result.push_back({LargestSizeSoFar + 1, IncreaseAction});
231 LargestSizeSoFar = v[i].first + 1;
232 }
233 }
234 result.push_back({LargestSizeSoFar + 1, DecreaseAction});
235 return result;
236 }
237
238 LegalizerInfo::SizeAndActionsVec
239 LegalizerInfo::decreaseToSmallerTypesAndIncreaseToSmallest(
240 const SizeAndActionsVec &v, LegalizeAction DecreaseAction,
241 LegalizeAction IncreaseAction) {
242 SizeAndActionsVec result;
243 if (v.size() == 0 || v[0].first != 1)
244 result.push_back({1, IncreaseAction});
245 for (size_t i = 0; i < v.size(); ++i) {
246 result.push_back(v[i]);
247 if (i + 1 == v.size() || v[i + 1].first != v[i].first + 1) {
248 result.push_back({v[i].first + 1, DecreaseAction});
249 }
250 }
251 return result;
252 }
253
254 LegalizerInfo::SizeAndAction
255 LegalizerInfo::findAction(const SizeAndActionsVec &Vec, const uint32_t Size) {
256 assert(Size >= 1);
257 // Find the last element in Vec that has a bitsize equal to or smaller than
258 // the requested bit size.
259 // That is the element just before the first element that is bigger than Size.
260 auto VecIt = std::upper_bound(
261 Vec.begin(), Vec.end(), Size,
262 [](const uint32_t Size, const SizeAndAction lhs) -> bool {
263 return Size < lhs.first;
264 });
265 assert(VecIt != Vec.begin() && "Does Vec not start with size 1?");
266 --VecIt;
267 int VecIdx = VecIt - Vec.begin();
268
269 LegalizeAction Action = Vec[VecIdx].second;
270 switch (Action) {
179271 case Legal:
180272 case Lower:
181273 case Libcall:
182274 case Custom:
183 return Aspect.Type;
275 return {Size, Action};
276 case FewerElements:
277 // FIXME: is this special case still needed and correct?
278 // Special case for scalarization:
279 if (Vec == SizeAndActionsVec({{1, FewerElements}}))
280 return {1, FewerElements};
184281 case NarrowScalar: {
185 return findLegalizableSize(
186 Aspect, [&](LLT Ty) -> LLT { return Ty.halfScalarSize(); });
187 }
188 case WidenScalar: {
189 return findLegalizableSize(Aspect, [&](LLT Ty) -> LLT {
190 return Ty.getSizeInBits() < 8 ? LLT::scalar(8) : Ty.doubleScalarSize();
191 });
192 }
193 case FewerElements: {
194 return findLegalizableSize(
195 Aspect, [&](LLT Ty) -> LLT { return Ty.halfElements(); });
196 }
282 // The following needs to be a loop, as for now, we do allow needing to
283 // go over "Unsupported" bit sizes before finding a legalizable bit size.
284 // e.g. (s8, WidenScalar), (s9, Unsupported), (s32, Legal). if Size==8,
285 // we need to iterate over s9, and then to s32 to return (s32, Legal).
286 // If we want to get rid of the below loop, we should have stronger asserts
287 // when building the SizeAndActionsVecs, probably not allowing
288 // "Unsupported" unless at the ends of the vector.
289 for (int i = VecIdx - 1; i >= 0; --i)
290 if (!needsLegalizingToDifferentSize(Vec[i].second) &&
291 Vec[i].second != Unsupported)
292 return {Vec[i].first, Action};
293 llvm_unreachable("");
294 }
295 case WidenScalar:
197296 case MoreElements: {
198 return findLegalizableSize(
199 Aspect, [&](LLT Ty) -> LLT { return Ty.doubleElements(); });
200 }
201 }
202 }
203
204 bool LegalizerInfo::legalizeCustom(MachineInstr &MI,
205 MachineRegisterInfo &MRI,
206 MachineIRBuilder &MIRBuilder) const {
207 return false;
208 }
297 // See above, the following needs to be a loop, at least for now.
298 for (std::size_t i = VecIdx + 1; i < Vec.size(); ++i)
299 if (!needsLegalizingToDifferentSize(Vec[i].second) &&
300 Vec[i].second != Unsupported)
301 return {Vec[i].first, Action};
302 llvm_unreachable("");
303 }
304 case Unsupported:
305 return {Size, Unsupported};
306 case NotFound:
307 llvm_unreachable("NotFound");
308 }
309 }
310
311 std::pair
312 LegalizerInfo::findScalarLegalAction(const InstrAspect &Aspect) const {
313 assert(Aspect.Type.isScalar() || Aspect.Type.isPointer());
314 if (Aspect.Opcode < FirstOp || Aspect.Opcode > LastOp)
315 return {NotFound, LLT()};
316 const unsigned OpcodeIdx = Aspect.Opcode - FirstOp;
317 if (Aspect.Type.isPointer() &&
318 AddrSpace2PointerActions[OpcodeIdx].find(Aspect.Type.getAddressSpace()) ==
319 AddrSpace2PointerActions[OpcodeIdx].end()) {
320 return {NotFound, LLT()};
321 }
322 const SmallVector &Actions =
323 Aspect.Type.isPointer()
324 ? AddrSpace2PointerActions[OpcodeIdx]
325 .find(Aspect.Type.getAddressSpace())
326 ->second
327 : ScalarActions[OpcodeIdx];
328 if (Aspect.Idx >= Actions.size())
329 return {NotFound, LLT()};
330 const SizeAndActionsVec &Vec = Actions[Aspect.Idx];
331 // FIXME: speed up this search, e.g. by using a results cache for repeated
332 // queries?
333 auto SizeAndAction = findAction(Vec, Aspect.Type.getSizeInBits());
334 return {SizeAndAction.second,
335 Aspect.Type.isScalar() ? LLT::scalar(SizeAndAction.first)
336 : LLT::pointer(Aspect.Type.getAddressSpace(),
337 SizeAndAction.first)};
338 }
339
340 std::pair
341 LegalizerInfo::findVectorLegalAction(const InstrAspect &Aspect) const {
342 assert(Aspect.Type.isVector());
343 // First legalize the vector element size, then legalize the number of
344 // lanes in the vector.
345 if (Aspect.Opcode < FirstOp || Aspect.Opcode > LastOp)
346 return {NotFound, Aspect.Type};
347 const unsigned OpcodeIdx = Aspect.Opcode - FirstOp;
348 const unsigned TypeIdx = Aspect.Idx;
349 if (TypeIdx >= ScalarInVectorActions[OpcodeIdx].size())
350 return {NotFound, Aspect.Type};
351 const SizeAndActionsVec &ElemSizeVec =
352 ScalarInVectorActions[OpcodeIdx][TypeIdx];
353
354 LLT IntermediateType;
355 auto ElementSizeAndAction =
356 findAction(ElemSizeVec, Aspect.Type.getScalarSizeInBits());
357 IntermediateType =
358 LLT::vector(Aspect.Type.getNumElements(), ElementSizeAndAction.first);
359 if (ElementSizeAndAction.second != Legal)
360 return {ElementSizeAndAction.second, IntermediateType};
361
362 auto i = NumElements2Actions[OpcodeIdx].find(
363 IntermediateType.getScalarSizeInBits());
364 if (i == NumElements2Actions[OpcodeIdx].end()) {
365 return {NotFound, IntermediateType};
366 }
367 const SizeAndActionsVec &NumElementsVec = (*i).second[TypeIdx];
368 auto NumElementsAndAction =
369 findAction(NumElementsVec, IntermediateType.getNumElements());
370 return {NumElementsAndAction.second,
371 LLT::vector(NumElementsAndAction.first,
372 IntermediateType.getScalarSizeInBits())};
373 }
4242 assert(isScalar() && "unexpected type");
4343 OS << "s" << getScalarSizeInBits();
4444 } else
45 llvm_unreachable("trying to print an invalid type");
45 OS << "LLT_invalid";
4646 }
4747
4848 const constexpr LLT::BitFieldInfo LLT::ScalarSizeFieldInfo;
2222
2323 using namespace llvm;
2424
25 /// FIXME: The following static functions are SizeChangeStrategy functions
26 /// that are meant to temporarily mimic the behaviour of the old legalization
27 /// based on doubling/halving non-legal types as closely as possible. This is
28 /// not entirly possible as only legalizing the types that are exactly a power
29 /// of 2 times the size of the legal types would require specifying all those
30 /// sizes explicitly.
31 /// In practice, not specifying those isn't a problem, and the below functions
32 /// should disappear quickly as we add support for legalizing non-power-of-2
33 /// sized types further.
34 static void
35 addAndInterleaveWithUnsupported(LegalizerInfo::SizeAndActionsVec &result,
36 const LegalizerInfo::SizeAndActionsVec &v) {
37 for (unsigned i = 0; i < v.size(); ++i) {
38 result.push_back(v[i]);
39 if (i + 1 < v[i].first && i + 1 < v.size() &&
40 v[i + 1].first != v[i].first + 1)
41 result.push_back({v[i].first + 1, LegalizerInfo::Unsupported});
42 }
43 }
44
45 static LegalizerInfo::SizeAndActionsVec
46 widen_1_narrow_128_ToLargest(const LegalizerInfo::SizeAndActionsVec &v) {
47 assert(v.size() >= 1);
48 assert(v[0].first > 2);
49 LegalizerInfo::SizeAndActionsVec result = {{1, LegalizerInfo::WidenScalar},
50 {2, LegalizerInfo::Unsupported}};
51 addAndInterleaveWithUnsupported(result, v);
52 auto Largest = result.back().first;
53 assert(Largest + 1 < 128);
54 result.push_back({Largest + 1, LegalizerInfo::Unsupported});
55 result.push_back({128, LegalizerInfo::NarrowScalar});
56 result.push_back({129, LegalizerInfo::Unsupported});
57 return result;
58 }
59
60 static LegalizerInfo::SizeAndActionsVec
61 widen_16(const LegalizerInfo::SizeAndActionsVec &v) {
62 assert(v.size() >= 1);
63 assert(v[0].first > 17);
64 LegalizerInfo::SizeAndActionsVec result = {{1, LegalizerInfo::Unsupported},
65 {16, LegalizerInfo::WidenScalar},
66 {17, LegalizerInfo::Unsupported}};
67 addAndInterleaveWithUnsupported(result, v);
68 auto Largest = result.back().first;
69 result.push_back({Largest + 1, LegalizerInfo::Unsupported});
70 return result;
71 }
72
73 static LegalizerInfo::SizeAndActionsVec
74 widen_1_8(const LegalizerInfo::SizeAndActionsVec &v) {
75 assert(v.size() >= 1);
76 assert(v[0].first > 9);
77 LegalizerInfo::SizeAndActionsVec result = {
78 {1, LegalizerInfo::WidenScalar}, {2, LegalizerInfo::Unsupported},
79 {8, LegalizerInfo::WidenScalar}, {9, LegalizerInfo::Unsupported}};
80 addAndInterleaveWithUnsupported(result, v);
81 auto Largest = result.back().first;
82 result.push_back({Largest + 1, LegalizerInfo::Unsupported});
83 return result;
84 }
85
86 static LegalizerInfo::SizeAndActionsVec
87 widen_1_8_16(const LegalizerInfo::SizeAndActionsVec &v) {
88 assert(v.size() >= 1);
89 assert(v[0].first > 17);
90 LegalizerInfo::SizeAndActionsVec result = {
91 {1, LegalizerInfo::WidenScalar}, {2, LegalizerInfo::Unsupported},
92 {8, LegalizerInfo::WidenScalar}, {9, LegalizerInfo::Unsupported},
93 {16, LegalizerInfo::WidenScalar}, {17, LegalizerInfo::Unsupported}};
94 addAndInterleaveWithUnsupported(result, v);
95 auto Largest = result.back().first;
96 result.push_back({Largest + 1, LegalizerInfo::Unsupported});
97 return result;
98 }
99
100 static LegalizerInfo::SizeAndActionsVec
101 widen_1_8_16_narrowToLargest(const LegalizerInfo::SizeAndActionsVec &v) {
102 assert(v.size() >= 1);
103 assert(v[0].first > 17);
104 LegalizerInfo::SizeAndActionsVec result = {
105 {1, LegalizerInfo::WidenScalar}, {2, LegalizerInfo::Unsupported},
106 {8, LegalizerInfo::WidenScalar}, {9, LegalizerInfo::Unsupported},
107 {16, LegalizerInfo::WidenScalar}, {17, LegalizerInfo::Unsupported}};
108 addAndInterleaveWithUnsupported(result, v);
109 auto Largest = result.back().first;
110 result.push_back({Largest + 1, LegalizerInfo::NarrowScalar});
111 return result;
112 }
113
114 static LegalizerInfo::SizeAndActionsVec
115 widen_1_8_16_32(const LegalizerInfo::SizeAndActionsVec &v) {
116 assert(v.size() >= 1);
117 assert(v[0].first > 33);
118 LegalizerInfo::SizeAndActionsVec result = {
119 {1, LegalizerInfo::WidenScalar}, {2, LegalizerInfo::Unsupported},
120 {8, LegalizerInfo::WidenScalar}, {9, LegalizerInfo::Unsupported},
121 {16, LegalizerInfo::WidenScalar}, {17, LegalizerInfo::Unsupported},
122 {32, LegalizerInfo::WidenScalar}, {33, LegalizerInfo::Unsupported}};
123 addAndInterleaveWithUnsupported(result, v);
124 auto Largest = result.back().first;
125 result.push_back({Largest + 1, LegalizerInfo::Unsupported});
126 return result;
127 }
128
25129 AArch64LegalizerInfo::AArch64LegalizerInfo() {
26130 using namespace TargetOpcode;
27131 const LLT p0 = LLT::pointer(0, 64);
41145 for (auto Ty : {s16, s32, s64, p0})
42146 setAction({G_PHI, Ty}, Legal);
43147
44 for (auto Ty : {s1, s8})
45 setAction({G_PHI, Ty}, WidenScalar);
148 setLegalizeScalarToDifferentSizeStrategy(G_PHI, 0, widen_1_8);
46149
47150 for (auto Ty : { s32, s64 })
48151 setAction({G_BSWAP, Ty}, Legal);
53156 for (auto Ty : {s32, s64, v2s32, v4s32, v2s64})
54157 setAction({BinOp, Ty}, Legal);
55158
56 for (auto Ty : {s1, s8, s16})
57 setAction({BinOp, Ty}, WidenScalar);
159 if (BinOp != G_ADD)
160 setLegalizeScalarToDifferentSizeStrategy(BinOp, 0,
161 widen_1_8_16_narrowToLargest);
58162 }
59163
60164 setAction({G_GEP, p0}, Legal);
61165 setAction({G_GEP, 1, s64}, Legal);
62166
63 for (auto Ty : {s1, s8, s16, s32})
64 setAction({G_GEP, 1, Ty}, WidenScalar);
167 setLegalizeScalarToDifferentSizeStrategy(G_GEP, 1, widen_1_8_16_32);
65168
66169 setAction({G_PTR_MASK, p0}, Legal);
67170
69172 for (auto Ty : {s32, s64})
70173 setAction({BinOp, Ty}, Legal);
71174
72 for (auto Ty : {s1, s8, s16})
73 setAction({BinOp, Ty}, WidenScalar);
175 setLegalizeScalarToDifferentSizeStrategy(BinOp, 0, widen_1_8_16);
74176 }
75177
76178 for (unsigned BinOp : {G_SREM, G_UREM})
77179 for (auto Ty : { s1, s8, s16, s32, s64 })
78180 setAction({BinOp, Ty}, Lower);
79181
80 for (unsigned Op : {G_SMULO, G_UMULO})
81 setAction({Op, s64}, Lower);
182 for (unsigned Op : {G_SMULO, G_UMULO}) {
183 setAction({Op, 0, s64}, Lower);
184 setAction({Op, 1, s1}, Legal);
185 }
82186
83187 for (unsigned Op : {G_UADDE, G_USUBE, G_SADDO, G_SSUBO, G_SMULH, G_UMULH}) {
84188 for (auto Ty : { s32, s64 })
100204 setAction({G_INSERT, Ty}, Legal);
101205 setAction({G_INSERT, 1, Ty}, Legal);
102206 }
207 setLegalizeScalarToDifferentSizeStrategy(G_INSERT, 0,
208 widen_1_8_16_narrowToLargest);
103209 for (auto Ty : {s1, s8, s16}) {
104 setAction({G_INSERT, Ty}, WidenScalar);
105210 setAction({G_INSERT, 1, Ty}, Legal);
106211 // FIXME: Can't widen the sources because that violates the constraints on
107212 // G_INSERT (It seems entirely reasonable that inputs shouldn't overlap).
117222 for (auto Ty : {s8, s16, s32, s64, p0, v2s32})
118223 setAction({MemOp, Ty}, Legal);
119224
120 setAction({MemOp, s1}, WidenScalar);
225 setLegalizeScalarToDifferentSizeStrategy(MemOp, 0,
226 widen_1_narrow_128_ToLargest);
121227
122228 // And everything's fine in addrspace 0.
123229 setAction({MemOp, 1, p0}, Legal);
131237
132238 setAction({G_CONSTANT, p0}, Legal);
133239
134 for (auto Ty : {s1, s8, s16})
135 setAction({TargetOpcode::G_CONSTANT, Ty}, WidenScalar);
136
137 setAction({TargetOpcode::G_FCONSTANT, s16}, WidenScalar);
240 setLegalizeScalarToDifferentSizeStrategy(G_CONSTANT, 0, widen_1_8_16);
241 setLegalizeScalarToDifferentSizeStrategy(G_FCONSTANT, 0, widen_16);
138242
139243 setAction({G_ICMP, 1, s32}, Legal);
140244 setAction({G_ICMP, 1, s64}, Legal);
141245 setAction({G_ICMP, 1, p0}, Legal);
142246
143 for (auto Ty : {s1, s8, s16}) {
144 setAction({G_ICMP, Ty}, WidenScalar);
145 setAction({G_FCMP, Ty}, WidenScalar);
146 setAction({G_ICMP, 1, Ty}, WidenScalar);
147 }
247 setLegalizeScalarToDifferentSizeStrategy(G_ICMP, 0, widen_1_8_16);
248 setLegalizeScalarToDifferentSizeStrategy(G_FCMP, 0, widen_1_8_16);
249 setLegalizeScalarToDifferentSizeStrategy(G_ICMP, 1, widen_1_8_16);
148250
149251 setAction({G_ICMP, s32}, Legal);
150252 setAction({G_FCMP, s32}, Legal);
158260 setAction({G_ANYEXT, Ty}, Legal);
159261 }
160262
161 for (auto Ty : { s1, s8, s16, s32 }) {
162 setAction({G_ZEXT, 1, Ty}, Legal);
163 setAction({G_SEXT, 1, Ty}, Legal);
164 setAction({G_ANYEXT, 1, Ty}, Legal);
165 }
166
167263 // FP conversions
168264 for (auto Ty : { s16, s32 }) {
169265 setAction({G_FPTRUNC, Ty}, Legal);
174270 setAction({G_FPTRUNC, 1, Ty}, Legal);
175271 setAction({G_FPEXT, Ty}, Legal);
176272 }
177
178 for (auto Ty : { s1, s8, s16, s32 })
179 setAction({G_TRUNC, Ty}, Legal);
180
181 for (auto Ty : { s8, s16, s32, s64 })
182 setAction({G_TRUNC, 1, Ty}, Legal);
183273
184274 // Conversions
185275 for (auto Ty : { s32, s64 }) {
188278 setAction({G_SITOFP, 1, Ty}, Legal);
189279 setAction({G_UITOFP, 1, Ty}, Legal);
190280 }
191 for (auto Ty : { s1, s8, s16 }) {
192 setAction({G_FPTOSI, 0, Ty}, WidenScalar);
193 setAction({G_FPTOUI, 0, Ty}, WidenScalar);
194 setAction({G_SITOFP, 1, Ty}, WidenScalar);
195 setAction({G_UITOFP, 1, Ty}, WidenScalar);
196 }
281 setLegalizeScalarToDifferentSizeStrategy(G_FPTOSI, 0, widen_1_8_16);
282 setLegalizeScalarToDifferentSizeStrategy(G_FPTOUI, 0, widen_1_8_16);
283 setLegalizeScalarToDifferentSizeStrategy(G_SITOFP, 1, widen_1_8_16);
284 setLegalizeScalarToDifferentSizeStrategy(G_UITOFP, 1, widen_1_8_16);
197285
198286 for (auto Ty : { s32, s64 }) {
199287 setAction({G_FPTOSI, 1, Ty}, Legal);
208296 setAction({G_BRINDIRECT, p0}, Legal);
209297
210298 // Select
211 for (auto Ty : {s1, s8, s16})
212 setAction({G_SELECT, Ty}, WidenScalar);
299 setLegalizeScalarToDifferentSizeStrategy(G_SELECT, 0, widen_1_8_16);
213300
214301 for (auto Ty : {s32, s64, p0})
215302 setAction({G_SELECT, Ty}, Legal);
2323
2424 using namespace llvm;
2525
26 /// FIXME: The following static functions are SizeChangeStrategy functions
27 /// that are meant to temporarily mimic the behaviour of the old legalization
28 /// based on doubling/halving non-legal types as closely as possible. This is
29 /// not entirly possible as only legalizing the types that are exactly a power
30 /// of 2 times the size of the legal types would require specifying all those
31 /// sizes explicitly.
32 /// In practice, not specifying those isn't a problem, and the below functions
33 /// should disappear quickly as we add support for legalizing non-power-of-2
34 /// sized types further.
35 static void
36 addAndInterleaveWithUnsupported(LegalizerInfo::SizeAndActionsVec &result,
37 const LegalizerInfo::SizeAndActionsVec &v) {
38 for (unsigned i = 0; i < v.size(); ++i) {
39 result.push_back(v[i]);
40 if (i + 1 < v[i].first && i + 1 < v.size() &&
41 v[i + 1].first != v[i].first + 1)
42 result.push_back({v[i].first + 1, LegalizerInfo::Unsupported});
43 }
44 }
45
46 static LegalizerInfo::SizeAndActionsVec
47 widen_8_16(const LegalizerInfo::SizeAndActionsVec &v) {
48 assert(v.size() >= 1);
49 assert(v[0].first > 17);
50 LegalizerInfo::SizeAndActionsVec result = {
51 {1, LegalizerInfo::Unsupported},
52 {8, LegalizerInfo::WidenScalar}, {9, LegalizerInfo::Unsupported},
53 {16, LegalizerInfo::WidenScalar}, {17, LegalizerInfo::Unsupported}};
54 addAndInterleaveWithUnsupported(result, v);
55 auto Largest = result.back().first;
56 result.push_back({Largest + 1, LegalizerInfo::Unsupported});
57 return result;
58 }
59
60 static LegalizerInfo::SizeAndActionsVec
61 widen_1_8_16(const LegalizerInfo::SizeAndActionsVec &v) {
62 assert(v.size() >= 1);
63 assert(v[0].first > 17);
64 LegalizerInfo::SizeAndActionsVec result = {
65 {1, LegalizerInfo::WidenScalar}, {2, LegalizerInfo::Unsupported},
66 {8, LegalizerInfo::WidenScalar}, {9, LegalizerInfo::Unsupported},
67 {16, LegalizerInfo::WidenScalar}, {17, LegalizerInfo::Unsupported}};
68 addAndInterleaveWithUnsupported(result, v);
69 auto Largest = result.back().first;
70 result.push_back({Largest + 1, LegalizerInfo::Unsupported});
71 return result;
72 }
73
2674 static bool AEABI(const ARMSubtarget &ST) {
2775 return ST.isTargetAEABI() || ST.isTargetGNUAEABI() || ST.isTargetMuslAEABI();
2876 }
4896 }
4997
5098 for (unsigned Op : {G_ADD, G_SUB, G_MUL, G_AND, G_OR, G_XOR}) {
51 for (auto Ty : {s1, s8, s16})
52 setAction({Op, Ty}, WidenScalar);
99 if (Op != G_ADD)
100 setLegalizeScalarToDifferentSizeStrategy(
101 Op, 0, widenToLargerTypesUnsupportedOtherwise);
53102 setAction({Op, s32}, Legal);
54103 }
55104
56105 for (unsigned Op : {G_SDIV, G_UDIV}) {
57 for (auto Ty : {s8, s16})
58 setAction({Op, Ty}, WidenScalar);
106 setLegalizeScalarToDifferentSizeStrategy(Op, 0,
107 widenToLargerTypesUnsupportedOtherwise);
59108 if (ST.hasDivideInARMMode())
60109 setAction({Op, s32}, Legal);
61110 else
63112 }
64113
65114 for (unsigned Op : {G_SREM, G_UREM}) {
66 for (auto Ty : {s8, s16})
67 setAction({Op, Ty}, WidenScalar);
115 setLegalizeScalarToDifferentSizeStrategy(Op, 0, widen_8_16);
68116 if (ST.hasDivideInARMMode())
69117 setAction({Op, s32}, Lower);
70118 else if (AEABI(ST))
73121 setAction({Op, s32}, Libcall);
74122 }
75123
76 for (unsigned Op : {G_SEXT, G_ZEXT}) {
124 for (unsigned Op : {G_SEXT, G_ZEXT, G_ANYEXT}) {
77125 setAction({Op, s32}, Legal);
78 for (auto Ty : {s1, s8, s16})
79 setAction({Op, 1, Ty}, Legal);
80126 }
81127
82128 for (unsigned Op : {G_ASHR, G_LSHR, G_SHL})
92138 setAction({G_BRCOND, s1}, Legal);
93139
94140 setAction({G_CONSTANT, s32}, Legal);
95 for (auto Ty : {s1, s8, s16})
96 setAction({G_CONSTANT, Ty}, WidenScalar);
141 setLegalizeScalarToDifferentSizeStrategy(G_CONSTANT, 0, widen_1_8_16);
97142
98143 setAction({G_ICMP, s1}, Legal);
99 for (auto Ty : {s8, s16})
100 setAction({G_ICMP, 1, Ty}, WidenScalar);
144 setLegalizeScalarToDifferentSizeStrategy(G_ICMP, 1,
145 widenToLargerTypesUnsupportedOtherwise);
101146 for (auto Ty : {s32, p0})
102147 setAction({G_ICMP, 1, Ty}, Legal);
103148
2121 using namespace llvm;
2222 using namespace TargetOpcode;
2323
24 /// FIXME: The following static functions are SizeChangeStrategy functions
25 /// that are meant to temporarily mimic the behaviour of the old legalization
26 /// based on doubling/halving non-legal types as closely as possible. This is
27 /// not entirly possible as only legalizing the types that are exactly a power
28 /// of 2 times the size of the legal types would require specifying all those
29 /// sizes explicitly.
30 /// In practice, not specifying those isn't a problem, and the below functions
31 /// should disappear quickly as we add support for legalizing non-power-of-2
32 /// sized types further.
33 static void
34 addAndInterleaveWithUnsupported(LegalizerInfo::SizeAndActionsVec &result,
35 const LegalizerInfo::SizeAndActionsVec &v) {
36 for (unsigned i = 0; i < v.size(); ++i) {
37 result.push_back(v[i]);
38 if (i + 1 < v[i].first && i + 1 < v.size() &&
39 v[i + 1].first != v[i].first + 1)
40 result.push_back({v[i].first + 1, LegalizerInfo::Unsupported});
41 }
42 }
43
44 static LegalizerInfo::SizeAndActionsVec
45 widen_1(const LegalizerInfo::SizeAndActionsVec &v) {
46 assert(v.size() >= 1);
47 assert(v[0].first > 1);
48 LegalizerInfo::SizeAndActionsVec result = {{1, LegalizerInfo::WidenScalar},
49 {2, LegalizerInfo::Unsupported}};
50 addAndInterleaveWithUnsupported(result, v);
51 auto Largest = result.back().first;
52 result.push_back({Largest + 1, LegalizerInfo::Unsupported});
53 return result;
54 }
55
2456 X86LegalizerInfo::X86LegalizerInfo(const X86Subtarget &STI,
2557 const X86TargetMachine &TM)
2658 : Subtarget(STI), TM(TM) {
3668 setLegalizerInfoAVX512DQ();
3769 setLegalizerInfoAVX512BW();
3870
71 setLegalizeScalarToDifferentSizeStrategy(G_PHI, 0, widen_1);
72 for (unsigned BinOp : {G_SUB, G_MUL, G_AND, G_OR, G_XOR})
73 setLegalizeScalarToDifferentSizeStrategy(BinOp, 0, widen_1);
74 for (unsigned MemOp : {G_LOAD, G_STORE})
75 setLegalizeScalarToDifferentSizeStrategy(MemOp, 0,
76 narrowToSmallerAndWidenToSmallest);
77 setLegalizeScalarToDifferentSizeStrategy(
78 G_GEP, 1, widenToLargerTypesUnsupportedOtherwise);
79 setLegalizeScalarToDifferentSizeStrategy(
80 G_CONSTANT, 0, widenToLargerTypesAndNarrowToLargest);
81
3982 computeTables();
4083 }
4184
4689 const LLT s8 = LLT::scalar(8);
4790 const LLT s16 = LLT::scalar(16);
4891 const LLT s32 = LLT::scalar(32);
49 const LLT s64 = LLT::scalar(64);
5092
5193 for (auto Ty : {p0, s1, s8, s16, s32})
5294 setAction({G_IMPLICIT_DEF, Ty}, Legal);
5496 for (auto Ty : {s8, s16, s32, p0})
5597 setAction({G_PHI, Ty}, Legal);
5698
57 setAction({G_PHI, s1}, WidenScalar);
58
59 for (unsigned BinOp : {G_ADD, G_SUB, G_MUL, G_AND, G_OR, G_XOR}) {
99 for (unsigned BinOp : {G_ADD, G_SUB, G_MUL, G_AND, G_OR, G_XOR})
60100 for (auto Ty : {s8, s16, s32})
61101 setAction({BinOp, Ty}, Legal);
62
63 setAction({BinOp, s1}, WidenScalar);
64 }
65102
66103 for (unsigned Op : {G_UADDE}) {
67104 setAction({Op, s32}, Legal);
72109 for (auto Ty : {s8, s16, s32, p0})
73110 setAction({MemOp, Ty}, Legal);
74111
75 setAction({MemOp, s1}, WidenScalar);
76112 // And everything's fine in addrspace 0.
77113 setAction({MemOp, 1, p0}, Legal);
78114 }
84120 setAction({G_GEP, p0}, Legal);
85121 setAction({G_GEP, 1, s32}, Legal);
86122
87 for (auto Ty : {s1, s8, s16})
88 setAction({G_GEP, 1, Ty}, WidenScalar);
89
90123 // Control-flow
91124 setAction({G_BRCOND, s1}, Legal);
92125
93126 // Constants
94127 for (auto Ty : {s8, s16, s32, p0})
95128 setAction({TargetOpcode::G_CONSTANT, Ty}, Legal);
96
97 setAction({TargetOpcode::G_CONSTANT, s1}, WidenScalar);
98 setAction({TargetOpcode::G_CONSTANT, s64}, NarrowScalar);
99129
100130 // Extensions
101131 for (auto Ty : {s8, s16, s32}) {
104134 setAction({G_ANYEXT, Ty}, Legal);
105135 }
106136
107 for (auto Ty : {s1, s8, s16}) {
108 setAction({G_ZEXT, 1, Ty}, Legal);
109 setAction({G_SEXT, 1, Ty}, Legal);
110 setAction({G_ANYEXT, 1, Ty}, Legal);
111 }
112
113137 // Comparison
114138 setAction({G_ICMP, s1}, Legal);
115139
122146 if (!Subtarget.is64Bit())
123147 return;
124148
125 const LLT s32 = LLT::scalar(32);
126149 const LLT s64 = LLT::scalar(64);
127150
128151 setAction({G_IMPLICIT_DEF, s64}, Legal);
144167 // Extensions
145168 for (unsigned extOp : {G_ZEXT, G_SEXT, G_ANYEXT}) {
146169 setAction({extOp, s64}, Legal);
147 setAction({extOp, 1, s32}, Legal);
148170 }
149171
150172 // Comparison
166166 %vec = load <2 x i16*>, <2 x i16*>* undef
167167 br label %block
168168 }
169
170 ; FALLBACK-WITH-REPORT-ERR-G_IMPLICIT_DEF-LEGALIZABLE: (FIXME: this is what is expected once we can legalize non-pow-of-2 G_IMPLICIT_DEF) remark: :0:0: unable to legalize instruction: %vreg1(s96) = G_INSERT %vreg2, %vreg0, 0; (in function: nonpow2_insertvalue_narrowing
171 ; FALLBACK-WITH-REPORT-ERR: remark: :0:0: unable to legalize instruction: %vreg2(s96) = G_IMPLICIT_DEF; (in function: nonpow2_insertvalue_narrowing
172 ; FALLBACK-WITH-REPORT-ERR: warning: Instruction selection used fallback path for nonpow2_insertvalue_narrowing
173 ; FALLBACK-WITH-REPORT-OUT-LABEL: nonpow2_insertvalue_narrowing:
174 %struct96 = type { float, float, float }
175 define void @nonpow2_insertvalue_narrowing(float %a) {
176 %dummy = insertvalue %struct96 undef, float %a, 0
177 ret void
178 }
179
180 ; FALLBACK-WITH-REPORT-ERR remark: :0:0: unable to legalize instruction: %vreg3(s96) = G_ADD %vreg2, %vreg2; (in function: nonpow2_add_narrowing
181 ; FALLBACK-WITH-REPORT-ERR: warning: Instruction selection used fallback path for nonpow2_add_narrowing
182 ; FALLBACK-WITH-REPORT-OUT-LABEL: nonpow2_add_narrowing:
183 define void @nonpow2_add_narrowing() {
184 %a = add i128 undef, undef
185 %b = trunc i128 %a to i96
186 %dummy = add i96 %b, %b
187 ret void
188 }
189
190 ; FALLBACK-WITH-REPORT-ERR: remark: :0:0: unable to legalize instruction: %vreg3(s96) = G_OR %vreg2, %vreg2; (in function: nonpow2_or_narrowing
191 ; FALLBACK-WITH-REPORT-ERR: warning: Instruction selection used fallback path for nonpow2_or_narrowing
192 ; FALLBACK-WITH-REPORT-OUT-LABEL: nonpow2_or_narrowing:
193 define void @nonpow2_or_narrowing() {
194 %a = add i128 undef, undef
195 %b = trunc i128 %a to i96
196 %dummy = or i96 %b, %b
197 ret void
198 }
199
200 ; FALLBACK-WITH-REPORT-ERR: remark: :0:0: unable to legalize instruction: %vreg0(s96) = G_LOAD %vreg1; mem:LD12[undef](align=16) (in function: nonpow2_load_narrowing
201 ; FALLBACK-WITH-REPORT-ERR: warning: Instruction selection used fallback path for nonpow2_load_narrowing
202 ; FALLBACK-WITH-REPORT-OUT-LABEL: nonpow2_load_narrowing:
203 define void @nonpow2_load_narrowing() {
204 %dummy = load i96, i96* undef
205 ret void
206 }
207
208 ; FALLBACK-WITH-REPORT-ERR: remark: :0:0: unable to legalize instruction: G_STORE %vreg3, %vreg0; mem:ST12[%c](align=16) (in function: nonpow2_store_narrowing
209 ; FALLBACK-WITH-REPORT-ERR: warning: Instruction selection used fallback path for nonpow2_store_narrowing
210 ; FALLBACK-WITH-REPORT-OUT-LABEL: nonpow2_store_narrowing:
211 define void @nonpow2_store_narrowing(i96* %c) {
212 %a = add i128 undef, undef
213 %b = trunc i128 %a to i96
214 store i96 %b, i96* %c
215 ret void
216 }
217
218 ; FALLBACK-WITH-REPORT-ERR: remark: :0:0: unable to legalize instruction: %vreg0(s96) = G_CONSTANT 0; (in function: nonpow2_constant_narrowing
219 ; FALLBACK-WITH-REPORT-ERR: warning: Instruction selection used fallback path for nonpow2_constant_narrowing
220 ; FALLBACK-WITH-REPORT-OUT-LABEL: nonpow2_constant_narrowing:
221 define void @nonpow2_constant_narrowing() {
222 store i96 0, i96* undef
223 ret void
224 }
225
226 ; Currently can't handle vector lengths that aren't an exact multiple of
227 ; natively supported vector lengths. Test that the fall-back works for those.
228 ; FALLBACK-WITH-REPORT-ERR-G_IMPLICIT_DEF-LEGALIZABLE: (FIXME: this is what is expected once we can legalize non-pow-of-2 G_IMPLICIT_DEF) remark: :0:0: unable to legalize instruction: %vreg1(<7 x s64>) = G_ADD %vreg0, %vreg0; (in function: nonpow2_vector_add_fewerelements
229 ; FALLBACK-WITH-REPORT-ERR: remark: :0:0: unable to legalize instruction: %vreg0(<7 x s64>) = G_IMPLICIT_DEF; (in function: nonpow2_vector_add_fewerelements
230 ; FALLBACK-WITH-REPORT-ERR: warning: Instruction selection used fallback path for nonpow2_vector_add_fewerelements
231 ; FALLBACK-WITH-REPORT-OUT-LABEL: nonpow2_vector_add_fewerelements:
232 define void @nonpow2_vector_add_fewerelements() {
233 %dummy = add <7 x i64> undef, undef
234 ret void
235 }
77 entry:
88 ret void
99 }
10 define void @test_scalar_add_big_nonpow2() {
11 entry:
12 ret void
13 }
1014 define void @test_scalar_add_small() {
1115 entry:
1216 ret void
1317 }
1418 define void @test_vector_add() {
19 entry:
20 ret void
21 }
22 define void @test_vector_add_nonpow2() {
1523 entry:
1624 ret void
1725 }
5765 ...
5866
5967 ---
68 name: test_scalar_add_big_nonpow2
69 registers:
70 - { id: 0, class: _ }
71 - { id: 1, class: _ }
72 - { id: 2, class: _ }
73 - { id: 3, class: _ }
74 - { id: 4, class: _ }
75 - { id: 5, class: _ }
76 - { id: 6, class: _ }
77 - { id: 7, class: _ }
78 - { id: 8, class: _ }
79 - { id: 9, class: _ }
80 body: |
81 bb.0.entry:
82 liveins: %x0, %x1, %x2, %x3
83 ; CHECK-LABEL: name: test_scalar_add_big_nonpow2
84 ; CHECK-NOT: G_MERGE_VALUES
85 ; CHECK-NOT: G_UNMERGE_VALUES
86 ; CHECK-DAG: [[CARRY0_32:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
87 ; CHECK-DAG: [[CARRY0:%[0-9]+]]:_(s1) = G_TRUNC [[CARRY0_32]]
88 ; CHECK: [[RES_LO:%[0-9]+]]:_(s64), [[CARRY1:%[0-9]+]]:_(s1) = G_UADDE %0, %1, [[CARRY0]]
89 ; CHECK: [[RES_MI:%[0-9]+]]:_(s64), [[CARRY2:%[0-9]+]]:_(s1) = G_UADDE %1, %2, [[CARRY1]]
90 ; CHECK: [[RES_HI:%[0-9]+]]:_(s64), {{%.*}}(s1) = G_UADDE %2, %3, [[CARRY2]]
91 ; CHECK-NOT: G_MERGE_VALUES
92 ; CHECK-NOT: G_UNMERGE_VALUES
93 ; CHECK: %x0 = COPY [[RES_LO]]
94 ; CHECK: %x1 = COPY [[RES_MI]]
95 ; CHECK: %x2 = COPY [[RES_HI]]
96
97 %0(s64) = COPY %x0
98 %1(s64) = COPY %x1
99 %2(s64) = COPY %x2
100 %3(s64) = COPY %x3
101 %4(s192) = G_MERGE_VALUES %0, %1, %2
102 %5(s192) = G_MERGE_VALUES %1, %2, %3
103 %6(s192) = G_ADD %4, %5
104 %7(s64), %8(s64), %9(s64) = G_UNMERGE_VALUES %6
105 %x0 = COPY %7
106 %x1 = COPY %8
107 %x2 = COPY %9
108 ...
109
110 ---
60111 name: test_scalar_add_small
61112 registers:
62113 - { id: 0, class: _ }
123174 %q0 = COPY %7
124175 %q1 = COPY %8
125176 ...
177 ---
178 name: test_vector_add_nonpow2
179 registers:
180 - { id: 0, class: _ }
181 - { id: 1, class: _ }
182 - { id: 2, class: _ }
183 - { id: 3, class: _ }
184 - { id: 4, class: _ }
185 - { id: 5, class: _ }
186 - { id: 6, class: _ }
187 - { id: 7, class: _ }
188 - { id: 8, class: _ }
189 - { id: 9, class: _ }
190 body: |
191 bb.0.entry:
192 liveins: %q0, %q1, %q2, %q3
193 ; CHECK-LABEL: name: test_vector_add_nonpow2
194 ; CHECK-NOT: G_EXTRACT
195 ; CHECK-NOT: G_SEQUENCE
196 ; CHECK: [[RES_LO:%[0-9]+]]:_(<2 x s64>) = G_ADD %0, %1
197 ; CHECK: [[RES_MI:%[0-9]+]]:_(<2 x s64>) = G_ADD %1, %2
198 ; CHECK: [[RES_HI:%[0-9]+]]:_(<2 x s64>) = G_ADD %2, %3
199 ; CHECK-NOT: G_EXTRACT
200 ; CHECK-NOT: G_SEQUENCE
201 ; CHECK: %q0 = COPY [[RES_LO]]
202 ; CHECK: %q1 = COPY [[RES_MI]]
203 ; CHECK: %q2 = COPY [[RES_HI]]
204
205 %0(<2 x s64>) = COPY %q0
206 %1(<2 x s64>) = COPY %q1
207 %2(<2 x s64>) = COPY %q2
208 %3(<2 x s64>) = COPY %q3
209 %4(<6 x s64>) = G_MERGE_VALUES %0, %1, %2
210 %5(<6 x s64>) = G_MERGE_VALUES %1, %2, %3
211 %6(<6 x s64>) = G_ADD %4, %5
212 %7(<2 x s64>), %8(<2 x s64>), %9(<2 x s64>) = G_UNMERGE_VALUES %6
213 %q0 = COPY %7
214 %q1 = COPY %8
215 %q2 = COPY %9
216 ...
88 define void @test_inserts_4() { ret void }
99 define void @test_inserts_5() { ret void }
1010 define void @test_inserts_6() { ret void }
11 define void @test_inserts_nonpow2() { ret void }
1112 ...
1213
1314 ---
140141 %4:_(s128) = G_INSERT %3, %2, 32
141142 RET_ReallyLR
142143 ...
144
145 ---
146 name: test_inserts_nonpow2
147 body: |
148 bb.0:
149 liveins: %x0, %x1, %x2
150
151
152 ; CHECK-LABEL: name: test_inserts_nonpow2
153 ; CHECK: %5:_(s192) = G_MERGE_VALUES %3(s64), %1(s64), %2(s64)
154 %0:_(s64) = COPY %x0
155 %1:_(s64) = COPY %x1
156 %2:_(s64) = COPY %x2
157 %3:_(s64) = COPY %x3
158 %4:_(s192) = G_MERGE_VALUES %0, %1, %2
159 %5:_(s192) = G_INSERT %4, %3, 0
160 RET_ReallyLR
161 ...
969969 - { id: 1, class: gprb }
970970 - { id: 2, class: gprb }
971971 - { id: 3, class: gprb }
972 body: |
973 bb.0:
974 liveins: %r0, %r1
972 - { id: 4, class: gprb }
973 body: |
974 bb.0:
975 liveins: %r0, %r1, %r2
975976
976977 %0(p0) = COPY %r0
977978 ; CHECK: [[VREGX:%[0-9]+]]:gpr = COPY %r0
979980 %1(p0) = COPY %r1
980981 ; CHECK: [[VREGY:%[0-9]+]]:gpr = COPY %r1
981982
982 %2(s1) = G_TRUNC %1(p0)
983 ; CHECK: [[VREGC:%[0-9]+]]:gpr = COPY [[VREGY]]
984
985 %3(p0) = G_SELECT %2(s1), %0, %1
986 ; CHECK: CMPri [[VREGC]], 0, 14, _, implicit-def %cpsr
983 %2(s32) = COPY %r2
984 ; CHECK: [[VREGC:%[0-9]+]]:gpr = COPY %r2
985
986 %3(s1) = G_TRUNC %2(s32)
987 ; CHECK: [[VREGD:%[0-9]+]]:gpr = COPY [[VREGC]]
988
989 %4(p0) = G_SELECT %3(s1), %0, %1
990 ; CHECK: CMPri [[VREGD]], 0, 14, _, implicit-def %cpsr
987991 ; CHECK: [[RES:%[0-9]+]]:gpr = MOVCCr [[VREGX]], [[VREGY]], 0, %cpsr
988992
989 %r0 = COPY %3(p0)
993 %r0 = COPY %4(p0)
990994 ; CHECK: %r0 = COPY [[RES]]
991995
992996 BX_RET 14, _, implicit %r0
4848 using namespace TargetOpcode;
4949 LegalizerInfo L;
5050 // Typical RISCy set of operations based on AArch64.
51 L.setAction({G_ADD, LLT::scalar(8)}, LegalizerInfo::WidenScalar);
52 L.setAction({G_ADD, LLT::scalar(16)}, LegalizerInfo::WidenScalar);
53 L.setAction({G_ADD, LLT::scalar(32)}, LegalizerInfo::Legal);
54 L.setAction({G_ADD, LLT::scalar(64)}, LegalizerInfo::Legal);
51 for (auto Op : {G_ADD, G_SUB}) {
52 for (unsigned Size : {32, 64})
53 L.setAction({Op, 0, LLT::scalar(Size)}, LegalizerInfo::Legal);
54 L.setLegalizeScalarToDifferentSizeStrategy(
55 Op, 0, LegalizerInfo::widenToLargerTypesAndNarrowToLargest);
56 }
57
5558 L.computeTables();
5659
57 // Check we infer the correct types and actually do what we're told.
58 ASSERT_EQ(L.getAction({G_ADD, LLT::scalar(8)}),
59 std::make_pair(LegalizerInfo::WidenScalar, LLT::scalar(32)));
60 ASSERT_EQ(L.getAction({G_ADD, LLT::scalar(16)}),
61 std::make_pair(LegalizerInfo::WidenScalar, LLT::scalar(32)));
62 ASSERT_EQ(L.getAction({G_ADD, LLT::scalar(32)}),
63 std::make_pair(LegalizerInfo::Legal, LLT::scalar(32)));
64 ASSERT_EQ(L.getAction({G_ADD, LLT::scalar(64)}),
65 std::make_pair(LegalizerInfo::Legal, LLT::scalar(64)));
60 for (auto &opcode : {G_ADD, G_SUB}) {
61 // Check we infer the correct types and actually do what we're told.
62 ASSERT_EQ(L.getAction({opcode, LLT::scalar(8)}),
63 std::make_pair(LegalizerInfo::WidenScalar, LLT::scalar(32)));
64 ASSERT_EQ(L.getAction({opcode, LLT::scalar(16)}),
65 std::make_pair(LegalizerInfo::WidenScalar, LLT::scalar(32)));
66 ASSERT_EQ(L.getAction({opcode, LLT::scalar(32)}),
67 std::make_pair(LegalizerInfo::Legal, LLT::scalar(32)));
68 ASSERT_EQ(L.getAction({opcode, LLT::scalar(64)}),
69 std::make_pair(LegalizerInfo::Legal, LLT::scalar(64)));
6670
67 // Make sure the default for over-sized types applies.
68 ASSERT_EQ(L.getAction({G_ADD, LLT::scalar(128)}),
69 std::make_pair(LegalizerInfo::NarrowScalar, LLT::scalar(64)));
71 // Make sure the default for over-sized types applies.
72 ASSERT_EQ(L.getAction({opcode, LLT::scalar(128)}),
73 std::make_pair(LegalizerInfo::NarrowScalar, LLT::scalar(64)));
74 // Make sure we also handle unusual sizes
75 ASSERT_EQ(L.getAction({opcode, LLT::scalar(1)}),
76 std::make_pair(LegalizerInfo::WidenScalar, LLT::scalar(32)));
77 ASSERT_EQ(L.getAction({opcode, LLT::scalar(31)}),
78 std::make_pair(LegalizerInfo::WidenScalar, LLT::scalar(32)));
79 ASSERT_EQ(L.getAction({opcode, LLT::scalar(33)}),
80 std::make_pair(LegalizerInfo::WidenScalar, LLT::scalar(64)));
81 ASSERT_EQ(L.getAction({opcode, LLT::scalar(63)}),
82 std::make_pair(LegalizerInfo::WidenScalar, LLT::scalar(64)));
83 ASSERT_EQ(L.getAction({opcode, LLT::scalar(65)}),
84 std::make_pair(LegalizerInfo::NarrowScalar, LLT::scalar(64)));
85 }
7086 }
7187
7288 TEST(LegalizerInfoTest, VectorRISC) {
7389 using namespace TargetOpcode;
7490 LegalizerInfo L;
7591 // Typical RISCy set of operations based on ARM.
76 L.setScalarInVectorAction(G_ADD, LLT::scalar(8), LegalizerInfo::Legal);
77 L.setScalarInVectorAction(G_ADD, LLT::scalar(16), LegalizerInfo::Legal);
78 L.setScalarInVectorAction(G_ADD, LLT::scalar(32), LegalizerInfo::Legal);
79
8092 L.setAction({G_ADD, LLT::vector(8, 8)}, LegalizerInfo::Legal);
8193 L.setAction({G_ADD, LLT::vector(16, 8)}, LegalizerInfo::Legal);
8294 L.setAction({G_ADD, LLT::vector(4, 16)}, LegalizerInfo::Legal);
8395 L.setAction({G_ADD, LLT::vector(8, 16)}, LegalizerInfo::Legal);
8496 L.setAction({G_ADD, LLT::vector(2, 32)}, LegalizerInfo::Legal);
8597 L.setAction({G_ADD, LLT::vector(4, 32)}, LegalizerInfo::Legal);
98
99 L.setLegalizeVectorElementToDifferentSizeStrategy(
100 G_ADD, 0, LegalizerInfo::widenToLargerTypesUnsupportedOtherwise);
101
102 L.setAction({G_ADD, 0, LLT::scalar(32)}, LegalizerInfo::Legal);
103
86104 L.computeTables();
87105
88106 // Check we infer the correct types and actually do what we're told for some
89107 // simple cases.
108 ASSERT_EQ(L.getAction({G_ADD, LLT::vector(8, 8)}),
109 std::make_pair(LegalizerInfo::Legal, LLT::vector(8, 8)));
110 ASSERT_EQ(L.getAction({G_ADD, LLT::vector(8, 7)}),
111 std::make_pair(LegalizerInfo::WidenScalar, LLT::vector(8, 8)));
90112 ASSERT_EQ(L.getAction({G_ADD, LLT::vector(2, 8)}),
91113 std::make_pair(LegalizerInfo::MoreElements, LLT::vector(8, 8)));
92 ASSERT_EQ(L.getAction({G_ADD, LLT::vector(8, 8)}),
93 std::make_pair(LegalizerInfo::Legal, LLT::vector(8, 8)));
94 ASSERT_EQ(
95 L.getAction({G_ADD, LLT::vector(8, 32)}),
96 std::make_pair(LegalizerInfo::FewerElements, LLT::vector(4, 32)));
114 ASSERT_EQ(L.getAction({G_ADD, LLT::vector(8, 32)}),
115 std::make_pair(LegalizerInfo::FewerElements, LLT::vector(4, 32)));
116 // Check a few non-power-of-2 sizes:
117 ASSERT_EQ(L.getAction({G_ADD, LLT::vector(3, 3)}),
118 std::make_pair(LegalizerInfo::WidenScalar, LLT::vector(3, 8)));
119 ASSERT_EQ(L.getAction({G_ADD, LLT::vector(3, 8)}),
120 std::make_pair(LegalizerInfo::MoreElements, LLT::vector(8, 8)));
97121 }
98122
99123 TEST(LegalizerInfoTest, MultipleTypes) {
100124 using namespace TargetOpcode;
101125 LegalizerInfo L;
102126 LLT p0 = LLT::pointer(0, 64);
103 LLT s32 = LLT::scalar(32);
104127 LLT s64 = LLT::scalar(64);
105128
106129 // Typical RISCy set of operations based on AArch64.
107130 L.setAction({G_PTRTOINT, 0, s64}, LegalizerInfo::Legal);
108131 L.setAction({G_PTRTOINT, 1, p0}, LegalizerInfo::Legal);
109132
110 L.setAction({G_PTRTOINT, 0, s32}, LegalizerInfo::WidenScalar);
133 L.setLegalizeScalarToDifferentSizeStrategy(
134 G_PTRTOINT, 0, LegalizerInfo::widenToLargerTypesAndNarrowToLargest);
135
111136 L.computeTables();
112137
113138 // Check we infer the correct types and actually do what we're told.
115140 std::make_pair(LegalizerInfo::Legal, s64));
116141 ASSERT_EQ(L.getAction({G_PTRTOINT, 1, p0}),
117142 std::make_pair(LegalizerInfo::Legal, p0));
143 // Make sure we also handle unusual sizes
144 ASSERT_EQ(L.getAction({G_PTRTOINT, 0, LLT::scalar(65)}),
145 std::make_pair(LegalizerInfo::NarrowScalar, s64));
146 ASSERT_EQ(L.getAction({G_PTRTOINT, 1, LLT::pointer(0, 32)}),
147 std::make_pair(LegalizerInfo::Unsupported, LLT::pointer(0, 32)));
118148 }
119149
120150 TEST(LegalizerInfoTest, MultipleSteps) {
121151 using namespace TargetOpcode;
122152 LegalizerInfo L;
123 LLT s16 = LLT::scalar(16);
124153 LLT s32 = LLT::scalar(32);
125154 LLT s64 = LLT::scalar(64);
126155
127 L.setAction({G_UREM, 0, s16}, LegalizerInfo::WidenScalar);
156 L.setLegalizeScalarToDifferentSizeStrategy(
157 G_UREM, 0, LegalizerInfo::widenToLargerTypesUnsupportedOtherwise);
128158 L.setAction({G_UREM, 0, s32}, LegalizerInfo::Lower);
129159 L.setAction({G_UREM, 0, s64}, LegalizerInfo::Lower);
130160
135165 ASSERT_EQ(L.getAction({G_UREM, LLT::scalar(32)}),
136166 std::make_pair(LegalizerInfo::Lower, LLT::scalar(32)));
137167 }
168
169 TEST(LegalizerInfoTest, SizeChangeStrategy) {
170 using namespace TargetOpcode;
171 LegalizerInfo L;
172 for (unsigned Size : {1, 8, 16, 32})
173 L.setAction({G_UREM, 0, LLT::scalar(Size)}, LegalizerInfo::Legal);
174
175 L.setLegalizeScalarToDifferentSizeStrategy(
176 G_UREM, 0, LegalizerInfo::widenToLargerTypesUnsupportedOtherwise);
177 L.computeTables();
178
179 // Check we infer the correct types and actually do what we're told.
180 for (unsigned Size : {1, 8, 16, 32}) {
181 ASSERT_EQ(L.getAction({G_UREM, LLT::scalar(Size)}),
182 std::make_pair(LegalizerInfo::Legal, LLT::scalar(Size)));
183 }
184 ASSERT_EQ(L.getAction({G_UREM, LLT::scalar(2)}),
185 std::make_pair(LegalizerInfo::WidenScalar, LLT::scalar(8)));
186 ASSERT_EQ(L.getAction({G_UREM, LLT::scalar(7)}),
187 std::make_pair(LegalizerInfo::WidenScalar, LLT::scalar(8)));
188 ASSERT_EQ(L.getAction({G_UREM, LLT::scalar(9)}),
189 std::make_pair(LegalizerInfo::WidenScalar, LLT::scalar(16)));
190 ASSERT_EQ(L.getAction({G_UREM, LLT::scalar(17)}),
191 std::make_pair(LegalizerInfo::WidenScalar, LLT::scalar(32)));
192 ASSERT_EQ(L.getAction({G_UREM, LLT::scalar(31)}),
193 std::make_pair(LegalizerInfo::WidenScalar, LLT::scalar(32)));
194 ASSERT_EQ(L.getAction({G_UREM, LLT::scalar(33)}),
195 std::make_pair(LegalizerInfo::Unsupported, LLT::scalar(33)));
138196 }
197 }
3535
3636 for (unsigned S : {1U, 17U, 32U, 64U, 0xfffffU}) {
3737 const LLT Ty = LLT::scalar(S);
38 const LLT HalfTy = (S % 2) == 0 ? Ty.halfScalarSize() : Ty;
39 const LLT DoubleTy = Ty.doubleScalarSize();
4038
4139 // Test kind.
42 for (const LLT TestTy : {Ty, HalfTy, DoubleTy}) {
43 ASSERT_TRUE(TestTy.isValid());
44 ASSERT_TRUE(TestTy.isScalar());
40 ASSERT_TRUE(Ty.isValid());
41 ASSERT_TRUE(Ty.isScalar());
4542
46 ASSERT_FALSE(TestTy.isPointer());
47 ASSERT_FALSE(TestTy.isVector());
48 }
43 ASSERT_FALSE(Ty.isPointer());
44 ASSERT_FALSE(Ty.isVector());
4945
5046 // Test sizes.
5147 EXPECT_EQ(S, Ty.getSizeInBits());
5248 EXPECT_EQ(S, Ty.getScalarSizeInBits());
5349
54 EXPECT_EQ(S*2, DoubleTy.getSizeInBits());
55 EXPECT_EQ(S*2, DoubleTy.getScalarSizeInBits());
56
57 if ((S % 2) == 0) {
58 EXPECT_EQ(S/2, HalfTy.getSizeInBits());
59 EXPECT_EQ(S/2, HalfTy.getScalarSizeInBits());
60 }
61
6250 // Test equality operators.
6351 EXPECT_TRUE(Ty == Ty);
6452 EXPECT_FALSE(Ty != Ty);
65
66 EXPECT_NE(Ty, DoubleTy);
6753
6854 // Test Type->LLT conversion.
6955 Type *IRTy = IntegerType::get(C, S);
8975 // Test getElementType().
9076 EXPECT_EQ(STy, VTy.getElementType());
9177
92 const LLT HalfSzTy = ((S % 2) == 0) ? VTy.halfScalarSize() : VTy;
93 const LLT DoubleSzTy = VTy.doubleScalarSize();
78 // Test kind.
79 ASSERT_TRUE(VTy.isValid());
80 ASSERT_TRUE(VTy.isVector());
9481
95 // halfElements requires an even number of elements.
96 const LLT HalfEltIfEvenTy = ((Elts % 2) == 0) ? VTy.halfElements() : VTy;
97 const LLT DoubleEltTy = VTy.doubleElements();
98
99 // Test kind.
100 for (const LLT TestTy : {VTy, HalfSzTy, DoubleSzTy, DoubleEltTy}) {
101 ASSERT_TRUE(TestTy.isValid());
102 ASSERT_TRUE(TestTy.isVector());
103
104 ASSERT_FALSE(TestTy.isScalar());
105 ASSERT_FALSE(TestTy.isPointer());
106 }
107
108 // Test halving elements to a scalar.
109 {
110 ASSERT_TRUE(HalfEltIfEvenTy.isValid());
111 ASSERT_FALSE(HalfEltIfEvenTy.isPointer());
112 if (Elts > 2) {
113 ASSERT_TRUE(HalfEltIfEvenTy.isVector());
114 } else {
115 ASSERT_FALSE(HalfEltIfEvenTy.isVector());
116 EXPECT_EQ(STy, HalfEltIfEvenTy);
117 }
118 }
119
82 ASSERT_FALSE(VTy.isScalar());
83 ASSERT_FALSE(VTy.isPointer());
12084
12185 // Test sizes.
12286 EXPECT_EQ(S * Elts, VTy.getSizeInBits());
12387 EXPECT_EQ(S, VTy.getScalarSizeInBits());
12488 EXPECT_EQ(Elts, VTy.getNumElements());
125
126 if ((S % 2) == 0) {
127 EXPECT_EQ((S / 2) * Elts, HalfSzTy.getSizeInBits());
128 EXPECT_EQ(S / 2, HalfSzTy.getScalarSizeInBits());
129 EXPECT_EQ(Elts, HalfSzTy.getNumElements());
130 }
131
132 EXPECT_EQ((S * 2) * Elts, DoubleSzTy.getSizeInBits());
133 EXPECT_EQ(S * 2, DoubleSzTy.getScalarSizeInBits());
134 EXPECT_EQ(Elts, DoubleSzTy.getNumElements());
135
136 if ((Elts % 2) == 0) {
137 EXPECT_EQ(S * (Elts / 2), HalfEltIfEvenTy.getSizeInBits());
138 EXPECT_EQ(S, HalfEltIfEvenTy.getScalarSizeInBits());
139 if (Elts > 2) {
140 EXPECT_EQ(Elts / 2, HalfEltIfEvenTy.getNumElements());
141 }
142 }
143
144 EXPECT_EQ(S * (Elts * 2), DoubleEltTy.getSizeInBits());
145 EXPECT_EQ(S, DoubleEltTy.getScalarSizeInBits());
146 EXPECT_EQ(Elts * 2, DoubleEltTy.getNumElements());
14789
14890 // Test equality operators.
14991 EXPECT_TRUE(VTy == VTy);
15294 // Test inequality operators on..
15395 // ..different kind.
15496 EXPECT_NE(VTy, STy);
155 // ..different #elts.
156 EXPECT_NE(VTy, DoubleEltTy);
157 // ..different scalar size.
158 EXPECT_NE(VTy, DoubleSzTy);
15997
16098 // Test Type->LLT conversion.
16199 Type *IRSTy = IntegerType::get(C, S);