llvm.org GIT mirror llvm / 27c2851
[globalisel][legalizer] Adapt LegalizerInfo to support inter-type dependencies and other things. Summary: As discussed in D42244, we have difficulty describing the legality of some operations. We're not able to specify relationships between types. For example, declaring the following setAction({..., 0, s32}, Legal) setAction({..., 0, s64}, Legal) setAction({..., 1, s32}, Legal) setAction({..., 1, s64}, Legal) currently declares these type combinations as legal: {s32, s32} {s64, s32} {s32, s64} {s64, s64} but we currently have no means to say that, for example, {s64, s32} is not legal. Some operations such as G_INSERT/G_EXTRACT/G_MERGE_VALUES/ G_UNMERGE_VALUES have relationships between the types that are currently described incorrectly. Additionally, G_LOAD/G_STORE currently have no means to legalize non-atomics differently to atomics. The necessary information is in the MMO but we have no way to use this in the legalizer. Similarly, there is currently no way for the register type and the memory type to differ so there is no way to cleanly represent extending-load/truncating-store in a way that can't be broken by optimizers (resulting in illegal MIR). It's also difficult to control the legalization strategy. We've added support for legalizing non-power of 2 types but there's still some hardcoded assumptions about the strategy. The main one I've noticed is that type0 is always legalized before type1 which is not a good strategy for `type0 = G_EXTRACT type1, ...` if you need to widen the container. It will converge on the same result eventually but it will take a much longer route when legalizing type0 than if you legalize type1 first. Lastly, the definition of legality and the legalization strategy is kept separate which is not ideal. It's helpful to be able to look at a one piece of code and see both what is legal and the method the legalizer will use to make illegal MIR more legal. This patch adds a layer onto the LegalizerInfo (to be removed when all targets have been migrated) which resolves all these issues. Here are the rules for shift and division: for (unsigned BinOp : {G_LSHR, G_ASHR, G_SDIV, G_UDIV}) getActionDefinitions(BinOp) .legalFor({s32, s64}) // If type0 is s32/s64 then it's Legal .clampScalar(0, s32, s64) // If type0 is <s32 then WidenScalar to s32 // If type0 is >s64 then NarrowScalar to s64 .widenScalarToPow2(0) // Round type0 scalars up to powers of 2 .unsupported(); // Otherwise, it's unsupported This describes everything needed to both define legality and describe how to make illegal things legal. Here's an example of a complex rule: getActionDefinitions(G_INSERT) .unsupportedIf([=](const LegalityQuery &Query) { // If type0 is smaller than type1 then it's unsupported return Query.Types[0].getSizeInBits() <= Query.Types[1].getSizeInBits(); }) .legalIf([=](const LegalityQuery &Query) { // If type0 is s32/s64/p0 and type1 is a power of 2 other than 2 or 4 then it's legal // We don't need to worry about large type1's because unsupportedIf caught that. const LLT &Ty0 = Query.Types[0]; const LLT &Ty1 = Query.Types[1]; if (Ty0 != s32 && Ty0 != s64 && Ty0 != p0) return false; return isPowerOf2_32(Ty1.getSizeInBits()) && (Ty1.getSizeInBits() == 1 || Ty1.getSizeInBits() >= 8); }) .clampScalar(0, s32, s64) .widenScalarToPow2(0) .maxScalarIf(typeInSet(0, {s32}), 1, s16) // If type0 is s32 and type1 is bigger than s16 then NarrowScalar type1 to s16 .maxScalarIf(typeInSet(0, {s64}), 1, s32) // If type0 is s64 and type1 is bigger than s32 then NarrowScalar type1 to s32 .widenScalarToPow2(1) // Round type1 scalars up to powers of 2 .unsupported(); This uses a lambda to say that G_INSERT is unsupported when type0 is bigger than type1 (in practice, this would be a default rule for G_INSERT). It also uses one to describe the legal cases. This particular predicate is equivalent to: .legalFor({{s32, s1}, {s32, s8}, {s32, s16}, {s64, s1}, {s64, s8}, {s64, s16}, {s64, s32}}) In terms of performance, I saw a slight (~6%) performance improvement when AArch64 was around 30% ported but it's pretty much break even right now. I'm going to take a look at constexpr as a means to reduce the initialization cost. Future work: * Make it possible for opcodes to share rulesets. There's no need for G_LSHR/G_ASHR/G_SDIV/G_UDIV to have separate rule and ruleset objects. There's no technical barrier to this, it just hasn't been done yet. * Replace the type-index numbers with an enum to get .clampScalar(Type0, s32, s64) * Better names for things like .maxScalarIf() (clampMaxScalar?) and the vector rules. * Improve initialization cost using constexpr Possible future work: * It's possible to make these rulesets change the MIR directly instead of returning a description of how to change the MIR. This should remove a little overhead caused by parsing the description and routing to the right code, but the real motivation is that it removes the need for LegalizeAction::Custom. With Custom removed, there's no longer a requirement that Custom legalization change the opcode to something that's considered legal. Reviewers: ab, t.p.northover, qcolombet, rovka, aditya_nandakumar, volkan, reames, bogner Reviewed By: bogner Subscribers: hintonda, bogner, aemerson, mgorny, javed.absar, kristof.beyls, llvm-commits Differential Revision: https://reviews.llvm.org/D42251 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@323681 91177308-0d34-0410-b5e6-96231b3b80d8 Daniel Sanders 2 years ago
7 changed file(s) with 858 addition(s) and 390 deletion(s). Raw diff Collapse all Expand all
2020 #include "llvm/ADT/STLExtras.h"
2121 #include "llvm/ADT/SmallVector.h"
2222 #include "llvm/CodeGen/TargetOpcodes.h"
23 #include "llvm/Support/raw_ostream.h"
2324 #include "llvm/Support/LowLevelTypeImpl.h"
2425 #include
2526 #include
7980
8081 /// Sentinel value for when no action was found in the specified table.
8182 NotFound,
83
84 /// Fall back onto the old rules.
85 /// TODO: Remove this once we've migrated
86 UseLegacyRules,
8287 };
8388 } // end namespace LegalizeActions
8489
108113 struct LegalityQuery {
109114 unsigned Opcode;
110115 ArrayRef Types;
116
117 raw_ostream &print(raw_ostream &OS) const;
118 };
119
120 /// The result of a query. It either indicates a final answer of Legal or
121 /// Unsupported or describes an action that must be taken to make an operation
122 /// more legal.
123 struct LegalizeActionStep {
124 /// The action to take or the final answer.
125 LegalizeAction Action;
126 /// If describing an action, the type index to change. Otherwise zero.
127 unsigned TypeIdx;
128 /// If describing an action, the new type for TypeIdx. Otherwise LLT{}.
129 LLT NewType;
130
131 LegalizeActionStep(LegalizeAction Action, unsigned TypeIdx,
132 const LLT &NewType)
133 : Action(Action), TypeIdx(TypeIdx), NewType(NewType) {}
134
135 bool operator==(const LegalizeActionStep &RHS) const {
136 return std::tie(Action, TypeIdx, NewType) ==
137 std::tie(RHS.Action, RHS.TypeIdx, RHS.NewType);
138 }
139 };
140
141 using LegalityPredicate = std::function;
142 using LegalizeMutation =
143 std::function(const LegalityQuery &)>;
144
145 namespace LegalityPredicates {
146 LegalityPredicate all(LegalityPredicate P0, LegalityPredicate P1);
147 LegalityPredicate typeInSet(unsigned TypeIdx,
148 std::initializer_list TypesInit);
149 LegalityPredicate
150 typePairInSet(unsigned TypeIdx0, unsigned TypeIdx1,
151 std::initializer_list> TypesInit);
152 LegalityPredicate isScalar(unsigned TypeIdx);
153 LegalityPredicate narrowerThan(unsigned TypeIdx, unsigned Size);
154 LegalityPredicate widerThan(unsigned TypeIdx, unsigned Size);
155 LegalityPredicate sizeNotPow2(unsigned TypeIdx);
156 LegalityPredicate numElementsNotPow2(unsigned TypeIdx);
157 } // end namespace LegalityPredicates
158
159 namespace LegalizeMutations {
160 LegalizeMutation identity(unsigned TypeIdx, LLT Ty);
161 LegalizeMutation widenScalarToNextPow2(unsigned TypeIdx, unsigned Min = 0);
162 LegalizeMutation moreElementsToNextPow2(unsigned TypeIdx, unsigned Min = 0);
163 } // end namespace LegalizeMutations
164
165 class LegalizeRule {
166 LegalityPredicate Predicate;
167 LegalizeAction Action;
168 LegalizeMutation Mutation;
169
170 public:
171 LegalizeRule(LegalityPredicate Predicate, LegalizeAction Action,
172 LegalizeMutation Mutation = nullptr)
173 : Predicate(Predicate), Action(Action), Mutation(Mutation) {}
174
175 bool match(const LegalityQuery &Query) const {
176 return Predicate(Query);
177 }
178
179 LegalizeAction getAction() const { return Action; }
180 std::pair determineMutation(const LegalityQuery &Query) const {
181 if (Mutation)
182 return Mutation(Query);
183 return std::make_pair(0, LLT{});
184 }
185 };
186
187 class LegalizeRuleSet {
188 /// When non-zero, the opcode we are an alias of
189 unsigned AliasOf;
190 /// If true, there is another opcode that aliases this one
191 bool IsAliasedByAnother;
192 SmallVector Rules;
193
194 void add(const LegalizeRule &Rule) {
195 assert(AliasOf == 0 &&
196 "RuleSet is aliased, change the representative opcode instead");
197 Rules.push_back(Rule);
198 }
199
200 static bool always(const LegalityQuery &) { return true; }
201
202 LegalizeRuleSet &actionIf(LegalizeAction Action,
203 LegalityPredicate Predicate) {
204 add({Predicate, Action});
205 return *this;
206 }
207 LegalizeRuleSet &actionIf(LegalizeAction Action, LegalityPredicate Predicate,
208 LegalizeMutation Mutation) {
209 add({Predicate, Action, Mutation});
210 return *this;
211 }
212 LegalizeRuleSet &actionFor(LegalizeAction Action,
213 std::initializer_list Types) {
214 using namespace LegalityPredicates;
215 return actionIf(Action, typeInSet(0, Types));
216 }
217 LegalizeRuleSet &
218 actionFor(LegalizeAction Action,
219 std::initializer_list> Types) {
220 using namespace LegalityPredicates;
221 return actionIf(Action, typePairInSet(0, 1, Types));
222 }
223 LegalizeRuleSet &actionForCartesianProduct(LegalizeAction Action,
224 std::initializer_list Types) {
225 using namespace LegalityPredicates;
226 return actionIf(Action, all(typeInSet(0, Types), typeInSet(1, Types)));
227 }
228 LegalizeRuleSet &
229 actionForCartesianProduct(LegalizeAction Action,
230 std::initializer_list Types0,
231 std::initializer_list Types1) {
232 using namespace LegalityPredicates;
233 return actionIf(Action, all(typeInSet(0, Types0), typeInSet(1, Types1)));
234 }
235
236 public:
237 LegalizeRuleSet() : AliasOf(0), IsAliasedByAnother(false), Rules() {}
238
239 bool isAliasedByAnother() { return IsAliasedByAnother; }
240 void setIsAliasedByAnother() { IsAliasedByAnother = true; }
241 void aliasTo(unsigned Opcode) {
242 assert((AliasOf == 0 || AliasOf == Opcode) &&
243 "Opcode is already aliased to another opcode");
244 assert(Rules.empty() && "Aliasing will discard rules");
245 AliasOf = Opcode;
246 }
247 unsigned getAlias() const { return AliasOf; }
248
249 // Primitive rule definition functions
250 LegalizeRuleSet &legalIf(LegalityPredicate Predicate) {
251 return actionIf(LegalizeAction::Legal, Predicate);
252 }
253 LegalizeRuleSet &legalFor(std::initializer_list Types) {
254 return actionFor(LegalizeAction::Legal, Types);
255 }
256 LegalizeRuleSet &legalFor(std::initializer_list> Types) {
257 return actionFor(LegalizeAction::Legal, Types);
258 }
259 LegalizeRuleSet &legalForCartesianProduct(std::initializer_list Types) {
260 return actionForCartesianProduct(LegalizeAction::Legal, Types);
261 }
262 LegalizeRuleSet &legalForCartesianProduct(std::initializer_list Types0,
263 std::initializer_list Types1) {
264 return actionForCartesianProduct(LegalizeAction::Legal, Types0, Types1);
265 }
266
267 LegalizeRuleSet &libcallIf(LegalityPredicate Predicate) {
268 return actionIf(LegalizeAction::Libcall, Predicate);
269 }
270 LegalizeRuleSet &libcallFor(std::initializer_list Types) {
271 return actionFor(LegalizeAction::Libcall, Types);
272 }
273 LegalizeRuleSet &
274 libcallFor(std::initializer_list> Types) {
275 return actionFor(LegalizeAction::Libcall, Types);
276 }
277 LegalizeRuleSet &
278 libcallForCartesianProduct(std::initializer_list Types) {
279 return actionForCartesianProduct(LegalizeAction::Libcall, Types);
280 }
281 LegalizeRuleSet &
282 libcallForCartesianProduct(std::initializer_list Types0,
283 std::initializer_list Types1) {
284 return actionForCartesianProduct(LegalizeAction::Libcall, Types0, Types1);
285 }
286
287 LegalizeRuleSet &widenScalarIf(LegalityPredicate Predicate,
288 LegalizeMutation Mutation) {
289 return actionIf(LegalizeAction::WidenScalar, Predicate, Mutation);
290 }
291 LegalizeRuleSet &narrowScalarIf(LegalityPredicate Predicate,
292 LegalizeMutation Mutation) {
293 return actionIf(LegalizeAction::NarrowScalar, Predicate, Mutation);
294 }
295
296 LegalizeRuleSet &moreElementsIf(LegalityPredicate Predicate,
297 LegalizeMutation Mutation) {
298 return actionIf(LegalizeAction::MoreElements, Predicate, Mutation);
299 }
300 LegalizeRuleSet &fewerElementsIf(LegalityPredicate Predicate,
301 LegalizeMutation Mutation) {
302 return actionIf(LegalizeAction::FewerElements, Predicate, Mutation);
303 }
304
305 LegalizeRuleSet &unsupported() {
306 return actionIf(LegalizeAction::Unsupported, always);
307 }
308 LegalizeRuleSet &unsupportedIf(LegalityPredicate Predicate) {
309 return actionIf(LegalizeAction::Unsupported, Predicate);
310 }
311
312 LegalizeRuleSet &customIf(LegalityPredicate Predicate) {
313 return actionIf(LegalizeAction::Custom, Predicate);
314 }
315 LegalizeRuleSet &customForCartesianProduct(std::initializer_list Types) {
316 return actionForCartesianProduct(LegalizeAction::Custom, Types);
317 }
318 LegalizeRuleSet &
319 customForCartesianProduct(std::initializer_list Types0,
320 std::initializer_list Types1) {
321 return actionForCartesianProduct(LegalizeAction::Custom, Types0, Types1);
322 }
323
324 // Convenience rule definition functions
325 LegalizeRuleSet &widenScalarToNextPow2(unsigned TypeIdx, unsigned MinSize = 0) {
326 using namespace LegalityPredicates;
327 return widenScalarIf(
328 sizeNotPow2(TypeIdx),
329 LegalizeMutations::widenScalarToNextPow2(TypeIdx, MinSize));
330 }
331
332 LegalizeRuleSet &narrowScalar(unsigned TypeIdx, LegalizeMutation Mutation) {
333 using namespace LegalityPredicates;
334 return narrowScalarIf(isScalar(TypeIdx), Mutation);
335 }
336
337 LegalizeRuleSet &minScalar(unsigned TypeIdx, const LLT &Ty) {
338 using namespace LegalityPredicates;
339 using namespace LegalizeMutations;
340 return widenScalarIf(narrowerThan(TypeIdx, Ty.getSizeInBits()),
341 LegalizeMutations::identity(TypeIdx, Ty));
342 }
343
344 LegalizeRuleSet &maxScalar(unsigned TypeIdx, const LLT &Ty) {
345 using namespace LegalityPredicates;
346 using namespace LegalizeMutations;
347 return narrowScalarIf(widerThan(TypeIdx, Ty.getSizeInBits()),
348 LegalizeMutations::identity(TypeIdx, Ty));
349 }
350
351 LegalizeRuleSet &maxScalarIf(LegalityPredicate Predicate, unsigned TypeIdx, const LLT &Ty) {
352 using namespace LegalityPredicates;
353 using namespace LegalizeMutations;
354 return narrowScalarIf(
355 [=](const LegalityQuery &Query) {
356 return widerThan(TypeIdx, Ty.getSizeInBits()) &&
357 Predicate(Query);
358 },
359 LegalizeMutations::identity(TypeIdx, Ty));
360 }
361
362 LegalizeRuleSet &clampScalar(unsigned TypeIdx, const LLT &MinTy, const LLT &MaxTy) {
363 assert(MinTy.isScalar() && MaxTy.isScalar() && "Expected scalar types");
364
365 return minScalar(TypeIdx, MinTy)
366 .maxScalar(TypeIdx, MaxTy);
367 }
368
369 LegalizeRuleSet &moreElementsToNextPow2(unsigned TypeIdx) {
370 using namespace LegalityPredicates;
371 return moreElementsIf(numElementsNotPow2(TypeIdx),
372 LegalizeMutations::moreElementsToNextPow2(TypeIdx));
373 }
374
375 LegalizeRuleSet &clampMinNumElements(unsigned TypeIdx, const LLT &EltTy,
376 unsigned MinElements) {
377 return moreElementsIf(
378 [=](const LegalityQuery &Query) {
379 LLT VecTy = Query.Types[TypeIdx];
380 return VecTy.getElementType() == EltTy &&
381 VecTy.getNumElements() < MinElements;
382 },
383 [=](const LegalityQuery &Query) {
384 LLT VecTy = Query.Types[TypeIdx];
385 return std::make_pair(
386 TypeIdx, LLT::vector(MinElements, VecTy.getScalarSizeInBits()));
387 });
388 }
389 LegalizeRuleSet &clampMaxNumElements(unsigned TypeIdx, const LLT &EltTy,
390 unsigned MaxElements) {
391 return fewerElementsIf(
392 [=](const LegalityQuery &Query) {
393 LLT VecTy = Query.Types[TypeIdx];
394 return VecTy.getElementType() == EltTy &&
395 VecTy.getNumElements() > MaxElements;
396 },
397 [=](const LegalityQuery &Query) {
398 LLT VecTy = Query.Types[TypeIdx];
399 return std::make_pair(
400 TypeIdx, LLT::vector(MaxElements, VecTy.getScalarSizeInBits()));
401 });
402 }
403 LegalizeRuleSet &clampNumElements(unsigned TypeIdx, const LLT &MinTy,
404 const LLT &MaxTy) {
405 assert(MinTy.getElementType() == MaxTy.getElementType() &&
406 "Expected element types to agree");
407
408 const LLT &EltTy = MinTy.getElementType();
409 return clampMinNumElements(TypeIdx, EltTy, MinTy.getNumElements())
410 .clampMaxNumElements(TypeIdx, EltTy, MaxTy.getNumElements());
411 }
412
413 LegalizeRuleSet &fallback() {
414 add({always, LegalizeAction::UseLegacyRules});
415 return *this;
416 }
417
418 LegalizeActionStep apply(const LegalityQuery &Query) const;
111419 };
112420
113421 class LegalizerInfo {
114422 public:
115 /// The result of a query. It either indicates a final answer of Legal or
116 /// Unsupported or describes an action that must be taken to make an operation
117 /// more legal.
118 struct LegalizeActionStep {
119 /// The action to take or the final answer.
120 LegalizeAction Action;
121 /// If describing an action, the type index to change. Otherwise zero.
122 unsigned TypeIdx;
123 /// If describing an action, the new type for TypeIdx. Otherwise LLT{}.
124 LLT NewType;
125
126 LegalizeActionStep(LegalizeAction Action, unsigned TypeIdx,
127 const LLT &NewType)
128 : Action(Action), TypeIdx(TypeIdx), NewType(NewType) {}
129
130 bool operator==(const LegalizeActionStep &RHS) const {
131 return std::tie(Action, TypeIdx, NewType) ==
132 std::tie(RHS.Action, RHS.TypeIdx, RHS.NewType);
133 }
134 };
135
136423 LegalizerInfo();
137424 virtual ~LegalizerInfo() = default;
425
426 unsigned getOpcodeIdxForOpcode(unsigned Opcode) const;
427 unsigned getActionDefinitionsIdx(unsigned Opcode) const;
138428
139429 /// Compute any ancillary tables needed to quickly decide how an operation
140430 /// should be handled. This must be called after all "set*Action"methods but
298588 decreaseToSmallerTypesAndIncreaseToSmallest(const SizeAndActionsVec &v,
299589 LegalizeAction DecreaseAction,
300590 LegalizeAction IncreaseAction);
591
592 const LegalizeRuleSet &getActionDefinitions(unsigned Opcode) const;
593 LegalizeRuleSet &getActionDefinitionsBuilder(unsigned Opcode);
594 LegalizeRuleSet &
595 getActionDefinitionsBuilder(std::initializer_list Opcodes);
596 void aliasActionDefinitions(unsigned OpcodeTo, unsigned OpcodeFrom);
301597
302598 /// Determine what action should be taken to legalize the described
303599 /// instruction. Requires computeTables to have been called.
502798 AddrSpace2PointerActions[LastOp - FirstOp + 1];
503799 std::unordered_map>
504800 NumElements2Actions[LastOp - FirstOp + 1];
801
802 LegalizeRuleSet RulesForOpcode[LastOp - FirstOp + 1];
505803 };
506804
507805 } // end namespace llvm.
55 IRTranslator.cpp
66 InstructionSelect.cpp
77 InstructionSelector.cpp
8 LegalityPredicates.cpp
9 LegalizeMutations.cpp
10 Legalizer.cpp
811 LegalizerHelper.cpp
9 Legalizer.cpp
1012 LegalizerInfo.cpp
1113 Localizer.cpp
1214 MachineIRBuilder.cpp
0 //===- lib/CodeGen/GlobalISel/LegalizerPredicates.cpp - Predicates --------===//
1 //
2 // The LLVM Compiler Infrastructure
3 //
4 // This file is distributed under the University of Illinois Open Source
5 // License. See LICENSE.TXT for details.
6 //
7 //===----------------------------------------------------------------------===//
8 //
9 // A library of predicate factories to use for LegalityPredicate.
10 //
11 //===----------------------------------------------------------------------===//
12
13 #include "llvm/CodeGen/GlobalISel/LegalizerInfo.h"
14
15 using namespace llvm;
16
17 LegalityPredicate
18 LegalityPredicates::all(LegalityPredicate P0, LegalityPredicate P1) {
19 return [=](const LegalityQuery &Query) {
20 return P0(Query) && P1(Query);
21 };
22 }
23
24 LegalityPredicate
25 LegalityPredicates::typeInSet(unsigned TypeIdx,
26 std::initializer_list TypesInit) {
27 SmallVector Types = TypesInit;
28 return [=](const LegalityQuery &Query) {
29 return std::find(Types.begin(), Types.end(), Query.Types[TypeIdx]) != Types.end();
30 };
31 }
32
33 LegalityPredicate LegalityPredicates::typePairInSet(
34 unsigned TypeIdx0, unsigned TypeIdx1,
35 std::initializer_list> TypesInit) {
36 SmallVector, 4> Types = TypesInit;
37 return [=](const LegalityQuery &Query) {
38 std::pair Match = {Query.Types[TypeIdx0], Query.Types[TypeIdx1]};
39 return std::find(Types.begin(), Types.end(), Match) != Types.end();
40 };
41 }
42
43 LegalityPredicate LegalityPredicates::isScalar(unsigned TypeIdx) {
44 return [=](const LegalityQuery &Query) {
45 return Query.Types[TypeIdx].isScalar();
46 };
47 }
48
49 LegalityPredicate LegalityPredicates::narrowerThan(unsigned TypeIdx,
50 unsigned Size) {
51 return [=](const LegalityQuery &Query) {
52 const LLT &QueryTy = Query.Types[TypeIdx];
53 return QueryTy.isScalar() && QueryTy.getSizeInBits() < Size;
54 };
55 }
56
57 LegalityPredicate LegalityPredicates::widerThan(unsigned TypeIdx,
58 unsigned Size) {
59 return [=](const LegalityQuery &Query) {
60 const LLT &QueryTy = Query.Types[TypeIdx];
61 return QueryTy.isScalar() && QueryTy.getSizeInBits() > Size;
62 };
63 }
64
65 LegalityPredicate LegalityPredicates::sizeNotPow2(unsigned TypeIdx) {
66 return [=](const LegalityQuery &Query) {
67 const LLT &QueryTy = Query.Types[TypeIdx];
68 return QueryTy.isScalar() && !isPowerOf2_32(QueryTy.getSizeInBits());
69 };
70 }
71
72 LegalityPredicate LegalityPredicates::numElementsNotPow2(unsigned TypeIdx) {
73 return [=](const LegalityQuery &Query) {
74 const LLT &QueryTy = Query.Types[TypeIdx];
75 return QueryTy.isVector() && isPowerOf2_32(QueryTy.getNumElements());
76 };
77 }
0 //===- lib/CodeGen/GlobalISel/LegalizerMutations.cpp - Mutations ----------===//
1 //
2 // The LLVM Compiler Infrastructure
3 //
4 // This file is distributed under the University of Illinois Open Source
5 // License. See LICENSE.TXT for details.
6 //
7 //===----------------------------------------------------------------------===//
8 //
9 // A library of mutation factories to use for LegalityMutation.
10 //
11 //===----------------------------------------------------------------------===//
12
13 #include "llvm/CodeGen/GlobalISel/LegalizerInfo.h"
14
15 using namespace llvm;
16
17 LegalizeMutation LegalizeMutations::identity(unsigned TypeIdx, LLT Ty) {
18 return
19 [=](const LegalityQuery &Query) { return std::make_pair(TypeIdx, Ty); };
20 }
21
22 LegalizeMutation LegalizeMutations::widenScalarToNextPow2(unsigned TypeIdx,
23 unsigned Min) {
24 return [=](const LegalityQuery &Query) {
25 unsigned NewSizeInBits =
26 1 << Log2_32_Ceil(Query.Types[TypeIdx].getSizeInBits());
27 if (NewSizeInBits < Min)
28 NewSizeInBits = Min;
29 return std::make_pair(TypeIdx, LLT::scalar(NewSizeInBits));
30 };
31 }
32
33 LegalizeMutation LegalizeMutations::moreElementsToNextPow2(unsigned TypeIdx,
34 unsigned Min) {
35 return [=](const LegalityQuery &Query) {
36 const LLT &VecTy = Query.Types[TypeIdx];
37 unsigned NewNumElements = 1 << Log2_32_Ceil(VecTy.getNumElements());
38 if (NewNumElements < Min)
39 NewNumElements = Min;
40 return std::make_pair(
41 TypeIdx, LLT::vector(NewNumElements, VecTy.getScalarSizeInBits()));
42 };
43 }
2323 #include "llvm/CodeGen/MachineRegisterInfo.h"
2424 #include "llvm/CodeGen/TargetOpcodes.h"
2525 #include "llvm/MC/MCInstrDesc.h"
26 #include "llvm/Support/Debug.h"
2627 #include "llvm/Support/ErrorHandling.h"
2728 #include "llvm/Support/LowLevelTypeImpl.h"
2829 #include "llvm/Support/MathExtras.h"
3132
3233 using namespace llvm;
3334 using namespace LegalizeActions;
35
36 #define DEBUG_TYPE "legalizer-info"
37
38 raw_ostream &LegalityQuery::print(raw_ostream &OS) const {
39 OS << Opcode << ", {";
40 for (const auto &Type : Types) {
41 OS << Type << ", ";
42 }
43 OS << "}";
44 return OS;
45 }
46
47 LegalizeActionStep LegalizeRuleSet::apply(const LegalityQuery &Query) const {
48 DEBUG(dbgs() << "Applying legalizer ruleset to: "; Query.print(dbgs());
49 dbgs() << "\n");
50 if (Rules.empty()) {
51 DEBUG(dbgs() << ".. fallback to legacy rules (no rules defined)\n");
52 return {LegalizeAction::UseLegacyRules, 0, LLT{}};
53 }
54 for (const auto &Rule : Rules) {
55 if (Rule.match(Query)) {
56 DEBUG(dbgs() << ".. match\n");
57 std::pair Mutation = Rule.determineMutation(Query);
58 DEBUG(dbgs() << ".. .. " << (unsigned)Rule.getAction() << ", "
59 << Mutation.first << ", " << Mutation.second << "\n");
60 assert(Query.Types[Mutation.first] != Mutation.second &&
61 "Simple loop detected");
62 return {Rule.getAction(), Mutation.first, Mutation.second};
63 } else
64 DEBUG(dbgs() << ".. no match\n");
65 }
66 DEBUG(dbgs() << ".. unsupported\n");
67 return {LegalizeAction::Unsupported, 0, LLT{}};
68 }
3469
3570 LegalizerInfo::LegalizerInfo() : TablesInitialized(false) {
3671 // Set defaults.
187222 return MRI.getType(MI.getOperand(OpIdx).getReg());
188223 }
189224
190 LegalizerInfo::LegalizeActionStep
225 unsigned LegalizerInfo::getOpcodeIdxForOpcode(unsigned Opcode) const {
226 assert(Opcode >= FirstOp && Opcode <= LastOp && "Unsupported opcode");
227 return Opcode - FirstOp;
228 }
229
230 unsigned LegalizerInfo::getActionDefinitionsIdx(unsigned Opcode) const {
231 unsigned OpcodeIdx = getOpcodeIdxForOpcode(Opcode);
232 if (unsigned Alias = RulesForOpcode[OpcodeIdx].getAlias()) {
233 DEBUG(dbgs() << ".. opcode " << Opcode << " is aliased to " << Alias
234 << "\n");
235 OpcodeIdx = getOpcodeIdxForOpcode(Alias);
236 DEBUG(dbgs() << ".. opcode " << Alias << " is aliased to "
237 << RulesForOpcode[OpcodeIdx].getAlias() << "\n");
238 assert(RulesForOpcode[OpcodeIdx].getAlias() == 0 && "Cannot chain aliases");
239 }
240
241 return OpcodeIdx;
242 }
243
244 const LegalizeRuleSet &
245 LegalizerInfo::getActionDefinitions(unsigned Opcode) const {
246 unsigned OpcodeIdx = getActionDefinitionsIdx(Opcode);
247 return RulesForOpcode[OpcodeIdx];
248 }
249
250 LegalizeRuleSet &LegalizerInfo::getActionDefinitionsBuilder(unsigned Opcode) {
251 unsigned OpcodeIdx = getActionDefinitionsIdx(Opcode);
252 auto &Result = RulesForOpcode[OpcodeIdx];
253 assert(!Result.isAliasedByAnother() && "Modifying this opcode will modify aliases");
254 return Result;
255 }
256
257 LegalizeRuleSet &LegalizerInfo::getActionDefinitionsBuilder(
258 std::initializer_list Opcodes) {
259 unsigned Representative = *Opcodes.begin();
260
261 for (auto I = Opcodes.begin() + 1, E = Opcodes.end(); I != E; ++I)
262 aliasActionDefinitions(Representative, *I);
263
264 auto &Return = getActionDefinitionsBuilder(Representative);
265 Return.setIsAliasedByAnother();
266 return Return;
267 }
268
269 void LegalizerInfo::aliasActionDefinitions(unsigned OpcodeTo,
270 unsigned OpcodeFrom) {
271 assert(OpcodeTo != OpcodeFrom && "Cannot alias to self");
272 assert(OpcodeTo >= FirstOp && OpcodeTo <= LastOp && "Unsupported opcode");
273 const unsigned OpcodeFromIdx = getOpcodeIdxForOpcode(OpcodeFrom);
274 RulesForOpcode[OpcodeFromIdx].aliasTo(OpcodeTo);
275 }
276
277 LegalizeActionStep
191278 LegalizerInfo::getAction(const LegalityQuery &Query) const {
279 LegalizeActionStep Step = getActionDefinitions(Query.Opcode).apply(Query);
280 if (Step.Action != LegalizeAction::UseLegacyRules) {
281 return Step;
282 }
283
192284 for (unsigned i = 0; i < Query.Types.size(); ++i) {
193285 auto Action = getAspectAction({Query.Opcode, i, Query.Types[i]});
194 if (Action.first != Legal)
286 if (Action.first != Legal) {
287 DEBUG(dbgs() << ".. (legacy) Type " << i << " Action="
288 << (unsigned)Action.first << ", " << Action.second << "\n");
195289 return {Action.first, i, Action.second};
196 }
290 } else
291 DEBUG(dbgs() << ".. (legacy) Type " << i << " Legal\n");
292 }
293 DEBUG(dbgs() << ".. (legacy) Legal\n");
197294 return {Legal, 0, LLT{}};
198295 }
199296
200 LegalizerInfo::LegalizeActionStep
297 LegalizeActionStep
201298 LegalizerInfo::getAction(const MachineInstr &MI,
202299 const MachineRegisterInfo &MRI) const {
203300 SmallVector Types;
322419 case Unsupported:
323420 return {Size, Unsupported};
324421 case NotFound:
422 case UseLegacyRules:
325423 llvm_unreachable("NotFound");
326424 }
327425 llvm_unreachable("Action has an unknown enum value");
332430 assert(Aspect.Type.isScalar() || Aspect.Type.isPointer());
333431 if (Aspect.Opcode < FirstOp || Aspect.Opcode > LastOp)
334432 return {NotFound, LLT()};
335 const unsigned OpcodeIdx = Aspect.Opcode - FirstOp;
433 const unsigned OpcodeIdx = getOpcodeIdxForOpcode(Aspect.Opcode);
336434 if (Aspect.Type.isPointer() &&
337435 AddrSpace2PointerActions[OpcodeIdx].find(Aspect.Type.getAddressSpace()) ==
338436 AddrSpace2PointerActions[OpcodeIdx].end()) {
363461 // lanes in the vector.
364462 if (Aspect.Opcode < FirstOp || Aspect.Opcode > LastOp)
365463 return {NotFound, Aspect.Type};
366 const unsigned OpcodeIdx = Aspect.Opcode - FirstOp;
464 const unsigned OpcodeIdx = getOpcodeIdxForOpcode(Aspect.Opcode);
367465 const unsigned TypeIdx = Aspect.Idx;
368466 if (TypeIdx >= ScalarInVectorActions[OpcodeIdx].size())
369467 return {NotFound, Aspect.Type};
2323
2424 using namespace llvm;
2525 using namespace LegalizeActions;
26
27 /// FIXME: The following static functions are SizeChangeStrategy functions
28 /// that are meant to temporarily mimic the behaviour of the old legalization
29 /// based on doubling/halving non-legal types as closely as possible. This is
30 /// not entirly possible as only legalizing the types that are exactly a power
31 /// of 2 times the size of the legal types would require specifying all those
32 /// sizes explicitly.
33 /// In practice, not specifying those isn't a problem, and the below functions
34 /// should disappear quickly as we add support for legalizing non-power-of-2
35 /// sized types further.
36 static void
37 addAndInterleaveWithUnsupported(LegalizerInfo::SizeAndActionsVec &result,
38 const LegalizerInfo::SizeAndActionsVec &v) {
39 for (unsigned i = 0; i < v.size(); ++i) {
40 result.push_back(v[i]);
41 if (i + 1 < v[i].first && i + 1 < v.size() &&
42 v[i + 1].first != v[i].first + 1)
43 result.push_back({v[i].first + 1, Unsupported});
44 }
45 }
46
47 static LegalizerInfo::SizeAndActionsVec
48 widen_1_narrow_128_ToLargest(const LegalizerInfo::SizeAndActionsVec &v) {
49 assert(v.size() >= 1);
50 assert(v[0].first > 2);
51 LegalizerInfo::SizeAndActionsVec result = {{1, WidenScalar},
52 {2, Unsupported}};
53 addAndInterleaveWithUnsupported(result, v);
54 auto Largest = result.back().first;
55 assert(Largest + 1 < 128);
56 result.push_back({Largest + 1, Unsupported});
57 result.push_back({128, NarrowScalar});
58 result.push_back({129, Unsupported});
59 return result;
60 }
61
62 static LegalizerInfo::SizeAndActionsVec
63 widen_16(const LegalizerInfo::SizeAndActionsVec &v) {
64 assert(v.size() >= 1);
65 assert(v[0].first > 17);
66 LegalizerInfo::SizeAndActionsVec result = {{1, Unsupported},
67 {16, WidenScalar},
68 {17, Unsupported}};
69 addAndInterleaveWithUnsupported(result, v);
70 auto Largest = result.back().first;
71 result.push_back({Largest + 1, Unsupported});
72 return result;
73 }
74
75 static LegalizerInfo::SizeAndActionsVec
76 widen_1_8(const LegalizerInfo::SizeAndActionsVec &v) {
77 assert(v.size() >= 1);
78 assert(v[0].first > 9);
79 LegalizerInfo::SizeAndActionsVec result = {
80 {1, WidenScalar}, {2, Unsupported},
81 {8, WidenScalar}, {9, Unsupported}};
82 addAndInterleaveWithUnsupported(result, v);
83 auto Largest = result.back().first;
84 result.push_back({Largest + 1, Unsupported});
85 return result;
86 }
87
88 static LegalizerInfo::SizeAndActionsVec
89 widen_1_8_16(const LegalizerInfo::SizeAndActionsVec &v) {
90 assert(v.size() >= 1);
91 assert(v[0].first > 17);
92 LegalizerInfo::SizeAndActionsVec result = {
93 {1, WidenScalar}, {2, Unsupported},
94 {8, WidenScalar}, {9, Unsupported},
95 {16, WidenScalar}, {17, Unsupported}};
96 addAndInterleaveWithUnsupported(result, v);
97 auto Largest = result.back().first;
98 result.push_back({Largest + 1, Unsupported});
99 return result;
100 }
101
102 static LegalizerInfo::SizeAndActionsVec
103 widen_1_8_16_narrowToLargest(const LegalizerInfo::SizeAndActionsVec &v) {
104 assert(v.size() >= 1);
105 assert(v[0].first > 17);
106 LegalizerInfo::SizeAndActionsVec result = {
107 {1, WidenScalar}, {2, Unsupported},
108 {8, WidenScalar}, {9, Unsupported},
109 {16, WidenScalar}, {17, Unsupported}};
110 addAndInterleaveWithUnsupported(result, v);
111 auto Largest = result.back().first;
112 result.push_back({Largest + 1, NarrowScalar});
113 return result;
114 }
115
116 static LegalizerInfo::SizeAndActionsVec
117 widen_1_8_16_32(const LegalizerInfo::SizeAndActionsVec &v) {
118 assert(v.size() >= 1);
119 assert(v[0].first > 33);
120 LegalizerInfo::SizeAndActionsVec result = {
121 {1, WidenScalar}, {2, Unsupported},
122 {8, WidenScalar}, {9, Unsupported},
123 {16, WidenScalar}, {17, Unsupported},
124 {32, WidenScalar}, {33, Unsupported}};
125 addAndInterleaveWithUnsupported(result, v);
126 auto Largest = result.back().first;
127 result.push_back({Largest + 1, Unsupported});
128 return result;
129 }
26 using namespace LegalityPredicates;
13027
13128 AArch64LegalizerInfo::AArch64LegalizerInfo(const AArch64Subtarget &ST) {
13229 using namespace TargetOpcode;
13734 const LLT s32 = LLT::scalar(32);
13835 const LLT s64 = LLT::scalar(64);
13936 const LLT s128 = LLT::scalar(128);
37 const LLT s256 = LLT::scalar(256);
38 const LLT s512 = LLT::scalar(512);
39 const LLT v16s8 = LLT::vector(16, 8);
40 const LLT v8s8 = LLT::vector(8, 8);
41 const LLT v4s8 = LLT::vector(4, 8);
42 const LLT v8s16 = LLT::vector(8, 16);
43 const LLT v4s16 = LLT::vector(4, 16);
44 const LLT v2s16 = LLT::vector(2, 16);
14045 const LLT v2s32 = LLT::vector(2, 32);
14146 const LLT v4s32 = LLT::vector(4, 32);
14247 const LLT v2s64 = LLT::vector(2, 64);
14348
144 for (auto Ty : {p0, s1, s8, s16, s32, s64})
145 setAction({G_IMPLICIT_DEF, Ty}, Legal);
146
147 for (auto Ty : {s16, s32, s64, p0})
148 setAction({G_PHI, Ty}, Legal);
149
150 setLegalizeScalarToDifferentSizeStrategy(G_PHI, 0, widen_1_8);
151
152 for (auto Ty : { s32, s64 })
153 setAction({G_BSWAP, Ty}, Legal);
154
155 for (unsigned BinOp : {G_ADD, G_SUB, G_MUL, G_AND, G_OR, G_XOR, G_SHL}) {
156 // These operations naturally get the right answer when used on
157 // GPR32, even if the actual type is narrower.
158 for (auto Ty : {s32, s64, v2s32, v4s32, v2s64})
159 setAction({BinOp, Ty}, Legal);
160
161 if (BinOp != G_ADD)
162 setLegalizeScalarToDifferentSizeStrategy(BinOp, 0,
163 widen_1_8_16_narrowToLargest);
164 }
165
166 setAction({G_GEP, p0}, Legal);
167 setAction({G_GEP, 1, s64}, Legal);
168
169 setLegalizeScalarToDifferentSizeStrategy(G_GEP, 1, widen_1_8_16_32);
170
171 setAction({G_PTR_MASK, p0}, Legal);
172
173 for (unsigned BinOp : {G_LSHR, G_ASHR, G_SDIV, G_UDIV}) {
174 for (auto Ty : {s32, s64})
175 setAction({BinOp, Ty}, Legal);
176
177 setLegalizeScalarToDifferentSizeStrategy(BinOp, 0, widen_1_8_16);
178 }
49 getActionDefinitionsBuilder(G_IMPLICIT_DEF)
50 .legalFor({p0, s1, s8, s16, s32, s64})
51 .clampScalar(0, s1, s64)
52 .widenScalarToNextPow2(0, 8);
53
54 getActionDefinitionsBuilder(G_PHI)
55 .legalFor({p0, s16, s32, s64})
56 .clampScalar(0, s16, s64)
57 .widenScalarToNextPow2(0);
58
59 getActionDefinitionsBuilder(G_BSWAP)
60 .legalFor({s32, s64})
61 .clampScalar(0, s16, s64)
62 .widenScalarToNextPow2(0);
63
64 getActionDefinitionsBuilder({G_ADD, G_SUB, G_MUL, G_AND, G_OR, G_XOR, G_SHL})
65 .legalFor({s32, s64, v2s32, v4s32, v2s64})
66 .clampScalar(0, s32, s64)
67 .widenScalarToNextPow2(0)
68 .clampNumElements(0, v2s32, v4s32)
69 .clampNumElements(0, v2s64, v2s64)
70 .moreElementsToNextPow2(0);
71
72 getActionDefinitionsBuilder(G_GEP)
73 .legalFor({{p0, s64}})
74 .clampScalar(1, s64, s64);
75
76 getActionDefinitionsBuilder(G_PTR_MASK).legalFor({p0});
77
78 getActionDefinitionsBuilder({G_LSHR, G_ASHR, G_SDIV, G_UDIV})
79 .legalFor({s32, s64})
80 .clampScalar(0, s32, s64)
81 .widenScalarToNextPow2(0);
17982
18083 for (unsigned BinOp : {G_SREM, G_UREM})
18184 for (auto Ty : { s1, s8, s16, s32, s64 })
18689 setAction({Op, 1, s1}, Legal);
18790 }
18891
189 for (unsigned Op : {G_UADDE, G_USUBE, G_SADDO, G_SSUBO, G_SMULH, G_UMULH}) {
190 for (auto Ty : { s32, s64 })
191 setAction({Op, Ty}, Legal);
192
193 setAction({Op, 1, s1}, Legal);
194 }
195
196 for (unsigned BinOp : {G_FADD, G_FSUB, G_FMA, G_FMUL, G_FDIV})
197 for (auto Ty : {s32, s64})
198 setAction({BinOp, Ty}, Legal);
199
200 for (unsigned BinOp : {G_FREM, G_FPOW}) {
201 setAction({BinOp, s32}, Libcall);
202 setAction({BinOp, s64}, Libcall);
203 }
204
205 for (auto Ty : {s32, s64, p0}) {
206 setAction({G_INSERT, Ty}, Legal);
207 setAction({G_INSERT, 1, Ty}, Legal);
208 }
209 setLegalizeScalarToDifferentSizeStrategy(G_INSERT, 0,
210 widen_1_8_16_narrowToLargest);
211 for (auto Ty : {s1, s8, s16}) {
212 setAction({G_INSERT, 1, Ty}, Legal);
213 // FIXME: Can't widen the sources because that violates the constraints on
214 // G_INSERT (It seems entirely reasonable that inputs shouldn't overlap).
215 }
216
217 for (auto Ty : {s1, s8, s16, s32, s64, p0})
218 setAction({G_EXTRACT, Ty}, Legal);
219
220 for (auto Ty : {s32, s64})
221 setAction({G_EXTRACT, 1, Ty}, Legal);
222
223 for (unsigned MemOp : {G_LOAD, G_STORE}) {
224 for (auto Ty : {s8, s16, s32, s64, p0, v2s32})
225 setAction({MemOp, Ty}, Legal);
226
227 setLegalizeScalarToDifferentSizeStrategy(MemOp, 0,
228 widen_1_narrow_128_ToLargest);
229
230 // And everything's fine in addrspace 0.
231 setAction({MemOp, 1, p0}, Legal);
232 }
92 getActionDefinitionsBuilder({G_SMULH, G_UMULH}).legalFor({s32, s64});
93
94 getActionDefinitionsBuilder({G_UADDE, G_USUBE, G_SADDO, G_SSUBO})
95 .legalFor({{s32, s1}, {s64, s1}});
96
97 getActionDefinitionsBuilder({G_FADD, G_FSUB, G_FMA, G_FMUL, G_FDIV})
98 .legalFor({s32, s64});
99
100 getActionDefinitionsBuilder({G_FREM, G_FPOW}).libcallFor({s32, s64});
101
102 getActionDefinitionsBuilder(G_INSERT)
103 .unsupportedIf([=](const LegalityQuery &Query) {
104 return Query.Types[0].getSizeInBits() <= Query.Types[1].getSizeInBits();
105 })
106 .legalIf([=](const LegalityQuery &Query) {
107 const LLT &Ty0 = Query.Types[0];
108 const LLT &Ty1 = Query.Types[1];
109 if (Ty0 != s32 && Ty0 != s64 && Ty0 != p0)
110 return false;
111 return isPowerOf2_32(Ty1.getSizeInBits()) &&
112 (Ty1.getSizeInBits() == 1 || Ty1.getSizeInBits() >= 8);
113 })
114 .clampScalar(0, s32, s64)
115 .widenScalarToNextPow2(0)
116 .maxScalarIf(typeInSet(0, {s32}), 1, s16)
117 .maxScalarIf(typeInSet(0, {s64}), 1, s32)
118 .widenScalarToNextPow2(1);
119
120 getActionDefinitionsBuilder(G_EXTRACT)
121 .unsupportedIf([=](const LegalityQuery &Query) {
122 return Query.Types[0].getSizeInBits() >= Query.Types[1].getSizeInBits();
123 })
124 .legalIf([=](const LegalityQuery &Query) {
125 const LLT &Ty0 = Query.Types[0];
126 const LLT &Ty1 = Query.Types[1];
127 if (Ty1 != s32 && Ty1 != s64)
128 return false;
129 if (Ty1 == p0)
130 return true;
131 return isPowerOf2_32(Ty0.getSizeInBits()) &&
132 (Ty0.getSizeInBits() == 1 || Ty0.getSizeInBits() >= 8);
133 })
134 .clampScalar(1, s32, s64)
135 .widenScalarToNextPow2(1)
136 .maxScalarIf(typeInSet(1, {s32}), 0, s16)
137 .maxScalarIf(typeInSet(1, {s64}), 0, s32)
138 .widenScalarToNextPow2(0);
139
140 getActionDefinitionsBuilder({G_LOAD, G_STORE})
141 .legalFor(
142 {{s8, p0}, {s16, p0}, {s32, p0}, {s64, p0}, {p0, p0}, {v2s32, p0}})
143 .clampScalar(0, s8, s64)
144 .widenScalarToNextPow2(0)
145 .clampNumElements(0, v2s32, v2s32);
233146
234147 // Constants
235 for (auto Ty : {s32, s64}) {
236 setAction({TargetOpcode::G_CONSTANT, Ty}, Legal);
237 setAction({TargetOpcode::G_FCONSTANT, Ty}, Legal);
238 }
239
240 setAction({G_CONSTANT, p0}, Legal);
241
242 setLegalizeScalarToDifferentSizeStrategy(G_CONSTANT, 0, widen_1_8_16);
243 setLegalizeScalarToDifferentSizeStrategy(G_FCONSTANT, 0, widen_16);
244
245 setAction({G_ICMP, 1, s32}, Legal);
246 setAction({G_ICMP, 1, s64}, Legal);
247 setAction({G_ICMP, 1, p0}, Legal);
248
249 setLegalizeScalarToDifferentSizeStrategy(G_ICMP, 0, widen_1_8_16);
250 setLegalizeScalarToDifferentSizeStrategy(G_FCMP, 0, widen_1_8_16);
251 setLegalizeScalarToDifferentSizeStrategy(G_ICMP, 1, widen_1_8_16);
252
253 setAction({G_ICMP, s32}, Legal);
254 setAction({G_FCMP, s32}, Legal);
255 setAction({G_FCMP, 1, s32}, Legal);
256 setAction({G_FCMP, 1, s64}, Legal);
148 getActionDefinitionsBuilder(G_CONSTANT)
149 .legalFor({p0, s32, s64})
150 .clampScalar(0, s32, s64)
151 .widenScalarToNextPow2(0);
152 getActionDefinitionsBuilder(G_FCONSTANT)
153 .legalFor({s32, s64})
154 .clampScalar(0, s32, s64);
155
156 getActionDefinitionsBuilder(G_ICMP)
157 .legalFor({{s32, s32}, {s32, s64}, {s32, p0}})
158 .clampScalar(0, s32, s32)
159 .clampScalar(1, s32, s64)
160 .widenScalarToNextPow2(1);
161
162 getActionDefinitionsBuilder(G_FCMP)
163 .legalFor({{s32, s32}, {s32, s64}})
164 .clampScalar(0, s32, s32)
165 .clampScalar(1, s32, s64)
166 .widenScalarToNextPow2(1);
257167
258168 // Extensions
259 for (auto Ty : { s1, s8, s16, s32, s64 }) {
260 setAction({G_ZEXT, Ty}, Legal);
261 setAction({G_SEXT, Ty}, Legal);
262 setAction({G_ANYEXT, Ty}, Legal);
263 }
169 getActionDefinitionsBuilder({G_ZEXT, G_SEXT, G_ANYEXT})
170 .legalFor({s1, s8, s16, s32, s64})
171 .maxScalar(0, s64)
172 .widenScalarToNextPow2(0);
264173
265174 // FP conversions
266 for (auto Ty : { s16, s32 }) {
267 setAction({G_FPTRUNC, Ty}, Legal);
268 setAction({G_FPEXT, 1, Ty}, Legal);
269 }
270
271 for (auto Ty : { s32, s64 }) {
272 setAction({G_FPTRUNC, 1, Ty}, Legal);
273 setAction({G_FPEXT, Ty}, Legal);
274 }
175 getActionDefinitionsBuilder(G_FPTRUNC).legalFor(
176 {{s16, s32}, {s16, s64}, {s32, s64}});
177 getActionDefinitionsBuilder(G_FPEXT).legalFor(
178 {{s32, s16}, {s64, s16}, {s64, s32}});
275179
276180 // Conversions
277 for (auto Ty : { s32, s64 }) {
278 setAction({G_FPTOSI, 0, Ty}, Legal);
279 setAction({G_FPTOUI, 0, Ty}, Legal);
280 setAction({G_SITOFP, 1, Ty}, Legal);
281 setAction({G_UITOFP, 1, Ty}, Legal);
282 }
283 setLegalizeScalarToDifferentSizeStrategy(G_FPTOSI, 0, widen_1_8_16);
284 setLegalizeScalarToDifferentSizeStrategy(G_FPTOUI, 0, widen_1_8_16);
285 setLegalizeScalarToDifferentSizeStrategy(G_SITOFP, 1, widen_1_8_16);
286 setLegalizeScalarToDifferentSizeStrategy(G_UITOFP, 1, widen_1_8_16);
287
288 for (auto Ty : { s32, s64 }) {
289 setAction({G_FPTOSI, 1, Ty}, Legal);
290 setAction({G_FPTOUI, 1, Ty}, Legal);
291 setAction({G_SITOFP, 0, Ty}, Legal);
292 setAction({G_UITOFP, 0, Ty}, Legal);
293 }
181 getActionDefinitionsBuilder({G_FPTOSI, G_FPTOUI})
182 .legalForCartesianProduct({s32, s64})
183 .clampScalar(0, s32, s64)
184 .widenScalarToNextPow2(0)
185 .clampScalar(1, s32, s64)
186 .widenScalarToNextPow2(1);
187
188 getActionDefinitionsBuilder({G_SITOFP, G_UITOFP})
189 .legalForCartesianProduct({s32, s64})
190 .clampScalar(1, s32, s64)
191 .widenScalarToNextPow2(1)
192 .clampScalar(0, s32, s64)
193 .widenScalarToNextPow2(0);
294194
295195 // Control-flow
296 for (auto Ty : {s1, s8, s16, s32})
297 setAction({G_BRCOND, Ty}, Legal);
298 setAction({G_BRINDIRECT, p0}, Legal);
196 getActionDefinitionsBuilder(G_BRCOND).legalFor({s1, s8, s16, s32});
197 getActionDefinitionsBuilder(G_BRINDIRECT).legalFor({p0});
299198
300199 // Select
301 setLegalizeScalarToDifferentSizeStrategy(G_SELECT, 0, widen_1_8_16);
302
303 for (auto Ty : {s32, s64, p0})
304 setAction({G_SELECT, Ty}, Legal);
305
306 setAction({G_SELECT, 1, s1}, Legal);
200 getActionDefinitionsBuilder(G_SELECT)
201 .legalFor({{s32, s1}, {s64, s1}, {p0, s1}})
202 .clampScalar(0, s32, s64)
203 .widenScalarToNextPow2(0);
307204
308205 // Pointer-handling
309 setAction({G_FRAME_INDEX, p0}, Legal);
310 setAction({G_GLOBAL_VALUE, p0}, Legal);
311
312 for (auto Ty : {s1, s8, s16, s32, s64})
313 setAction({G_PTRTOINT, 0, Ty}, Legal);
314
315 setAction({G_PTRTOINT, 1, p0}, Legal);
316
317 setAction({G_INTTOPTR, 0, p0}, Legal);
318 setAction({G_INTTOPTR, 1, s64}, Legal);
206 getActionDefinitionsBuilder(G_FRAME_INDEX).legalFor({p0});
207 getActionDefinitionsBuilder(G_GLOBAL_VALUE).legalFor({p0});
208
209 getActionDefinitionsBuilder(G_PTRTOINT)
210 .legalForCartesianProduct({s1, s8, s16, s32, s64}, {p0})
211 .maxScalar(0, s64)
212 .widenScalarToNextPow2(0, /*Min*/ 8);
213
214 getActionDefinitionsBuilder(G_INTTOPTR)
215 .unsupportedIf([&](const LegalityQuery &Query) {
216 return Query.Types[0].getSizeInBits() != Query.Types[1].getSizeInBits();
217 })
218 .legalFor({s64, p0});
319219
320220 // Casts for 32 and 64-bit width type are just copies.
321221 // Same for 128-bit width type, except they are on the FPR bank.
322 for (auto Ty : {s1, s8, s16, s32, s64, s128}) {
323 setAction({G_BITCAST, 0, Ty}, Legal);
324 setAction({G_BITCAST, 1, Ty}, Legal);
325 }
326
327 // For the sake of copying bits around, the type does not really
328 // matter as long as it fits a register.
329 for (int EltSize = 8; EltSize <= 64; EltSize *= 2) {
330 setAction({G_BITCAST, 0, LLT::vector(128/EltSize, EltSize)}, Legal);
331 setAction({G_BITCAST, 1, LLT::vector(128/EltSize, EltSize)}, Legal);
332 if (EltSize >= 64)
333 continue;
334
335 setAction({G_BITCAST, 0, LLT::vector(64/EltSize, EltSize)}, Legal);
336 setAction({G_BITCAST, 1, LLT::vector(64/EltSize, EltSize)}, Legal);
337 if (EltSize >= 32)
338 continue;
339
340 setAction({G_BITCAST, 0, LLT::vector(32/EltSize, EltSize)}, Legal);
341 setAction({G_BITCAST, 1, LLT::vector(32/EltSize, EltSize)}, Legal);
342 }
343
344 setAction({G_VASTART, p0}, Legal);
222 getActionDefinitionsBuilder(G_BITCAST)
223 // FIXME: This is wrong since G_BITCAST is not allowed to change the
224 // number of bits but it's what the previous code described and fixing
225 // it breaks tests.
226 .legalForCartesianProduct({s1, s8, s16, s32, s64, s128, v16s8, v8s8, v4s8,
227 v8s16, v4s16, v2s16, v4s32, v2s32, v2s64});
228
229 getActionDefinitionsBuilder(G_VASTART).legalFor({p0});
345230
346231 // va_list must be a pointer, but most sized types are pretty easy to handle
347232 // as the destination.
348 setAction({G_VAARG, 1, p0}, Legal);
349
350 for (auto Ty : {s8, s16, s32, s64, p0})
351 setAction({G_VAARG, Ty}, Custom);
233 getActionDefinitionsBuilder(G_VAARG)
234 .customForCartesianProduct({s8, s16, s32, s64, p0}, {p0})
235 .clampScalar(0, s8, s64)
236 .widenScalarToNextPow2(0, /*Min*/ 8);
237
238 if (ST.hasLSE()) {
239 getActionDefinitionsBuilder(G_ATOMIC_CMPXCHG)
240 .legalForCartesianProduct({s8, s16, s32, s64}, {p0});
241 }
242 getActionDefinitionsBuilder(G_ATOMIC_CMPXCHG);
352243
353244 if (ST.hasLSE()) {
354245 for (auto Ty : {s8, s16, s32, s64}) {
355246 setAction({G_ATOMIC_CMPXCHG_WITH_SUCCESS, Ty}, Lower);
356 setAction({G_ATOMIC_CMPXCHG, Ty}, Legal);
357247 }
358 setAction({G_ATOMIC_CMPXCHG, 1, p0}, Legal);
359
360 for (unsigned Op :
361 {G_ATOMICRMW_XCHG, G_ATOMICRMW_ADD, G_ATOMICRMW_SUB, G_ATOMICRMW_AND,
362 G_ATOMICRMW_OR, G_ATOMICRMW_XOR, G_ATOMICRMW_MIN, G_ATOMICRMW_MAX,
363 G_ATOMICRMW_UMIN, G_ATOMICRMW_UMAX}) {
364 for (auto Ty : {s8, s16, s32, s64}) {
365 setAction({Op, Ty}, Legal);
248
249 getActionDefinitionsBuilder(
250 {G_ATOMICRMW_XCHG, G_ATOMICRMW_ADD, G_ATOMICRMW_SUB, G_ATOMICRMW_AND,
251 G_ATOMICRMW_OR, G_ATOMICRMW_XOR, G_ATOMICRMW_MIN, G_ATOMICRMW_MAX,
252 G_ATOMICRMW_UMIN, G_ATOMICRMW_UMAX})
253 .legalForCartesianProduct({s8, s16, s32, s64}, {p0});
254 }
255
256 // Merge/Unmerge
257 for (unsigned Op : {G_MERGE_VALUES, G_UNMERGE_VALUES}) {
258 unsigned BigTyIdx = Op == G_MERGE_VALUES ? 0 : 1;
259 unsigned LitTyIdx = Op == G_MERGE_VALUES ? 1 : 0;
260
261 auto notValidElt = [](const LegalityQuery &Query, unsigned TypeIdx) {
262 const LLT &Ty = Query.Types[TypeIdx];
263 if (Ty.isVector()) {
264 const LLT &EltTy = Ty.getElementType();
265 if (EltTy.getSizeInBits() < 8 || EltTy.getSizeInBits() > 64)
266 return true;
267 if (!isPowerOf2_32(EltTy.getSizeInBits()))
268 return true;
366269 }
367 setAction({Op, 1, p0}, Legal);
368 }
270 return false;
271 };
272 auto scalarize =
273 [](const LegalityQuery &Query, unsigned TypeIdx) {
274 const LLT &Ty = Query.Types[TypeIdx];
275 return std::make_pair(TypeIdx, Ty.getElementType());
276 };
277
278 // FIXME: This rule is horrible, but specifies the same as what we had
279 // before with the particularly strange definitions removed (e.g.
280 // s8 = G_MERGE_VALUES s32, s32).
281 // Part of the complexity comes from these ops being extremely flexible. For
282 // example, you can build/decompose vectors with it, concatenate vectors,
283 // etc. and in addition to this you can also bitcast with it at the same
284 // time. We've been considering breaking it up into multiple ops to make it
285 // more manageable throughout the backend.
286 getActionDefinitionsBuilder(Op)
287 // Break up vectors with weird elements into scalars
288 .fewerElementsIf(
289 [=](const LegalityQuery &Query) { return notValidElt(Query, 0); },
290 [=](const LegalityQuery &Query) { return scalarize(Query, 0); })
291 .fewerElementsIf(
292 [=](const LegalityQuery &Query) { return notValidElt(Query, 1); },
293 [=](const LegalityQuery &Query) { return scalarize(Query, 1); })
294 // Clamp the big scalar to s8-s512 and make it either a power of 2, 192,
295 // or 384.
296 .clampScalar(BigTyIdx, s8, s512)
297 .widenScalarIf(
298 [=](const LegalityQuery &Query) {
299 const LLT &Ty = Query.Types[BigTyIdx];
300 return !isPowerOf2_32(Ty.getSizeInBits()) &&
301 Ty.getSizeInBits() % 64 != 0;
302 },
303 [=](const LegalityQuery &Query) {
304 // Pick the next power of 2, or a multiple of 64 over 128.
305 // Whichever is smaller.
306 const LLT &Ty = Query.Types[BigTyIdx];
307 unsigned NewSizeInBits = 1
308 << Log2_32_Ceil(Ty.getSizeInBits() + 1);
309 if (NewSizeInBits >= 256) {
310 unsigned RoundedTo = alignTo<64>(Ty.getSizeInBits() + 1);
311 if (RoundedTo < NewSizeInBits)
312 NewSizeInBits = RoundedTo;
313 }
314 return std::make_pair(BigTyIdx, LLT::scalar(NewSizeInBits));
315 })
316 // Clamp the little scalar to s8-s256 and make it a power of 2. It's not
317 // worth considering the multiples of 64 since 2*192 and 2*384 are not
318 // valid.
319 .clampScalar(LitTyIdx, s8, s256)
320 .widenScalarToNextPow2(LitTyIdx, /*Min*/ 8)
321 // So at this point, we have s8, s16, s32, s64, s128, s192, s256, s384,
322 // s512, , , , or .
323 // At this point it's simple enough to accept the legal types.
324 .legalIf([=](const LegalityQuery &Query) {
325 const LLT &BigTy = Query.Types[BigTyIdx];
326 const LLT &LitTy = Query.Types[LitTyIdx];
327 if (BigTy.isVector() && BigTy.getSizeInBits() < 32)
328 return false;
329 if (LitTy.isVector() && LitTy.getSizeInBits() < 32)
330 return false;
331 return BigTy.getSizeInBits() % LitTy.getSizeInBits() == 0;
332 })
333 // Any vectors left are the wrong size. Scalarize them.
334 .fewerElementsIf([](const LegalityQuery &Query) { return true; },
335 [](const LegalityQuery &Query) {
336 return std::make_pair(
337 0, Query.Types[0].getElementType());
338 })
339 .fewerElementsIf([](const LegalityQuery &Query) { return true; },
340 [](const LegalityQuery &Query) {
341 return std::make_pair(
342 1, Query.Types[1].getElementType());
343 });
369344 }
370
371 // Merge/Unmerge
372 for (unsigned Op : {G_MERGE_VALUES, G_UNMERGE_VALUES})
373 for (int Sz : {8, 16, 32, 64, 128, 192, 256, 384, 512}) {
374 LLT ScalarTy = LLT::scalar(Sz);
375 setAction({Op, ScalarTy}, Legal);
376 setAction({Op, 1, ScalarTy}, Legal);
377 if (Sz < 32)
378 continue;
379 for (int EltSize = 8; EltSize <= 64; EltSize *= 2) {
380 if (EltSize >= Sz)
381 continue;
382 LLT VecTy = LLT::vector(Sz / EltSize, EltSize);
383 setAction({Op, VecTy}, Legal);
384 setAction({Op, 1, VecTy}, Legal);
385 }
386 }
387345
388346 computeTables();
389347 }
2828 case Custom: OS << "Custom"; break;
2929 case Unsupported: OS << "Unsupported"; break;
3030 case NotFound: OS << "NotFound";
31 case UseLegacyRules: OS << "UseLegacyRules"; break;
3132 }
3233 return OS;
3334 }
6162 for (unsigned opcode : {G_ADD, G_SUB}) {
6263 // Check we infer the correct types and actually do what we're told.
6364 ASSERT_EQ(L.getAction({opcode, {LLT::scalar(8)}}),
64 LegalizerInfo::LegalizeActionStep(WidenScalar, 0, LLT::scalar(32)));
65 LegalizeActionStep(WidenScalar, 0, LLT::scalar(32)));
6566 ASSERT_EQ(L.getAction({opcode, {LLT::scalar(16)}}),
66 LegalizerInfo::LegalizeActionStep(WidenScalar, 0, LLT::scalar(32)));
67 LegalizeActionStep(WidenScalar, 0, LLT::scalar(32)));
6768 ASSERT_EQ(L.getAction({opcode, {LLT::scalar(32)}}),
68 LegalizerInfo::LegalizeActionStep(Legal, 0, LLT{}));
69 LegalizeActionStep(Legal, 0, LLT{}));
6970 ASSERT_EQ(L.getAction({opcode, {LLT::scalar(64)}}),
70 LegalizerInfo::LegalizeActionStep(Legal, 0, LLT{}));
71 LegalizeActionStep(Legal, 0, LLT{}));
7172
7273 // Make sure the default for over-sized types applies.
73 ASSERT_EQ(
74 L.getAction({opcode, {LLT::scalar(128)}}),
75 LegalizerInfo::LegalizeActionStep(NarrowScalar, 0, LLT::scalar(64)));
74 ASSERT_EQ(L.getAction({opcode, {LLT::scalar(128)}}),
75 LegalizeActionStep(NarrowScalar, 0, LLT::scalar(64)));
7676 // Make sure we also handle unusual sizes
77 ASSERT_EQ(
78 L.getAction({opcode, {LLT::scalar(1)}}),
79 LegalizerInfo::LegalizeActionStep(WidenScalar, 0, LLT::scalar(32)));
80 ASSERT_EQ(
81 L.getAction({opcode, {LLT::scalar(31)}}),
82 LegalizerInfo::LegalizeActionStep(WidenScalar, 0, LLT::scalar(32)));
83 ASSERT_EQ(
84 L.getAction({opcode, {LLT::scalar(33)}}),
85 LegalizerInfo::LegalizeActionStep(WidenScalar, 0, LLT::scalar(64)));
86 ASSERT_EQ(
87 L.getAction({opcode, {LLT::scalar(63)}}),
88 LegalizerInfo::LegalizeActionStep(WidenScalar, 0, LLT::scalar(64)));
89 ASSERT_EQ(
90 L.getAction({opcode, {LLT::scalar(65)}}),
91 LegalizerInfo::LegalizeActionStep(NarrowScalar, 0, LLT::scalar(64)));
77 ASSERT_EQ(L.getAction({opcode, {LLT::scalar(1)}}),
78 LegalizeActionStep(WidenScalar, 0, LLT::scalar(32)));
79 ASSERT_EQ(L.getAction({opcode, {LLT::scalar(31)}}),
80 LegalizeActionStep(WidenScalar, 0, LLT::scalar(32)));
81 ASSERT_EQ(L.getAction({opcode, {LLT::scalar(33)}}),
82 LegalizeActionStep(WidenScalar, 0, LLT::scalar(64)));
83 ASSERT_EQ(L.getAction({opcode, {LLT::scalar(63)}}),
84 LegalizeActionStep(WidenScalar, 0, LLT::scalar(64)));
85 ASSERT_EQ(L.getAction({opcode, {LLT::scalar(65)}}),
86 LegalizeActionStep(NarrowScalar, 0, LLT::scalar(64)));
9287 }
9388 }
9489
113108 // Check we infer the correct types and actually do what we're told for some
114109 // simple cases.
115110 ASSERT_EQ(L.getAction({G_ADD, {LLT::vector(8, 8)}}),
116 LegalizerInfo::LegalizeActionStep(Legal, 0, LLT{}));
117 ASSERT_EQ(
118 L.getAction({G_ADD, {LLT::vector(8, 7)}}),
119 LegalizerInfo::LegalizeActionStep(WidenScalar, 0, LLT::vector(8, 8)));
120 ASSERT_EQ(
121 L.getAction({G_ADD, {LLT::vector(2, 8)}}),
122 LegalizerInfo::LegalizeActionStep(MoreElements, 0, LLT::vector(8, 8)));
123 ASSERT_EQ(
124 L.getAction({G_ADD, {LLT::vector(8, 32)}}),
125 LegalizerInfo::LegalizeActionStep(FewerElements, 0, LLT::vector(4, 32)));
111 LegalizeActionStep(Legal, 0, LLT{}));
112 ASSERT_EQ(L.getAction({G_ADD, {LLT::vector(8, 7)}}),
113 LegalizeActionStep(WidenScalar, 0, LLT::vector(8, 8)));
114 ASSERT_EQ(L.getAction({G_ADD, {LLT::vector(2, 8)}}),
115 LegalizeActionStep(MoreElements, 0, LLT::vector(8, 8)));
116 ASSERT_EQ(L.getAction({G_ADD, {LLT::vector(8, 32)}}),
117 LegalizeActionStep(FewerElements, 0, LLT::vector(4, 32)));
126118 // Check a few non-power-of-2 sizes:
127 ASSERT_EQ(
128 L.getAction({G_ADD, {LLT::vector(3, 3)}}),
129 LegalizerInfo::LegalizeActionStep(WidenScalar, 0, LLT::vector(3, 8)));
130 ASSERT_EQ(
131 L.getAction({G_ADD, {LLT::vector(3, 8)}}),
132 LegalizerInfo::LegalizeActionStep(MoreElements, 0, LLT::vector(8, 8)));
119 ASSERT_EQ(L.getAction({G_ADD, {LLT::vector(3, 3)}}),
120 LegalizeActionStep(WidenScalar, 0, LLT::vector(3, 8)));
121 ASSERT_EQ(L.getAction({G_ADD, {LLT::vector(3, 8)}}),
122 LegalizeActionStep(MoreElements, 0, LLT::vector(8, 8)));
133123 }
134124
135125 TEST(LegalizerInfoTest, MultipleTypes) {
149139
150140 // Check we infer the correct types and actually do what we're told.
151141 ASSERT_EQ(L.getAction({G_PTRTOINT, {s64, p0}}),
152 LegalizerInfo::LegalizeActionStep(Legal, 0, LLT{}));
142 LegalizeActionStep(Legal, 0, LLT{}));
153143
154144 // Make sure we also handle unusual sizes
155145 ASSERT_EQ(
156146 L.getAction({G_PTRTOINT, {LLT::scalar(65), s64}}),
157 LegalizerInfo::LegalizeActionStep(NarrowScalar, 0, s64));
147 LegalizeActionStep(NarrowScalar, 0, s64));
158148 ASSERT_EQ(
159149 L.getAction({G_PTRTOINT, {s64, LLT::pointer(0, 32)}}),
160 LegalizerInfo::LegalizeActionStep(Unsupported, 1, LLT::pointer(0, 32)));
150 LegalizeActionStep(Unsupported, 1, LLT::pointer(0, 32)));
161151 }
162152
163153 TEST(LegalizerInfoTest, MultipleSteps) {
174164 L.computeTables();
175165
176166 ASSERT_EQ(L.getAction({G_UREM, {LLT::scalar(16)}}),
177 LegalizerInfo::LegalizeActionStep(WidenScalar, 0, LLT::scalar(32)));
167 LegalizeActionStep(WidenScalar, 0, LLT::scalar(32)));
178168 ASSERT_EQ(L.getAction({G_UREM, {LLT::scalar(32)}}),
179 LegalizerInfo::LegalizeActionStep(Lower, 0, LLT::scalar(32)));
169 LegalizeActionStep(Lower, 0, LLT::scalar(32)));
180170 }
181171
182172 TEST(LegalizerInfoTest, SizeChangeStrategy) {
192182 // Check we infer the correct types and actually do what we're told.
193183 for (unsigned Size : {1, 8, 16, 32}) {
194184 ASSERT_EQ(L.getAction({G_UREM, {LLT::scalar(Size)}}),
195 LegalizerInfo::LegalizeActionStep(Legal, 0, LLT{}));
185 LegalizeActionStep(Legal, 0, LLT{}));
196186 }
197187 ASSERT_EQ(L.getAction({G_UREM, {LLT::scalar(2)}}),
198 LegalizerInfo::LegalizeActionStep(WidenScalar, 0, LLT::scalar(8)));
188 LegalizeActionStep(WidenScalar, 0, LLT::scalar(8)));
199189 ASSERT_EQ(L.getAction({G_UREM, {LLT::scalar(7)}}),
200 LegalizerInfo::LegalizeActionStep(WidenScalar, 0, LLT::scalar(8)));
190 LegalizeActionStep(WidenScalar, 0, LLT::scalar(8)));
201191 ASSERT_EQ(L.getAction({G_UREM, {LLT::scalar(9)}}),
202 LegalizerInfo::LegalizeActionStep(WidenScalar, 0, LLT::scalar(16)));
192 LegalizeActionStep(WidenScalar, 0, LLT::scalar(16)));
203193 ASSERT_EQ(L.getAction({G_UREM, {LLT::scalar(17)}}),
204 LegalizerInfo::LegalizeActionStep(WidenScalar, 0, LLT::scalar(32)));
194 LegalizeActionStep(WidenScalar, 0, LLT::scalar(32)));
205195 ASSERT_EQ(L.getAction({G_UREM, {LLT::scalar(31)}}),
206 LegalizerInfo::LegalizeActionStep(WidenScalar, 0, LLT::scalar(32)));
196 LegalizeActionStep(WidenScalar, 0, LLT::scalar(32)));
207197 ASSERT_EQ(L.getAction({G_UREM, {LLT::scalar(33)}}),
208 LegalizerInfo::LegalizeActionStep(Unsupported, 0, LLT::scalar(33)));
209 }
210 }
198 LegalizeActionStep(Unsupported, 0, LLT::scalar(33)));
199 }
200 }