llvm.org GIT mirror llvm / 9fc5a53
[PM] Fix a pretty nasty bug where the new pass manager would invalidate passes too many time. I think this is actually the issue that someone raised with me at the developer's meeting and in an email, but that we never really got to the bottom of. Having all the testing utilities made it much easier to dig down and uncover the core issue. When a pass manager is running many passes over a single function, we need it to invalidate the analyses between each run so that they can be re-computed as needed. We also need to track the intersection of preserved higher-level analyses across all the passes that we run (for example, if there is one module analysis which all the function analyses preserve, we want to track that and propagate it). Unfortunately, this interacted poorly with any enclosing pass adaptor between two IR units. It would see the intersection of preserved analyses, and need to invalidate any other analyses, but some of the un-preserved analyses might have already been invalidated *and recomputed*! We would fail to propagate the fact that the analysis had already been invalidated. The solution to this struck me as really strange at first, but the more I thought about it, the more natural it seemed. After a nice discussion with Duncan about it on IRC, it seemed even nicer. The idea is that invalidating an analysis *causes* it to be preserved! Preserving the lack of result is trivial. If it is recomputed, great. Until something *else* invalidates it again, we're good. The consequence of this is that the invalidate methods on the analysis manager which operate over many passes now consume their PreservedAnalyses object, update it to "preserve" every analysis pass to which it delivers an invalidation (regardless of whether the pass chooses to be removed, or handles the invalidation itself by updating itself). Then we return this augmented set from the invalidate routine, letting the pass manager take the result and use the intersection of *that* across each pass run to compute the final preserved set. This accounts for all the places where the early invalidation of an analysis has already "preserved" it for a future run. I've beefed up the testing and adjusted the assertions to show that we no longer repeatedly invalidate or compute the analyses across nested pass managers. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225333 91177308-0d34-0410-b5e6-96231b3b80d8 Chandler Carruth 4 years ago
5 changed file(s) with 159 addition(s) and 42 deletion(s). Raw diff Collapse all Expand all
119119 void invalidateImpl(void *PassID, LazyCallGraph::SCC &C);
120120
121121 /// \brief Invalidate the results for a function..
122 void invalidateImpl(LazyCallGraph::SCC &C, const PreservedAnalyses &PA);
122 PreservedAnalyses invalidateImpl(LazyCallGraph::SCC &C, PreservedAnalyses PA);
123123
124124 /// \brief List of function analysis pass IDs and associated concept pointers.
125125 ///
342342
343343 // We know that the CGSCC pass couldn't have invalidated any other
344344 // SCC's analyses (that's the contract of a CGSCC pass), so
345 // directly handle the CGSCC analysis manager's invalidation here.
345 // directly handle the CGSCC analysis manager's invalidation here. We
346 // also update the preserved set of analyses to reflect that invalidated
347 // analyses are now safe to preserve.
346348 // FIXME: This isn't quite correct. We need to handle the case where the
347349 // pass updated the CG, particularly some child of the current SCC, and
348350 // invalidate its analyses.
349 CGAM.invalidate(C, PassPA);
351 PassPA = CGAM.invalidate(C, std::move(PassPA));
350352
351353 // Then intersect the preserved set so that invalidation of module
352354 // analyses will eventually occur when the module pass completes.
561563 // We know that the function pass couldn't have invalidated any other
562564 // function's analyses (that's the contract of a function pass), so
563565 // directly handle the function analysis manager's invalidation here.
566 // Also, update the preserved analyses to reflect that once invalidated
567 // these can again be preserved.
564568 if (FAM)
565 FAM->invalidate(N->getFunction(), PassPA);
569 PassPA = FAM->invalidate(N->getFunction(), std::move(PassPA));
566570
567571 // Then intersect the preserved set so that invalidation of module
568572 // analyses will eventually occur when the module pass completes.
9191 }
9292
9393 /// \brief Mark a particular pass as preserved, adding it to the set.
94 template void preserve() {
94 template void preserve() { preserve(PassT::ID()); }
95
96 /// \brief Mark an abstract PassID as preserved, adding it to the set.
97 void preserve(void *PassID) {
9598 if (!areAllPreserved())
96 PreservedPassIDs.insert(PassT::ID());
99 PreservedPassIDs.insert(PassID);
97100 }
98101
99102 /// \brief Intersect this set with another in place.
382385 ///
383386 /// Walk through all of the analyses pertaining to this unit of IR and
384387 /// invalidate them unless they are preserved by the PreservedAnalyses set.
385 void invalidate(IRUnitT IR, const PreservedAnalyses &PA) {
386 derived_this()->invalidateImpl(IR, PA);
388 /// We accept the PreservedAnalyses set by value and update it with each
389 /// analyis pass which has been successfully invalidated and thus can be
390 /// preserved going forward. The updated set is returned.
391 PreservedAnalyses invalidate(IRUnitT IR, PreservedAnalyses PA) {
392 return derived_this()->invalidateImpl(IR, std::move(PA));
387393 }
388394
389395 protected:
450456 void invalidateImpl(void *PassID, Module &M);
451457
452458 /// \brief Invalidate results across a module.
453 void invalidateImpl(Module &M, const PreservedAnalyses &PA);
459 PreservedAnalyses invalidateImpl(Module &M, PreservedAnalyses PA);
454460
455461 /// \brief Map type from module analysis pass ID to pass result concept
456462 /// pointer.
514520 void invalidateImpl(void *PassID, Function &F);
515521
516522 /// \brief Invalidate the results for a function..
517 void invalidateImpl(Function &F, const PreservedAnalyses &PA);
523 PreservedAnalyses invalidateImpl(Function &F, PreservedAnalyses PA);
518524
519525 /// \brief List of function analysis pass IDs and associated concept pointers.
520526 ///
737743
738744 // We know that the function pass couldn't have invalidated any other
739745 // function's analyses (that's the contract of a function pass), so
740 // directly handle the function analysis manager's invalidation here.
746 // directly handle the function analysis manager's invalidation here and
747 // update our preserved set to reflect that these have already been
748 // handled.
741749 if (FAM)
742 FAM->invalidate(*I, PassPA);
750 PassPA = FAM->invalidate(*I, std::move(PassPA));
743751
744752 // Then intersect the preserved set so that invalidation of module
745753 // analyses will eventually occur when the module pass completes.
2828 dbgs() << "Running CGSCC pass: " << Passes[Idx]->name() << "\n";
2929
3030 PreservedAnalyses PassPA = Passes[Idx]->run(C, AM);
31
32 // If we have an active analysis manager at this level we want to ensure we
33 // update it as each pass runs and potentially invalidates analyses. We
34 // also update the preserved set of analyses based on what analyses we have
35 // already handled the invalidation for here and don't need to invalidate
36 // when finished.
3137 if (AM)
32 AM->invalidate(C, PassPA);
38 PassPA = AM->invalidate(C, std::move(PassPA));
39
40 // Finally, we intersect the final preserved analyses to compute the
41 // aggregate preserved set for this pass manager.
3342 PA.intersect(std::move(PassPA));
3443 }
3544
93102 CGSCCAnalysisResults.erase(RI);
94103 }
95104
96 void CGSCCAnalysisManager::invalidateImpl(LazyCallGraph::SCC &C,
97 const PreservedAnalyses &PA) {
105 PreservedAnalyses CGSCCAnalysisManager::invalidateImpl(LazyCallGraph::SCC &C,
106 PreservedAnalyses PA) {
98107 // Short circuit for a common case of all analyses being preserved.
99108 if (PA.areAllPreserved())
100 return;
109 return std::move(PA);
101110
102111 if (DebugPM)
103112 dbgs() << "Invalidating all non-preserved analyses for SCC: " << C.getName()
109118 CGSCCAnalysisResultListT &ResultsList = CGSCCAnalysisResultLists[&C];
110119 for (CGSCCAnalysisResultListT::iterator I = ResultsList.begin(),
111120 E = ResultsList.end();
112 I != E;)
121 I != E;) {
122 void *PassID = I->first;
123
124 // Pass the invalidation down to the pass itself to see if it thinks it is
125 // necessary. The analysis pass can return false if no action on the part
126 // of the analysis manager is required for this invalidation event.
113127 if (I->second->invalidate(C, PA)) {
114128 if (DebugPM)
115 dbgs() << "Invalidating CGSCC analysis: " << lookupPass(I->first).name()
129 dbgs() << "Invalidating CGSCC analysis: " << lookupPass(PassID).name()
116130 << "\n";
117131
118132 InvalidatedPassIDs.push_back(I->first);
120134 } else {
121135 ++I;
122136 }
137
138 // After handling each pass, we mark it as preserved. Once we've
139 // invalidated any stale results, the rest of the system is allowed to
140 // start preserving this analysis again.
141 PA.preserve(PassID);
142 }
123143 while (!InvalidatedPassIDs.empty())
124144 CGSCCAnalysisResults.erase(
125145 std::make_pair(InvalidatedPassIDs.pop_back_val(), &C));
126146 CGSCCAnalysisResultLists.erase(&C);
147
148 return std::move(PA);
127149 }
128150
129151 char CGSCCAnalysisManagerModuleProxy::PassID;
2929 dbgs() << "Running module pass: " << Passes[Idx]->name() << "\n";
3030
3131 PreservedAnalyses PassPA = Passes[Idx]->run(M, AM);
32
33 // If we have an active analysis manager at this level we want to ensure we
34 // update it as each pass runs and potentially invalidates analyses. We
35 // also update the preserved set of analyses based on what analyses we have
36 // already handled the invalidation for here and don't need to invalidate
37 // when finished.
3238 if (AM)
33 AM->invalidate(M, PassPA);
39 PassPA = AM->invalidate(M, std::move(PassPA));
40
41 // Finally, we intersect the final preserved analyses to compute the
42 // aggregate preserved set for this pass manager.
3443 PA.intersect(std::move(PassPA));
3544
3645 M.getContext().yield();
7584 ModuleAnalysisResults.erase(PassID);
7685 }
7786
78 void ModuleAnalysisManager::invalidateImpl(Module &M,
79 const PreservedAnalyses &PA) {
87 PreservedAnalyses ModuleAnalysisManager::invalidateImpl(Module &M,
88 PreservedAnalyses PA) {
8089 // Short circuit for a common case of all analyses being preserved.
8190 if (PA.areAllPreserved())
82 return;
91 return std::move(PA);
8392
8493 if (DebugPM)
8594 dbgs() << "Invalidating all non-preserved analyses for module: "
8998 // invalidate iteration for DenseMap.
9099 for (ModuleAnalysisResultMapT::iterator I = ModuleAnalysisResults.begin(),
91100 E = ModuleAnalysisResults.end();
92 I != E; ++I)
101 I != E; ++I) {
102 void *PassID = I->first;
103
104 // Pass the invalidation down to the pass itself to see if it thinks it is
105 // necessary. The analysis pass can return false if no action on the part
106 // of the analysis manager is required for this invalidation event.
93107 if (I->second->invalidate(M, PA)) {
94108 if (DebugPM)
95109 dbgs() << "Invalidating module analysis: "
96 << lookupPass(I->first).name() << "\n";
110 << lookupPass(PassID).name() << "\n";
97111
98112 ModuleAnalysisResults.erase(I);
99113 }
114
115 // After handling each pass, we mark it as preserved. Once we've
116 // invalidated any stale results, the rest of the system is allowed to
117 // start preserving this analysis again.
118 PA.preserve(PassID);
119 }
120
121 return std::move(PA);
100122 }
101123
102124 PreservedAnalyses FunctionPassManager::run(Function &F,
111133 dbgs() << "Running function pass: " << Passes[Idx]->name() << "\n";
112134
113135 PreservedAnalyses PassPA = Passes[Idx]->run(F, AM);
136
137 // If we have an active analysis manager at this level we want to ensure we
138 // update it as each pass runs and potentially invalidates analyses. We
139 // also update the preserved set of analyses based on what analyses we have
140 // already handled the invalidation for here and don't need to invalidate
141 // when finished.
114142 if (AM)
115 AM->invalidate(F, PassPA);
143 PassPA = AM->invalidate(F, std::move(PassPA));
144
145 // Finally, we intersect the final preserved analyses to compute the
146 // aggregate preserved set for this pass manager.
116147 PA.intersect(std::move(PassPA));
117148
118149 F.getContext().yield();
178209 FunctionAnalysisResults.erase(RI);
179210 }
180211
181 void FunctionAnalysisManager::invalidateImpl(Function &F,
182 const PreservedAnalyses &PA) {
212 PreservedAnalyses
213 FunctionAnalysisManager::invalidateImpl(Function &F, PreservedAnalyses PA) {
183214 // Short circuit for a common case of all analyses being preserved.
184215 if (PA.areAllPreserved())
185 return;
216 return std::move(PA);
186217
187218 if (DebugPM)
188219 dbgs() << "Invalidating all non-preserved analyses for function: "
194225 FunctionAnalysisResultListT &ResultsList = FunctionAnalysisResultLists[&F];
195226 for (FunctionAnalysisResultListT::iterator I = ResultsList.begin(),
196227 E = ResultsList.end();
197 I != E;)
228 I != E;) {
229 void *PassID = I->first;
230
231 // Pass the invalidation down to the pass itself to see if it thinks it is
232 // necessary. The analysis pass can return false if no action on the part
233 // of the analysis manager is required for this invalidation event.
198234 if (I->second->invalidate(F, PA)) {
199235 if (DebugPM)
200236 dbgs() << "Invalidating function analysis: "
201 << lookupPass(I->first).name() << "\n";
237 << lookupPass(PassID).name() << "\n";
202238
203239 InvalidatedPassIDs.push_back(I->first);
204240 I = ResultsList.erase(I);
205241 } else {
206242 ++I;
207243 }
244
245 // After handling each pass, we mark it as preserved. Once we've
246 // invalidated any stale results, the rest of the system is allowed to
247 // start preserving this analysis again.
248 PA.preserve(PassID);
249 }
208250 while (!InvalidatedPassIDs.empty())
209251 FunctionAnalysisResults.erase(
210252 std::make_pair(InvalidatedPassIDs.pop_back_val(), &F));
211253 if (ResultsList.empty())
212254 FunctionAnalysisResultLists.erase(&F);
255
256 return std::move(PA);
213257 }
214258
215259 char FunctionAnalysisManagerModuleProxy::PassID;
194194 ; CHECK-DO-INVALIDATE-FUNCTION-ANALYSIS-RESULTS: Running function analysis: NoOpFunctionAnalysis
195195
196196 ; RUN: opt -disable-output -disable-verify -debug-pass-manager -debug-cgscc-pass-manager \
197 ; RUN: -passes='require,cgscc(require,function(require,invalidate,require),require>),require' %s 2>&1 \
197 ; RUN: -passes='require,module(require,function(require,invalidate,require),require>),require' %s 2>&1 \
198198 ; RUN: | FileCheck %s --check-prefix=CHECK-INVALIDATE-ALL
199199 ; CHECK-INVALIDATE-ALL: Starting module pass manager run.
200200 ; CHECK-INVALIDATE-ALL: Running module pass: No-op Analysis Requirement Pass
201201 ; CHECK-INVALIDATE-ALL: Running module analysis: NoOpModuleAnalysis
202 ; CHECK-INVALIDATE-ALL: Starting CGSCC pass manager run.
203 ; CHECK-INVALIDATE-ALL: Running CGSCC pass: No-op Analysis Requirement Pass
204 ; CHECK-INVALIDATE-ALL: Running CGSCC analysis: NoOpCGSCCAnalysis
202 ; CHECK-INVALIDATE-ALL: Starting module pass manager run.
203 ; CHECK-INVALIDATE-ALL: Running module pass: No-op Analysis Requirement Pass
204 ; CHECK-INVALIDATE-ALL-NOT: Running module analysis: NoOpModuleAnalysis
205205 ; CHECK-INVALIDATE-ALL: Starting function pass manager run.
206206 ; CHECK-INVALIDATE-ALL: Running function pass: No-op Analysis Requirement Pass
207207 ; CHECK-INVALIDATE-ALL: Running function analysis: NoOpFunctionAnalysis
212212 ; CHECK-INVALIDATE-ALL: Running function analysis: NoOpFunctionAnalysis
213213 ; CHECK-INVALIDATE-ALL: Finished function pass manager run.
214214 ; CHECK-INVALIDATE-ALL: Invalidating all non-preserved analyses for function
215 ; CHECK-INVALIDATE-ALL: Invalidating function analysis: NoOpFunctionAnalysis
216 ; CHECK-INVALIDATE-ALL: Invalidating all non-preserved analyses for SCC
217 ; CHECK-INVALIDATE-ALL: Invalidating CGSCC analysis: NoOpCGSCCAnalysis
218 ; CHECK-INVALIDATE-ALL: Running CGSCC pass: No-op Analysis Requirement Pass
219 ; CHECK-INVALIDATE-ALL: Running CGSCC analysis: NoOpCGSCCAnalysis
220 ; CHECK-INVALIDATE-ALL: Finished CGSCC pass manager run.
221 ; CHECK-INVALIDATE-ALL: Invalidating all non-preserved analyses for SCC
222 ; CHECK-INVALIDATE-ALL: Invalidating CGSCC analysis: NoOpCGSCCAnalysis
215 ; CHECK-INVALIDATE-ALL-NOT: Running function analysis: NoOpFunctionAnalysis
223216 ; CHECK-INVALIDATE-ALL: Invalidating all non-preserved analyses for module
224217 ; CHECK-INVALIDATE-ALL: Invalidating module analysis: NoOpModuleAnalysis
225218 ; CHECK-INVALIDATE-ALL: Running module pass: No-op Analysis Requirement Pass
226219 ; CHECK-INVALIDATE-ALL: Running module analysis: NoOpModuleAnalysis
227220 ; CHECK-INVALIDATE-ALL: Finished module pass manager run.
221 ; CHECK-INVALIDATE-ALL: Invalidating all non-preserved analyses for module
222 ; CHECK-INVALIDATE-ALL-NOT: Invalidating module analysis: NoOpModuleAnalysis
223 ; CHECK-INVALIDATE-ALL: Running module pass: No-op Analysis Requirement Pass
224 ; CHECK-INVALIDATE-ALL-NOT: Running module analysis: NoOpModuleAnalysis
225 ; CHECK-INVALIDATE-ALL: Finished module pass manager run.
226
227 ; RUN: opt -disable-output -disable-verify -debug-pass-manager -debug-cgscc-pass-manager \
228 ; RUN: -passes='require,module(require,cgscc(require,function(require,invalidate,require),require),require),require' %s 2>&1 \
229 ; RUN: | FileCheck %s --check-prefix=CHECK-INVALIDATE-ALL-CG
230 ; CHECK-INVALIDATE-ALL-CG: Starting module pass manager run.
231 ; CHECK-INVALIDATE-ALL-CG: Running module pass: No-op Analysis Requirement Pass
232 ; CHECK-INVALIDATE-ALL-CG: Running module analysis: NoOpModuleAnalysis
233 ; CHECK-INVALIDATE-ALL-CG: Starting module pass manager run.
234 ; CHECK-INVALIDATE-ALL-CG: Running module pass: No-op Analysis Requirement Pass
235 ; CHECK-INVALIDATE-ALL-CG-NOT: Running module analysis: NoOpModuleAnalysis
236 ; CHECK-INVALIDATE-ALL-CG: Starting CGSCC pass manager run.
237 ; CHECK-INVALIDATE-ALL-CG: Running CGSCC pass: No-op Analysis Requirement Pass
238 ; CHECK-INVALIDATE-ALL-CG: Running CGSCC analysis: NoOpCGSCCAnalysis
239 ; CHECK-INVALIDATE-ALL-CG: Starting function pass manager run.
240 ; CHECK-INVALIDATE-ALL-CG: Running function pass: No-op Analysis Requirement Pass
241 ; CHECK-INVALIDATE-ALL-CG: Running function analysis: NoOpFunctionAnalysis
242 ; CHECK-INVALIDATE-ALL-CG: Running function pass: InvalidateAllAnalysesPass
243 ; CHECK-INVALIDATE-ALL-CG: Invalidating all non-preserved analyses for function
244 ; CHECK-INVALIDATE-ALL-CG: Invalidating function analysis: NoOpFunctionAnalysis
245 ; CHECK-INVALIDATE-ALL-CG: Running function pass: No-op Analysis Requirement Pass
246 ; CHECK-INVALIDATE-ALL-CG: Running function analysis: NoOpFunctionAnalysis
247 ; CHECK-INVALIDATE-ALL-CG: Finished function pass manager run.
248 ; CHECK-INVALIDATE-ALL-CG: Invalidating all non-preserved analyses for function
249 ; CHECK-INVALIDATE-ALL-CG-NOT: Running function analysis: NoOpFunctionAnalysis
250 ; CHECK-INVALIDATE-ALL-CG: Invalidating all non-preserved analyses for SCC
251 ; CHECK-INVALIDATE-ALL-CG: Invalidating CGSCC analysis: NoOpCGSCCAnalysis
252 ; CHECK-INVALIDATE-ALL-CG: Running CGSCC pass: No-op Analysis Requirement Pass
253 ; CHECK-INVALIDATE-ALL-CG: Running CGSCC analysis: NoOpCGSCCAnalysis
254 ; CHECK-INVALIDATE-ALL-CG: Finished CGSCC pass manager run.
255 ; CHECK-INVALIDATE-ALL-CG: Invalidating all non-preserved analyses for SCC
256 ; CHECK-INVALIDATE-ALL-CG-NOT: Invalidating CGSCC analysis: NoOpCGSCCAnalysis
257 ; CHECK-INVALIDATE-ALL-CG: Invalidating all non-preserved analyses for module
258 ; CHECK-INVALIDATE-ALL-CG: Invalidating module analysis: NoOpModuleAnalysis
259 ; CHECK-INVALIDATE-ALL-CG: Running module pass: No-op Analysis Requirement Pass
260 ; CHECK-INVALIDATE-ALL-CG: Running module analysis: NoOpModuleAnalysis
261 ; CHECK-INVALIDATE-ALL-CG: Finished module pass manager run.
262 ; CHECK-INVALIDATE-ALL-CG: Invalidating all non-preserved analyses for module
263 ; CHECK-INVALIDATE-ALL-CG-NOT: Invalidating module analysis: NoOpModuleAnalysis
264 ; CHECK-INVALIDATE-ALL-CG: Running module pass: No-op Analysis Requirement Pass
265 ; CHECK-INVALIDATE-ALL-CG-NOT: Running module analysis: NoOpModuleAnalysis
266 ; CHECK-INVALIDATE-ALL-CG: Finished module pass manager run.
228267
229268 define void @foo() {
230269 ret void