Add scoped-noalias metadata
This commit adds scoped noalias metadata. The primary motivations for this
feature are:
1. To preserve noalias function attribute information when inlining
2. To provide the ability to model block-scope C99 restrict pointers
Neither of these two abilities are added here, only the necessary
infrastructure. In fact, there should be no change to existing functionality,
only the addition of new features. The logic that converts noalias function
parameters into this metadata during inlining will come in a follow-up commit.
What is added here is the ability to generally specify noalias memory-access
sets. Regarding the metadata, alias-analysis scopes are defined similar to TBAA
nodes:
!scope0 = metadata !{ metadata !"scope of foo()" }
!scope1 = metadata !{ metadata !"scope 1", metadata !scope0 }
!scope2 = metadata !{ metadata !"scope 2", metadata !scope0 }
!scope3 = metadata !{ metadata !"scope 2.1", metadata !scope2 }
!scope4 = metadata !{ metadata !"scope 2.2", metadata !scope2 }
Loads and stores can be tagged with an alias-analysis scope, and also, with a
noalias tag for a specific scope:
... = load %ptr1, !alias.scope !{ !scope1 }
... = load %ptr2, !alias.scope !{ !scope1, !scope2 }, !noalias !{ !scope1 }
When evaluating an aliasing query, if one of the instructions is associated
with an alias.scope id that is identical to the noalias scope associated with
the other instruction, or is a descendant (in the scope hierarchy) of the
noalias scope associated with the other instruction, then the two memory
accesses are assumed not to alias.
Note that is the first element of the scope metadata is a string, then it can
be combined accross functions and translation units. The string can be replaced
by a self-reference to create globally unqiue scope identifiers.
[Note: This overview is slightly stylized, since the metadata nodes really need
to just be numbers (!0 instead of !scope0), and the scope lists are also global
unnamed metadata.]
Existing noalias metadata in a callee is "cloned" for use by the inlined code.
This is necessary because the aliasing scopes are unique to each call site
(because of possible control dependencies on the aliasing properties). For
example, consider a function: foo(noalias a, noalias b) { *a = *b; } that gets
inlined into bar() { ... if (...) foo(a1, b1); ... if (...) foo(a2, b2); } --
now just because we know that a1 does not alias with b1 at the first call site,
and a2 does not alias with b2 at the second call site, we cannot let inlining
these functons have the metadata imply that a1 does not alias with b2.
llvm-svn: 213864
diff --git a/llvm/lib/Transforms/IPO/PassManagerBuilder.cpp b/llvm/lib/Transforms/IPO/PassManagerBuilder.cpp
index 701fb46..98477b5 100644
--- a/llvm/lib/Transforms/IPO/PassManagerBuilder.cpp
+++ b/llvm/lib/Transforms/IPO/PassManagerBuilder.cpp
@@ -107,6 +107,7 @@
// BasicAliasAnalysis wins if they disagree. This is intended to help
// support "obvious" type-punning idioms.
PM.add(createTypeBasedAliasAnalysisPass());
+ PM.add(createScopedNoAliasAAPass());
PM.add(createBasicAliasAnalysisPass());
}
diff --git a/llvm/lib/Transforms/Scalar/GVN.cpp b/llvm/lib/Transforms/Scalar/GVN.cpp
index 43ccb99..7135874 100644
--- a/llvm/lib/Transforms/Scalar/GVN.cpp
+++ b/llvm/lib/Transforms/Scalar/GVN.cpp
@@ -1791,6 +1791,19 @@
case LLVMContext::MD_tbaa:
ReplInst->setMetadata(Kind, MDNode::getMostGenericTBAA(IMD, ReplMD));
break;
+ case LLVMContext::MD_alias_scope:
+ case LLVMContext::MD_noalias:
+ // FIXME: If both the original and replacement value are part of the
+ // same control-flow region (meaning that the execution of one
+ // guarentees the executation of the other), then we can combine the
+ // noalias scopes here and do better than the general conservative
+ // answer.
+
+ // In general, GVN unifies expressions over different control-flow
+ // regions, and so we need a conservative combination of the noalias
+ // scopes.
+ ReplInst->setMetadata(Kind, MDNode::intersect(IMD, ReplMD));
+ break;
case LLVMContext::MD_range:
ReplInst->setMetadata(Kind, MDNode::getMostGenericRange(IMD, ReplMD));
break;
diff --git a/llvm/lib/Transforms/Scalar/Scalar.cpp b/llvm/lib/Transforms/Scalar/Scalar.cpp
index de724d4..12df676 100644
--- a/llvm/lib/Transforms/Scalar/Scalar.cpp
+++ b/llvm/lib/Transforms/Scalar/Scalar.cpp
@@ -203,6 +203,10 @@
unwrap(PM)->add(createTypeBasedAliasAnalysisPass());
}
+void LLVMAddScopedNoAliasAAPass(LLVMPassManagerRef PM) {
+ unwrap(PM)->add(createScopedNoAliasAAPass());
+}
+
void LLVMAddBasicAliasAnalysisPass(LLVMPassManagerRef PM) {
unwrap(PM)->add(createBasicAliasAnalysisPass());
}
diff --git a/llvm/lib/Transforms/Scalar/Scalarizer.cpp b/llvm/lib/Transforms/Scalar/Scalarizer.cpp
index 7a73f11..813041a 100644
--- a/llvm/lib/Transforms/Scalar/Scalarizer.cpp
+++ b/llvm/lib/Transforms/Scalar/Scalarizer.cpp
@@ -312,6 +312,8 @@
|| Tag == LLVMContext::MD_fpmath
|| Tag == LLVMContext::MD_tbaa_struct
|| Tag == LLVMContext::MD_invariant_load
+ || Tag == LLVMContext::MD_alias_scope
+ || Tag == LLVMContext::MD_noalias
|| Tag == ParallelLoopAccessMDKind);
}
diff --git a/llvm/lib/Transforms/Utils/InlineFunction.cpp b/llvm/lib/Transforms/Utils/InlineFunction.cpp
index f0a9f2b..f4152dd 100644
--- a/llvm/lib/Transforms/Utils/InlineFunction.cpp
+++ b/llvm/lib/Transforms/Utils/InlineFunction.cpp
@@ -13,10 +13,13 @@
//===----------------------------------------------------------------------===//
#include "llvm/Transforms/Utils/Cloning.h"
+#include "llvm/ADT/SmallSet.h"
#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/SetVector.h"
#include "llvm/ADT/StringExtras.h"
#include "llvm/Analysis/CallGraph.h"
#include "llvm/Analysis/InstructionSimplify.h"
+#include "llvm/Analysis/ValueTracking.h"
#include "llvm/IR/Attributes.h"
#include "llvm/IR/CallSite.h"
#include "llvm/IR/CFG.h"
@@ -28,6 +31,7 @@
#include "llvm/IR/Instructions.h"
#include "llvm/IR/IntrinsicInst.h"
#include "llvm/IR/Intrinsics.h"
+#include "llvm/IR/MDBuilder.h"
#include "llvm/IR/Module.h"
#include "llvm/Transforms/Utils/Local.h"
using namespace llvm;
@@ -260,6 +264,100 @@
InvokeDest->removePredecessor(II->getParent());
}
+/// CloneAliasScopeMetadata - When inlining a function that contains noalias
+/// scope metadata, this metadata needs to be cloned so that the inlined blocks
+/// have different "unqiue scopes" at every call site. Were this not done, then
+/// aliasing scopes from a function inlined into a caller multiple times could
+/// not be differentiated (and this would lead to miscompiles because the
+/// non-aliasing property communicated by the metadata could have
+/// call-site-specific control dependencies).
+static void CloneAliasScopeMetadata(CallSite CS, ValueToValueMapTy &VMap) {
+ const Function *CalledFunc = CS.getCalledFunction();
+ SetVector<const MDNode *> MD;
+
+ // Note: We could only clone the metadata if it is already used in the
+ // caller. I'm omitting that check here because it might confuse
+ // inter-procedural alias analysis passes. We can revisit this if it becomes
+ // an efficiency or overhead problem.
+
+ for (Function::const_iterator I = CalledFunc->begin(), IE = CalledFunc->end();
+ I != IE; ++I)
+ for (BasicBlock::const_iterator J = I->begin(), JE = I->end(); J != JE; ++J) {
+ if (const MDNode *M = J->getMetadata(LLVMContext::MD_alias_scope))
+ MD.insert(M);
+ if (const MDNode *M = J->getMetadata(LLVMContext::MD_noalias))
+ MD.insert(M);
+ }
+
+ if (MD.empty())
+ return;
+
+ // Walk the existing metadata, adding the complete (perhaps cyclic) chain to
+ // the set.
+ SmallVector<const Value *, 16> Queue(MD.begin(), MD.end());
+ while (!Queue.empty()) {
+ const MDNode *M = cast<MDNode>(Queue.pop_back_val());
+ for (unsigned i = 0, ie = M->getNumOperands(); i != ie; ++i)
+ if (const MDNode *M1 = dyn_cast<MDNode>(M->getOperand(i)))
+ if (MD.insert(M1))
+ Queue.push_back(M1);
+ }
+
+ // Now we have a complete set of all metadata in the chains used to specify
+ // the noalias scopes and the lists of those scopes.
+ SmallVector<MDNode *, 16> DummyNodes;
+ DenseMap<const MDNode *, TrackingVH<MDNode> > MDMap;
+ for (SetVector<const MDNode *>::iterator I = MD.begin(), IE = MD.end();
+ I != IE; ++I) {
+ MDNode *Dummy = MDNode::getTemporary(CalledFunc->getContext(),
+ ArrayRef<Value*>());
+ DummyNodes.push_back(Dummy);
+ MDMap[*I] = Dummy;
+ }
+
+ // Create new metadata nodes to replace the dummy nodes, replacing old
+ // metadata references with either a dummy node or an already-created new
+ // node.
+ for (SetVector<const MDNode *>::iterator I = MD.begin(), IE = MD.end();
+ I != IE; ++I) {
+ SmallVector<Value *, 4> NewOps;
+ for (unsigned i = 0, ie = (*I)->getNumOperands(); i != ie; ++i) {
+ const Value *V = (*I)->getOperand(i);
+ if (const MDNode *M = dyn_cast<MDNode>(V))
+ NewOps.push_back(MDMap[M]);
+ else
+ NewOps.push_back(const_cast<Value *>(V));
+ }
+
+ MDNode *NewM = MDNode::get(CalledFunc->getContext(), NewOps),
+ *TempM = MDMap[*I];
+
+ TempM->replaceAllUsesWith(NewM);
+ }
+
+ // Now replace the metadata in the new inlined instructions with the
+ // repacements from the map.
+ for (ValueToValueMapTy::iterator VMI = VMap.begin(), VMIE = VMap.end();
+ VMI != VMIE; ++VMI) {
+ if (!VMI->second)
+ continue;
+
+ Instruction *NI = dyn_cast<Instruction>(VMI->second);
+ if (!NI)
+ continue;
+
+ if (MDNode *M = NI->getMetadata(LLVMContext::MD_alias_scope))
+ NI->setMetadata(LLVMContext::MD_alias_scope, MDMap[M]);
+
+ if (MDNode *M = NI->getMetadata(LLVMContext::MD_noalias))
+ NI->setMetadata(LLVMContext::MD_noalias, MDMap[M]);
+ }
+
+ // Now that everything has been replaced, delete the dummy nodes.
+ for (unsigned i = 0, ie = DummyNodes.size(); i != ie; ++i)
+ MDNode::deleteTemporary(DummyNodes[i]);
+}
+
/// UpdateCallGraphAfterInlining - Once we have cloned code over from a callee
/// into the caller, update the specified callgraph to reflect the changes we
/// made. Note that it's possible that not all code was copied over, so only
@@ -648,6 +746,9 @@
// Update inlined instructions' line number information.
fixupLineNumbers(Caller, FirstNewBlock, TheCall);
+
+ // Clone existing noalias metadata if necessary.
+ CloneAliasScopeMetadata(CS, VMap);
}
// If there are any alloca instructions in the block that used to be the entry
diff --git a/llvm/lib/Transforms/Vectorize/BBVectorize.cpp b/llvm/lib/Transforms/Vectorize/BBVectorize.cpp
index 28ec83bf..9ee0ffb 100644
--- a/llvm/lib/Transforms/Vectorize/BBVectorize.cpp
+++ b/llvm/lib/Transforms/Vectorize/BBVectorize.cpp
@@ -2982,6 +2982,10 @@
case LLVMContext::MD_tbaa:
K->setMetadata(Kind, MDNode::getMostGenericTBAA(JMD, KMD));
break;
+ case LLVMContext::MD_alias_scope:
+ case LLVMContext::MD_noalias:
+ K->setMetadata(Kind, MDNode::intersect(JMD, KMD));
+ break;
case LLVMContext::MD_fpmath:
K->setMetadata(Kind, MDNode::getMostGenericFPMath(JMD, KMD));
break;
diff --git a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
index b60ce61..a88eb49 100644
--- a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
+++ b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
@@ -535,6 +535,8 @@
// non-speculated memory access when the condition was false, this would be
// caught by the runtime overlap checks).
if (Kind != LLVMContext::MD_tbaa &&
+ Kind != LLVMContext::MD_alias_scope &&
+ Kind != LLVMContext::MD_noalias &&
Kind != LLVMContext::MD_fpmath)
continue;
diff --git a/llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp b/llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp
index 53a43d9..bbfdb4e 100644
--- a/llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp
+++ b/llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp
@@ -230,6 +230,10 @@
case LLVMContext::MD_tbaa:
MD = MDNode::getMostGenericTBAA(MD, IMD);
break;
+ case LLVMContext::MD_alias_scope:
+ case LLVMContext::MD_noalias:
+ MD = MDNode::intersect(MD, IMD);
+ break;
case LLVMContext::MD_fpmath:
MD = MDNode::getMostGenericFPMath(MD, IMD);
break;