[VPlan] Clean up tryToCreate(Widen)Recipe. (NFC)
This patch includes some clean-ups to tryToCreateRecipe, suggested in
D77973.
It includes:
* Renaming tryToCreateRecipe to tryToCreateWidenRecipe.
* Move VPBB insertion logic to caller of tryToCreateWidenRecipe.
* Hoists instruction checks to tryToCreateWidenRecipe, making it
clearer which instructions are handled by which recipe, simplifying
the checks by using early exits.
* Split up handling of induction PHIs and truncates using inductions.
Reviewers: gilr, rengolin, Ayal, hsaito
Reviewed By: Ayal
Differential Revision: https://reviews.llvm.org/D78287
diff --git a/llvm/lib/Transforms/Vectorize/VPRecipeBuilder.h b/llvm/lib/Transforms/Vectorize/VPRecipeBuilder.h
index 4651802..7c2b899 100644
--- a/llvm/lib/Transforms/Vectorize/VPRecipeBuilder.h
+++ b/llvm/lib/Transforms/Vectorize/VPRecipeBuilder.h
@@ -54,6 +54,12 @@
// marked by having a nullptr entry in this map.
DenseMap<Instruction *, VPRecipeBase *> Ingredient2Recipe;
+ /// Check if \p I can be widened at the start of \p Range and possibly
+ /// decrease the range such that the returned value holds for the entire \p
+ /// Range. The function should not be called for memory instructions or calls.
+ bool shouldWiden(Instruction *I, VFRange &Range) const;
+
+public:
/// Set the recipe created for given ingredient. This operation is a no-op for
/// ingredients that were not marked using a nullptr entry in the map.
void setRecipe(Instruction *I, VPRecipeBase *R) {
@@ -64,12 +70,6 @@
Ingredient2Recipe[I] = R;
}
- /// Check if \p I can be widened at the start of \p Range and possibly
- /// decrease the range such that the returned value holds for the entire \p
- /// Range. The function should not be called for memory instructions or calls.
- bool shouldWiden(Instruction *I, VFRange &Range) const;
-
-public:
/// A helper function that computes the predicate of the block BB, assuming
/// that the header block of the loop is set to True. It returns the *entry*
/// mask for the block BB.
@@ -96,33 +96,30 @@
return Ingredient2Recipe[I];
}
- /// Check if \I is a memory instruction to be widened for \p Range.Start and
- /// potentially masked. Such instructions are handled by a recipe that takes
- /// an additional VPInstruction for the mask.
+ /// Check if the load or store instruction \p I should widened for \p
+ /// Range.Start and potentially masked. Such instructions are handled by a
+ /// recipe that takes an additional VPInstruction for the mask.
VPWidenMemoryInstructionRecipe *
tryToWidenMemory(Instruction *I, VFRange &Range, VPlanPtr &Plan);
- /// Check if an induction recipe should be constructed for \I within the given
- /// VF \p Range. If so build and return it. If not, return null. \p Range.End
- /// may be decreased to ensure same decision from \p Range.Start to
- /// \p Range.End.
- VPWidenIntOrFpInductionRecipe *tryToOptimizeInduction(Instruction *I,
- VFRange &Range);
+ /// Check if an induction recipe should be constructed for \I. If so build and
+ /// return it. If not, return null.
+ VPWidenIntOrFpInductionRecipe *tryToOptimizeInductionPHI(PHINode *Phi);
+
+ /// Optimize the special case where the operand of \p I is a constant integer
+ /// induction variable.
+ VPWidenIntOrFpInductionRecipe *tryToOptimizeInductionTruncate(TruncInst *I,
+ VFRange &Range);
/// Handle non-loop phi nodes. Currently all such phi nodes are turned into
/// a sequence of select instructions as the vectorizer currently performs
/// full if-conversion.
- VPBlendRecipe *tryToBlend(Instruction *I, VPlanPtr &Plan);
+ VPBlendRecipe *tryToBlend(PHINode *Phi, VPlanPtr &Plan);
- /// Handle call instruction. If \p I is a call that can be widened for \p
- /// Range.Start, return a new VPWidenCallRecipe. Range.End may be decreased to
- /// ensure same decision from \p Range.Start to \p Range.End.
- VPWidenCallRecipe *tryToWidenCall(Instruction *I, VFRange &Range,
- VPlan &Plan);
- /// Check if \p I is a SelectInst and return a VPWidenSelectRecipe if it is.
- /// The function should only be called if the cost-model indicates that
- /// widening should be performed.
- VPWidenSelectRecipe *tryToWidenSelect(Instruction *I);
+ /// Handle call instructions. If \p CI can be widened for \p Range.Start,
+ /// return a new VPWidenCallRecipe. Range.End may be decreased to ensure same
+ /// decision from \p Range.Start to \p Range.End.
+ VPWidenCallRecipe *tryToWidenCall(CallInst *CI, VFRange &Range, VPlan &Plan);
/// Check if \p I has an opcode that can be widened and return a VPWidenRecipe
/// if it can. The function should only be called if the cost-model indicates
@@ -143,9 +140,9 @@
Builder(Builder) {}
/// Check if a recipe can be create for \p I withing the given VF \p Range.
- /// If a recipe can be created, it adds it to \p VPBB.
- bool tryToCreateRecipe(Instruction *Instr, VFRange &Range, VPlanPtr &Plan,
- VPBasicBlock *VPBB);
+ /// If a recipe can be created, return it. Otherwise return nullptr.
+ VPRecipeBase *tryToCreateWidenRecipe(Instruction *Instr, VFRange &Range,
+ VPlanPtr &Plan);
/// Build a VPReplicationRecipe for \p I and enclose it within a Region if it
/// is predicated. \return \p VPBB augmented with this new recipe if \p I is