LoopVectorizationLegality checks if it is legal to vectorize a loop, and to what vectorization factor. More...

#include "llvm/Transforms/Vectorize/LoopVectorizationLegality.h"

Public Types
using	ReductionList = MapVector<PHINode *, RecurrenceDescriptor>
	ReductionList contains the reduction descriptors for all of the reductions that were found in the loop.
using	InductionList = MapVector<PHINode *, InductionDescriptor>
	InductionList saves induction variables and maps them to the induction descriptor.
using	RecurrenceSet = SmallPtrSet<const PHINode *, 8>
	RecurrenceSet contains the phi nodes that are recurrences other than inductions and reductions.

Public Member Functions
	LoopVectorizationLegality (Loop L, PredicatedScalarEvolution &PSE, DominatorTree DT, TargetTransformInfo TTI, TargetLibraryInfo TLI, Function F, LoopAccessInfoManager &LAIs, LoopInfo LI, OptimizationRemarkEmitter ORE, LoopVectorizationRequirements R, LoopVectorizeHints H, DemandedBits DB, AssumptionCache AC, BlockFrequencyInfo BFI, ProfileSummaryInfo PSI, AAResults AA)
bool	canVectorize (bool UseVPlanNativePath)
	Returns true if it is legal to vectorize this loop.
bool	canVectorizeFPMath (bool EnableStrictReductions)
	Returns true if it is legal to vectorize the FP math operations in this loop.
bool	canFoldTailByMasking () const
	Return true if we can vectorize this loop while folding its tail by masking.
void	prepareToFoldTailByMasking ()
	Mark all respective loads/stores for masking.
PHINode *	getPrimaryInduction ()
	Returns the primary induction variable.
const ReductionList &	getReductionVars () const
	Returns the reduction variables found in the loop.
const RecurrenceDescriptor &	getRecurrenceDescriptor (PHINode *PN) const
	Returns the recurrence descriptor associated with a given phi node `PN`, expecting one to exist.
const InductionList &	getInductionVars () const
	Returns the induction variables found in the loop.
RecurrenceSet &	getFixedOrderRecurrences ()
	Return the fixed-order recurrences found in the loop.
IntegerType *	getWidestInductionType ()
	Returns the widest induction type.
bool	isInvariantStoreOfReduction (StoreInst *SI)
	Returns True if given store is a final invariant store of one of the reductions found in the loop.
bool	isInvariantAddressOfReduction (Value *V)
	Returns True if given address is invariant and is used to store recurrent expression.
bool	isInductionPhi (const Value *V) const
	Returns True if V is a Phi node of an induction variable in this loop.
const InductionDescriptor *	getIntOrFpInductionDescriptor (PHINode *Phi) const
	Returns a pointer to the induction descriptor, if `Phi` is an integer or floating point induction.
const InductionDescriptor *	getPointerInductionDescriptor (PHINode *Phi) const
	Returns a pointer to the induction descriptor, if `Phi` is pointer induction.
bool	isCastedInductionVariable (const Value *V) const
	Returns True if V is a cast that is part of an induction def-use chain, and had been proven to be redundant under a runtime guard (in other words, the cast has the same SCEV expression as the induction phi).
bool	isInductionVariable (const Value *V) const
	Returns True if V can be considered as an induction variable in this loop.
bool	isReductionVariable (PHINode *PN) const
	Returns True if PN is a reduction variable in this loop.
bool	isFixedOrderRecurrence (const PHINode *Phi) const
	Returns True if Phi is a fixed-order recurrence in this loop.
bool	blockNeedsPredication (BasicBlock *BB) const
	Return true if the block BB needs to be predicated in order for the loop to be vectorized.
int	isConsecutivePtr (Type AccessTy, Value Ptr) const
	Check if this pointer is consecutive when vectorizing.
bool	isInvariant (Value *V) const
	Returns true if `V` is invariant across all loop iterations according to SCEV.
bool	isUniform (Value *V, ElementCount VF) const
	Returns true if value V is uniform across `VF` lanes, when `VF` is provided, and otherwise if `V` is invariant across all loop iterations.
bool	isUniformMemOp (Instruction &I, ElementCount VF) const
	A uniform memory op is a load or store which accesses the same memory location on all `VF` lanes, if `VF` is provided and otherwise if the memory location is invariant.
const RuntimePointerChecking *	getRuntimePointerChecking () const
	Returns the information that we collected about runtime memory check.
const LoopAccessInfo *	getLAI () const
bool	isSafeForAnyVectorWidth () const
uint64_t	getMaxSafeVectorWidthInBits () const
bool	hasUncountableEarlyExit () const
	Returns true if the loop has exactly one uncountable early exit, i.e.
BasicBlock *	getUncountableEarlyExitingBlock () const
	Returns the uncountable early exiting block, if there is exactly one.
bool	hasUncountableExitWithSideEffects () const
	Returns true if this is an early exit loop with state-changing or potentially-faulting operations and the condition for the uncountable exit must be determined before any of the state changes or potentially faulting operations take place.
bool	isSafeForAnyStoreLoadForwardDistances () const
	Return true if there is store-load forwarding dependencies.
uint64_t	getMaxStoreLoadForwardSafeDistanceInBits () const
	Return safe power-of-2 number of elements, which do not prevent store-load forwarding and safe to operate simultaneously.
bool	isMaskRequired (const Instruction *I) const
	Returns true if vector representation of the instruction `I` requires mask.
bool	hasVectorCallVariants () const
	Returns true if there is at least one function call in the loop which has a vectorized variant available.
unsigned	getNumStores () const
unsigned	getNumLoads () const
std::optional< const HistogramInfo * >	getHistogramInfo (Instruction *I) const
	Returns a HistogramInfo* for the given instruction if it was determined to be part of a load -> update -> store sequence where multiple lanes may be working on the same memory address.
bool	hasHistograms () const
	Returns a list of all known histogram operations in the loop.
const SmallPtrSetImpl< const Instruction * > &	getPotentiallyFaultingLoads () const
	Returns potentially faulting loads.
PredicatedScalarEvolution *	getPredicatedScalarEvolution () const
Loop *	getLoop () const
LoopInfo *	getLoopInfo () const
AssumptionCache *	getAssumptionCache () const
ScalarEvolution *	getScalarEvolution () const
DominatorTree *	getDominatorTree () const
const SmallVector< BasicBlock *, 4 > &	getCountableExitingBlocks () const
	Returns all exiting blocks with a countable exit, i.e.

Detailed Description

LoopVectorizationLegality checks if it is legal to vectorize a loop, and to what vectorization factor.

This class does not look at the profitability of vectorization, only the legality. This class has two main kinds of checks:

Memory checks - The code in canVectorizeMemory checks if vectorization will change the order of memory accesses in a way that will change the correctness of the program.
Scalars checks - The code in canVectorizeInstrs and canVectorizeMemory checks for a number of different conditions, such as the availability of a single induction variable, that all types are supported and vectorize-able, etc. This code reflects the capabilities of InnerLoopVectorizer. This class is also used by InnerLoopVectorizer for identifying induction variable and the different reduction variables.

Definition at line 252 of file LoopVectorizationLegality.h.

Member Typedef Documentation

◆ InductionList

using llvm::LoopVectorizationLegality::InductionList = MapVector<PHINode *, InductionDescriptor>

InductionList saves induction variables and maps them to the induction descriptor.

Definition at line 273 of file LoopVectorizationLegality.h.

◆ RecurrenceSet

using llvm::LoopVectorizationLegality::RecurrenceSet = SmallPtrSet<const PHINode *, 8>

RecurrenceSet contains the phi nodes that are recurrences other than inductions and reductions.

Definition at line 277 of file LoopVectorizationLegality.h.

◆ ReductionList

using llvm::LoopVectorizationLegality::ReductionList = MapVector<PHINode *, RecurrenceDescriptor>

ReductionList contains the reduction descriptors for all of the reductions that were found in the loop.

Definition at line 269 of file LoopVectorizationLegality.h.

Constructor & Destructor Documentation

◆ LoopVectorizationLegality()

llvm::LoopVectorizationLegality::LoopVectorizationLegality	(	Loop *	L,
		PredicatedScalarEvolution &	PSE,
		DominatorTree *	DT,
		TargetTransformInfo *	TTI,
		TargetLibraryInfo *	TLI,
		Function *	F,
		LoopAccessInfoManager &	LAIs,
		LoopInfo *	LI,
		OptimizationRemarkEmitter *	ORE,
		LoopVectorizationRequirements *	R,
		LoopVectorizeHints *	H,
		DemandedBits *	DB,
		AssumptionCache *	AC,
		BlockFrequencyInfo *	BFI,
		ProfileSummaryInfo *	PSI,
		AAResults *	AA )

inline

Definition at line 254 of file LoopVectorizationLegality.h.

References F, and H.

Member Function Documentation

◆ blockNeedsPredication()

bool llvm::LoopVectorizationLegality::blockNeedsPredication ( BasicBlock * BB ) const

Return true if the block BB needs to be predicated in order for the loop to be vectorized.

Definition at line 1423 of file LoopVectorizationLegality.cpp.

References assert(), llvm::LoopAccessInfo::blockNeedsPredication(), getUncountableEarlyExitingBlock(), hasUncountableEarlyExit(), llvm::is_contained(), and llvm::predecessors().

Referenced by isUniformMemOp().

◆ canFoldTailByMasking()

bool llvm::LoopVectorizationLegality::canFoldTailByMasking ( ) const

Return true if we can vectorize this loop while folding its tail by masking.

Definition at line 2068 of file LoopVectorizationLegality.cpp.

References llvm::cast(), llvm::SmallPtrSetImpl< PtrType >::count(), llvm::dbgs(), getInductionVars(), getReductionVars(), llvm::SmallPtrSetImpl< PtrType >::insert(), LLVM_DEBUG, and llvm::Value::users().

◆ canVectorize()

bool llvm::LoopVectorizationLegality::canVectorize ( bool UseVPlanNativePath )

Returns true if it is legal to vectorize this loop.

This does not mean that it is profitable to vectorize this loop, only that it is legal to do so. Temporarily taking UseVPlanNativePath parameter. If true, take the new code path being implemented for outer loop vectorization (should be functional for inner loop vectorization) based on VPlan. If false, good old LV code.

Definition at line 1940 of file LoopVectorizationLegality.cpp.

References assert(), llvm::dbgs(), DEBUG_TYPE, llvm::LoopVectorizeHints::FK_Enabled, hasUncountableEarlyExit(), hasUncountableExitWithSideEffects(), llvm::isa(), LLVM_DEBUG, PragmaVectorizeSCEVCheckThreshold, llvm::reportVectorizationFailure(), and VectorizeSCEVCheckThreshold.

Referenced by llvm::LoopVectorizePass::processLoop().

◆ canVectorizeFPMath()

bool llvm::LoopVectorizationLegality::canVectorizeFPMath ( bool EnableStrictReductions )

Returns true if it is legal to vectorize the FP math operations in this loop.

Vectorizing is legal if we allow reordering of FP operations, or if we can use in-order reductions.

Definition at line 1332 of file LoopVectorizationLegality.cpp.

References llvm::all_of(), llvm::any_of(), llvm::InductionDescriptor::getExactFPMathInst(), getInductionVars(), getReductionVars(), llvm::RecurrenceDescriptor::hasExactFPMath(), and llvm::RecurrenceDescriptor::isOrdered().

Referenced by llvm::LoopVectorizePass::processLoop().

◆ getAssumptionCache()

AssumptionCache * llvm::LoopVectorizationLegality::getAssumptionCache ( ) const

inline

Definition at line 473 of file LoopVectorizationLegality.h.

Referenced by llvm::ARMTTIImpl::preferPredicateOverEpilogue().

◆ getCountableExitingBlocks()

const SmallVector< BasicBlock *, 4 > & llvm::LoopVectorizationLegality::getCountableExitingBlocks ( ) const

inline

Returns all exiting blocks with a countable exit, i.e.

the exit-not-taken count is known exactly at compile time.

Definition at line 481 of file LoopVectorizationLegality.h.

Referenced by llvm::LoopVectorizePass::processLoop().

◆ getDominatorTree()

DominatorTree * llvm::LoopVectorizationLegality::getDominatorTree ( ) const

inline

Definition at line 477 of file LoopVectorizationLegality.h.

Referenced by llvm::ARMTTIImpl::preferPredicateOverEpilogue().

◆ getFixedOrderRecurrences()

RecurrenceSet & llvm::LoopVectorizationLegality::getFixedOrderRecurrences ( )

inline

Return the fixed-order recurrences found in the loop.

Definition at line 319 of file LoopVectorizationLegality.h.

Referenced by llvm::AArch64TTIImpl::preferPredicateOverEpilogue().

◆ getHistogramInfo()

std::optional< const HistogramInfo * > llvm::LoopVectorizationLegality::getHistogramInfo ( Instruction * I ) const

inline

Returns a HistogramInfo* for the given instruction if it was determined to be part of a load -> update -> store sequence where multiple lanes may be working on the same memory address.

Definition at line 448 of file LoopVectorizationLegality.h.

References I.

◆ getInductionVars()

const InductionList & llvm::LoopVectorizationLegality::getInductionVars ( ) const

inline

Returns the induction variables found in the loop.

Definition at line 316 of file LoopVectorizationLegality.h.

Referenced by canFoldTailByMasking(), canVectorizeFPMath(), fixScalarResumeValuesFromBypass(), getIntOrFpInductionDescriptor(), and getPointerInductionDescriptor().

◆ getIntOrFpInductionDescriptor()

const InductionDescriptor * llvm::LoopVectorizationLegality::getIntOrFpInductionDescriptor ( PHINode * Phi ) const

Returns a pointer to the induction descriptor, if Phi is an integer or floating point induction.

Definition at line 1388 of file LoopVectorizationLegality.cpp.

References llvm::MapVector< KeyT, ValueT, MapType, VectorType >::find(), getInductionVars(), llvm::InductionDescriptor::IK_FpInduction, llvm::InductionDescriptor::IK_IntInduction, and isInductionPhi().

◆ getLAI()

const LoopAccessInfo * llvm::LoopVectorizationLegality::getLAI ( ) const

inline

Definition at line 393 of file LoopVectorizationLegality.h.

Referenced by llvm::ARMTTIImpl::preferPredicateOverEpilogue(), llvm::LoopVectorizePass::processLoop(), and processLoopInVPlanNativePath().

◆ getLoop()

Loop * llvm::LoopVectorizationLegality::getLoop ( ) const

inline

Definition at line 469 of file LoopVectorizationLegality.h.

Referenced by llvm::AArch64TTIImpl::preferPredicateOverEpilogue(), and llvm::ARMTTIImpl::preferPredicateOverEpilogue().

◆ getLoopInfo()

LoopInfo * llvm::LoopVectorizationLegality::getLoopInfo ( ) const

inline

Definition at line 471 of file LoopVectorizationLegality.h.

Referenced by llvm::ARMTTIImpl::preferPredicateOverEpilogue().

◆ getMaxSafeVectorWidthInBits()

uint64_t llvm::LoopVectorizationLegality::getMaxSafeVectorWidthInBits ( ) const

inline

Definition at line 400 of file LoopVectorizationLegality.h.

◆ getMaxStoreLoadForwardSafeDistanceInBits()

uint64_t llvm::LoopVectorizationLegality::getMaxStoreLoadForwardSafeDistanceInBits ( ) const

inline

Return safe power-of-2 number of elements, which do not prevent store-load forwarding and safe to operate simultaneously.

Definition at line 428 of file LoopVectorizationLegality.h.

◆ getNumLoads()

unsigned llvm::LoopVectorizationLegality::getNumLoads ( ) const

inline

Definition at line 443 of file LoopVectorizationLegality.h.

◆ getNumStores()

unsigned llvm::LoopVectorizationLegality::getNumStores ( ) const

inline

Definition at line 442 of file LoopVectorizationLegality.h.

◆ getPointerInductionDescriptor()

const InductionDescriptor * llvm::LoopVectorizationLegality::getPointerInductionDescriptor ( PHINode * Phi ) const

Returns a pointer to the induction descriptor, if Phi is pointer induction.

Definition at line 1399 of file LoopVectorizationLegality.cpp.

References llvm::MapVector< KeyT, ValueT, MapType, VectorType >::find(), getInductionVars(), llvm::InductionDescriptor::IK_PtrInduction, and isInductionPhi().

◆ getPotentiallyFaultingLoads()

const SmallPtrSetImpl< const Instruction * > & llvm::LoopVectorizationLegality::getPotentiallyFaultingLoads ( ) const

inline

Returns potentially faulting loads.

Definition at line 461 of file LoopVectorizationLegality.h.

Referenced by llvm::LoopVectorizePass::processLoop().

◆ getPredicatedScalarEvolution()

PredicatedScalarEvolution * llvm::LoopVectorizationLegality::getPredicatedScalarEvolution ( ) const

inline

Definition at line 465 of file LoopVectorizationLegality.h.

Referenced by llvm::AArch64TTIImpl::preferPredicateOverEpilogue().

◆ getPrimaryInduction()

PHINode * llvm::LoopVectorizationLegality::getPrimaryInduction ( )

inline

Returns the primary induction variable.

Definition at line 302 of file LoopVectorizationLegality.h.

Referenced by fixScalarResumeValuesFromBypass().

◆ getRecurrenceDescriptor()

const RecurrenceDescriptor & llvm::LoopVectorizationLegality::getRecurrenceDescriptor ( PHINode * PN ) const

inline

Returns the recurrence descriptor associated with a given phi node PN, expecting one to exist.

Definition at line 309 of file LoopVectorizationLegality.h.

References assert(), and isReductionVariable().

◆ getReductionVars()

const ReductionList & llvm::LoopVectorizationLegality::getReductionVars ( ) const

inline

Returns the reduction variables found in the loop.

Definition at line 305 of file LoopVectorizationLegality.h.

Referenced by canFoldTailByMasking(), canVectorizeFPMath(), isInvariantAddressOfReduction(), isInvariantStoreOfReduction(), and llvm::AArch64TTIImpl::preferPredicateOverEpilogue().

◆ getRuntimePointerChecking()

const RuntimePointerChecking * llvm::LoopVectorizationLegality::getRuntimePointerChecking ( ) const

inline

Returns the information that we collected about runtime memory check.

Definition at line 389 of file LoopVectorizationLegality.h.

◆ getScalarEvolution()

ScalarEvolution * llvm::LoopVectorizationLegality::getScalarEvolution ( ) const

inline

Definition at line 475 of file LoopVectorizationLegality.h.

Referenced by llvm::ARMTTIImpl::preferPredicateOverEpilogue().

◆ getUncountableEarlyExitingBlock()

BasicBlock * llvm::LoopVectorizationLegality::getUncountableEarlyExitingBlock ( ) const

inline

Returns the uncountable early exiting block, if there is exactly one.

Definition at line 409 of file LoopVectorizationLegality.h.

Referenced by blockNeedsPredication().

◆ getWidestInductionType()

IntegerType * llvm::LoopVectorizationLegality::getWidestInductionType ( )

inline

Returns the widest induction type.

Definition at line 322 of file LoopVectorizationLegality.h.

◆ hasHistograms()

bool llvm::LoopVectorizationLegality::hasHistograms ( ) const

inline

Returns a list of all known histogram operations in the loop.

Definition at line 457 of file LoopVectorizationLegality.h.

Referenced by llvm::LoopVectorizePass::processLoop().

◆ hasUncountableEarlyExit()

bool llvm::LoopVectorizationLegality::hasUncountableEarlyExit ( ) const

inline

Returns true if the loop has exactly one uncountable early exit, i.e.

an uncountable exit that isn't the latch block.

Definition at line 406 of file LoopVectorizationLegality.h.

Referenced by blockNeedsPredication(), canVectorize(), and llvm::LoopVectorizePass::processLoop().

◆ hasUncountableExitWithSideEffects()

bool llvm::LoopVectorizationLegality::hasUncountableExitWithSideEffects ( ) const

inline

Returns true if this is an early exit loop with state-changing or potentially-faulting operations and the condition for the uncountable exit must be determined before any of the state changes or potentially faulting operations take place.

Definition at line 417 of file LoopVectorizationLegality.h.

Referenced by canVectorize().

◆ hasVectorCallVariants()

bool llvm::LoopVectorizationLegality::hasVectorCallVariants ( ) const

inline

Returns true if there is at least one function call in the loop which has a vectorized variant available.

Definition at line 440 of file LoopVectorizationLegality.h.

◆ isCastedInductionVariable()

bool llvm::LoopVectorizationLegality::isCastedInductionVariable ( const Value * V ) const

Returns True if V is a cast that is part of an induction def-use chain, and had been proven to be redundant under a runtime guard (in other words, the cast has the same SCEV expression as the induction phi).

Definition at line 1408 of file LoopVectorizationLegality.cpp.

References llvm::dyn_cast().

Referenced by isInductionVariable().

◆ isConsecutivePtr()

int llvm::LoopVectorizationLegality::isConsecutivePtr	(	Type *	AccessTy,
		Value *	Ptr ) const

Check if this pointer is consecutive when vectorizing.

This happens when the last index of the GEP is the induction variable, or that the pointer itself is an induction variable. This check allows us to vectorize A[idx] into a wide load/store. Returns: 0 - Stride is unknown or non-consecutive. 1 - Address is consecutive. -1 - Address is consecutive, and decreasing. NOTE: This method must only be used before modifying the original scalar loop. Do not use after invoking 'createVectorizedLoopSkeleton' (PR34965).

Definition at line 455 of file LoopVectorizationLegality.cpp.

References llvm::getPtrStride(), llvm::IRPass, Ptr, and llvm::shouldOptimizeForSize().

◆ isFixedOrderRecurrence()

bool llvm::LoopVectorizationLegality::isFixedOrderRecurrence ( const PHINode * Phi ) const

Returns True if Phi is a fixed-order recurrence in this loop.

Definition at line 1418 of file LoopVectorizationLegality.cpp.

◆ isInductionPhi()

bool llvm::LoopVectorizationLegality::isInductionPhi ( const Value * V ) const

Returns True if V is a Phi node of an induction variable in this loop.

Definition at line 1378 of file LoopVectorizationLegality.cpp.

References llvm::dyn_cast_or_null().

Referenced by getIntOrFpInductionDescriptor(), getPointerInductionDescriptor(), and isInductionVariable().

◆ isInductionVariable()

bool llvm::LoopVectorizationLegality::isInductionVariable ( const Value * V ) const

Returns True if V can be considered as an induction variable in this loop.

V can be the induction phi, or some redundant cast in the def-use chain of the inducion phi.

Definition at line 1414 of file LoopVectorizationLegality.cpp.

References isCastedInductionVariable(), and isInductionPhi().

◆ isInvariant()

bool llvm::LoopVectorizationLegality::isInvariant ( Value * V ) const

Returns true if V is invariant across all loop iterations according to SCEV.

Definition at line 473 of file LoopVectorizationLegality.cpp.

Referenced by isUniform().

◆ isInvariantAddressOfReduction()

bool llvm::LoopVectorizationLegality::isInvariantAddressOfReduction ( Value * V )

Returns True if given address is invariant and is used to store recurrent expression.

Definition at line 1365 of file LoopVectorizationLegality.cpp.

References llvm::any_of(), llvm::StoreInst::getPointerOperand(), getReductionVars(), llvm::ScalarEvolution::getSCEV(), and llvm::RecurrenceDescriptor::IntermediateStore.

◆ isInvariantStoreOfReduction()

bool llvm::LoopVectorizationLegality::isInvariantStoreOfReduction ( StoreInst * SI )

Returns True if given store is a final invariant store of one of the reductions found in the loop.

Definition at line 1358 of file LoopVectorizationLegality.cpp.

References llvm::any_of(), getReductionVars(), and llvm::RecurrenceDescriptor::IntermediateStore.

◆ isMaskRequired()

bool llvm::LoopVectorizationLegality::isMaskRequired ( const Instruction * I ) const

inline

Returns true if vector representation of the instruction I requires mask.

Definition at line 434 of file LoopVectorizationLegality.h.

References I.

◆ isReductionVariable()

bool llvm::LoopVectorizationLegality::isReductionVariable ( PHINode * PN ) const

inline

Returns True if PN is a reduction variable in this loop.

Definition at line 354 of file LoopVectorizationLegality.h.

Referenced by getRecurrenceDescriptor().

◆ isSafeForAnyStoreLoadForwardDistances()

bool llvm::LoopVectorizationLegality::isSafeForAnyStoreLoadForwardDistances ( ) const

inline

Return true if there is store-load forwarding dependencies.

Definition at line 422 of file LoopVectorizationLegality.h.

◆ isSafeForAnyVectorWidth()

bool llvm::LoopVectorizationLegality::isSafeForAnyVectorWidth ( ) const

inline

Definition at line 395 of file LoopVectorizationLegality.h.

◆ isUniform()

bool llvm::LoopVectorizationLegality::isUniform	(	Value *	V,
		ElementCount	VF ) const

Returns true if value V is uniform across VF lanes, when VF is provided, and otherwise if V is invariant across all loop iterations.

Definition at line 568 of file LoopVectorizationLegality.cpp.

References llvm::all_of(), llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::getKnownMinValue(), llvm::ScalarEvolution::getSCEV(), I, llvm::isa(), isInvariant(), llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::isScalable(), llvm::ElementCount::isScalar(), llvm::ScalarEvolution::isSCEVable(), llvm::reverse(), and llvm::seq().

Referenced by isUniformMemOp().

◆ isUniformMemOp()

bool llvm::LoopVectorizationLegality::isUniformMemOp	(	Instruction &	I,
		ElementCount	VF ) const

A uniform memory op is a load or store which accesses the same memory location on all VF lanes, if VF is provided and otherwise if the memory location is invariant.

Definition at line 601 of file LoopVectorizationLegality.cpp.

References blockNeedsPredication(), llvm::getLoadStorePointerOperand(), I, isUniform(), and Ptr.

◆ prepareToFoldTailByMasking()

void llvm::LoopVectorizationLegality::prepareToFoldTailByMasking ( )

Mark all respective loads/stores for masking.

Must only be called when tail-folding is possible.

Definition at line 2136 of file LoopVectorizationLegality.cpp.

References assert().

The documentation for this class was generated from the following files:

include/llvm/Transforms/Vectorize/LoopVectorizationLegality.h
lib/Transforms/Vectorize/LoopVectorizationLegality.cpp

Public Types

Public Member Functions

Detailed Description

Member Typedef Documentation

◆ InductionList

◆ RecurrenceSet

◆ ReductionList

Constructor & Destructor Documentation

◆ LoopVectorizationLegality()

Member Function Documentation

◆ blockNeedsPredication()

◆ canFoldTailByMasking()

◆ canVectorize()

◆ canVectorizeFPMath()

◆ getAssumptionCache()

◆ getCountableExitingBlocks()

◆ getDominatorTree()

◆ getFixedOrderRecurrences()

◆ getHistogramInfo()

◆ getInductionVars()

◆ getIntOrFpInductionDescriptor()

◆ getLAI()

◆ getLoop()

◆ getLoopInfo()

◆ getMaxSafeVectorWidthInBits()

◆ getMaxStoreLoadForwardSafeDistanceInBits()

◆ getNumLoads()

◆ getNumStores()

◆ getPointerInductionDescriptor()

◆ getPotentiallyFaultingLoads()

◆ getPredicatedScalarEvolution()

◆ getPrimaryInduction()

◆ getRecurrenceDescriptor()

◆ getReductionVars()

◆ getRuntimePointerChecking()

◆ getScalarEvolution()

◆ getUncountableEarlyExitingBlock()

◆ getWidestInductionType()

◆ hasHistograms()

◆ hasUncountableEarlyExit()

◆ hasUncountableExitWithSideEffects()

◆ hasVectorCallVariants()

◆ isCastedInductionVariable()

◆ isConsecutivePtr()

◆ isFixedOrderRecurrence()

◆ isInductionPhi()

◆ isInductionVariable()

◆ isInvariant()

◆ isInvariantAddressOfReduction()

◆ isInvariantStoreOfReduction()

◆ isMaskRequired()

◆ isReductionVariable()

◆ isSafeForAnyStoreLoadForwardDistances()

◆ isSafeForAnyVectorWidth()

◆ isUniform()

◆ isUniformMemOp()

◆ prepareToFoldTailByMasking()