Changeset View
Changeset View
Standalone View
Standalone View
llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
| Show First 20 Lines • Show All 1,421 Lines • ▼ Show 20 Lines | Align NVPTXTargetLowering::getArgumentAlignment(SDValue Callee, | ||||
| if (!DirectCallee) { | if (!DirectCallee) { | ||||
| // We don't have a direct function symbol, but that may be because of | // We don't have a direct function symbol, but that may be because of | ||||
| // constant cast instructions in the call. | // constant cast instructions in the call. | ||||
| // With bitcast'd call targets, the instruction will be the call | // With bitcast'd call targets, the instruction will be the call | ||||
| if (const auto *CI = dyn_cast<CallInst>(CB)) { | if (const auto *CI = dyn_cast<CallInst>(CB)) { | ||||
| // Check if we have call alignment metadata | // Check if we have call alignment metadata | ||||
| if (getAlign(*CI, Idx, Alignment)) | if (getAlign(*CI, Idx, Alignment)) | ||||
tra: Would we ever have an instruction other than `Call` which would also carry explicit alignment… | |||||
Not Done ReplyInline Actionsallocas, loads, and stores, though I'm not sure if that's what you're asking. Could you expand a little on what you mean? The code here is only dealing with argument alignment so in isolation the answer is "no", but I have a feeling you might be hinting at something deeper ? ldrumm: allocas, loads, and stores, though I'm not sure if that's what you're asking. Could you expand… | |||||
Not Done ReplyInline ActionsIf CallBase can be an instruction other than Call, will we be able to get argument alignment via getAlign() and if so, should we? Another observation here is that previously digging through bitcasts was done under if ( dyn_cast<CallInst>(CB)) while now it's done for any CB instruction. It looks like it should work (getMaybeBitcastedCallee will bail out on other instructions, if it can't get ), but was that done that intentionally? If looking through bitcasts is applicable to the Call instruction only, I'd prefer to keep it under the if to make it obvious. tra: If `CallBase` can be an instruction other than `Call`, will we be able to get argument… | |||||
I don't think this can ever be anything other than a call. It's within if (const auto *CI = dyn_cast<CallInst>(CB)) {. The getMaybeBitcastedCallee is indeed used in more possible places but I figured this to be safe in the current usage. ldrumm: I don't think this can ever be anything other than a call. It's within `if (const auto *CI =… | |||||
| return Align(Alignment); | return Align(Alignment); | ||||
| const Value *CalleeV = CI->getCalledOperand(); | |||||
| // Ignore any bitcast instructions | |||||
| while (isa<ConstantExpr>(CalleeV)) { | |||||
| const ConstantExpr *CE = cast<ConstantExpr>(CalleeV); | |||||
| if (!CE->isCast()) | |||||
| break; | |||||
| // Look through the bitcast | |||||
| CalleeV = cast<ConstantExpr>(CalleeV)->getOperand(0); | |||||
| } | |||||
| // We have now looked past all of the bitcasts. Do we finally have a | |||||
| // Function? | |||||
| if (const auto *CalleeF = dyn_cast<Function>(CalleeV)) | |||||
| DirectCallee = CalleeF; | |||||
| } | } | ||||
| DirectCallee = getMaybeBitcastedCallee(CB); | |||||
| } | } | ||||
| // Check for function alignment information if we found that the | // Check for function alignment information if we found that the | ||||
| // ultimate target is a Function | // ultimate target is a Function | ||||
| if (DirectCallee) { | if (DirectCallee) { | ||||
| if (getAlign(*DirectCallee, Idx, Alignment)) | if (getAlign(*DirectCallee, Idx, Alignment)) | ||||
| return Align(Alignment); | return Align(Alignment); | ||||
| // If alignment information is not available, fall back to the | // If alignment information is not available, fall back to the | ||||
| ▲ Show 20 Lines • Show All 60 Lines • ▼ Show 20 Lines | for (unsigned i = 0, e = Args.size(); i != e; ++i, ++OIdx) { | ||||
| if (IsByVal) { | if (IsByVal) { | ||||
| // The ByValAlign in the Outs[OIdx].Flags is always set at this point, | // The ByValAlign in the Outs[OIdx].Flags is always set at this point, | ||||
| // so we don't need to worry whether it's naturally aligned or not. | // so we don't need to worry whether it's naturally aligned or not. | ||||
| // See TargetLowering::LowerCallTo(). | // See TargetLowering::LowerCallTo(). | ||||
| ArgAlign = Outs[OIdx].Flags.getNonZeroByValAlign(); | ArgAlign = Outs[OIdx].Flags.getNonZeroByValAlign(); | ||||
| // Try to increase alignment to enhance vectorization options. | // Try to increase alignment to enhance vectorization options. | ||||
| ArgAlign = std::max(ArgAlign, getFunctionParamOptimizedAlign( | ArgAlign = std::max(ArgAlign, getFunctionParamOptimizedAlign( | ||||
| CB->getCalledFunction(), ETy, DL)); | getMaybeBitcastedCallee(CB), ETy, DL)); | ||||
| // Enforce minumum alignment of 4 to work around ptxas miscompile | // Enforce minumum alignment of 4 to work around ptxas miscompile | ||||
| // for sm_50+. See corresponding alignment adjustment in | // for sm_50+. See corresponding alignment adjustment in | ||||
| // emitFunctionParamList() for details. | // emitFunctionParamList() for details. | ||||
| ArgAlign = std::max(ArgAlign, Align(4)); | ArgAlign = std::max(ArgAlign, Align(4)); | ||||
| } else { | } else { | ||||
| ArgAlign = getArgumentAlignment(Callee, CB, Ty, ParamCount + 1, DL); | ArgAlign = getArgumentAlignment(Callee, CB, Ty, ParamCount + 1, DL); | ||||
| } | } | ||||
| ▲ Show 20 Lines • Show All 2,803 Lines • ▼ Show 20 Lines | |||||
| /// alignment. To allow using 128-bit vectorized loads/stores, this function | /// alignment. To allow using 128-bit vectorized loads/stores, this function | ||||
| /// ensures that alignment is 16 or greater. | /// ensures that alignment is 16 or greater. | ||||
| Align NVPTXTargetLowering::getFunctionParamOptimizedAlign( | Align NVPTXTargetLowering::getFunctionParamOptimizedAlign( | ||||
| const Function *F, Type *ArgTy, const DataLayout &DL) const { | const Function *F, Type *ArgTy, const DataLayout &DL) const { | ||||
| const uint64_t ABITypeAlign = DL.getABITypeAlign(ArgTy).value(); | const uint64_t ABITypeAlign = DL.getABITypeAlign(ArgTy).value(); | ||||
| // If a function has linkage different from internal or private, we | // If a function has linkage different from internal or private, we | ||||
| // must use default ABI alignment as external users rely on it. | // must use default ABI alignment as external users rely on it. | ||||
| if (!F->hasLocalLinkage()) | if (!(F && F->hasLocalLinkage())) | ||||
Nit: if (!(F && F->hasLocalLinkage())) would be a bit easier to understand, IMO. tra: Nit: `if (!(F && F->hasLocalLinkage()))` would be a bit easier to understand, IMO.
| |||||
| return Align(ABITypeAlign); | return Align(ABITypeAlign); | ||||
| assert(!isKernelFunction(*F) && "Expect kernels to have non-local linkage"); | assert(!isKernelFunction(*F) && "Expect kernels to have non-local linkage"); | ||||
| return Align(std::max(uint64_t(16), ABITypeAlign)); | return Align(std::max(uint64_t(16), ABITypeAlign)); | ||||
| } | } | ||||
| /// isLegalAddressingMode - Return true if the addressing mode represented | /// isLegalAddressingMode - Return true if the addressing mode represented | ||||
| /// by AM is legal for this target, for a load/store of the specified type. | /// by AM is legal for this target, for a load/store of the specified type. | ||||
| ▲ Show 20 Lines • Show All 918 Lines • Show Last 20 Lines | |||||
Would we ever have an instruction other than Call which would also carry explicit alignment metadata?