[RISCV] Custom type legalize i8/i16 UDIV/UREM/SDIV on RV64 so we can use divuw/remuw/divw.

[craig.topper]

Summary

This makes our i8/i16 codegen more similar to the i32 codegen.

I've also added computeKnownBits support for DIVUW/REMUW so
that we can remove zero extending ANDs from the output. Without
this we end up turning DIVUW/REMUW back into DIVU/REMU via some
isel patterns.

[frasercrmck]

LGTM, though I'd prefer the linter's suggestion over that manually-formatted code. You might be able to improve it somewhat with choice use of parens?

[luismarques]

LGTM.

[jrtc27]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

2187

Do these do the right thing for 0 (which RISC-V defines as saturating)? Or do we not care because it's UB in the IR?

[craig.topper]

In D95322#2519185, @frasercrmck wrote:

LGTM, though I'd prefer the linter's suggestion over that manually-formatted code. You might be able to improve it somewhat with choice use of parens?

Weirdly that came from clang-format. I just rebuilt it and double checked. Not sure if that means phabricator is using an older version of clang-format?

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

2187

I think they do the right thing, but I've added a note that the behavior is undefined so we don't have to worry about someone changing the KnownBits implementation in the future. If we find a use case for making them not undefined, we can revisit.

[nathanchance]

This patch causes issues with building the Linux kernel. cvise spits out:

$ cat pci.i
char pci_cache_line_size, pci_set_cacheline_size_dev_cacheline_size;
pci_set_cacheline_size_dev() {
  if (pci_set_cacheline_size_dev_cacheline_size % pci_cache_line_size)
    return 0;
  return 2;
}

$ clang -O2 --target=riscv64-linux-gnu -c -o /dev/null pci.i
clang: /home/nathan/cbl/github/tc-build/llvm-project/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:5744: llvm::SDValue llvm::SelectionDAG::getNode(unsigned int, const llvm::SDLoc &, llvm::EVT, llvm::SDValue, llvm::SDValue, llvm::SDValue, const llvm::SDNodeFlags): Assertion `N1.getValueType() == N2.getValueType() && "SETCC operands must have the same type!"' failed.
PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.      Program arguments: /home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang -O2 --target=riscv64-linux-gnu -c -o /dev/null pci.i
1.      <eof> parser at end of file
2.      Code generation
3.      Running pass 'Function Pass Manager' on module 'pci.i'.
4.      Running pass 'RISCV DAG->DAG Pattern Instruction Selection' on function '@pci_set_cacheline_size_dev'
 #0 0x00000000029a6673 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x29a6673)
 #1 0x00000000029a443e llvm::sys::RunSignalHandlers() (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x29a443e)
 #2 0x00000000029a5a0d llvm::sys::CleanupOnSignal(unsigned long) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x29a5a0d)
 #3 0x0000000002935ea3 (anonymous namespace)::CrashRecoveryContextImpl::HandleCrash(int, unsigned long) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x2935ea3)
 #4 0x0000000002935fde CrashRecoverySignalHandler(int) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x2935fde)
 #5 0x00007f27c68973c0 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x153c0)
 #6 0x00007f27c635c18b raise (/lib/x86_64-linux-gnu/libc.so.6+0x4618b)
 #7 0x00007f27c633b859 abort (/lib/x86_64-linux-gnu/libc.so.6+0x25859)
 #8 0x00007f27c633b729 (/lib/x86_64-linux-gnu/libc.so.6+0x25729)
 #9 0x00007f27c634cf36 (/lib/x86_64-linux-gnu/libc.so.6+0x36f36)
#10 0x0000000003855672 llvm::SelectionDAG::getNode(unsigned int, llvm::SDLoc const&, llvm::EVT, llvm::SDValue, llvm::SDValue, llvm::SDValue, llvm::SDNodeFlags) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x3855672)
#11 0x00000000038e3ac1 llvm::DAGTypeLegalizer::PromoteIntRes_SETCC(llvm::SDNode*) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x38e3ac1)
#12 0x00000000038e04ca llvm::DAGTypeLegalizer::PromoteIntegerResult(llvm::SDNode*, unsigned int) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x38e04ca)
#13 0x0000000003892b61 llvm::DAGTypeLegalizer::run() (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x3892b61)
#14 0x0000000003898985 llvm::SelectionDAG::LegalizeTypes() (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x3898985)
#15 0x0000000003880a97 llvm::SelectionDAGISel::CodeGenAndEmitDAG() (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x3880a97)
#16 0x000000000387f5c9 llvm::SelectionDAGISel::SelectAllBasicBlocks(llvm::Function const&) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x387f5c9)
#17 0x000000000387c287 llvm::SelectionDAGISel::runOnMachineFunction(llvm::MachineFunction&) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x387c287)
#18 0x0000000001ec42dd llvm::MachineFunctionPass::runOnFunction(llvm::Function&) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x1ec42dd)
#19 0x0000000002300ba8 llvm::FPPassManager::runOnFunction(llvm::Function&) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x2300ba8)
#20 0x0000000002307381 llvm::FPPassManager::runOnModule(llvm::Module&) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x2307381)
#21 0x00000000023011cc llvm::legacy::PassManagerImpl::run(llvm::Module&) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x23011cc)
#22 0x0000000002bdff36 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::DataLayout const&, llvm::Module*, clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x2bdff36)
#23 0x000000000343d5fc clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x343d5fc)
#24 0x0000000003b62a24 clang::ParseAST(clang::Sema&, bool, bool) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x3b62a24)
#25 0x00000000033a15f0 clang::FrontendAction::Execute() (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x33a15f0)
#26 0x00000000032fd85a clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x32fd85a)
#27 0x00000000034375b8 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x34375b8)
#28 0x000000000183851b cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x183851b)
#29 0x0000000001836432 ExecuteCC1Tool(llvm::SmallVectorImpl<char const*>&) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x1836432)
#30 0x00000000031abbd2 void llvm::function_ref<void ()>::callback_fn<clang::driver::CC1Command::Execute(llvm::ArrayRef<llvm::Optional<llvm::StringRef> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, bool*) const::$_1>(long) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x31abbd2)
#31 0x0000000002935db7 llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x2935db7)
#32 0x00000000031ab2e7 clang::driver::CC1Command::Execute(llvm::ArrayRef<llvm::Optional<llvm::StringRef> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, bool*) const (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x31ab2e7)
#33 0x0000000003172495 clang::driver::Compilation::ExecuteCommand(clang::driver::Command const&, clang::driver::Command const*&) const (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x3172495)
#34 0x0000000003172787 clang::driver::Compilation::ExecuteJobs(clang::driver::JobList const&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*> >&) const (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x3172787)
#35 0x000000000318bb38 clang::driver::Driver::ExecuteCompilation(clang::driver::Compilation&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*> >&) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x318bb38)
#36 0x0000000001835d8d main (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x1835d8d)
#37 0x00007f27c633d0b3 __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b3)
#38 0x000000000183325e _start (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x183325e)
clang-12: error: clang frontend command failed with exit code 134 (use -v to see invocation)
ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project b208e5bcd0be5ffb6346b1eab30ad372782bbe4b)
Target: riscv64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/nathan/cbl/github/tc-build/build/llvm/stage1/bin
clang-12: note: diagnostic msg: Error generating preprocessed source(s) - no preprocessable inputs.

[craig.topper]

In D95322#2523189, @nathanchance wrote:

This patch causes issues with building the Linux kernel. cvise spits out:

$ cat pci.i
char pci_cache_line_size, pci_set_cacheline_size_dev_cacheline_size;
pci_set_cacheline_size_dev() {
  if (pci_set_cacheline_size_dev_cacheline_size % pci_cache_line_size)
    return 0;
  return 2;
}

$ clang -O2 --target=riscv64-linux-gnu -c -o /dev/null pci.i
clang: /home/nathan/cbl/github/tc-build/llvm-project/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:5744: llvm::SDValue llvm::SelectionDAG::getNode(unsigned int, const llvm::SDLoc &, llvm::EVT, llvm::SDValue, llvm::SDValue, llvm::SDValue, const llvm::SDNodeFlags): Assertion `N1.getValueType() == N2.getValueType() && "SETCC operands must have the same type!"' failed.
PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.      Program arguments: /home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang -O2 --target=riscv64-linux-gnu -c -o /dev/null pci.i
1.      <eof> parser at end of file
2.      Code generation
3.      Running pass 'Function Pass Manager' on module 'pci.i'.
4.      Running pass 'RISCV DAG->DAG Pattern Instruction Selection' on function '@pci_set_cacheline_size_dev'
 #0 0x00000000029a6673 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x29a6673)
 #1 0x00000000029a443e llvm::sys::RunSignalHandlers() (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x29a443e)
 #2 0x00000000029a5a0d llvm::sys::CleanupOnSignal(unsigned long) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x29a5a0d)
 #3 0x0000000002935ea3 (anonymous namespace)::CrashRecoveryContextImpl::HandleCrash(int, unsigned long) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x2935ea3)
 #4 0x0000000002935fde CrashRecoverySignalHandler(int) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x2935fde)
 #5 0x00007f27c68973c0 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x153c0)
 #6 0x00007f27c635c18b raise (/lib/x86_64-linux-gnu/libc.so.6+0x4618b)
 #7 0x00007f27c633b859 abort (/lib/x86_64-linux-gnu/libc.so.6+0x25859)
 #8 0x00007f27c633b729 (/lib/x86_64-linux-gnu/libc.so.6+0x25729)
 #9 0x00007f27c634cf36 (/lib/x86_64-linux-gnu/libc.so.6+0x36f36)
#10 0x0000000003855672 llvm::SelectionDAG::getNode(unsigned int, llvm::SDLoc const&, llvm::EVT, llvm::SDValue, llvm::SDValue, llvm::SDValue, llvm::SDNodeFlags) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x3855672)
#11 0x00000000038e3ac1 llvm::DAGTypeLegalizer::PromoteIntRes_SETCC(llvm::SDNode*) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x38e3ac1)
#12 0x00000000038e04ca llvm::DAGTypeLegalizer::PromoteIntegerResult(llvm::SDNode*, unsigned int) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x38e04ca)
#13 0x0000000003892b61 llvm::DAGTypeLegalizer::run() (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x3892b61)
#14 0x0000000003898985 llvm::SelectionDAG::LegalizeTypes() (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x3898985)
#15 0x0000000003880a97 llvm::SelectionDAGISel::CodeGenAndEmitDAG() (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x3880a97)
#16 0x000000000387f5c9 llvm::SelectionDAGISel::SelectAllBasicBlocks(llvm::Function const&) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x387f5c9)
#17 0x000000000387c287 llvm::SelectionDAGISel::runOnMachineFunction(llvm::MachineFunction&) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x387c287)
#18 0x0000000001ec42dd llvm::MachineFunctionPass::runOnFunction(llvm::Function&) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x1ec42dd)
#19 0x0000000002300ba8 llvm::FPPassManager::runOnFunction(llvm::Function&) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x2300ba8)
#20 0x0000000002307381 llvm::FPPassManager::runOnModule(llvm::Module&) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x2307381)
#21 0x00000000023011cc llvm::legacy::PassManagerImpl::run(llvm::Module&) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x23011cc)
#22 0x0000000002bdff36 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::DataLayout const&, llvm::Module*, clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x2bdff36)
#23 0x000000000343d5fc clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x343d5fc)
#24 0x0000000003b62a24 clang::ParseAST(clang::Sema&, bool, bool) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x3b62a24)
#25 0x00000000033a15f0 clang::FrontendAction::Execute() (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x33a15f0)
#26 0x00000000032fd85a clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x32fd85a)
#27 0x00000000034375b8 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x34375b8)
#28 0x000000000183851b cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x183851b)
#29 0x0000000001836432 ExecuteCC1Tool(llvm::SmallVectorImpl<char const*>&) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x1836432)
#30 0x00000000031abbd2 void llvm::function_ref<void ()>::callback_fn<clang::driver::CC1Command::Execute(llvm::ArrayRef<llvm::Optional<llvm::StringRef> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, bool*) const::$_1>(long) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x31abbd2)
#31 0x0000000002935db7 llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x2935db7)
#32 0x00000000031ab2e7 clang::driver::CC1Command::Execute(llvm::ArrayRef<llvm::Optional<llvm::StringRef> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, bool*) const (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x31ab2e7)
#33 0x0000000003172495 clang::driver::Compilation::ExecuteCommand(clang::driver::Command const&, clang::driver::Command const*&) const (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x3172495)
#34 0x0000000003172787 clang::driver::Compilation::ExecuteJobs(clang::driver::JobList const&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*> >&) const (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x3172787)
#35 0x000000000318bb38 clang::driver::Driver::ExecuteCompilation(clang::driver::Compilation&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*> >&) (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x318bb38)
#36 0x0000000001835d8d main (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x1835d8d)
#37 0x00007f27c633d0b3 __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b3)
#38 0x000000000183325e _start (/home/nathan/cbl/github/tc-build/build/llvm/stage1/bin/clang+0x183325e)
clang-12: error: clang frontend command failed with exit code 134 (use -v to see invocation)
ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project b208e5bcd0be5ffb6346b1eab30ad372782bbe4b)
Target: riscv64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/nathan/cbl/github/tc-build/build/llvm/stage1/bin
clang-12: note: diagnostic msg: Error generating preprocessed source(s) - no preprocessable inputs.

Thanks for the report. Should be fixed after f9d7f77267bca055c6cc480065ca7dd9f768b948

[Jim]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

2201

I met a bug that removes a and mask.

typedef unsigned char uint8x2_t __attribute__((vector_size(2)));

uint8x2_t udiv(uint8x2_t *a, uint8x2_t *b) {
  return *a / *b;
}

Assembly look like:
divu a1, a3, a1
divu a0, a0, a2
slli a1, a1, 8
or a0, a0, a1 >> missing "and a0, a0, 256" before or operation

If element 0 of *b is zero, it a division by zero. a0 would be 0xffffffff.
So the result of or operation is incorrect.
Is it a undefined behavior?

[jrtc27]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

2201

For vectors, if any element of the divisor is zero, the operation has undefined behavior.

From LangRef.

llvm/lib/Target/RISCV/RISCVISelLowering.h

enum NodeType : unsigned {
   SplitF64,
   TAIL,
   // RV64I shifts, directly matching the semantics of the named RISC-V
   // instructions.
   SLLW,
   SRAW,
   SRLW,
   // 32-bit operations from RV64M that can't be simply matched with a pattern
-  // at instruction selection time.
+  // at instruction selection time. These have undefined behavior for division
+  // by 0 or overflow (divw) like their target independent counterparts.
   DIVW,
   DIVUW,
   REMUW,
   // RV64IB rotates, directly matching the semantics of the named RISC-V
   // instructions.
   ROLW,
   RORW,
   // RV64IB funnel shifts, with the semantics of the named RISC-V instructions,

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

if (!Subtarget.hasStdExtM()) {
     setOperationAction(ISD::SDIV, XLenVT, Expand);
     setOperationAction(ISD::UDIV, XLenVT, Expand);
     setOperationAction(ISD::SREM, XLenVT, Expand);
     setOperationAction(ISD::UREM, XLenVT, Expand);
   }
 
   if (Subtarget.is64Bit() && Subtarget.hasStdExtM()) {
     setOperationAction(ISD::MUL, MVT::i32, Custom);
+
+    setOperationAction(ISD::SDIV, MVT::i8, Custom);
+    setOperationAction(ISD::UDIV, MVT::i8, Custom);
+    setOperationAction(ISD::UREM, MVT::i8, Custom);
+    setOperationAction(ISD::SDIV, MVT::i16, Custom);
+    setOperationAction(ISD::UDIV, MVT::i16, Custom);
+    setOperationAction(ISD::UREM, MVT::i16, Custom);
     setOperationAction(ISD::SDIV, MVT::i32, Custom);
     setOperationAction(ISD::UDIV, MVT::i32, Custom);
     setOperationAction(ISD::UREM, MVT::i32, Custom);
   }
 
   setOperationAction(ISD::SDIVREM, XLenVT, Expand);
   setOperationAction(ISD::UDIVREM, XLenVT, Expand);
   setOperationAction(ISD::SMUL_LOHI, XLenVT, Expand);
static RISCVISD::NodeType getRISCVWOpcode(unsigned Opcode) {
   }
 }
 
 // Converts the given 32-bit operation to a target-specific SelectionDAG node.
 // Because i32 isn't a legal type for RV64, these operations would otherwise
 // be promoted to i64, making it difficult to select the SLLW/DIVUW/.../*W
 // later one because the fact the operation was originally of type i32 is
 // lost.
-static SDValue customLegalizeToWOp(SDNode *N, SelectionDAG &DAG) {
+static SDValue customLegalizeToWOp(SDNode *N, SelectionDAG &DAG,
+                                   unsigned ExtOpc = ISD::ANY_EXTEND) {
   SDLoc DL(N);
   RISCVISD::NodeType WOpcode = getRISCVWOpcode(N->getOpcode());
-  SDValue NewOp0 = DAG.getNode(ISD::ANY_EXTEND, DL, MVT::i64, N->getOperand(0));
-  SDValue NewOp1 = DAG.getNode(ISD::ANY_EXTEND, DL, MVT::i64, N->getOperand(1));
+  SDValue NewOp0 = DAG.getNode(ExtOpc, DL, MVT::i64, N->getOperand(0));
+  SDValue NewOp1 = DAG.getNode(ExtOpc, DL, MVT::i64, N->getOperand(1));
   SDValue NewRes = DAG.getNode(WOpcode, DL, MVT::i64, NewOp0, NewOp1);
   // ReplaceNodeResults requires we maintain the same type for the return value.
   return DAG.getNode(ISD::TRUNCATE, DL, MVT::i32, NewRes);
 }
 
 // Converts the given 32-bit operation to a i64 operation with signed extension
 // semantic to reduce the signed extension instructions.
 static SDValue customLegalizeToWOpWithSExt(SDNode *N, SelectionDAG &DAG) {
void RISCVTargetLowering::ReplaceNodeResults(SDNode *N,
   case ISD::ROTL:
   case ISD::ROTR:
     assert(N->getValueType(0) == MVT::i32 && Subtarget.is64Bit() &&
            "Unexpected custom legalisation");
     Results.push_back(customLegalizeToWOp(N, DAG));
     break;
   case ISD::SDIV:
   case ISD::UDIV:
-  case ISD::UREM:
-    assert(N->getValueType(0) == MVT::i32 && Subtarget.is64Bit() &&
-           Subtarget.hasStdExtM() && "Unexpected custom legalisation");
+  case ISD::UREM: {
+    MVT VT = N->getSimpleValueType(0);
+    assert((VT == MVT::i8 || VT == MVT::i16 || VT == MVT::i32) &&
+           Subtarget.is64Bit() && Subtarget.hasStdExtM() &&
+           "Unexpected custom legalisation");
     if (N->getOperand(0).getOpcode() == ISD::Constant ||
         N->getOperand(1).getOpcode() == ISD::Constant)
       return;
-    Results.push_back(customLegalizeToWOp(N, DAG));
+
+    // If the input is i32, use ANY_EXTEND since the W instructions don't read
+    // the upper 32 bits. For other types we need to sign or zero extend
+    // based on the opcode.
+    unsigned ExtOpc = ISD::ANY_EXTEND;
+    if (VT != MVT::i32)
+      ExtOpc = N->getOpcode() == ISD::SDIV ? ISD::SIGN_EXTEND
+                                           : ISD::ZERO_EXTEND;
+
+    Results.push_back(customLegalizeToWOp(N, DAG, ExtOpc));
     break;
+  }
   case ISD::BITCAST: {
     assert(((N->getValueType(0) == MVT::i32 && Subtarget.is64Bit() &&
              Subtarget.hasStdExtF()) ||
             (N->getValueType(0) == MVT::i16 && Subtarget.hasStdExtZfh())) &&
            "Unexpected custom legalisation");
     SDValue Op0 = N->getOperand(0);
     if (N->getValueType(0) == MVT::i16 && Subtarget.hasStdExtZfh()) {
       if (Op0.getValueType() != MVT::f16)
bool RISCVTargetLowering::targetShrinkDemandedConstant(
   return TLO.CombineTo(Op, NewOp);
 }
 
 void RISCVTargetLowering::computeKnownBitsForTargetNode(const SDValue Op,
                                                         KnownBits &Known,
                                                         const APInt &DemandedElts,
                                                         const SelectionDAG &DAG,
                                                         unsigned Depth) const {
+  unsigned BitWidth = Known.getBitWidth();
   unsigned Opc = Op.getOpcode();
   assert((Opc >= ISD::BUILTIN_OP_END ||
           Opc == ISD::INTRINSIC_WO_CHAIN ||
           Opc == ISD::INTRINSIC_W_CHAIN ||
           Opc == ISD::INTRINSIC_VOID) &&
          "Should use MaskedValueIsZero if you don't know whether Op"
          " is a target node!");
 
   Known.resetAll();
   switch (Opc) {
   default: break;
+  case RISCVISD::REMUW: {
+    KnownBits Known2;
+    Known = DAG.computeKnownBits(Op.getOperand(0), DemandedElts, Depth + 1);
+    Known2 = DAG.computeKnownBits(Op.getOperand(1), DemandedElts, Depth + 1);
+    // We only care about the lower 32 bits.
+    Known = KnownBits::urem(Known.trunc(32), Known2.trunc(32));
+    // Restore the original width by sign extending.
+    Known = Known.sext(BitWidth);
+    break;
+  }
+  case RISCVISD::DIVUW: {
+    KnownBits Known2;
+    Known = DAG.computeKnownBits(Op.getOperand(0), DemandedElts, Depth + 1);
+    Known2 = DAG.computeKnownBits(Op.getOperand(1), DemandedElts, Depth + 1);
+    // We only care about the lower 32 bits.
+    Known = KnownBits::udiv(Known.trunc(32), Known2.trunc(32));
+    // Restore the original width by sign extending.
+    Known = Known.sext(BitWidth);
+    break;
+  }
   case RISCVISD::READ_VLENB:
     // We assume VLENB is at least 8 bytes.
     // FIXME: The 1.0 draft spec defines minimum VLEN as 128 bits.
     Known.Zero.setLowBits(3);
     break;
   }
 }
 

llvm/test/CodeGen/RISCV/rv64m-exhaustive-w-insts.ll

 ; RV64IM-NEXT:    ret
   %1 = udiv i32 %a, %b
   ret i32 %1
 }
 
 define zeroext i8 @zext_divuw_zext_zext_i8(i8 zeroext %a, i8 zeroext %b) nounwind {
 ; RV64IM-LABEL: zext_divuw_zext_zext_i8:
 ; RV64IM:       # %bb.0:
-; RV64IM-NEXT:    divu a0, a0, a1
+; RV64IM-NEXT:    divuw a0, a0, a1
 ; RV64IM-NEXT:    ret
   %1 = udiv i8 %a, %b
   ret i8 %1
 }
 
 define zeroext i16 @zext_divuw_zext_zext_i16(i16 zeroext %a, i16 zeroext %b) nounwind {
 ; RV64IM-LABEL: zext_divuw_zext_zext_i16:
 ; RV64IM:       # %bb.0:
-; RV64IM-NEXT:    divu a0, a0, a1
+; RV64IM-NEXT:    divuw a0, a0, a1
 ; RV64IM-NEXT:    ret
   %1 = udiv i16 %a, %b
   ret i16 %1
 }
 
 define i32 @aext_divw_aext_aext(i32 %a, i32 %b) nounwind {
 ; RV64IM-LABEL: aext_divw_aext_aext:
 ; RV64IM:       # %bb.0:

 ; RV64IM-NEXT:    ret
   %1 = sdiv i32 %a, %b
   ret i32 %1
 }
 
 define signext i8 @sext_divw_sext_sext_i8(i8 signext %a, i8 signext %b) nounwind {
 ; RV64IM-LABEL: sext_divw_sext_sext_i8:
 ; RV64IM:       # %bb.0:
-; RV64IM-NEXT:    div a0, a0, a1
-; RV64IM-NEXT:    slli a0, a0, 56
-; RV64IM-NEXT:    srai a0, a0, 56
+; RV64IM-NEXT:    divw a0, a0, a1
 ; RV64IM-NEXT:    ret
   %1 = sdiv i8 %a, %b
   ret i8 %1
 }
 
 define signext i16 @sext_divw_sext_sext_i16(i16 signext %a, i16 signext %b) nounwind {
 ; RV64IM-LABEL: sext_divw_sext_sext_i16:
 ; RV64IM:       # %bb.0:
-; RV64IM-NEXT:    div a0, a0, a1
-; RV64IM-NEXT:    slli a0, a0, 48
-; RV64IM-NEXT:    srai a0, a0, 48
+; RV64IM-NEXT:    divw a0, a0, a1
 ; RV64IM-NEXT:    ret
   %1 = sdiv i16 %a, %b
   ret i16 %1
 }
 
 define i32 @aext_remw_aext_aext(i32 %a, i32 %b) nounwind {
 ; RV64IM-LABEL: aext_remw_aext_aext:
 ; RV64IM:       # %bb.0:

 ; RV64IM-NEXT:    ret
   %1 = urem i32 %a, %b
   ret i32 %1
 }
 
 define zeroext i8 @zext_remuw_zext_zext_i8(i8 zeroext %a, i8 zeroext %b) nounwind {
 ; RV64IM-LABEL: zext_remuw_zext_zext_i8:
 ; RV64IM:       # %bb.0:
-; RV64IM-NEXT:    remu a0, a0, a1
+; RV64IM-NEXT:    remuw a0, a0, a1
 ; RV64IM-NEXT:    ret
   %1 = urem i8 %a, %b
   ret i8 %1
 }
 
 define zeroext i16 @zext_remuw_zext_zext_i16(i16 zeroext %a, i16 zeroext %b) nounwind {
 ; RV64IM-LABEL: zext_remuw_zext_zext_i16:
 ; RV64IM:       # %bb.0:
-; RV64IM-NEXT:    remu a0, a0, a1
+; RV64IM-NEXT:    remuw a0, a0, a1
 ; RV64IM-NEXT:    ret
   %1 = urem i16 %a, %b
   ret i16 %1
 }