[mips] Clang generates unaligned offset for MSA instruction st.d

[mamidzic]

Summary

Issue was found during compilation of V8 code inside android project with clang-3.9.
The folowing messages was shown:

/tmp/preprocessor-v8-msa-repro-f130c4.s: Assembler messages:
/tmp/preprocessor-v8-msa-repro-f130c4.s:33112: Error: operand 2 out of range `st.d $w0,23($16)'
clang-3.9: error: assembler command failed with exit code 1 (use -v to see invocation)

This issue is also manifesting on the latest upstream clang. More information here.
This patch implements checks for unaligned and out of range offsets for ST.* and LD.* Mips MSA instructions.

[dsanders]

lib/Target/Mips/MipsInstrInfo.td

1076–1081

For consistency with an existing microMIPS ComplexPattern (addrimm4lsl2) and the mem_simm_lsl* operands could we spell this 'addrimm10lsl2'?

lib/Target/Mips/MipsSEISelDAGToDAG.cpp

483

EliminateFI -> eliminateFrameIndex?

488–492

We shouldn't modify offsets in this code, we should only accept valid ones. Modifying the offset breaks the address calculation. One example of this is the O32 calling convention which can legitimately produce offsets that are unsuitable for the offset field of ld.[bhwd] and st.[bhwd] instructions (the ABI's stack alignment is insufficient to be able to guarantee that a vector argument/result is fully aligned). These instructions can still be used so long as the offset is added to the base address in a separate instruction since the spec permits unaligned addresses.

This code needs to fall back on selectAddrDefault() whenever the offset is inappropriate for the instructions instead of modifying the offset. We also need to do the alignment check in selectAddrFrameIndexOffset() so that we can account for the increased range that the scaled offsets allow.

test/CodeGen/Mips/msa/i5_ld_st.ll

68

As noted above, it's incorrect to use an offset other than 9. The expected code should be something like:

addiu $3, $2, 9
ld.h $4, 0($3)

Similarly for the ld.[hwd] and st.[hwd] cases.

[mamidzic]

lib/Target/Mips/MipsSEISelDAGToDAG.cpp

483

Well both exist. There is MipsRegisterInfo::eliminateFrameIndex and MipsSERegisterInfo::eliminateFI. eliminateFrameIndex only calls eliminateFI and then the offset is actually being calculated.

488–492

I have done the fall back on selectAddrDefault() when the offset is wrong, but I'm not sure what did you mean with the last sentence. Could you elaborate a bit?

[mamidzic]

Changed the names of operands from addrimm10al[2,4,8] to addrimm10lsl[1,2,3].
Removed the changing of offsets in selectIntAddrMSAAlign(). If the unaligned offset is detected, selectAddrDefault() is now called.
Changed tests so that they reflect the changes in the patch.

[dsanders]

lib/Target/Mips/MipsSEISelDAGToDAG.cpp

483

I couldn't find it because I was searching for 'EliminateFI' from the comment instead of the 'eliminateFI' found in our backend. It's probably best to refer to eliminateFrameIndex since that's the interface with the target independent code but 'eliminateFI' is ok too as long as we correct the upper case 'E'

488–492

Thanks.

I have done the fall back on selectAddrDefault() when the offset is wrong, but I'm not sure what did you mean with the last sentence. Could you elaborate a bit?

The current code checks for an offset that is a simm10 and is a multiple of the alignment but it should be checking for a simm10/simm11/simm12/simm13 (depending on the alignment) that is a multiple of the alignment. To put this another way, we need to check that the offset is a multiple of the alignment and that Offset / Alignment is a simm10.

For example, the offsets have to follow these constraints:

Instruction	>=	<=	Multiple of
lw.b	-1024	1023	1
lw.h	-2048	2046	2
lw.w	-4096	4092	4
lw.d	-8192	8184	8

but the current implementation implements these rules:

Instruction	>=	<=	Multiple of
lw.b	-1024	1023	1
lw.h	-1024	1022	2
lw.w	-1024	1020	4
lw.d	-1024	1016	8

[mamidzic]

Changed name of selectIntAddrMSA to selectIntAddrMSAAlign1, also relocated to be grouped with other MSAAlign functions.
Changed name of selectAddrRegImm10 to SelectAddrRegImmMSA because it's only used with st.* and ld.* instructions and it's not always Imm10. Also relocated it to be grouped with other MSA related functions.
Implemented checking of offset range for st.* and ld.* instructions.
Added valid and invalid offset range tests for st.* and ld.* instructions.
Fixed some indentation mistakes and typos.

[mamidzic]

lib/Target/Mips/MipsSEISelDAGToDAG.cpp

488–492

If I'm not mistaken, simm10 range is from -512 to 511 so I assumed you meant:

Instruction	>=	<=	Multiple of
lw.b	-512	511	1
lw.h	-1024	1022	2
lw.w	-2048	2044	4
lw.d	-4096	4088	8

[dsanders]

I think we're nearly there. My main concern is the fragility of the alignment check if we do it where it currently is.

lib/Target/Mips/MipsISelDAGToDAG.h

81–94

One last naming nit: We ought to switch out the mention of MSA for a mention of the immediate size and shift amount (e.g. selectAddrSimm10Lsl2) since these can be re-used for non-MSA purposes too. Removing the reference to 'MSA' should prevent people thinking it has MSA-specific special cases in the future.

lib/Target/Mips/MipsSEISelDAGToDAG.cpp

482–502

If we have a shift amount rather than the alignment (like in the MC layer) then we can directly use something like '10 + ShiftAmount' rather than having to convert.

488–492

Yes, you're right.

507–515

I think we still need to sink the alignment check into selectAddrFrameIndexOffset() to make this check robust.

In the current code, if selectAddrRegImmMSA() returns 'Base' as something other than a FrameIndexSDNode such as (add ptr, 3) then the current code will accept invalid offsets again. Admittedly, the current implementation of selectAddrRegImmMSA() and the functions it calls can only match FrameIndexSDNode nodes right now but we should improve that as soon as possible.

[mamidzic]

Removed selectIntAddrMSAAlign and selectAddrRegImmMSA
Changed name of following functions:

selectIntAddrMSAAlign1->selectIntAddrSImm10
selectIntAddrMSAAlign2->selectIntAddrSImm10Lsl1
selectIntAddrMSAAlign4->selectIntAddrSImm10Lsl2
selectIntAddrMSAAlign8->selectIntAddrSImm10Lsl3

Added ShiftAmount parameter to the selectAddrFrameIndexOffset.
Alignment check is now done in selectAddrFrameIndexOffset.

[mamidzic]

Hi, are there any news regarding this patch?

[dsanders]

LGTM with a nit

lib/Target/Mips/MipsSEISelDAGToDAG.cpp

312

Please add a '!= 0' to make the intent of the check more obvious

lib/Target/Mips/MipsISelDAGToDAG.h

private:
   virtual bool selectIntAddrMM(SDValue Addr, SDValue &Base,
                                SDValue &Offset) const;
 
   virtual bool selectIntAddrLSL2MM(SDValue Addr, SDValue &Base,
                                    SDValue &Offset) const;
 
   /// Match addr+simm10 and addr
   virtual bool selectIntAddrMSA(SDValue Addr, SDValue &Base,
                                 SDValue &Offset) const;
 
+  virtual bool selectIntAddrMSAAlign(SDValue Addr, SDValue &Base,
+                                     SDValue &Offset, unsigned Align) const;
+
+  virtual bool selectIntAddrMSAAlign2(SDValue Addr, SDValue &Base,
+                                      SDValue &Offset) const;
+
+  virtual bool selectIntAddrMSAAlign4(SDValue Addr, SDValue &Base,
+                                      SDValue &Offset) const;
+
+  virtual bool selectIntAddrMSAAlign8(SDValue Addr, SDValue &Base,
+                                      SDValue &Offset) const;
+
   virtual bool selectAddr16(SDNode *Parent, SDValue N, SDValue &Base,
                             SDValue &Offset, SDValue &Alias);
 
   /// \brief Select constant vector splats.
   virtual bool selectVSplat(SDNode *N, APInt &Imm,
                             unsigned MinSizeInBits) const;
   /// \brief Select constant vector splats whose value fits in a uimm1.
   virtual bool selectVSplatUimm1(SDValue N, SDValue &Imm) const;

lib/Target/Mips/MipsISelDAGToDAG.cpp

 }
 
 bool MipsDAGToDAGISel::selectIntAddrMSA(SDValue Addr, SDValue &Base,
                                         SDValue &Offset) const {
   llvm_unreachable("Unimplemented function.");
   return false;
 }
 
+bool MipsDAGToDAGISel::selectIntAddrMSAAlign(
+     SDValue Addr, SDValue &Base, SDValue &Offset, unsigned Align) const {
+  llvm_unreachable("Unimplemented function.");
+  return false;
+}
+
+bool MipsDAGToDAGISel::selectIntAddrMSAAlign2(SDValue Addr, SDValue &Base,
+                                              SDValue &Offset) const {
+  llvm_unreachable("Unimplemented function.");
+  return false;
+}
+
+bool MipsDAGToDAGISel::selectIntAddrMSAAlign4(SDValue Addr, SDValue &Base,
+                                              SDValue &Offset) const {
+  llvm_unreachable("Unimplemented function.");
+  return false;
+}
+
+bool MipsDAGToDAGISel::selectIntAddrMSAAlign8(SDValue Addr, SDValue &Base,
+                                              SDValue &Offset) const {
+  llvm_unreachable("Unimplemented function.");
+  return false;
+}
+
 bool MipsDAGToDAGISel::selectAddr16(SDNode *Parent, SDValue N, SDValue &Base,
                                     SDValue &Offset, SDValue &Alias) {
   llvm_unreachable("Unimplemented function.");
   return false;
 }
 
 bool MipsDAGToDAGISel::selectVSplat(SDNode *N, APInt &Imm,
                                     unsigned MinSizeInBits) const {

lib/Target/Mips/MipsInstrInfo.td

 
 def addrRegReg :
   ComplexPattern<iPTR, 2, "selectAddrRegReg", [frameindex]>;
 
 def addrDefault :
   ComplexPattern<iPTR, 2, "selectAddrDefault", [frameindex]>;
 
 def addrimm10 : ComplexPattern<iPTR, 2, "selectIntAddrMSA", [frameindex]>;
+def addrimm10al2 : ComplexPattern<iPTR, 2, "selectIntAddrMSAAlign2",
+                                  [frameindex]>;
+def addrimm10al4 : ComplexPattern<iPTR, 2, "selectIntAddrMSAAlign4",
+                                  [frameindex]>;
+def addrimm10al8 : ComplexPattern<iPTR, 2, "selectIntAddrMSAAlign8",
+                                  [frameindex]>;
 
 //===----------------------------------------------------------------------===//
 // Instructions specific format
 //===----------------------------------------------------------------------===//
 
 // Arithmetic and logical instructions with 3 register operands.
 class ArithLogicR<string opstr, RegisterOperand RO, bit isComm = 0,
                   InstrItinClass Itin = NoItinerary,

lib/Target/Mips/MipsMSAInstrInfo.td

class LD_DESC_BASE<string instr_asm, SDPatternOperator OpNode,
   dag InOperandList = (ins MemOpnd:$addr);
   string AsmString = !strconcat(instr_asm, "\t$wd, $addr");
   list<dag> Pattern = [(set ROWD:$wd, (TyNode (OpNode Addr:$addr)))];
   InstrItinClass Itinerary = itin;
   string DecoderMethod = "DecodeMSA128Mem";
 }
 
 class LD_B_DESC : LD_DESC_BASE<"ld.b", load, v16i8, MSA128BOpnd, mem_simm10>;
-class LD_H_DESC : LD_DESC_BASE<"ld.h", load, v8i16, MSA128HOpnd, mem_simm10_lsl1>;
-class LD_W_DESC : LD_DESC_BASE<"ld.w", load, v4i32, MSA128WOpnd, mem_simm10_lsl2>;
-class LD_D_DESC : LD_DESC_BASE<"ld.d", load, v2i64, MSA128DOpnd, mem_simm10_lsl3>;
+class LD_H_DESC : LD_DESC_BASE<"ld.h", load, v8i16, MSA128HOpnd,
+                               mem_simm10_lsl1, addrimm10al2>;
+class LD_W_DESC : LD_DESC_BASE<"ld.w", load, v4i32, MSA128WOpnd,
+                               mem_simm10_lsl2, addrimm10al4>;
+class LD_D_DESC : LD_DESC_BASE<"ld.d", load, v2i64, MSA128DOpnd,
+                               mem_simm10_lsl3, addrimm10al8>;
 
 class LDI_B_DESC : MSA_I10_LDI_DESC_BASE<"ldi.b", MSA128BOpnd>;
 class LDI_H_DESC : MSA_I10_LDI_DESC_BASE<"ldi.h", MSA128HOpnd>;
 class LDI_W_DESC : MSA_I10_LDI_DESC_BASE<"ldi.w", MSA128WOpnd>;
 class LDI_D_DESC : MSA_I10_LDI_DESC_BASE<"ldi.d", MSA128DOpnd>;
 
 class LSA_DESC_BASE<string instr_asm, RegisterOperand RORD,
                     RegisterOperand RORS = RORD, RegisterOperand RORT = RORD,
class ST_DESC_BASE<string instr_asm, SDPatternOperator OpNode,
   dag InOperandList = (ins ROWD:$wd, MemOpnd:$addr);
   string AsmString = !strconcat(instr_asm, "\t$wd, $addr");
   list<dag> Pattern = [(OpNode (TyNode ROWD:$wd), Addr:$addr)];
   InstrItinClass Itinerary = itin;
   string DecoderMethod = "DecodeMSA128Mem";
 }
 
 class ST_B_DESC : ST_DESC_BASE<"st.b", store, v16i8, MSA128BOpnd, mem_simm10>;
-class ST_H_DESC : ST_DESC_BASE<"st.h", store, v8i16, MSA128HOpnd, mem_simm10_lsl1>;
-class ST_W_DESC : ST_DESC_BASE<"st.w", store, v4i32, MSA128WOpnd, mem_simm10_lsl2>;
-class ST_D_DESC : ST_DESC_BASE<"st.d", store, v2i64, MSA128DOpnd, mem_simm10_lsl3>;
+class ST_H_DESC : ST_DESC_BASE<"st.h", store, v8i16, MSA128HOpnd,
+                               mem_simm10_lsl1, addrimm10al2>;
+class ST_W_DESC : ST_DESC_BASE<"st.w", store, v4i32, MSA128WOpnd,
+                               mem_simm10_lsl2, addrimm10al4>;
+class ST_D_DESC : ST_DESC_BASE<"st.d", store, v2i64, MSA128DOpnd,
+                               mem_simm10_lsl3, addrimm10al8>;
 
 class SUBS_S_B_DESC : MSA_3R_DESC_BASE<"subs_s.b", int_mips_subs_s_b,
                                        MSA128BOpnd>;
 class SUBS_S_H_DESC : MSA_3R_DESC_BASE<"subs_s.h", int_mips_subs_s_h,
                                        MSA128HOpnd>;
 class SUBS_S_W_DESC : MSA_3R_DESC_BASE<"subs_s.w", int_mips_subs_s_w,
                                        MSA128WOpnd>;
 class SUBS_S_D_DESC : MSA_3R_DESC_BASE<"subs_s.d", int_mips_subs_s_d,

 
 // Patterns.
 class MSAPat<dag pattern, dag result, list<Predicate> pred = [HasMSA]> :
   Pat<pattern, result>, Requires<pred>;
 
 def : MSAPat<(extractelt (v4i32 MSA128W:$ws), immZExt4:$idx),
              (COPY_S_W MSA128W:$ws, immZExt4:$idx)>;
 
-def : MSAPat<(v8f16 (load addrimm10:$addr)), (LD_H addrimm10:$addr)>;
-def : MSAPat<(v4f32 (load addrimm10:$addr)), (LD_W addrimm10:$addr)>;
-def : MSAPat<(v2f64 (load addrimm10:$addr)), (LD_D addrimm10:$addr)>;
-
-def ST_FH : MSAPat<(store (v8f16 MSA128H:$ws), addrimm10:$addr),
-                   (ST_H MSA128H:$ws, addrimm10:$addr)>;
-def ST_FW : MSAPat<(store (v4f32 MSA128W:$ws), addrimm10:$addr),
-                   (ST_W MSA128W:$ws, addrimm10:$addr)>;
-def ST_FD : MSAPat<(store (v2f64 MSA128D:$ws), addrimm10:$addr),
-                   (ST_D MSA128D:$ws, addrimm10:$addr)>;
+def : MSAPat<(v8f16 (load addrimm10al2:$addr)), (LD_H addrimm10al2:$addr)>;
+def : MSAPat<(v4f32 (load addrimm10al4:$addr)), (LD_W addrimm10al4:$addr)>;
+def : MSAPat<(v2f64 (load addrimm10al8:$addr)), (LD_D addrimm10al8:$addr)>;
+
+def ST_FH : MSAPat<(store (v8f16 MSA128H:$ws), addrimm10al2:$addr),
+                   (ST_H MSA128H:$ws, addrimm10al2:$addr)>;
+def ST_FW : MSAPat<(store (v4f32 MSA128W:$ws), addrimm10al4:$addr),
+                   (ST_W MSA128W:$ws, addrimm10al4:$addr)>;
+def ST_FD : MSAPat<(store (v2f64 MSA128D:$ws), addrimm10al8:$addr),
+                   (ST_D MSA128D:$ws, addrimm10al8:$addr)>;
 
 class MSA_FABS_PSEUDO_DESC_BASE<RegisterOperand ROWD,
                                 RegisterOperand ROWS = ROWD,
                                 InstrItinClass itin = NoItinerary> :
   MSAPseudo<(outs ROWD:$wd),
             (ins ROWS:$ws),
             [(set ROWD:$wd, (fabs ROWS:$ws))]> {
   InstrItinClass Itinerary = itin;

lib/Target/Mips/MipsSEISelDAGToDAG.h

bool selectIntAddrMM(SDValue Addr, SDValue &Base,
                        SDValue &Offset) const override;
 
   bool selectIntAddrLSL2MM(SDValue Addr, SDValue &Base,
                            SDValue &Offset) const override;
 
   bool selectIntAddrMSA(SDValue Addr, SDValue &Base,
                         SDValue &Offset) const override;
 
+  bool selectIntAddrMSAAlign(SDValue Addr, SDValue &Base,
+                             SDValue &Offset, unsigned Align) const override;
+
+  bool selectIntAddrMSAAlign2(SDValue Addr, SDValue &Base,
+                              SDValue &Offset) const override;
+
+  bool selectIntAddrMSAAlign4(SDValue Addr, SDValue &Base,
+                              SDValue &Offset) const override;
+
+  bool selectIntAddrMSAAlign8(SDValue Addr, SDValue &Base,
+                              SDValue &Offset) const override;
+
   /// \brief Select constant vector splats.
   bool selectVSplat(SDNode *N, APInt &Imm,
                     unsigned MinSizeInBits) const override;
   /// \brief Select constant vector splats whose value fits in a given integer.
   bool selectVSplatCommon(SDValue N, SDValue &Imm, bool Signed,
                                   unsigned ImmBitSize) const;
   /// \brief Select constant vector splats whose value fits in a uimm1.
   bool selectVSplatUimm1(SDValue N, SDValue &Imm) const override;

lib/Target/Mips/MipsSEISelDAGToDAG.cpp

if (isIntN(OffsetBits, CN->getSExtValue())) {
       // If the first operand is a FI, get the TargetFI Node
       if (FrameIndexSDNode *FIN = dyn_cast<FrameIndexSDNode>
                                   (Addr.getOperand(0)))
         Base = CurDAG->getTargetFrameIndex(FIN->getIndex(), ValTy);
       else
         Base = Addr.getOperand(0);
 
       Offset = CurDAG->getTargetConstant(CN->getZExtValue(), SDLoc(Addr),
                                          ValTy);
       return true;
     }
   }
   return false;
 }
 
 /// ComplexPattern used on MipsInstrInfo
 /// Used on Mips Load/Store instructions
if (selectAddrRegImm10(Addr, Base, Offset))
     return true;
 
   if (selectAddrDefault(Addr, Base, Offset))
     return true;
 
   return false;
 }
 
+bool MipsSEDAGToDAGISel::selectIntAddrMSAAlign(
+     SDValue Addr, SDValue &Base, SDValue &Offset, unsigned Align) const {
+  if (selectAddrRegImm10(Addr, Base, Offset)){
+    // If base is a FI, additional offset calculation is done in EliminateFI.
+    if (!isa<FrameIndexSDNode>(Base)) {
+      if (ConstantSDNode *CN = dyn_cast<ConstantSDNode>(Offset)) {
+        unsigned CnstOff = CN->getZExtValue();
+        unsigned OffsetDiff =  OffsetToAlignment(CnstOff, Align);
+        // If offset is not aligned, change it to the first greater offset
+        // that is aligned.
+        if(OffsetDiff)
+          Offset = CurDAG->getTargetConstant(CnstOff + OffsetDiff, SDLoc(Addr),
+                                             Addr.getValueType());
+      }
+    }
+
+    return true;
+  }
+
+  return selectAddrDefault(Addr, Base, Offset);
+}
+
+bool MipsSEDAGToDAGISel::selectIntAddrMSAAlign2(SDValue Addr, SDValue &Base,
+                                                SDValue &Offset) const {
+  return selectIntAddrMSAAlign(Addr, Base, Offset, 2);
+}
+
+bool MipsSEDAGToDAGISel::selectIntAddrMSAAlign4(SDValue Addr, SDValue &Base,
+                                                SDValue &Offset) const {
+  return selectIntAddrMSAAlign(Addr, Base, Offset, 4);
+}
+
+bool MipsSEDAGToDAGISel::selectIntAddrMSAAlign8(SDValue Addr, SDValue &Base,
+                                                SDValue &Offset) const {
+  return selectIntAddrMSAAlign(Addr, Base, Offset, 8);
+}
+
 // Select constant vector splats.
 //
 // Returns true and sets Imm if:
 // * MSA is enabled
 // * N is a ISD::BUILD_VECTOR representing a constant splat
 bool MipsSEDAGToDAGISel::selectVSplat(SDNode *N, APInt &Imm,
                                       unsigned MinSizeInBits) const {
   if (!Subtarget->hasMSA())

test/CodeGen/Mips/msa/i5_ld_st.ll

 
 declare <16 x i8> @llvm.mips.ld.b(i8*, i32) nounwind
 
 ; CHECK: llvm_mips_ld_b_test:
 ; CHECK: ld.b [[R1:\$w[0-9]+]], 16(
 ; CHECK: st.b
 ; CHECK: .size llvm_mips_ld_b_test
 ;
+
+define void @llvm_mips_ld_b_unaligned_test() nounwind {
+entry:
+  %0 = bitcast <16 x i8>* @llvm_mips_ld_b_ARG to i8*
+  %1 = tail call <16 x i8> @llvm.mips.ld.b(i8* %0, i32 9)
+  store <16 x i8> %1, <16 x i8>* @llvm_mips_ld_b_RES
+  ret void
+}
+
+; CHECK: llvm_mips_ld_b_unaligned_test:
+; CHECK: ld.b [[R1:\$w[0-9]+]], 9(
+; CHECK: st.b
+; CHECK: .size llvm_mips_ld_b_unaligned_test
+;
+
 @llvm_mips_ld_h_ARG = global <8 x i16> <i16 0, i16 1, i16 2, i16 3, i16 4, i16 5, i16 6, i16 7>, align 16
 @llvm_mips_ld_h_RES = global <8 x i16> <i16 0, i16 0, i16 0, i16 0, i16 0, i16 0, i16 0, i16 0>, align 16
 
 define void @llvm_mips_ld_h_test() nounwind {
 entry:
   %0 = bitcast <8 x i16>* @llvm_mips_ld_h_ARG to i8*
   %1 = tail call <8 x i16> @llvm.mips.ld.h(i8* %0, i32 16)
   store <8 x i16> %1, <8 x i16>* @llvm_mips_ld_h_RES
   ret void
 }
 
 declare <8 x i16> @llvm.mips.ld.h(i8*, i32) nounwind
 
 ; CHECK: llvm_mips_ld_h_test:
 ; CHECK: ld.h [[R1:\$w[0-9]+]], 16(
 ; CHECK: st.h
 ; CHECK: .size llvm_mips_ld_h_test
 ;
+
+define void @llvm_mips_ld_h_unaligned_test() nounwind {
+entry:
+  %0 = bitcast <8 x i16>* @llvm_mips_ld_h_ARG to i8*
+  %1 = tail call <8 x i16> @llvm.mips.ld.h(i8* %0, i32 9)
+  store <8 x i16> %1, <8 x i16>* @llvm_mips_ld_h_RES
+  ret void
+}
+
+; CHECK: llvm_mips_ld_h_unaligned_test:
+; CHECK: ld.h [[R1:\$w[0-9]+]], 10(
+; CHECK: st.h
+; CHECK: .size llvm_mips_ld_h_unaligned_test
+;
+
 @llvm_mips_ld_w_ARG = global <4 x i32> <i32 0, i32 1, i32 2, i32 3>, align 16
 @llvm_mips_ld_w_RES = global <4 x i32> <i32 0, i32 0, i32 0, i32 0>, align 16
 
 define void @llvm_mips_ld_w_test() nounwind {
 entry:
   %0 = bitcast <4 x i32>* @llvm_mips_ld_w_ARG to i8*
   %1 = tail call <4 x i32> @llvm.mips.ld.w(i8* %0, i32 16)
   store <4 x i32> %1, <4 x i32>* @llvm_mips_ld_w_RES
   ret void
 }
 
 declare <4 x i32> @llvm.mips.ld.w(i8*, i32) nounwind
 
 ; CHECK: llvm_mips_ld_w_test:
 ; CHECK: ld.w [[R1:\$w[0-9]+]], 16(
 ; CHECK: st.w
 ; CHECK: .size llvm_mips_ld_w_test
 ;
 @llvm_mips_ld_d_ARG = global <2 x i64> <i64 0, i64 1>, align 16
 @llvm_mips_ld_d_RES = global <2 x i64> <i64 0, i64 0>, align 16
 
+define void @llvm_mips_ld_w_unaligned_test() nounwind {
+entry:
+  %0 = bitcast <4 x i32>* @llvm_mips_ld_w_ARG to i8*
+  %1 = tail call <4 x i32> @llvm.mips.ld.w(i8* %0, i32 9)
+  store <4 x i32> %1, <4 x i32>* @llvm_mips_ld_w_RES
+  ret void
+}
+
+; CHECK: llvm_mips_ld_w_unaligned_test:
+; CHECK: ld.w [[R1:\$w[0-9]+]], 12(
+; CHECK: st.w
+; CHECK: .size llvm_mips_ld_w_unaligned_test
+;
+
 define void @llvm_mips_ld_d_test() nounwind {
 entry:
   %0 = bitcast <2 x i64>* @llvm_mips_ld_d_ARG to i8*
   %1 = tail call <2 x i64> @llvm.mips.ld.d(i8* %0, i32 16)
   store <2 x i64> %1, <2 x i64>* @llvm_mips_ld_d_RES
   ret void
 }
 
 declare <2 x i64> @llvm.mips.ld.d(i8*, i32) nounwind
 
 ; CHECK: llvm_mips_ld_d_test:
 ; CHECK: ld.d [[R1:\$w[0-9]+]], 16(
 ; CHECK: st.d
 ; CHECK: .size llvm_mips_ld_d_test
 ;
+
+define void @llvm_mips_ld_d_unaligned_ttest() nounwind {
+entry:
+  %0 = bitcast <2 x i64>* @llvm_mips_ld_d_ARG to i8*
+  %1 = tail call <2 x i64> @llvm.mips.ld.d(i8* %0, i32 9)
+  store <2 x i64> %1, <2 x i64>* @llvm_mips_ld_d_RES
+  ret void
+}
+
+; CHECK: llvm_mips_ld_d_unaligned_ttest:
+; CHECK: ld.d [[R1:\$w[0-9]+]], 16(
+; CHECK: st.d
+; CHECK: .size llvm_mips_ld_d_unaligned_ttest
+;
+
 @llvm_mips_st_b_ARG = global <16 x i8> <i8 0, i8 1, i8 2, i8 3, i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15>, align 16
 @llvm_mips_st_b_RES = global <16 x i8> <i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0>, align 16
 
 define void @llvm_mips_st_b_test() nounwind {
 entry:
   %0 = load <16 x i8>, <16 x i8>* @llvm_mips_st_b_ARG
   %1 = bitcast <16 x i8>* @llvm_mips_st_b_RES to i8*
   tail call void @llvm.mips.st.b(<16 x i8> %0, i8* %1, i32 16)
   ret void
 }
 
 declare void @llvm.mips.st.b(<16 x i8>, i8*, i32) nounwind
 
 ; CHECK: llvm_mips_st_b_test:
 ; CHECK: ld.b
 ; CHECK: st.b [[R1:\$w[0-9]+]], 16(
 ; CHECK: .size llvm_mips_st_b_test
 ;
+
+define void @llvm_mips_st_b_unaligned_test() nounwind {
+entry:
+  %0 = load <16 x i8>, <16 x i8>* @llvm_mips_st_b_ARG
+  %1 = bitcast <16 x i8>* @llvm_mips_st_b_RES to i8*
+  tail call void @llvm.mips.st.b(<16 x i8> %0, i8* %1, i32 9)
+  ret void
+}
+
+; CHECK: llvm_mips_st_b_unaligned_test:
+; CHECK: ld.b
+; CHECK: st.b [[R1:\$w[0-9]+]], 9(
+; CHECK: .size llvm_mips_st_b_unaligned_test
+;
+
 @llvm_mips_st_h_ARG = global <8 x i16> <i16 0, i16 1, i16 2, i16 3, i16 4, i16 5, i16 6, i16 7>, align 16
 @llvm_mips_st_h_RES = global <8 x i16> <i16 0, i16 0, i16 0, i16 0, i16 0, i16 0, i16 0, i16 0>, align 16
 
 define void @llvm_mips_st_h_test() nounwind {
 entry:
   %0 = load <8 x i16>, <8 x i16>* @llvm_mips_st_h_ARG
   %1 = bitcast <8 x i16>* @llvm_mips_st_h_RES to i8*
   tail call void @llvm.mips.st.h(<8 x i16> %0, i8* %1, i32 16)
   ret void
 }
 
 declare void @llvm.mips.st.h(<8 x i16>, i8*, i32) nounwind
 
 ; CHECK: llvm_mips_st_h_test:
 ; CHECK: ld.h
 ; CHECK: st.h [[R1:\$w[0-9]+]], 16(
 ; CHECK: .size llvm_mips_st_h_test
 ;
+
+define void @llvm_mips_st_h_unaligned_test() nounwind {
+entry:
+  %0 = load <8 x i16>, <8 x i16>* @llvm_mips_st_h_ARG
+  %1 = bitcast <8 x i16>* @llvm_mips_st_h_RES to i8*
+  tail call void @llvm.mips.st.h(<8 x i16> %0, i8* %1, i32 9)
+  ret void
+}
+
+; CHECK: llvm_mips_st_h_unaligned_test:
+; CHECK: ld.h
+; CHECK: st.h [[R1:\$w[0-9]+]], 10(
+; CHECK: .size llvm_mips_st_h_unaligned_test
+;
+
 @llvm_mips_st_w_ARG = global <4 x i32> <i32 0, i32 1, i32 2, i32 3>, align 16
 @llvm_mips_st_w_RES = global <4 x i32> <i32 0, i32 0, i32 0, i32 0>, align 16
 
 define void @llvm_mips_st_w_test() nounwind {
 entry:
   %0 = load <4 x i32>, <4 x i32>* @llvm_mips_st_w_ARG
   %1 = bitcast <4 x i32>* @llvm_mips_st_w_RES to i8*
   tail call void @llvm.mips.st.w(<4 x i32> %0, i8* %1, i32 16)
   ret void
 }
 
 declare void @llvm.mips.st.w(<4 x i32>, i8*, i32) nounwind
 
 ; CHECK: llvm_mips_st_w_test:
 ; CHECK: ld.w
 ; CHECK: st.w [[R1:\$w[0-9]+]], 16(
 ; CHECK: .size llvm_mips_st_w_test
 ;
+
+define void @llvm_mips_st_w_unaligned_test() nounwind {
+entry:
+  %0 = load <4 x i32>, <4 x i32>* @llvm_mips_st_w_ARG
+  %1 = bitcast <4 x i32>* @llvm_mips_st_w_RES to i8*
+  tail call void @llvm.mips.st.w(<4 x i32> %0, i8* %1, i32 9)
+  ret void
+}
+
+; CHECK: llvm_mips_st_w_unaligned_test:
+; CHECK: ld.w
+; CHECK: st.w [[R1:\$w[0-9]+]], 12(
+; CHECK: .size llvm_mips_st_w_unaligned_test
+;
+
 @llvm_mips_st_d_ARG = global <2 x i64> <i64 0, i64 1>, align 16
 @llvm_mips_st_d_RES = global <2 x i64> <i64 0, i64 0>, align 16
 
 define void @llvm_mips_st_d_test() nounwind {
 entry:
   %0 = load <2 x i64>, <2 x i64>* @llvm_mips_st_d_ARG
   %1 = bitcast <2 x i64>* @llvm_mips_st_d_RES to i8*
   tail call void @llvm.mips.st.d(<2 x i64> %0, i8* %1, i32 16)
   ret void
 }
 
 declare void @llvm.mips.st.d(<2 x i64>, i8*, i32) nounwind
 
 ; CHECK: llvm_mips_st_d_test:
 ; CHECK: ld.d
 ; CHECK: st.d [[R1:\$w[0-9]+]], 16(
 ; CHECK: .size llvm_mips_st_d_test
 ;
+
+define void @llvm_mips_st_d_unaligned_test() nounwind {
+entry:
+  %0 = load <2 x i64>, <2 x i64>* @llvm_mips_st_d_ARG
+  %1 = bitcast <2 x i64>* @llvm_mips_st_d_RES to i8*
+  tail call void @llvm.mips.st.d(<2 x i64> %0, i8* %1, i32 9)
+  ret void
+}
+
+; CHECK: llvm_mips_st_d_unaligned_test:
+; CHECK: ld.d
+; CHECK: st.d [[R1:\$w[0-9]+]], 16(
+; CHECK: .size llvm_mips_st_d_unaligned_test
+;