Implementation of global.get/set for reftypes in LLVM IR

[pmatos]

Summary

This change implements new DAG notes GLOBAL_GET/GLOBAL_SET, and
lowering methods for load and stores of reference types from IR
globals. Once the lowering creates the new nodes, tablegen pattern
matches those and converts them to Wasm global.get/set.

[pmatos]

Fix getLegalType for simple MVTs

[pmatos]

Attempt to specify externref as a pointer type with a fixed size of 32, instead of opaque.

This doesn't work properly yet as visitLoad fails to getConstant for an externref because externref is not an integer - as in the MVT::externref is not defined as an integer externref.

Requires some more investigation and experimentation.

[pmatos]

Complete patch ready for review.

Includes testcases, handles externref and funcrefs.
Also handles lowering of a call of a funcref by lowering to a table.set + call_indirect.

[pmatos]

A few data points on the patch. Externrefs live in address space 1 and funcrefs in address space 3. Externref globals live in address space 2 and funcref globals live in address space 4.

On global.set, global.get:

• The patch creates new nodes for global.get and set. These nodes are create during lowering of loads (generating global.get) and stores (generating global.set).
• Selection happens through a Pattern that creates the respective GLOBAL_SET/GET_EXTERNREF/FUNCREF.

On lowering of funcref calls:

• A new node for table set is created. This is created during the call lowering and added to the chain of the call. Later on it's selected through a Pattern and the instruction TABLE_SET_FUNCREF is generated.

On middle end store/load optimizations: LLVM didn't like too much optimizing stores and loads of 0 size, therefore some optimization skipping was put in place so that we don't try to optimize loads and stores when the VT is sizeless.

[pmatos]

Remove commented code that shouldn't be there in the first place.

[lebedev.ri]

Why are IR-level changes needed if these are generated during lowering?
Will this play well with typeless pointers?

[pmatos]

In D95425#2632834, @lebedev.ri wrote:

Why are IR-level changes needed if these are generated during lowering?
Will this play well with typeless pointers?

Are you talking here about the changes in Codegen/? Many of the load/store optimizations expect a size for the pointee and when there's none LLVM crashes. All of the changes here is to skip optimizations that wouldn't work on sizeless pointees. I haven't tested this on typeless pointers, but I can say that only the size is checked therefore unless the typeless pointers point to a sizeless pointee, then nothing should change.

[wingo]

Really neat patch, glad to finally see codegen for externref!!! Comments are a bunch of nits / questions, really -- I am not an LLVM expert so please take them with a salt lick.

llvm/include/llvm/CodeGen/ValueTypes.h

213–215

Should this return false when isZeroSized() ?

llvm/lib/CodeGen/MachineOperand.cpp

1166–1167

Perhaps add isZeroSized predicate here, for readability.

Generally speaking, it might be nice to add assert(!isZeroSized()) to the implementation of all relevant getSize() methods, just to make sure you (and future developers) are alerted when they fail to explicitly handle the externref case.

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

4105 ↗

(On Diff #331366)

isZeroSized. or !hasSize ? Dunno.

4118 ↗

(On Diff #331366)

Comments like this make me think there is a property that we're currently calling "zero sized" but which probably has a better name. Anyway just a thought.

llvm/lib/IR/LLVMContextImpl.cpp

35 ↗

(On Diff #331366)

Probably something we don't want clang-format to do?

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1358

Symbolic names for address spaces needed here and below

1386

What is an externref store? A store of an externref value?

2383

What is this?

2394

What does alignment mean in the context of reference types? We can't spill them to the stack. Are you just specifying a least common denominator here? Comment needed in any case.

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.h

48

Need a big comment somewhere describing the relationship between address spaces and reftypes. Use symbolic names instead of 1 and 3 ?

llvm/lib/Target/WebAssembly/WebAssemblyUtilities.cpp

110 ↗

(On Diff #331366)

This would not be the case for the call_ref table, right? Rather it would have fixed size 1 and be defined as comdat.

llvm/test/CodeGen/WebAssembly/externref-globalget.ll

5

A design document for how this works would really be helpful :) So LLVM is going to consider that any pointer to address space 1 has MVT exterrnref at lowering-time? It's interesting to go down this route rather than adding a new llvm::Type.

llvm/test/CodeGen/WebAssembly/funcref-call.ll

18

We should clear the table entry after the call, to prevent hidden GC roots. It's overhead now but I would make it unconditional, in anticipation of call_ref.

[tlively]

In D95425#2636691, @pmatos wrote:

In D95425#2632834, @lebedev.ri wrote:

Why are IR-level changes needed if these are generated during lowering?
Will this play well with typeless pointers?

Are you talking here about the changes in Codegen/? Many of the load/store optimizations expect a size for the pointee and when there's none LLVM crashes. All of the changes here is to skip optimizations that wouldn't work on sizeless pointees. I haven't tested this on typeless pointers, but I can say that only the size is checked therefore unless the typeless pointers point to a sizeless pointee, then nothing should change.

I think this is referring to the changes in IR/ and Bitcode/ and llvm-c/. I am also surprised that there are changes in those locations. What motivated them?

[tlively]

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1388–1389

This seems a little strange. Why not allow 0? Once we support user-defined tables and not just individual globals, will this need to allow any non-negative offset?

2394

I would also expect Align(1) to make more sense here, since table slot indices do not need to be divisible by any particular alignment.

llvm/lib/Target/WebAssembly/WebAssemblyInstrCall.td

86–92 ↗

(On Diff #331366)

Is this used anywhere?

llvm/test/CodeGen/WebAssembly/externref-globalget.ll

5

We essentially consider changes to LLVM IR to be out of scope to the extent possible. If we only touch code in the WebAssembly backend and maybe fix a few bugs in target independent code, then we are comfortable signing off on patches within the team, but if we do anything non-trivial in target independent code, we need sign off from the wider LLVM community. Changes to LLVM IR would require a full-blown RFC process and I expect they would not be accepted.

llvm/test/CodeGen/WebAssembly/externref-undef.ll

14–15

I would expect this to declare an externref local and return it uninitialized or to return a ref.null. It would be good to at least add a TODO comment for fixing this.

[wingo]

llvm/test/CodeGen/WebAssembly/externref-globalget.ll

5

I guess my open question is really as regards the C mapping -- would the idea be for the front-end to also specially recognize pointers in address space 1 ? If there were a first-class IR type for these values, as is the case for SVE, you'd have a straightforward answer. More discussion in https://docs.google.com/document/d/1aVX0tQChxA2Tlno2KmEUjdruLoNgpwzV5Kv335HvNmI/edit#heading=h.u50o9qhzzjy6.

[pmatos]

llvm/lib/Target/WebAssembly/WebAssemblyInstrCall.td

86–92 ↗

(On Diff #331366)

Thanks for pointing this one out - I will remove this.

[pmatos]

Small fixes:

• Set Align(1) for reference types
• Remove unused node
• Make webassembly target description string dependent on feature string (fixes clang tests)

[pmatos]

Reformat patch with clang-format-diff.py

[lebedev.ri]

Once again, are the changes to the LLVM IR (LLVMTypeKind & friends) actually needed?
I'm not seeing any usages of that in the tests.

[pmatos]

In D95425#2643730, @lebedev.ri wrote:

Once again, are the changes to the LLVM IR (LLVMTypeKind & friends) actually needed?
I'm not seeing any usages of that in the tests.

Hi, I was going refer to those in a separate comment. My apologies for the delay.
Those were required to get this to work due to a call in SelectionDAGBuilder.cpp (current like 9966) to VT.getTypeForEVT. This requires me to create a type for the EVT externref and funcref. This forced the change to ValueTypes.cpp. Once you create the Type for externref and funcref the other changes come along either by necessity or to avoid warnings that certain cases were not handled in switches.

[pmatos]

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1386

That should say externref global stores.

llvm/lib/Target/WebAssembly/WebAssemblyUtilities.cpp

110 ↗

(On Diff #331366)

I refactored that. Comdat is being set in getOrCreateFuncrefCallTableSymbol

[tlively]

In D95425#2643765, @pmatos wrote:

In D95425#2643730, @lebedev.ri wrote:

Once again, are the changes to the LLVM IR (LLVMTypeKind & friends) actually needed?
I'm not seeing any usages of that in the tests.

Hi, I was going refer to those in a separate comment. My apologies for the delay.
Those were required to get this to work due to a call in SelectionDAGBuilder.cpp (current like 9966) to VT.getTypeForEVT. This requires me to create a type for the EVT externref and funcref. This forced the change to ValueTypes.cpp. Once you create the Type for externref and funcref the other changes come along either by necessity or to avoid warnings that certain cases were not handled in switches.

It would make sense for VT.getTypeForEVT to return the appropriate address space pointer types that we are lowering to externref and funcref, right?

[pmatos]

In D95425#2646638, @tlively wrote:

In D95425#2643765, @pmatos wrote:

In D95425#2643730, @lebedev.ri wrote:

Once again, are the changes to the LLVM IR (LLVMTypeKind & friends) actually needed?
I'm not seeing any usages of that in the tests.

Hi, I was going refer to those in a separate comment. My apologies for the delay.
Those were required to get this to work due to a call in SelectionDAGBuilder.cpp (current like 9966) to VT.getTypeForEVT. This requires me to create a type for the EVT externref and funcref. This forced the change to ValueTypes.cpp. Once you create the Type for externref and funcref the other changes come along either by necessity or to avoid warnings that certain cases were not handled in switches.

It would make sense for VT.getTypeForEVT to return the appropriate address space pointer types that we are lowering to externref and funcref, right?

Unsure here to be honest because it's not clear to me where this TypeForEVT fits in the optimizations done by LLVM, but I can give it a try to use the type we are loweing to instead of creating a new one specifically for externref and funcref.

[pmatos]

llvm/include/llvm/CodeGen/ValueTypes.h

213–215

I agree that this should probably return false in that case.

llvm/lib/CodeGen/MachineOperand.cpp

1166–1167

Cannot really use isZeroSized here because this is for MachineMemOperands - we defined isZeroSized for VTs.

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

4105 ↗

(On Diff #331366)

Sounds like I should have used isZeroSized here.

4118 ↗

(On Diff #331366)

but isn't the property really if it's zero sized? we don't want to mention any target dependent property here like reference type, so this feels like the most generic name.

llvm/lib/IR/LLVMContextImpl.cpp

35 ↗

(On Diff #331366)

right - wonder how to stop clang-format to not do this.

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1388–1389

This is for globals, so Offset is undef. I guess once we do tables, we'll need to enable other offsets here.

2383

At some point in an load/store optimization this is called and the default implementation crashes on opaque types, so we need to override this. Although we could use here isZeroSized().

[pmatos]

Clarify many of the issues requested in previous review comments.

[pmatos]

Fix table name for funcref calls

[pmatos]

Are there any further comments to this patch before landing?

[wingo]

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1490

Given that this node defines one value, the chain, should this be MVT::other?

[sbc100]

llvm/lib/Target/WebAssembly/WebAssemblyInstrMemory.td

120 ↗

(On Diff #337379)

The other names in this file seem to either use UPPER_SNAKE_CASE or CamelCase. Any reason these new one use lower_snake_case?

llvm/test/CodeGen/WebAssembly/externref-inttoptr.ll

12

Missing newline at EOF here and in externref-ptrtoint.ll

[tlively]

I'm still not comfortable approving this patch with changes to llvm-c/, Bitcode/, or IR/. If those changes can't be backed out, this will require an RFC on llvm-dev, but I would be surprised if that were necessary.

[pmatos]

In D95425#2699048, @tlively wrote:

I'm still not comfortable approving this patch with changes to llvm-c/, Bitcode/, or IR/. If those changes can't be backed out, this will require an RFC on llvm-dev, but I would be surprised if that were necessary.

I understand - I will look into a workaround to avoid the call of getTypeForEVT I mentioned earlier on.

[pmatos]

Apparently getTypeForEVT in LowerArguments is only necessary to call the getABIAlignmentForCallingConv. I was thinking that since reference types really are pointers to different address spaces we could return their TypeForEVT as being int32. That will then return as the alignment for reference types the default alignment for int32. This should help with avoiding us touching middle end code. I will experiment in this direction.

[tlively]

In D95425#2711969, @pmatos wrote:

Apparently getTypeForEVT in LowerArguments is only necessary to call the getABIAlignmentForCallingConv. I was thinking that since reference types really are pointers to different address spaces we could return their TypeForEVT as being int32. That will then return as the alignment for reference types the default alignment for int32. This should help with avoiding us touching middle end code. I will experiment in this direction.

Why not return the original pointer to opaque type, i.e. PointerType::get(StructType::create(Context), 1)?

[pmatos]

Use PointerType to avoid defining types for externref and funcref.

[pmatos]

llvm/lib/Target/WebAssembly/WebAssemblyInstrMemory.td

120 ↗

(On Diff #337379)

@sbc100 I think it's quite mixed in this file at least. In the beginning of the file we have def regPlusImm followed by def or_is_add. These are for PatFrags. I modelled these after @tlively 's SIMD SDTypeProfile that uses: def wasm_load_splat_t for example.

[pmatos]

Take care of some syntax linting/style comments

[pmatos]

Add missing newlines at the end of test files

[pmatos]

There are still two things missing:

1. Clear the table entry after the call_indirect to avoid hidden GC roots as mentioned by @wingo
2. Understand how to handle a ret %extern undef - comment by @tlively

[wingo]

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

69

should be just "if"

1131

Should this be TABLE_SET_FUNCREF and not just TABLE_SET ? Given that we're producing a DAG node that then needs to be selected, I would think the type-specific stuff could happen in the pattern-matching.

1490

Pretty sure this comment still applies

[pmatos]

llvm/test/CodeGen/WebAssembly/externref-undef.ll

14–15

Returning a ref.null or an uninitialized externref would make more sense if the return type would be %externref. However, in this case this is an %extern value, which really is an opaque type and should never really happen.

[pmatos]

Fix else if and VT type as suggested by @wingo.

[pmatos]

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1131

In this case, I think we cannot generate a TABLE_SET. I think the TABLE_SET you are referring to is the machine instruction defined in the backend which is not matched from here. From here, you can only match the nodes you create and that's WebAssemblyISD::TABLE_SET_FUNCREF. I am still not totally clear why this is, but I never had any luck generating the machine instructions defined in the backend directly from lowering so there might be a way but it's not straightforward. So we generate here the custom machine node we created and then we pattern match that in tablegen to generate the instruction.

1490

Sorry - I missed this. You're right, this should be MVT::Other.

[pmatos]

Make TABLE_SET node polymorphic. Deal with types through the Pats. One for funcref and one for externref.

[pmatos]

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1131

Apologies I misunderstood your comment. We can make the node polymorphic and deal with the types in the Pat Frags. I will do that next.

[tlively]

This is looking really good!

llvm/lib/CodeGen/MachineOperand.cpp

1170

Is there a test that demonstrates this printing? Also, I'm not sure specifically calling out zero-sized operands in the text dump is that useful. Can zero-sized operands still be given an alignment? If so, it might even be good to continue printing the alignment for them.

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.cpp

132–134

Is this necessary given that the symbol is set to be undefined? Perhaps it would be better to make it defined here and also set comdat so that the linker doesn't need to do anything special to make sure it exists in the final binary. (Or possibly I'm misunderstanding what these settings mean!) @sbc100 wdyt?

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1447

It would be good to mark these helper functions static.

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.h

59–61

Why are the address spaces for globals integral? It doesn't make sense to do arithmetic on the "address" of a global afaict. Does using getelementptr to index tables in the global address space require the address space to be integral? If so, that might be a good reason to make the address space integral, but it might also be a good reason to put tables and globals in different address spaces instead. Also, IMO it would be better to use just a single address space for all globals; that way we could support globals with other types like i32, i64, etc. without even more address spaces.

llvm/lib/Target/WebAssembly/WebAssemblyInstrInfo.td

1 ↗

(On Diff #340787)

Unintentional whitespace change?

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

130

This check should incorporate the CPU as well using getSubtargetImpl(CPU, FS)->hasReferenceTypes().

llvm/test/CodeGen/WebAssembly/externref-undef.ll

14–15

Ah, that makes sense.

[sbc100]

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.cpp

132–134

I think don't want this here. Firstly I think this will mark the symbol as being a comdat group symbol .. which is not the same thing as being part of a comdat group ( which I think is that is indented here). Secondly, I don't think it makes sense for an undefined symbol to be part of a comdat group. I think what you want here, if the symbol was defined is setWeak(). Given that is not defined I dont think you even want that.

[wingo]

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

4114 ↗

(On Diff #340787)

Does this path actually fire for you? In my local tests, if you are loading from EXTERNREF_GLOBAL, then the EltVT ends up being i32.

[pmatos]

Rebase this on D101608 from @wingo and clear table slot after call

[pmatos]

Fix patch submitted to phab...

[pmatos]

Fix patch set... again!

[pmatos]

Fix broken merge of table ins reordering commit

[pmatos]

Removed extraneous changes to previous patches

[wingo]

clang/lib/Basic/Targets/WebAssembly.h

150 ↗

(On Diff #342708)

This change is already in D101608; rebase?

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

4083 ↗

(On Diff #342729)

It turns out that all the changes here to visitLoad and visitStore are no longer needed. The issue was that before we made getPointerMemTy virtual, it was hard-coded to return an integer of whatever the pointer bit size was. But for externref and funcref values, where we're using pointers in non-default address spaces to represent opaque values, that's not what we want: an externref "in memory" isn't an i32. Having made getPointerMemTy virtual and returning externref / funcref for those values, now we avoid the zero-extension paths, and even the ISD::ADD node doesn't cause us problems.

4257 ↗

(On Diff #342729)

Can revert this too, as mentioned above.

llvm/lib/CodeGen/TargetLoweringBase.cpp

1709

This one I would think that we want to return true, that the access is "fast", for access to zero-sized types.

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

2387

I just checked and it would seem that the getPointerMemTy change means that this override no longer appears to be needed.

llvm/lib/Target/WebAssembly/WebAssemblyInstrTable.td

15

I think you may need hasSideEffects = 0 for these annotations to have an effect.

19

Not something for this patch, but this is certainly bogus: surely we mean table.get\t$table, $i and we have an i32 index argument?

59

Consider changing to use the approach used for GLOBAL_GET from the parent commit, and folding these into the multiclass so that we don't have to repeat the code for externref and funcref.

[tlively]

llvm/lib/Target/WebAssembly/WebAssemblyInstrTable.td

15

I would be surprised if this were true!

19

Agree about the i32 index argument, but it is correct to have the result as part of the string for the register-based output format.

llvm/test/CodeGen/WebAssembly/externref-globalget.ll

5

I forget if we had or resolved this conversation elsewhere already, but I would expect clang to emit pointers in the correct address spaces, yes. The SVE folks had to go though multiple years of effort to add scalable vectors to LLVM IR because there was no existing way to model their semantics in IR such that they could implement all the vectorization optimizations they wanted. We are lucky that we can get away with using address space pointers directly without having to add new types to the IR.

[pmatos]

llvm/lib/CodeGen/MachineOperand.cpp

1170

The reason I changed this here is due to the call of getSize() crashing for zeroSized arguments. I have not added test for the printing. I don't think zero-sized operands can still be given an alignment. We are actually setting the alignment to

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.cpp

132–134

It seems I was mistaken - I thought that by setting it as Comdat, the linker would merge symbols from different compilation units if they existed. Maybe what I need is Weak then.

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.h

59–61

Current set of patches should have addressed your concerns here.

llvm/lib/Target/WebAssembly/WebAssemblyInstrTable.td

15

Why would this be the case? If I remember correctly, I added mayLoad and mayStore here so that lowering includes a chain. And this works without the need for hasSideEffects. Unless you think this is required for other reaons, but mayLoad works with it.

19

Not sure I understand why this should be $i instead of $table. Isn't $table represented as an index anyway? A table32_op is defined as Operand<i32> so I don't quite understand what we gain by changing this.

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

130

What exactly do you mean here? Not sure what is the difference here. You mean adding something like : hasReferenceTypes(FS) || getSubtargetImpl(CPU, FS)->hasReferenceTypes()?

[pmatos]

• Simplifies some of code in the middle-end.
• Removes setComdat and adds setWeak

Small improvements to address comments in review.

[pmatos]

Making the linter happy...

[wingo]

llvm/lib/Target/WebAssembly/WebAssemblyInstrTable.td

15

Yeah I misunderstood, I was thinking that mayLoad and mayStore were a subset of the side effect annotations of hasSideEffects, which defaults to 1 effectively (grep for guessInstructionProperties). But no, the side effects modelled here are the disjoint bits mayLoad, mayStore, and hasSideEffects; mayRaiseFPException would appear to be a subset of hasSideEffects. Still, may best to explicitly set hasSideEffects to 1 for table.get and table.set just as documentation that they can trap if the index is out-of-bounds.

[pmatos]

Rebase and update minor details and fix a few lint warnings...
Non-functional changes only.

[pmatos]

@deadalnix @tlively @sbc100 anything missing here to get this landed?

[tlively]

It would be good to add dedicated tests for table.get and table.set as well.

llvm/lib/CodeGen/MachineOperand.cpp

1170

Looks like this comment was cut off! I wonder if it would be better to skip printing opaque for now since we have no test of it/ Also, if another target comes along with a zero-sized type, I don't know whether "opaque" will necessarily describe it.

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

558–559

Can you add the .wat of the code being generated here to the comment?

llvm/lib/Target/WebAssembly/WebAssemblyInstrTable.td

19

I would still expect to see a register for the result and immediate inputs for the table symbol and the table slot index here.

llvm/test/CodeGen/WebAssembly/externref-undef.ll

14–15

It would be good to add this explanation as a comment in the test file.

llvm/test/CodeGen/WebAssembly/funcref-call.ll

16

Missing a table element index here.

[pmatos]

@tlively thanks for the comments. I am currently taking some time off but I will address your concerns upon my return.

[pmatos]

Fix details requested by @tlively in latest comemnts.

[pmatos]

Hopefully we are close to landing this. :)

llvm/lib/Target/WebAssembly/WebAssemblyInstrTable.td

19

Not sure I understood everything properly as the only thing missing was the index in the register based version. I added that now.

llvm/test/CodeGen/WebAssembly/funcref-call.ll

16

I am not sure whether that's the case. According to the spec, table.set only gets a table. Two elements are popped from the stack: the index i32.const 0 and the value local.get 0.

[tlively]

Thanks for your patience through all the review! Let's get this landed :)

llvm/lib/Target/WebAssembly/WebAssemblyInstrTable.td

19

Ah, sorry about that. I was confused and was thinking that the table index was an immediate, but of course you're right that it is not.

llvm/test/CodeGen/WebAssembly/funcref-call.ll

16

Yep, sorry for the confusion.

[DavidSpickett]

Hi @pmatos, this has caused SVE related failures on our AArch64 bot (and probably others):
https://lab.llvm.org/buildbot/#/builders/43/builds/7187

Not an SVE expert myself but if you need help figuring it out I can find someone to help out I'm sure.

[DavidSpickett]

Also affecting riscv-v (riscvv?) so I've reverted the change.

https://lab.llvm.org/buildbot/#/builders/67/builds/3087

[pmatos]

In D95425#2810027, @DavidSpickett wrote:

Also affecting riscv-v (riscvv?) so I've reverted the change.

https://lab.llvm.org/buildbot/#/builders/67/builds/3087

Apologies for the breakages and thanks for reverting. I will take a look at it and if I need help, I will get back to you.

clang/lib/Basic/Targets/WebAssembly.cpp

if (Feature == "+tail-call") {
       HasTailCall = true;
       continue;
     }
     if (Feature == "-tail-call") {
       HasTailCall = false;
       continue;
     }
     if (Feature == "+reference-types") {
+      // FIXME: Ensure address spaces 10 and 20 are marked as non-integral in
+      // the datalayout string.
       HasReferenceTypes = true;
       continue;
     }
     if (Feature == "-reference-types") {
       HasReferenceTypes = false;
       continue;
     }
 

llvm/include/llvm/CodeGen/TargetLowering.h

public:
   /// FIXME: The default needs to be removed once all the code is updated.
   virtual MVT getPointerTy(const DataLayout &DL, uint32_t AS = 0) const {
     return MVT::getIntegerVT(DL.getPointerSizeInBits(AS));
   }
 
   /// Return the in-memory pointer type for the given address space, defaults to
   /// the pointer type from the data layout.  FIXME: The default needs to be
   /// removed once all the code is updated.
-  MVT getPointerMemTy(const DataLayout &DL, uint32_t AS = 0) const {
+  virtual MVT getPointerMemTy(const DataLayout &DL, uint32_t AS = 0) const {
     return MVT::getIntegerVT(DL.getPointerSizeInBits(AS));
   }
 
   /// Return the type for frame index, which is determined by
   /// the alloca address space specified through the data layout.
   MVT getFrameIndexTy(const DataLayout &DL) const {
     return getPointerTy(DL, DL.getAllocaAddrSpace());
   }

llvm/include/llvm/CodeGen/ValueTypes.h

EVT changeTypeToInteger() {
       if (isVector())
         return changeVectorElementTypeToInteger();
 
       if (isSimple())
         return getSimpleVT().changeTypeToInteger();
       return changeExtendedTypeToInteger();
     }
 
+    /// Test if the given EVT has zero size
+    bool isZeroSized() const { return getSizeInBits() == 0; }
+
     /// Test if the given EVT is simple (as opposed to being extended).
     bool isSimple() const {
       return V.SimpleTy != MVT::INVALID_SIMPLE_VALUE_TYPE;
     }
 
     /// Test if the given EVT is extended (as opposed to being simple).
     bool isExtended() const {
       return !isSimple();
public:
     }
 
     /// Return true if this is an overloaded type for TableGen.
     bool isOverloaded() const {
       return (V==MVT::iAny || V==MVT::fAny || V==MVT::vAny || V==MVT::iPTRAny);
     }
 
     /// Return true if the bit size is a multiple of 8.
-    bool isByteSized() const { return getSizeInBits().isKnownMultipleOf(8); }
+    bool isByteSized() const {
+      return !isZeroSized() && getSizeInBits().isKnownMultipleOf(8);
+    }
 
     /// Return true if the size is a power-of-two number of bytes.
     bool isRound() const {
       if (isScalableVector())
         return false;
       unsigned BitSize = getSizeInBits();
       return BitSize >= 8 && !(BitSize & (BitSize - 1));
     }

llvm/include/llvm/Support/MachineValueType.h

TypeSize getSizeInBits() const {
       case v1024f32:  return TypeSize::Fixed(32768);
       case v2048i32:
       case v2048f32:  return TypeSize::Fixed(65536);
       case funcref:
       case externref: return TypeSize::Fixed(0); // opaque type
       }
     }
 
+    /// Test if the given MVT has zero size
+    bool isZeroSized() const {
+      return !getSizeInBits().isScalable() && getFixedSizeInBits() == 0;
+    }
+
     /// Return the size of the specified fixed width value type in bits. The
     /// function will assert if the type is scalable.
     uint64_t getFixedSizeInBits() const {
       return getSizeInBits().getFixedSize();
     }
 
     uint64_t getScalarSizeInBits() const {
       return getScalarType().getSizeInBits().getFixedSize();

llvm/lib/CodeGen/CodeGenPrepare.cpp

bool CodeGenPrepare::optimizeLoadExt(LoadInst *Load) {
   SmallVector<Instruction *, 8> WorkList;
   SmallPtrSet<Instruction *, 16> Visited;
   SmallVector<Instruction *, 8> AndsToMaybeRemove;
   for (auto *U : Load->users())
     WorkList.push_back(cast<Instruction>(U));
 
   EVT LoadResultVT = TLI->getValueType(*DL, Load->getType());
   unsigned BitWidth = LoadResultVT.getSizeInBits();
+  // If the BitWidth is 0, do not try to optimize the type
+  if (BitWidth == 0)
+    return false;
+
   APInt DemandBits(BitWidth, 0);
   APInt WidestAndBits(BitWidth, 0);
 
   while (!WorkList.empty()) {
     Instruction *I = WorkList.back();
     WorkList.pop_back();
 
     // Break use-def graph loops.

llvm/lib/CodeGen/MachineOperand.cpp

if (const Value *Val = getValue()) {
     }
   } else if (getOpaqueValue() == nullptr && getOffset() != 0) {
     OS << ((isLoad() && isStore()) ? " on "
            : isLoad()              ? " from "
                                    : " into ")
        << "unknown-address";
   }
   MachineOperand::printOperandOffset(OS, getOffset());
-  if (getAlign() != getSize())
+  if (getSize() > 0 && getAlign() != getSize())
       OS << ", align " << getAlign().value();
   if (getAlign() != getBaseAlign())
     OS << ", basealign " << getBaseAlign().value();
   auto AAInfo = getAAInfo();
   if (AAInfo.TBAA) {
     OS << ", !tbaa ";
     AAInfo.TBAA->printAsOperand(OS, MST);
   }
   if (AAInfo.Scope) {
     OS << ", !alias.scope ";
     AAInfo.Scope->printAsOperand(OS, MST);
   }

llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

bool DAGCombiner::parallelizeChainedStores(StoreSDNode *St) {
   // We must have a base and an offset.
   if (!BasePtr.getBase().getNode())
     return false;
 
   // Do not handle stores to undef base pointers.
   if (BasePtr.getBase().isUndef())
     return false;
 
+  // Do not handle stores to opaque types
+  if (St->getMemoryVT().isZeroSized())
+    return false;
+
   // BaseIndexOffset assumes that offsets are fixed-size, which
   // is not valid for scalable vectors where the offsets are
   // scaled by `vscale`, so bail out early.
   if (St->getMemoryVT().isScalableVector())
     return false;
 
   // Add ST's interval.
   Intervals.insert(0, (St->getMemoryVT().getSizeInBits() + 7) / 8, Unit);

llvm/lib/CodeGen/TargetLoweringBase.cpp

bool TargetLoweringBase::allowsMemoryAccessForAlignment(
     LLVMContext &Context, const DataLayout &DL, EVT VT, unsigned AddrSpace,
     Align Alignment, MachineMemOperand::Flags Flags, bool *Fast) const {
   // Check if the specified alignment is sufficient based on the data layout.
   // TODO: While using the data layout works in practice, a better solution
   // would be to implement this check directly (make this a virtual function).
   // For example, the ABI alignment may change based on software platform while
   // this function should only be affected by hardware implementation.
   Type *Ty = VT.getTypeForEVT(Context);
-  if (Alignment >= DL.getABITypeAlign(Ty)) {
+  if (VT.isZeroSized() || Alignment >= DL.getABITypeAlign(Ty)) {
     // Assume that an access that meets the ABI-specified alignment is fast.
     if (Fast != nullptr)
       *Fast = true;
     return true;
   }
 
   // This is a misaligned access.
   return allowsMisalignedMemoryAccesses(VT, AddrSpace, Alignment, Flags, Fast);

llvm/lib/CodeGen/ValueTypes.cpp

Type *EVT::getTypeForEVT(LLVMContext &Context) const {
   case MVT::bf16:     return Type::getBFloatTy(Context);
   case MVT::f32:     return Type::getFloatTy(Context);
   case MVT::f64:     return Type::getDoubleTy(Context);
   case MVT::f80:     return Type::getX86_FP80Ty(Context);
   case MVT::f128:    return Type::getFP128Ty(Context);
   case MVT::ppcf128: return Type::getPPC_FP128Ty(Context);
   case MVT::x86mmx:  return Type::getX86_MMXTy(Context);
   case MVT::x86amx:  return Type::getX86_AMXTy(Context);
+  case MVT::externref:
+    return PointerType::get(StructType::create(Context), 10);
+  case MVT::funcref:
+    return PointerType::get(StructType::create(Context), 20);
   case MVT::v1i1:
     return FixedVectorType::get(Type::getInt1Ty(Context), 1);
   case MVT::v2i1:
     return FixedVectorType::get(Type::getInt1Ty(Context), 2);
   case MVT::v4i1:
     return FixedVectorType::get(Type::getInt1Ty(Context), 4);
   case MVT::v8i1:
     return FixedVectorType::get(Type::getInt1Ty(Context), 8);

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.h

 const MachineOperand &getCalleeOp(const MachineInstr &MI);
 
 /// Returns the __indirect_function_table, for use in call_indirect and in
 /// function bitcasts.
 MCSymbolWasm *
 getOrCreateFunctionTableSymbol(MCContext &Ctx,
                                const WebAssemblySubtarget *Subtarget);
 
+/// Returns the __funcref_call_table, for use in funcref calls when lowered to
+/// table.set + call_indirect.
+MCSymbolWasm *
+getOrCreateFuncrefCallTableSymbol(MCContext &Ctx,
+                                  const WebAssemblySubtarget *Subtarget);
+
 /// Find a catch instruction from an EH pad. Returns null if no catch
 /// instruction found or the catch is in an invalid location.
 MachineInstr *findCatch(MachineBasicBlock *EHPad);
 
 } // end namespace WebAssembly
 
 } // end namespace llvm
 
 #endif

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.cpp

if (Sym) {
     Sym->setUndefined();
   }
   // MVP object files can't have symtab entries for tables.
   if (!(Subtarget && Subtarget->hasReferenceTypes()))
     Sym->setOmitFromLinkingSection();
   return Sym;
 }
 
+MCSymbolWasm *WebAssembly::getOrCreateFuncrefCallTableSymbol(
+    MCContext &Ctx, const WebAssemblySubtarget *Subtarget) {
+  StringRef Name = "__funcref_call_table";
+  MCSymbolWasm *Sym = cast_or_null<MCSymbolWasm>(Ctx.lookupSymbol(Name));
+  if (Sym) {
+    if (!Sym->isFunctionTable())
+      Ctx.reportError(SMLoc(), "symbol is not a wasm funcref table");
+  } else {
+    Sym = cast<MCSymbolWasm>(Ctx.getOrCreateSymbol(Name));
+
+    // Setting Weak ensure only one table is left after linking when multiple
+    // modules define the table.
+    Sym->setWeak(true);
+
+    wasm::WasmLimits Limits = {0, 1, 1};
+    wasm::WasmTableType TableType = {wasm::WASM_TYPE_FUNCREF, Limits};
+    Sym->setType(wasm::WASM_SYMBOL_TYPE_TABLE);
+    Sym->setTableType(TableType);
+  }
+  // MVP object files can't have symtab entries for tables.
+  if (!(Subtarget && Subtarget->hasReferenceTypes()))
+    Sym->setOmitFromLinkingSection();
+  return Sym;
+}
+
 // Find a catch instruction from an EH pad.
 MachineInstr *WebAssembly::findCatch(MachineBasicBlock *EHPad) {
   assert(EHPad->isEHPad());
   auto Pos = EHPad->begin();
   // Skip any label or debug instructions. Also skip 'end' marker instructions
   // that may exist after marker placement in CFGStackify.
   while (Pos != EHPad->end() &&
          (Pos->isLabel() || Pos->isDebugInstr() || isMarker(Pos->getOpcode())))
     Pos++;
   if (Pos != EHPad->end() && WebAssembly::isCatch(Pos->getOpcode()))
     return &*Pos;
   return nullptr;
 }

llvm/lib/Target/WebAssembly/WebAssemblyFastISel.cpp

MVT::SimpleValueType getLegalType(MVT::SimpleValueType VT) {
     case MVT::i1:
     case MVT::i8:
     case MVT::i16:
       return MVT::i32;
     case MVT::i32:
     case MVT::i64:
     case MVT::f32:
     case MVT::f64:
+      return VT;
     case MVT::funcref:
     case MVT::externref:
+      if (Subtarget->hasReferenceTypes())
         return VT;
+      break;
     case MVT::f16:
       return MVT::f32;
     case MVT::v16i8:
     case MVT::v8i16:
     case MVT::v4i32:
     case MVT::v4f32:
     case MVT::v2i64:
     case MVT::v2f64:

llvm/lib/Target/WebAssembly/WebAssemblyISD.def

 HANDLE_NODETYPE(CATCH)
 HANDLE_NODETYPE(MEMORY_COPY)
 HANDLE_NODETYPE(MEMORY_FILL)
 
 // Memory intrinsics
 HANDLE_MEM_NODETYPE(LOAD_SPLAT)
 HANDLE_MEM_NODETYPE(GLOBAL_GET)
 HANDLE_MEM_NODETYPE(GLOBAL_SET)
+HANDLE_MEM_NODETYPE(TABLE_SET)

llvm/lib/Target/WebAssembly/WebAssemblyISelDAGToDAG.cpp

public:
   }
 
   void PreprocessISelDAG() override;
 
   void Select(SDNode *Node) override;
 
   bool SelectInlineAsmMemoryOperand(const SDValue &Op, unsigned ConstraintID,
                                     std::vector<SDValue> &OutOps) override;
+  bool SelectExternRefAddr(const SDValue &Addr, const SDValue &Base);
 
 // Include the pieces autogenerated from the target description.
 #include "WebAssemblyGenDAGISel.inc"
 
 private:
   // add select functions here...
 };
 } // end anonymous namespace

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.h

 
 class WebAssemblySubtarget;
 
 class WebAssemblyTargetLowering final : public TargetLowering {
 public:
   WebAssemblyTargetLowering(const TargetMachine &TM,
                             const WebAssemblySubtarget &STI);
 
+  enum WasmAddressSpace : unsigned {
+    // WebAssembly uses the following address spaces:
+    // AS 0 : is the default address space for values in linear memory
+    DEFAULT = 0,
+    // AS 1 : is a non-integral address space for global variables
+    GLOBAL = 1,
+    // AS 10 : is a non-integral address space for externref values
+    EXTERNREF = 10,
+    // AS 20 : is a non-integral address space for funcref values
+    FUNCREF = 20,
+  };
+
+  MVT getPointerTy(const DataLayout &DL, uint32_t AS = 0) const override {
+    if (AS == WasmAddressSpace::EXTERNREF)
+      return MVT::externref;
+    else if (AS == WasmAddressSpace::FUNCREF)
+      return MVT::funcref;
+    return TargetLowering::getPointerTy(DL, AS);
+  }
+  MVT getPointerMemTy(const DataLayout &DL, uint32_t AS = 0) const override {
+    if (AS == WasmAddressSpace::EXTERNREF)
+      return MVT::externref;
+    else if (AS == WasmAddressSpace::FUNCREF)
+      return MVT::funcref;
+    return TargetLowering::getPointerMemTy(DL, AS);
+  }
+
 private:
   /// Keep a pointer to the WebAssemblySubtarget around so that we can make the
   /// right decision when generating code for different targets.
   const WebAssemblySubtarget *Subtarget;
 
   AtomicExpansionKind shouldExpandAtomicRMWInIR(AtomicRMWInst *) const override;
   FastISel *createFastISel(FunctionLoweringInfo &FuncInfo,
                            const TargetLibraryInfo *LibInfo) const override;
   MVT getScalarShiftAmountTy(const DataLayout &DL, EVT) const override;
   MachineBasicBlock *
   EmitInstrWithCustomInserter(MachineInstr &MI,
                               MachineBasicBlock *MBB) const override;
   const char *getTargetNodeName(unsigned Opcode) const override;
   std::pair<unsigned, const TargetRegisterClass *>
   getRegForInlineAsmConstraint(const TargetRegisterInfo *TRI,
                                StringRef Constraint, MVT VT) const override;
   bool isCheapToSpeculateCttz() const override;
   bool isCheapToSpeculateCtlz() const override;
   bool isLegalAddressingMode(const DataLayout &DL, const AddrMode &AM, Type *Ty,
                              unsigned AS,
                              Instruction *I = nullptr) const override;
+  bool isFuncref(const Value *Op) const;
   bool allowsMisalignedMemoryAccesses(EVT, unsigned AddrSpace, Align Alignment,
                                       MachineMemOperand::Flags Flags,
                                       bool *Fast) const override;
   bool isIntDivCheap(EVT VT, AttributeList Attr) const override;
   bool isVectorLoadExtDesirable(SDValue ExtVal) const override;
   EVT getSetCCResultType(const DataLayout &DL, LLVMContext &Context,
                          EVT VT) const override;
   bool getTgtMemIntrinsic(IntrinsicInfo &Info, const CallInst &I,

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

 //=- WebAssemblyISelLowering.cpp - WebAssembly DAG Lowering Implementation -==//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
 /// \file
 /// This file implements the WebAssemblyTargetLowering class.
 ///
 //===----------------------------------------------------------------------===//
 
 #include "WebAssemblyISelLowering.h"
 #include "MCTargetDesc/WebAssemblyMCTargetDesc.h"
+#include "Utils/WebAssemblyTypeUtilities.h"
 #include "Utils/WebAssemblyUtilities.h"
 #include "WebAssemblyMachineFunctionInfo.h"
 #include "WebAssemblySubtarget.h"
 #include "WebAssemblyTargetMachine.h"
 #include "llvm/CodeGen/CallingConvLower.h"
 #include "llvm/CodeGen/MachineInstrBuilder.h"
 #include "llvm/CodeGen/MachineJumpTableInfo.h"
 #include "llvm/CodeGen/MachineModuleInfo.h"
WebAssemblyTargetLowering::WebAssemblyTargetLowering(
   addRegisterClass(MVT::f64, &WebAssembly::F64RegClass);
   if (Subtarget->hasSIMD128()) {
     addRegisterClass(MVT::v16i8, &WebAssembly::V128RegClass);
     addRegisterClass(MVT::v8i16, &WebAssembly::V128RegClass);
     addRegisterClass(MVT::v4i32, &WebAssembly::V128RegClass);
     addRegisterClass(MVT::v4f32, &WebAssembly::V128RegClass);
     addRegisterClass(MVT::v2i64, &WebAssembly::V128RegClass);
     addRegisterClass(MVT::v2f64, &WebAssembly::V128RegClass);
   }
+  if (Subtarget->hasReferenceTypes()) {
+    addRegisterClass(MVT::externref, &WebAssembly::EXTERNREFRegClass);
+    addRegisterClass(MVT::funcref, &WebAssembly::FUNCREFRegClass);
+  }
   // Compute derived properties from the register classes.
   computeRegisterProperties(Subtarget->getRegisterInfo());
 
   // Transform loads and stores to pointers in address space 1 to loads and
   // stores to WebAssembly global variables, outside linear memory.
   for (auto T : {MVT::i32, MVT::i64, MVT::f32, MVT::f64}) {
     setOperationAction(ISD::LOAD, T, Custom);
     setOperationAction(ISD::STORE, T, Custom);
   }
   if (Subtarget->hasSIMD128()) {
     for (auto T : {MVT::v16i8, MVT::v8i16, MVT::v4i32, MVT::v4f32, MVT::v2i64,
                    MVT::v2f64}) {
       setOperationAction(ISD::LOAD, T, Custom);
       setOperationAction(ISD::STORE, T, Custom);
     }
   }
+  if (Subtarget->hasReferenceTypes()) {
+    for (auto T : {MVT::externref, MVT::funcref}) {
+      setOperationAction(ISD::LOAD, T, Custom);
+      setOperationAction(ISD::STORE, T, Custom);
+    }
+  }
 
   setOperationAction(ISD::GlobalAddress, MVTPtr, Custom);
   setOperationAction(ISD::GlobalTLSAddress, MVTPtr, Custom);
   setOperationAction(ISD::ExternalSymbol, MVTPtr, Custom);
   setOperationAction(ISD::JumpTable, MVTPtr, Custom);
   setOperationAction(ISD::BlockAddress, MVTPtr, Custom);
   setOperationAction(ISD::BRIND, MVT::Other, Custom);
 
LowerCallResults(MachineInstr &CallResults, DebugLoc DL, MachineBasicBlock *BB,
   MachineInstr &CallParams = *CallResults.getPrevNode();
   assert(CallParams.getOpcode() == WebAssembly::CALL_PARAMS);
   assert(CallResults.getOpcode() == WebAssembly::CALL_RESULTS ||
          CallResults.getOpcode() == WebAssembly::RET_CALL_RESULTS);
 
   bool IsIndirect = CallParams.getOperand(0).isReg();
   bool IsRetCall = CallResults.getOpcode() == WebAssembly::RET_CALL_RESULTS;
 
+  bool IsFuncrefCall = false;
+  if (IsIndirect) {
+    Register Reg = CallParams.getOperand(0).getReg();
+    const MachineFunction *MF = BB->getParent();
+    const MachineRegisterInfo &MRI = MF->getRegInfo();
+    const TargetRegisterClass *TRC = MRI.getRegClass(Reg);
+    IsFuncrefCall = (TRC == &WebAssembly::FUNCREFRegClass);
+    assert(!IsFuncrefCall || Subtarget->hasReferenceTypes());
+  }
+
   unsigned CallOp;
   if (IsIndirect && IsRetCall) {
     CallOp = WebAssembly::RET_CALL_INDIRECT;
   } else if (IsIndirect) {
     CallOp = WebAssembly::CALL_INDIRECT;
   } else if (IsRetCall) {
     CallOp = WebAssembly::RET_CALL;
   } else {
LowerCallResults(MachineInstr &CallResults, DebugLoc DL, MachineBasicBlock *BB,
 
   for (auto Def : CallResults.defs())
     MIB.add(Def);
 
   if (IsIndirect) {
     // Placeholder for the type index.
     MIB.addImm(0);
     // The table into which this call_indirect indexes.
-    MCSymbolWasm *Table =
-        WebAssembly::getOrCreateFunctionTableSymbol(MF.getContext(), Subtarget);
+    MCSymbolWasm *Table = IsFuncrefCall
+                              ? WebAssembly::getOrCreateFuncrefCallTableSymbol(
+                                    MF.getContext(), Subtarget)
+                              : WebAssembly::getOrCreateFunctionTableSymbol(
+                                    MF.getContext(), Subtarget);
     if (Subtarget->hasReferenceTypes()) {
       MIB.addSym(Table);
     } else {
       // For the MVP there is at most one table whose number is 0, but we can't
       // write a table symbol or issue relocations.  Instead we just ensure the
       // table is live and write a zero.
       Table->setNoStrip();
       MIB.addImm(0);
     }
   }
 
   for (auto Use : CallParams.uses())
     MIB.add(Use);
 
   BB->insert(CallResults.getIterator(), MIB);
   CallParams.eraseFromParent();
   CallResults.eraseFromParent();
 
+  // If this is a funcref call, to avoid hidden GC roots, we need to clear the
+  // table slot with ref.null upon call_indirect return.
+  //
+  // This generates the following code, which comes right after a call_indirect
+  // of a funcref:
+  //
+  //    i32.const 0
+  //    ref.null func
+  //    table.set __funcref_call_table
+  if (IsIndirect && IsFuncrefCall) {
+    MCSymbolWasm *Table = WebAssembly::getOrCreateFuncrefCallTableSymbol(
+        MF.getContext(), Subtarget);
+    Register RegZero =
+        MF.getRegInfo().createVirtualRegister(&WebAssembly::I32RegClass);
+    MachineInstr *Const0 =
+        BuildMI(MF, DL, TII.get(WebAssembly::CONST_I32), RegZero).addImm(0);
+    BB->insertAfter(MIB.getInstr()->getIterator(), Const0);
+
+    Register RegFuncref =
+        MF.getRegInfo().createVirtualRegister(&WebAssembly::FUNCREFRegClass);
+    MachineInstr *RefNull =
+        BuildMI(MF, DL, TII.get(WebAssembly::REF_NULL_FUNCREF), RegFuncref)
+            .addImm(static_cast<int32_t>(WebAssembly::HeapType::Funcref));
+    BB->insertAfter(Const0->getIterator(), RefNull);
+
+    MachineInstr *TableSet =
+        BuildMI(MF, DL, TII.get(WebAssembly::TABLE_SET_FUNCREF))
+            .addSym(Table)
+            .addReg(RegZero)
+            .addReg(RegFuncref);
+    BB->insertAfter(RefNull->getIterator(), TableSet);
+  }
+
   return BB;
 }
 
 MachineBasicBlock *WebAssemblyTargetLowering::EmitInstrWithCustomInserter(
     MachineInstr &MI, MachineBasicBlock *BB) const {
   const TargetInstrInfo &TII = *Subtarget->getInstrInfo();
   DebugLoc DL = MI.getDebugLoc();
 
for (const auto &In : Ins) {
     if (In.Flags.isInConsecutiveRegsLast())
       fail(DL, DAG,
            "WebAssembly hasn't implemented cons regs last return values");
     // Ignore In.getNonZeroOrigAlign() because all our arguments are passed in
     // registers.
     InTys.push_back(In.VT);
   }
 
+  // Lastly, if this is a call to a funcref we need to add an instruction
+  // table.set to the chain and transform the call.
+  if (CLI.CB && isFuncref(CLI.CB->getCalledOperand())) {
+    // In the absence of function references proposal where a funcref call is
+    // lowered to call_ref, using reference types we generate a table.set to set
+    // the funcref to a special table used solely for this purpose, followed by
+    // a call_indirect. Here we just generate the table set, and return the
+    // SDValue of the table.set so that LowerCall can finalize the lowering by
+    // generating the call_indirect.
+    SDValue Chain = Ops[0];
+
+    MCSymbolWasm *Table = WebAssembly::getOrCreateFuncrefCallTableSymbol(
+        MF.getContext(), Subtarget);
+    SDValue Sym = DAG.getMCSymbol(Table, PtrVT);
+    SDValue TableSlot = DAG.getConstant(0, DL, MVT::i32);
+    SDValue TableSetOps[] = {Chain, Sym, TableSlot, Callee};
+    SDValue TableSet = DAG.getMemIntrinsicNode(
+        WebAssemblyISD::TABLE_SET, DL, DAG.getVTList(MVT::Other), TableSetOps,
+        MVT::funcref,
+        // Machine Mem Operand args
+        MachinePointerInfo(WasmAddressSpace::FUNCREF),
+        CLI.CB->getCalledOperand()->getPointerAlignment(DAG.getDataLayout()),
+        MachineMemOperand::MOStore);
+
+    Ops[0] = TableSet; // The new chain is the TableSet itself
+  }
+
   if (CLI.IsTailCall) {
     // ret_calls do not return values to the current frame
     SDVTList NodeTys = DAG.getVTList(MVT::Other, MVT::Glue);
     return DAG.getNode(WebAssemblyISD::RET_CALL, DL, NodeTys, Ops);
   }
 
   InTys.push_back(MVT::Other);
   SDVTList InTyList = DAG.getVTList(InTys);
case ISD::LOAD:
     return LowerLoad(Op, DAG);
   case ISD::STORE:
     return LowerStore(Op, DAG);
   }
 }
 
 static bool IsWebAssemblyGlobal(SDValue Op) {
   if (const GlobalAddressSDNode *GA = dyn_cast<GlobalAddressSDNode>(Op))
     return WebAssembly::isWasmVarAddressSpace(GA->getAddressSpace());
 
   return false;
 }
 
 static Optional<unsigned> IsWebAssemblyLocal(SDValue Op, SelectionDAG &DAG) {
   const FrameIndexSDNode *FI = dyn_cast<FrameIndexSDNode>(Op);
   if (!FI)
     return None;
 
   auto &MF = DAG.getMachineFunction();
   return WebAssemblyFrameLowering::getLocalForStackObject(MF, FI->getIndex());
 }
 
+bool WebAssemblyTargetLowering::isFuncref(const Value *Op) const {
+  const Type *Ty = Op->getType();
+
+  return isa<PointerType>(Ty) &&
+         Ty->getPointerAddressSpace() == WasmAddressSpace::FUNCREF;
+}
+
 SDValue WebAssemblyTargetLowering::LowerStore(SDValue Op,
                                               SelectionDAG &DAG) const {
   SDLoc DL(Op);
   StoreSDNode *SN = cast<StoreSDNode>(Op.getNode());
   const SDValue &Value = SN->getValue();
   const SDValue &Base = SN->getBasePtr();
   const SDValue &Offset = SN->getOffset();
 
   if (IsWebAssemblyGlobal(Base)) {
     if (!Offset->isUndef())
       report_fatal_error("unexpected offset when storing to webassembly global",
                          false);
 
     SDVTList Tys = DAG.getVTList(MVT::Other);
     SDValue Ops[] = {SN->getChain(), Value, Base};
     return DAG.getMemIntrinsicNode(WebAssemblyISD::GLOBAL_SET, DL, Tys, Ops,
                                    SN->getMemoryVT(), SN->getMemOperand());
   }
 
if (Optional<unsigned> Local = IsWebAssemblyLocal(Base, DAG)) {
     SDValue Result = DAG.getMergeValues({LocalGet, LN->getChain()}, DL);
     assert(Result->getNumValues() == 2 && "Loads must carry a chain!");
     return Result;
   }
 
   return Op;
 }
 
 SDValue WebAssemblyTargetLowering::LowerCopyToReg(SDValue Op,
                                                   SelectionDAG &DAG) const {
   SDValue Src = Op.getOperand(2);
   if (isa<FrameIndexSDNode>(Src.getNode())) {
     // CopyToReg nodes don't support FrameIndex operands. Other targets select
     // the FI to some LEA-like instruction, but since we don't have that, we
     // need to insert some kind of instruction that can take an FI operand and
     // produces a value usable by CopyToReg (i.e. in a vreg). So insert a dummy
     // local.copy between Op and its FI operand.
SDValue WebAssemblyTargetLowering::LowerRETURNADDR(SDValue Op,
 
   if (!Subtarget->getTargetTriple().isOSEmscripten()) {
     fail(DL, DAG,
          "Non-Emscripten WebAssembly hasn't implemented "
          "__builtin_return_address");
     return SDValue();
   }
 
   if (verifyReturnAddressArgumentIsConstant(Op, DAG))
     return SDValue();
 
   unsigned Depth = cast<ConstantSDNode>(Op.getOperand(0))->getZExtValue();
   MakeLibCallOptions CallOptions;
   return makeLibCall(DAG, RTLIB::RETURN_ADDRESS, Op.getValueType(),
                      {DAG.getConstant(Depth, DL, MVT::i32)}, CallOptions, DL)
       .first;
 }
case ISD::ZERO_EXTEND:
     return performVectorExtendCombine(N, DCI);
   case ISD::SINT_TO_FP:
   case ISD::UINT_TO_FP:
   case ISD::EXTRACT_SUBVECTOR:
     return performVectorConvertLowCombine(N, DCI);
   case ISD::CONCAT_VECTORS:
     return performVectorTruncSatLowCombine(N, DCI);
   }
 }

llvm/lib/Target/WebAssembly/WebAssemblyInstrTable.td

 // WebAssemblyInstrTable.td - WebAssembly Table codegen support -*- tablegen -*-
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
 /// \file
 /// WebAssembly Table operand code-gen constructs.
 /// Instructions that handle tables
 //===----------------------------------------------------------------------===//
 
-
 multiclass TABLE<WebAssemblyRegClass rt> {
-  defm TABLE_GET_#rt : I<(outs rt:$res), (ins table32_op:$table),
+  let mayLoad = 1 in
+  defm TABLE_GET_#rt : I<(outs rt:$res), (ins table32_op:$table, I32:$i),
                          (outs), (ins table32_op:$table),
                          [],
-                         "table.get\t$res, $table",
+                         "table.get\t$res, $table, $i",
                          "table.get\t$table",
                          0x25>;
 
+  let mayStore = 1 in
   defm TABLE_SET_#rt : I<(outs), (ins table32_op:$table, I32:$i, rt:$val),
                          (outs), (ins table32_op:$table),
                          [],
                          "table.set\t$table, $i, $val",
                          "table.set\t$table",
                          0x26>;
 
   defm TABLE_GROW_#rt : I<(outs I32:$sz), (ins table32_op:$table, rt:$val, I32:$n),
defm TABLE_FILL_#rt : I<(outs), (ins table32_op:$table, I32:$i, rt:$val, I32:$n),
                           "table.fill\t$table",
                           0xfc11>;
 
 }
 
 defm "" : TABLE<FUNCREF>, Requires<[HasReferenceTypes]>;
 defm "" : TABLE<EXTERNREF>, Requires<[HasReferenceTypes]>;
 
+def wasm_table_set_t : SDTypeProfile<0, 3, []>;
+def wasm_table_set : SDNode<"WebAssemblyISD::TABLE_SET", wasm_table_set_t,
+                            [SDNPHasChain, SDNPMayStore, SDNPMemOperand]>;
+
+def : Pat<(wasm_table_set i32:$table, i32:$idx, funcref:$r),
+          (TABLE_SET_FUNCREF i32:$table, i32:$idx, funcref:$r)>,
+          Requires<[HasReferenceTypes]>;
+def : Pat<(wasm_table_set i32:$table, i32:$idx, externref:$r),
+          (TABLE_SET_EXTERNREF i32:$table, i32:$idx, externref:$r)>,
+          Requires<[HasReferenceTypes]>;
+
 defm TABLE_SIZE : I<(outs I32:$sz), (ins table32_op:$table),
                     (outs), (ins table32_op:$table),
                     [],
                     "table.size\t$sz, $table",
                     "table.size\t$table",
                     0xfc10>,
                     Requires<[HasReferenceTypes]>;
 
 
 defm TABLE_COPY : I<(outs), (ins table32_op:$table1, table32_op:$table2, I32:$d, I32:$s, I32:$n),
                     (outs), (ins table32_op:$table1, table32_op:$table2),
                     [],
                     "table.copy\t$table1, $table2, $d, $s, $n",
                     "table.copy\t$table1, $table2",
                     0xfc0e>,
                     Requires<[HasReferenceTypes]>;

llvm/lib/Target/WebAssembly/WebAssemblyMCInstLower.cpp

static wasm::ValType getType(const TargetRegisterClass *RC) {
   if (RC == &WebAssembly::I64RegClass)
     return wasm::ValType::I64;
   if (RC == &WebAssembly::F32RegClass)
     return wasm::ValType::F32;
   if (RC == &WebAssembly::F64RegClass)
     return wasm::ValType::F64;
   if (RC == &WebAssembly::V128RegClass)
     return wasm::ValType::V128;
+  if (RC == &WebAssembly::EXTERNREFRegClass)
+    return wasm::ValType::EXTERNREF;
+  if (RC == &WebAssembly::FUNCREFRegClass)
+    return wasm::ValType::FUNCREF;
   llvm_unreachable("Unexpected register class");
 }
 
 static void getFunctionReturns(const MachineInstr *MI,
                                SmallVectorImpl<wasm::ValType> &Returns) {
   const Function &F = MI->getMF()->getFunction();
   const TargetMachine &TM = MI->getMF()->getTarget();
   Type *RetTy = F.getReturnType();

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

if (!TT.isOSEmscripten()) {
     // that only works for Emscripten, so disable them if we aren't targeting
     // Emscripten.
     return Reloc::Static;
   }
 
   return *RM;
 }
 
+// Check for reference types in Feature String, in order to extend target
+// description string
+static bool hasReferenceTypes(const StringRef &FS) {
+  return FS.find("+reference-types") != StringRef::npos;
+}
+
 /// Create an WebAssembly architecture model.
 ///
 WebAssemblyTargetMachine::WebAssemblyTargetMachine(
     const Target &T, const Triple &TT, StringRef CPU, StringRef FS,
     const TargetOptions &Options, Optional<Reloc::Model> RM,
     Optional<CodeModel::Model> CM, CodeGenOpt::Level OL, bool JIT)
     : LLVMTargetMachine(T,
                         TT.isArch64Bit()
-                            ? "e-m:e-p:64:64-i64:64-n32:64-S128-ni:1"
-                            : "e-m:e-p:32:32-i64:64-n32:64-S128-ni:1",
+                            ? (hasReferenceTypes(FS)
+                                   ? "e-m:e-p:64:64-i64:64-n32:64-S128-ni:1:10:20"
+                                   : "e-m:e-p:64:64-i64:64-n32:64-S128-ni:1")
+                            : (hasReferenceTypes(FS)
+                                   ? "e-m:e-p:32:32-i64:64-n32:64-S128-ni:1:10:20"
+                                   : "e-m:e-p:32:32-i64:64-n32:64-S128-ni:1"),
                         TT, CPU, FS, Options, getEffectiveRelocModel(RM, TT),
                         getEffectiveCodeModel(CM, CodeModel::Large), OL),
       TLOF(new WebAssemblyTargetObjectFile()) {
   // WebAssembly type-checks instructions, but a noreturn function with a return
   // type that doesn't match the context will cause a check failure. So we lower
   // LLVM 'unreachable' to ISD::TRAP and then lower that to WebAssembly's
   // 'unreachable' instructions which is meant for that case.
   this->Options.TrapUnreachable = true;

llvm/test/CodeGen/WebAssembly/externref-globalget.ll

+; RUN: llc < %s --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types | FileCheck %s
+
+%extern = type opaque
+%externref = type %extern addrspace(10)* ;; addrspace 10 is nonintegral
+
+@externref_global = local_unnamed_addr addrspace(1) global %externref undef
+
+define %externref @return_externref_global() {
+  ;; this generates a global.get of @externref_global
+  %ref = load %externref, %externref addrspace(1)* @externref_global
+  ret %externref %ref
+}
+
+; CHECK-LABEL: return_externref_global:
+; CHECK-NEXT: functype       return_externref_global () -> (externref)
+; CHECK-NEXT: global.get externref_global
+; CHECK-NEXT: end_function
+
+; CHECK: .globl externref_global

llvm/test/CodeGen/WebAssembly/externref-globalset.ll

+; RUN: llc --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types < %s | FileCheck %s
+
+%extern = type opaque
+%externref = type %extern addrspace(10)* ;; addrspace 10 is nonintegral
+
+@externref_global = local_unnamed_addr addrspace(1) global %externref undef
+
+define void @set_externref_global(%externref %g) {
+  ;; this generates a global.set of @externref.global
+  store %externref %g, %externref addrspace(1)* @externref_global
+  ret void
+}
+
+; CHECK-LABEL: set_externref_global:
+; CHECK-NEXT: functype       set_externref_global (externref) -> ()
+; CHECK-NEXT: local.get      0
+; CHECK-NEXT: global.set     externref_global
+; CHECK-NEXT: end_function
+
+; CHECK: .globl externref_global

llvm/test/CodeGen/WebAssembly/externref-inttoptr.ll

+; RUN: not llc --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types < %s 2>&1 | FileCheck %s --check-prefix=CHECK-ERROR
+
+%extern = type opaque
+%externref = type %extern addrspace(10)*
+
+define %externref @int_to_externref(i32 %i) {
+  %ref = inttoptr i32 %i to %externref
+  ret %externref %ref
+}
+
+; CHECK-ERROR: inttoptr not supported for non-integral pointers

llvm/test/CodeGen/WebAssembly/externref-ptrtoint.ll

+; RUN: not llc --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types < %s 2>&1 | FileCheck %s --check-prefix=CHECK-ERROR
+
+%extern = type opaque
+%externref = type %extern addrspace(10)*
+
+define i32 @externref_to_int(%externref %ref) {
+  %i = ptrtoint %externref %ref to i32
+  ret i32 %i
+}
+
+; CHECK-ERROR: ptrtoint not supported for non-integral pointers

llvm/test/CodeGen/WebAssembly/externref-undef.ll

+; RUN: llc < %s --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types | FileCheck %s
+
+%extern = type opaque
+%externref = type %extern addrspace(10)* ;; addrspace 10 is nonintegral
+
+@externref_global = local_unnamed_addr addrspace(1) global %externref undef
+
+define %extern @return_extern_undef() {
+  ; Returning a ref.null or an uninitialized externref would make
+  ; more sense if the return type would be %externref. However, in
+  ; this case this is an %extern value, which really is an opaque
+  ; type and should never really happen.
+  ret %extern undef
+}
+
+; CHECK-LABEL: return_extern_undef:
+; CHECK-NEXT: functype       return_extern_undef () -> ()
+; CHECK-NEXT: end_function
+
+; CHECK: .globl externref_global
+

llvm/test/CodeGen/WebAssembly/externref-unsized-load.ll

+; RUN: not llc --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types < %s 2>&1 | FileCheck %s --check-prefix=CHECK-ERROR
+
+%extern = type opaque
+%externref = type %extern addrspace(10)*
+
+define void @load_extern(%externref %ref) {
+  %e = load %extern, %externref %ref
+  ret void
+}
+
+; CHECK-ERROR: error: loading unsized types is not allowed

llvm/test/CodeGen/WebAssembly/externref-unsized-store.ll

+; RUN: not llc --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types < %s 2>&1 | FileCheck %s --check-prefix=CHECK-ERROR
+
+%extern = type opaque
+%externref = type %extern addrspace(10)*
+
+define void @store_extern(%externref %ref) {
+  store %extern undef, %externref %ref
+  ret void
+}
+
+; CHECK-ERROR: error: storing unsized types is not allowed

llvm/test/CodeGen/WebAssembly/funcref-call.ll

+; RUN: llc < %s --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types | FileCheck %s
+
+%func = type void ()
+%funcref = type %func addrspace(20)* ;; addrspace 20 is nonintegral
+
+define void @call_funcref(%funcref %ref) {
+  call addrspace(20) void %ref() 
+  ret void
+}
+
+; CHECK-LABEL: call_funcref:
+; CHECK-NEXT: functype       call_funcref (funcref) -> ()
+; CHECK-NEXT: i32.const 0
+; CHECK-NEXT: local.get 0
+; CHECK-NEXT: table.set __funcref_call_table
+; CHECK-NEXT: local.get 0
+; CHECK-NEXT: call_indirect __funcref_call_table, () -> ()
+; CHECK-NEXT: i32.const 0
+; CHECK-NEXT: ref.null func
+; CHECK-NEXT: table.set __funcref_call_table
+; CHECK-NEXT: end_function
+
+; CHECK: .tabletype __funcref_call_table, funcref

llvm/test/CodeGen/WebAssembly/funcref-globalget.ll

+; RUN: llc < %s --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types | FileCheck %s
+
+%func = type opaque
+%funcref = type %func addrspace(20)* ;; addrspace 20 is nonintegral
+
+@funcref_global = local_unnamed_addr addrspace(1) global %funcref undef
+
+define %funcref @return_funcref_global() {
+  ;; this generates a global.get of @funcref_global
+  %ref = load %funcref, %funcref addrspace(1)* @funcref_global
+  ret %funcref %ref
+}
+
+; CHECK-LABEL: return_funcref_global:
+; CHECK-NEXT: .functype       return_funcref_global () -> (funcref)
+; CHECK-NEXT: global.get funcref_global
+; CHECK-NEXT: end_function
+
+; CHECK: .globl funcref_global

llvm/test/CodeGen/WebAssembly/funcref-globalset.ll

+; RUN: llc < %s --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types | FileCheck %s
+
+%func = type opaque
+%funcref = type %func addrspace(20)* ;; addrspace 20 is nonintegral
+
+@funcref_global = local_unnamed_addr addrspace(1) global %funcref undef
+
+define void @set_funcref_global(%funcref %g) {
+  ;; this generates a global.set of @funcref_global
+  store %funcref %g, %funcref addrspace(1)* @funcref_global
+  ret void
+}
+
+; CHECK-LABEL: set_funcref_global:
+; CHECK-NEXT: functype       set_funcref_global (funcref) -> ()
+; CHECK-NEXT: local.get      0
+; CHECK-NEXT: global.set     funcref_global
+; CHECK-NEXT: end_function
+
+; CHECK: .globl funcref_global