[XCOFF][AIX] report_fatal_error when an overflow section is needed

[jasonliu]

Summary

If there are more than 65534 relocation entries in a single section, we should generate an overflow section.
Since we don't support overflow section for now, we should generate an error.

[DiggerLin]

why the file llvm/test/CodeGen/PowerPC/aix-xcoff-reloc-overflow.ll so large ,it can not be loaded?

[jasonliu]

In D81104#2072208, @DiggerLin wrote:

why the file llvm/test/CodeGen/PowerPC/aix-xcoff-reloc-overflow.ll so large ,it can not be loaded?

I was able to load it by clicking Show File Contents.
I'm open to any ideas that could greatly reduce the file size and still trigger the error I put in.

[daltenty]

llvm/test/CodeGen/PowerPC/aix-xcoff-reloc-overflow.ll

6 ↗

(On Diff #268223)

We're able to sed and friends in tests, so I think we can do an expansion based on @hubert.reinterpretcast original preprocessor example. Replacing the repeated line with 'A' and using something like this seems like it would do the trick:

sed "s/A/ABA/g;s/A/ABA/g;s/A/ABA/g;s/A/ABA/g;s/A/ABA/g;s/A/ABA/g;s/A/ABA/g;s/A/ABA/g;s/A/ABA/g;s/A/ABA/g;s/A/ABA/g;s/A/ABA/g;s/A/ABA/g;s/A/ABA/g;s/A/ABA/g;s/A/ABA/g;" %s| tr B "\n" | sed "s/A/call\ void\ bitcast\ \(void (...)*\ @foo\ to\ void\ ()*)()/g"|llc -verify-machineinstrs -mtriple powerpc-ibm-aix-xcoff -mcpu=pwr4 -mattr=-altivec -filetype=obj -o %t.o

[jasonliu]

Use sed to generate the test.

[jasonliu]

llvm/test/CodeGen/PowerPC/aix-xcoff-reloc-overflow.ll

6 ↗

(On Diff #268223)

Thanks for the suggestion.
I could not 'sed' the same file that the 'sed' command resides. So I created an input file instead.

[hubert.reinterpretcast]

llvm/test/CodeGen/PowerPC/aix-xcoff-reloc-overflow.ll

6 ↗

(On Diff #268223)

Would grep -v have helped there?

[jasonliu]

llvm/test/CodeGen/PowerPC/aix-xcoff-reloc-overflow.ll

6 ↗

(On Diff #268223)

Hmm... I'm not sure how grep -v would help.
The problem I'm seeing is:
when this file get passed into the sed command, sed command would replace the A recursively inside this sed command, which would result in an output file like:

;; This test generates more than 65535 relocation entries in a single section,
;; which would trigger an overflow section to be generated in 32-bit mode.
;; Since overflow section is not supported yet, we will emit an error instead of
;; generating an invalid binary for now.

; RUN: sed "s/call void bitcast (void (...)* @foo to void ()*)()
call void bitcast (void (...)* @foo to void ()*)()
call void bitcast (void (...)* @foo to void ()*)()
call void bitcast (void (...)* @foo to void ()*)()
.... 
/g;" \
; RUN:     %p/Inputs/reloc-sed.ll | tr B "\n" | \
....

[jasonliu]

Remove input files and use grep -v

[hubert.reinterpretcast]

llvm/lib/MC/XCOFFObjectWriter.cpp

744

Semantically, this is not UINT16_MAX but an XCOFF-defined constant. Otherwise, the maximum representable value would be UINT16_MAX itself and not one less. RelocOverflow is currently defined in lib/Object/XCOFFObjectFile.cpp, but it should probably be defined in include/llvm/BinaryFormat/XCOFF.h.

752

Move the check to before the addition; use: Section->RelocationCount >= UINT16_MAX - Csect.Relocations.size(). After said move, the two if statements can be merged.

[hubert.reinterpretcast]

llvm/test/CodeGen/PowerPC/aix-xcoff-reloc-overflow.ll

7 ↗

(On Diff #268634)

I suggest testing exactly 65535:

; RUN: grep -v RUN: %s | \
; RUN:   sed >%t.overflow.ll 's/SIZE/65535/;s/MACRO/#/;s/#/################/g;s/#/################/g;s/#/################/g;s/#/################/g;s/#/#_/g;s/_#_\([^#]\)/\1/;s/_/, /g;s/#/i8* @c/g;'
; RUN: not --crash llc -verify-machineinstrs -mtriple powerpc-ibm-aix-xcoff \
; RUN:     -mcpu=pwr4 -mattr=-altivec -filetype=obj -o %t.o %t.overflow.ll 2>&1 | \
; RUN:   FileCheck --check-prefix=XCOFF32 %s
; XCOFF32: LLVM ERROR: relocation entries overflowed; overflow section is not implemented yet

; RUN: not --crash llc -verify-machineinstrs -mtriple powerpc64-ibm-aix-xcoff \
; RUN:     -mcpu=pwr4 -mattr=-altivec -filetype=obj -o %t.o %t.overflow.ll 2>&1 | \
; RUN:   FileCheck --check-prefix=XCOFF64 %s
; XCOFF64: LLVM ERROR: 64-bit XCOFF object files are not supported yet.

@c = external global i8, align 1
@arr = global [SIZE x i8*] [MACRO], align 8

Also, I think we should test for 65534.

[jasonliu]

Address Hubert's comment.

[jasonliu]

llvm/test/CodeGen/PowerPC/aix-xcoff-reloc-overflow.ll

7 ↗

(On Diff #268634)

Thanks. I was pleasantly surprised that the suggested test(4 llc invocation) finished within 30 seconds.

[daltenty]

LGTM

llvm/lib/MC/XCOFFObjectWriter.cpp

756

nit: Capitalization and period.

[jasonliu]

llvm/lib/MC/XCOFFObjectWriter.cpp

756

https://llvm.org/docs/CodingStandards.html#error-and-warning-messages
ask us not to capitalize first letter and end with period.

[hubert.reinterpretcast]

LGTM with minor comments.

llvm/test/CodeGen/PowerPC/aix-xcoff-huge-relocs.ll

9

Minor nit: Not sure if we want to align -mcpu with the llc options?

22

Same comment.

25

Same comment.

32

I think we'd need the NOT lines after the other lines as well.

llvm/include/llvm/BinaryFormat/XCOFF.h

 namespace XCOFF {
 
 // Constants used in the XCOFF definition.
 
 constexpr size_t FileNamePadSize = 6;
 constexpr size_t NameSize = 8;
 constexpr size_t SymbolTableEntrySize = 18;
 constexpr size_t RelocationSerializationSize32 = 10;
+constexpr uint16_t RelocOverflow = 65535;
 
 enum ReservedSectionNum : int16_t { N_DEBUG = -2, N_ABS = -1, N_UNDEF = 0 };
 
 // x_smclas field of x_csect from system header: /usr/include/syms.h
 /// Storage Mapping Class definitions.
 enum StorageMappingClass : uint8_t {
   //     READ ONLY CLASSES
   XMC_PR = 0,      ///< Program Code

llvm/lib/MC/XCOFFObjectWriter.cpp

for (auto *Section : Sections) {
     if (Section->Index == Section::UninitializedIndex)
       // Nothing to record for this Section.
       continue;
 
     for (const auto *Group : Section->Groups) {
       if (Group->empty())
         continue;
 
-      for (auto &Csect : *Group)
+      for (auto &Csect : *Group) {
+        if (Csect.Relocations.size() >= XCOFF::RelocOverflow ||
+            Section->RelocationCount >=
+                XCOFF::RelocOverflow - Csect.Relocations.size())
+          report_fatal_error(
+              "relocation entries overflowed; overflow section is "
+              "not implemented yet");
+
         Section->RelocationCount += Csect.Relocations.size();
       }
     }
+  }
 
   // Calculate the file offset to the relocation entries.
   uint64_t RawPointer = RelocationEntryOffset;
   for (auto Sec : Sections) {
     if (Sec->Index == Section::UninitializedIndex || !Sec->RelocationCount)
       continue;
 
     Sec->FileOffsetToRelocations = RawPointer;
     const uint32_t RelocationSizeInSec =
         Sec->RelocationCount * XCOFF::RelocationSerializationSize32;

llvm/lib/Object/XCOFFObjectFile.cpp

 #include <cstddef>
 #include <cstring>
 
 namespace llvm {
 namespace object {
 
 static const uint8_t FunctionSym = 0x20;
 static const uint8_t SymTypeMask = 0x07;
-static const uint16_t RelocOverflow = 65535;
 static const uint16_t NoRelMask = 0x0001;
 
 // Checks that [Ptr, Ptr + Size) bytes fall inside the memory buffer
 // 'M'. Returns a pointer to the underlying object on success.
 template <typename T>
 static Expected<const T *> getObject(MemoryBufferRef M, const void *Ptr,
                                      const uint64_t Size = sizeof(T)) {
   uintptr_t Addr = uintptr_t(Ptr);

 // section header contains the actual count of relocation entries in the s_paddr
 // field. STYP_OVRFLO headers contain the section index of their corresponding
 // sections as their raw "NumberOfRelocations" field value.
 Expected<uint32_t> XCOFFObjectFile::getLogicalNumberOfRelocationEntries(
     const XCOFFSectionHeader32 &Sec) const {
 
   uint16_t SectionIndex = &Sec - sectionHeaderTable32() + 1;
 
-  if (Sec.NumberOfRelocations < RelocOverflow)
+  if (Sec.NumberOfRelocations < XCOFF::RelocOverflow)
     return Sec.NumberOfRelocations;
   for (const auto &Sec : sections32()) {
     if (Sec.Flags == XCOFF::STYP_OVRFLO &&
         Sec.NumberOfRelocations == SectionIndex)
       return Sec.PhysicalAddress;
   }
   return errorCodeToError(object_error::parse_failed);
 }

llvm/test/CodeGen/PowerPC/aix-xcoff-huge-relocs.ll

+;; This test generates 65535 relocation entries in a single section,
+;; which would trigger an overflow section to be generated in 32-bit mode.
+;; Since overflow section is not supported yet, we will emit an error instead of
+;; generating an invalid binary for now.
+; RUN: grep -v RUN: %s | \
+; RUN:   sed >%t.overflow.ll 's/SIZE/65535/;s/MACRO/#/;s/#/################/g;s/#/################/g;s/#/################/g;s/#/################/g;s/#/#_/g;s/_#_\([^#]\)/\1/;s/_/, /g;s/#/i8* @c/g;'
+; RUN: not --crash llc -verify-machineinstrs -mtriple powerpc-ibm-aix-xcoff \
+; RUN:     -mcpu=pwr4 -mattr=-altivec -filetype=obj -o %t.o %t.overflow.ll 2>&1 | \
+; RUN:   FileCheck --check-prefix=OVERFLOW %s
+; OVERFLOW: LLVM ERROR: relocation entries overflowed; overflow section is not implemented yet
+
+;; This test generates 65534 relocation entries, an overflow section should 
+;; not be generated.
+; RUN: grep -v RUN: %s | \
+; RUN:   sed >%t.ll 's/SIZE/65534/;s/MACRO/#/;s/#/################/g;s/#/################/g;s/#/################/g;s/#/################/g;s/#/#_/g;s/_#_#_\([^#]\)/\1/;s/_/, /g;s/#/i8* @c/g;'
+; RUN: llc -verify-machineinstrs -mtriple powerpc-ibm-aix-xcoff \
+; RUN:     -mcpu=pwr4 -mattr=-altivec -filetype=obj -o %t.o %t.ll
+; RUN: llvm-readobj --section-headers %t.o | FileCheck --check-prefix=XCOFF32 %s
+
+; RUN: not --crash llc -verify-machineinstrs -mtriple powerpc64-ibm-aix-xcoff \
+; RUN:     -mcpu=pwr4 -mattr=-altivec -filetype=obj -o %t.o %t.overflow.ll 2>&1 | \
+; RUN:   FileCheck --check-prefix=XCOFF64 %s
+; RUN: not --crash llc -verify-machineinstrs -mtriple powerpc64-ibm-aix-xcoff \
+; RUN:     -mcpu=pwr4 -mattr=-altivec -filetype=obj -o %t.o %t.ll 2>&1 | \
+; RUN:   FileCheck --check-prefix=XCOFF64 %s
+; XCOFF64: LLVM ERROR: 64-bit XCOFF object files are not supported yet.
+
+@c = external global i8, align 1
+@arr = global [SIZE x i8*] [MACRO], align 8
+
+; XCOFF32-NOT:     Name: .ovrflo
+; XCOFF32-NOT:     Type: STYP_OVRFLO
+; XCOFF32:       Section {
+; XCOFF32:         Name: .data
+; XCOFF32-NEXT:    PhysicalAddress: 0x0
+; XCOFF32-NEXT:    VirtualAddress: 0x0
+; XCOFF32-NEXT:    Size: 0x3FFF8
+; XCOFF32-NEXT:    RawDataOffset: 0x64
+; XCOFF32-NEXT:    RelocationPointer: 0x4005C
+; XCOFF32-NEXT:    LineNumberPointer: 0x0
+; XCOFF32-NEXT:    NumberOfRelocations: 65534
+; XCOFF32-NEXT:    NumberOfLineNumbers: 0
+; XCOFF32-NEXT:    Type: STYP_DATA (0x40)
+; XCOFF32-NEXT:  }