[WIP] Add Zstd ELF support

[ckissane]

Summary

• Would add ZSTD compression as a elf section compression option with enum value 2

[ckissane]

simplifed codeflow

[MaskRay]

The lld/ change should be separate. And it needs tests. We should have several tests such as mixed zlib and zstd input, malformed zstd input, compressed level, etc. OutputSections should not have a new member. The parallel output part will be tricky, too (see D117853). I can handle the lld/ELF part and leave other stuff to you.

llvm-objcopy should probably be a separate patch as well.

• clang (Driver, cc1as) and MC (including llvm-mc --compress-debug-sections) in one patch
• lld/ELF
• llvm-objcopy

Context not available.

See https://llvm.org/docs/Phabricator.html#requesting-a-review-via-the-web-interface "To make reviews easier, please always include as much context as possible with your diff!"

llvm/include/llvm/BinaryFormat/ELF.h

1772

Zstandard algorithm

llvm/lib/MC/ELFObjectWriter.cpp

149

Having two variables is ugly. Just pass MAI->compressDebugSections() as an argument.

884

if (ZstdStyle)

llvm/lib/ObjCopy/ELF/ELFObject.cpp

565

clang-format

llvm/lib/Object/Decompressor.cpp

24

clang-format

Please check the prevailing style first.

llvm/tools/llvm-objcopy/ObjcopyOptions.cpp

744

clang-format

1014

clang-format

[ckissane]

remove extra struct member

[ckissane]

add more diff context

[ckissane]

In D128667#3612956, @MaskRay wrote:

The lld/ change should be separate. And it needs tests. We should have several tests such as mixed zlib and zstd input, malformed zstd input, compressed level, etc. OutputSections should not have a new member.

Got it, I have also removed the extra member.

In D128667#3612956, @MaskRay wrote:

The parallel output part will be tricky, too (see D117853). I can handle the lld/ELF part and leave other stuff to you.

Sounds good, and thank you for your offer to help!

In D128667#3612956, @MaskRay wrote:

llvm-objcopy should probably be a separate patch as well.

got it.

• clang (Driver, cc1as) and MC (including llvm-mc --compress-debug-sections) in one patch
• lld/ELF
• llvm-objcopy

Context not available.

See https://llvm.org/docs/Phabricator.html#requesting-a-review-via-the-web-interface "To make reviews easier, please always include as much context as possible with your diff!"

Doing so now

[MaskRay]

Tips: you may run git diff -U0 --no-color --relative 'HEAD^' -- | path/to/llvm-project/clang/tools/clang-format/clang-format-diff.py -p1 -i to clang-format your local change.

lld/ELF/Driver.cpp

957

https://llvm.org/docs/CodingStandards.html#don-t-use-braces-on-simple-single-statement-bodies-of-if-else-loop-statements

965

https://llvm.org/docs/CodingStandards.html#don-t-use-braces-on-simple-single-statement-bodies-of-if-else-loop-statements

lld/ELF/InputSection.h

389

Wwe should not increase the size of InputSection which may have great impact on memory usage. So I mention that I will prefer handling the lld/ELF part.

[ckissane]

format

[ckissane]

improve style on ELFWriter::maybeWriteCompression
fix typo in comment on zstd value for elf enum

[MaskRay]

llvm-objcopy --decompress-debug-sections clang.zstd clang.zstd.uncompress doesn't work.

% /tmp/out/custom1/bin/llvm-objcopy --decompress-debug-sections clang.zstd clang.zstd.uncompress
/tmp/out/custom1/bin/llvm-objcopy: error: 'clang.zstd': '.debug_loc': zlib error: Z_DATA_ERROR

Add a new test llvm/test/tools/llvm-objcopy/compress-debug-sections-zstd.test and check existing tests for zlib (SHF_COMPRESSED).
I'll create a patch to remove .zdebug support to simplify the code.

Tip: you can nullify a compression/uncompression place, then run ninja check-llvm to find all places in llvm/ where a test may be relevant.

[ckissane]

In D128667#3613222, @MaskRay wrote:

llvm-objcopy --decompress-debug-sections clang.zstd clang.zstd.uncompress doesn't work.

% /tmp/out/custom1/bin/llvm-objcopy --decompress-debug-sections clang.zstd clang.zstd.uncompress
/tmp/out/custom1/bin/llvm-objcopy: error: 'clang.zstd': '.debug_loc': zlib error: Z_DATA_ERROR

Add a new test llvm/test/tools/llvm-objcopy/compress-debug-sections-zstd.test and check existing tests for zlib (SHF_COMPRESSED).
I'll create a patch to remove .zdebug support to simplify the code.

Tip: you can nullify a compression/uncompression place, then run ninja check-llvm to find all places in llvm/ where a test may be relevant.

it seems that --decompress-debug-sections assumes always Z type compression. I can update the patch to fix this

[dblaikie]

Yeah, probably worth splitting this up a bit - somewhat hard to split up generation+parsing in LLVM itself (eg: llvm-mc+llvm-objdump), so maybe they're grouped together (alternatively the objdump support gets checked in first with precompiled/binary test files - I forget what I did when I added the compression support initially) - but lld at least can be separated out.

lld/ELF/InputSection.cpp

78–80

Perhaps we can delay this failure/error until the type of compressed section is parsed, then error if the specific kind of compression isn't available - rather than giving the user a more vague error here that doesn't tell them which kind of compression they need to be built into the linker.

(but yeah, if @MaskRay is going to take the lld support into a separate patch the feedback/discussion can happen there)

[jhenderson]

llvm/lib/ObjCopy/ELF/ELFObject.cpp

567–568

This StringRef construction is identical in both parts of the if, so should be pulled out of the if statement into a local variable.

llvm/tools/llvm-objcopy/ObjcopyOptions.cpp

1009–1019

This is identical (I think?) to a block further up in the file. Could we pull it out into a helper function, please? Something like the following:

Error checkCompressionAvailability() {
   /* moved code here */
}

// Usage would look like:
if (Error err = checkCompressionAvailability())
  return err;

[ckissane]

In D128667#3612956, @MaskRay wrote:

The lld/ change should be separate. And it needs tests. We should have several tests such as mixed zlib and zstd input, malformed zstd input, compressed level, etc. OutputSections should not have a new member. The parallel output part will be tricky, too (see D117853). I can handle the lld/ELF part and leave other stuff to you.

llvm-objcopy should probably be a separate patch as well.

• clang (Driver, cc1as) and MC (including llvm-mc --compress-debug-sections) in one patch
• lld/ELF
• llvm-objcopy

Context not available.

See https://llvm.org/docs/Phabricator.html#requesting-a-review-via-the-web-interface "To make reviews easier, please always include as much context as possible with your diff!"

@MaskRay I would gladly accept your help on lld/ELF and I'll break this patch up into 2 patches:

• clang (Driver, cc1as) and MC (including llvm-mc --compress-debug-sections) in one patch
• llvm-objcopy

as suggested.
Also, I believe you requested a git repo for the whole picture on adding zstd: https://github.com/ckissane/llvm-project/tree/ckissane.add-zstd-compression

[MaskRay]

edit: ELFCOMPRESS_ZSTD https://groups.google.com/g/generic-abi/c/satyPkuMisk has been approved.

[MaskRay]

llvm/tools/llvm-objcopy/ObjcopyOptions.cpp

1015

For --decompress-debug-sections, the zstd error should be reported only when a compressed debug section using zstd is found. zlib is similar.

I cleaned up legacy zdebug code from llvm-objcopy, fixed an uninitialized ch_reserved bug. Created D130458 as an alternative patch.

clang/lib/Driver/ToolChains/Clang.cpp

   const Arg *A = Args.getLastArg(options::OPT_gz_EQ);
   if (!A)
     return;
   if (checkDebugInfoOption(A, Args, D, TC)) {
     StringRef Value = A->getValue();
     if (Value == "none") {
       CmdArgs.push_back("--compress-debug-sections=none");
     } else if (Value == "zlib") {
-      if (llvm::compression::elf::isAvailable()) {
+      if (llvm::compression::zlib::isAvailable()) {
+        CmdArgs.push_back(
+            Args.MakeArgString("--compress-debug-sections=" + Twine(Value)));
+      } else {
+        D.Diag(diag::warn_debug_compression_unavailable);
+      }
+    } else if (Value == "zstd") {
+      if (llvm::compression::zstd::isAvailable()) {
         CmdArgs.push_back(
             Args.MakeArgString("--compress-debug-sections=" + Twine(Value)));
       } else {
         D.Diag(diag::warn_debug_compression_unavailable);
       }
     } else {
       D.Diag(diag::err_drv_unsupported_option_argument)
           << A->getOption().getName() << Value;

clang/lib/Driver/ToolChains/CommonArgs.cpp

     llvm::opt::ArgStringList &CmdArgs) {
   // GNU ld supports --compress-debug-sections=none|zlib|zlib-gnu|zlib-gabi
   // whereas zlib is an alias to zlib-gabi and zlib-gnu is obsoleted. Therefore
   // -gz=none|zlib are translated to --compress-debug-sections=none|zlib. -gz
   // is not translated since ld --compress-debug-sections option requires an
   // argument.
   if (const Arg *A = Args.getLastArg(options::OPT_gz_EQ)) {
     StringRef V = A->getValue();
-    if (V == "none" || V == "zlib")
+    if (V == "none" || V == "zlib" || V == "zstd")
       CmdArgs.push_back(Args.MakeArgString("--compress-debug-sections=" + V));
     else
       TC.getDriver().Diag(diag::err_drv_unsupported_option_argument)
           << A->getOption().getName() << V;
   }
 }
 
 void tools::AddTargetFeature(const ArgList &Args,

clang/lib/Driver/ToolChains/Gnu.cpp

   std::tie(RelocationModel, PICLevel, IsPIE) =
       ParsePICArgs(getToolChain(), Args);
 
   if (const Arg *A = Args.getLastArg(options::OPT_gz, options::OPT_gz_EQ)) {
     if (A->getOption().getID() == options::OPT_gz) {
       CmdArgs.push_back("--compress-debug-sections");
     } else {
       StringRef Value = A->getValue();
-      if (Value == "none" || Value == "zlib") {
+      if (Value == "none" || Value == "zlib" || Value == "zstd") {
         CmdArgs.push_back(
             Args.MakeArgString("--compress-debug-sections=" + Twine(Value)));
       } else {
         D.Diag(diag::err_drv_unsupported_option_argument)
             << A->getOption().getName() << Value;
       }
     }
   }

lld/ELF/Driver.cpp

     }
   }
 }
 
 static bool getCompressDebugSections(opt::InputArgList &args) {
   StringRef s = args.getLastArgValue(OPT_compress_debug_sections, "none");
   if (s == "none")
     return false;
-  if (s != "zlib")
-    error("unknown --compress-debug-sections value: " + s);
-  if (!compression::elf::isAvailable())
+
+  if (s == "zlib") {
+    if (compression::zlib::isAvailable()) {
+      return true;
+    }
     error("--compress-debug-sections: zlib is not available");
-  return true;
+    return false;
+  }
+
+  if (s == "zstd") {
+    if (compression::zstd::isAvailable()) {
+      return true;
+    }
+    error("--compress-debug-sections: zstd is not available");
+    return false;
+  }
+
+  error("unknown --compress-debug-sections value: " + s);
+  return false;
 }
 
 static StringRef getAliasSpelling(opt::Arg *arg) {
   if (const opt::Arg *alias = arg->getAlias())
     return alias->getSpelling();
   return arg->getSpelling();
 }
 

lld/ELF/InputSection.h

 //===- InputSection.h -------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLD_ELF_INPUT_SECTION_H
 #define LLD_ELF_INPUT_SECTION_H
 
 #include "Relocations.h"
 #include "lld/Common/LLVM.h"
 #include "llvm/ADT/CachedHashString.h"
 #include "llvm/ADT/DenseSet.h"
 #include "llvm/ADT/TinyPtrVector.h"
+#include "llvm/MC/MCTargetOptions.h"
 #include "llvm/Object/ELF.h"
 
 namespace lld {
 namespace elf {
 
 class InputFile;
 class Symbol;
 

   void parseCompressedHeader();
   void uncompress() const;
 
   // This field stores the uncompressed size of the compressed data in rawData,
   // or -1 if rawData is not compressed (either because the section wasn't
   // compressed in the first place, or because we ended up uncompressing it).
   // Since the feature is not used often, this is usually -1.
   mutable int64_t uncompressedSize = -1;
+
+  llvm::DebugCompressionType CompressionType;
 };
 
 // SectionPiece represents a piece of splittable section contents.
 // We allocate a lot of these and binary search on them. This means that they
 // have to be as compact as possible, which is why we don't store the size (can
 // be found by looking at the next one).
 struct SectionPiece {
   SectionPiece() = default;

 
 private:
   template <class ELFT, class RelTy>
   void copyRelocations(uint8_t *buf, llvm::ArrayRef<RelTy> rels);
 
   template <class ELFT> void copyShtGroup(uint8_t *buf);
 };
 
-static_assert(sizeof(InputSection) <= 160, "InputSection is too big");
+static_assert(sizeof(InputSection) <= 168, "InputSection is too big");
 
 inline bool isDebugSection(const InputSectionBase &sec) {
   return (sec.flags & llvm::ELF::SHF_ALLOC) == 0 &&
          sec.name.startswith(".debug");
 }
 
 // The list of all input sections.
 extern SmallVector<InputSectionBase *, 0> inputSections;

lld/ELF/InputSection.cpp

 #include "InputFiles.h"
 #include "OutputSections.h"
 #include "Relocations.h"
 #include "SymbolTable.h"
 #include "Symbols.h"
 #include "SyntheticSections.h"
 #include "Target.h"
 #include "lld/Common/CommonLinkerContext.h"
+#include "llvm/BinaryFormat/ELF.h"
+#include "llvm/MC/MCTargetOptions.h"
 #include "llvm/Support/Compiler.h"
 #include "llvm/Support/Compression.h"
 #include "llvm/Support/Endian.h"
 #include "llvm/Support/xxhash.h"
 #include <algorithm>
 #include <mutex>
 #include <vector>
 

   uint32_t v = std::max<uint32_t>(alignment, 1);
   if (!isPowerOf2_64(v))
     fatal(toString(this) + ": sh_addralign is not a power of 2");
   this->alignment = v;
 
   // If SHF_COMPRESSED is set, parse the header. The legacy .zdebug format is no
   // longer supported.
   if (flags & SHF_COMPRESSED) {
-    if (!compression::elf::isAvailable())
+    if (!compression::zlib::isAvailable() || !compression::zstd::isAvailable())
       error(toString(file) + ": contains a compressed section, " +
-            "but zlib is not available");
+            "but zlib or zstd is not available");
     invokeELFT(parseCompressedHeader);
   }
 }
 
 // Drop SHF_GROUP bit unless we are producing a re-linkable object file.
 // SHF_GROUP is a marker that a section belongs to some comdat group.
 // That flag doesn't make sense in an executable.
 static uint64_t getFlags(uint64_t flags) {

 void InputSectionBase::uncompress() const {
   size_t size = uncompressedSize;
   char *uncompressedBuf;
   {
     static std::mutex mu;
     std::lock_guard<std::mutex> lock(mu);
     uncompressedBuf = bAlloc().Allocate<char>(size);
   }
-
-  if (Error e = compression::elf::uncompress(toStringRef(rawData),
-                                             uncompressedBuf, size))
-    fatal(toString(this) +
-          ": uncompress failed: " + llvm::toString(std::move(e)));
+  // Note: this works off the assumption that the CType has been extracted from
+  // header pre this func
+  if (CompressionType == DebugCompressionType::Zstd) {
+    if (Error e = compression::zstd::uncompress(toStringRef(rawData),
+                                                uncompressedBuf, size))
+      fatal(toString(this) +
+            ": uncompress failed: " + llvm::toString(std::move(e)));
+  } else {
+    if (Error e = compression::zlib::uncompress(toStringRef(rawData),
+                                                uncompressedBuf, size))
+      fatal(toString(this) +
+            ": uncompress failed: " + llvm::toString(std::move(e)));
+  }
   rawData = makeArrayRef((uint8_t *)uncompressedBuf, size);
   uncompressedSize = -1;
 }
 
 template <class ELFT> RelsOrRelas<ELFT> InputSectionBase::relsOrRelas() const {
   if (relSecIdx == 0)
     return {};
   RelsOrRelas<ELFT> ret;

 
   // New-style header
   if (rawData.size() < sizeof(typename ELFT::Chdr)) {
     error(toString(this) + ": corrupted compressed section");
     return;
   }
 
   auto *hdr = reinterpret_cast<const typename ELFT::Chdr *>(rawData.data());
-  if (hdr->ch_type != ELFCOMPRESS_ZLIB) {
+  if (hdr->ch_type != ELFCOMPRESS_ZLIB && (hdr->ch_type != ELFCOMPRESS_ZSTD)) {
     error(toString(this) + ": unsupported compression type");
     return;
   }
-
   uncompressedSize = hdr->ch_size;
   alignment = std::max<uint32_t>(hdr->ch_addralign, 1);
   rawData = rawData.slice(sizeof(*hdr));
 }
 
 InputSection *InputSectionBase::getLinkOrderDep() const {
   assert(flags & SHF_LINK_ORDER);
   if (!link)

     copyShtGroup<ELFT>(buf);
     return;
   }
 
   // If this is a compressed section, uncompress section contents directly
   // to the buffer.
   if (uncompressedSize >= 0) {
     size_t size = uncompressedSize;
-    if (Error e = compression::elf::uncompress(toStringRef(rawData),
-                                               (char *)buf, size))
-      fatal(toString(this) +
-            ": uncompress failed: " + llvm::toString(std::move(e)));
+    if (CompressionType == DebugCompressionType::Zstd) {
+      if (Error e = compression::zstd::uncompress(toStringRef(rawData),
+                                                  (char *)buf, size))
+        fatal(toString(this) +
+              ": uncompress failed: " + llvm::toString(std::move(e)));
+    } else {
+      if (Error e = compression::zlib::uncompress(toStringRef(rawData),
+                                                  (char *)buf, size))
+        fatal(toString(this) +
+              ": uncompress failed: " + llvm::toString(std::move(e)));
+    }
     uint8_t *bufEnd = buf + size;
     relocate<ELFT>(buf, bufEnd);
     return;
   }
 
   // Copy section contents from source object file to output file
   // and then apply relocations.
   memcpy(buf, rawData.data(), rawData.size());

lld/ELF/OutputSections.h

 //===- OutputSections.h -----------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLD_ELF_OUTPUT_SECTIONS_H
 #define LLD_ELF_OUTPUT_SECTIONS_H
 
 #include "InputSection.h"
 #include "LinkerScript.h"
 #include "lld/Common/LLVM.h"
+#include "llvm/MC/MCTargetOptions.h"
 
 #include <array>
 
 namespace lld {
 namespace elf {
 
 struct PhdrEntry;
 
 struct CompressedData {
   std::unique_ptr<SmallVector<uint8_t, 0>[]> shards;
   uint32_t numShards = 0;
   uint32_t checksum = 0;
   uint64_t uncompressedSize;
+  llvm::DebugCompressionType CompressionType;
 };
 
 // This represents a section in an output file.
 // It is composed of multiple InputSections.
 // The writer creates multiple OutputSections and assign them unique,
 // non-overlapping file offsets and VAs.
 class OutputSection final : public SectionBase {
 public:

llvm/include/llvm/BinaryFormat/ELF.h

   Elf64_Word n_namesz;
   Elf64_Word n_descsz;
   Elf64_Word n_type;
 };
 
 // Legal values for ch_type field of compressed section header.
 enum {
   ELFCOMPRESS_ZLIB = 1,            // ZLIB/DEFLATE algorithm.
+  ELFCOMPRESS_ZSTD = 2,            // Zstandard algorithm.
   ELFCOMPRESS_LOOS = 0x60000000,   // Start of OS-specific.
   ELFCOMPRESS_HIOS = 0x6fffffff,   // End of OS-specific.
   ELFCOMPRESS_LOPROC = 0x70000000, // Start of processor-specific.
   ELFCOMPRESS_HIPROC = 0x7fffffff  // End of processor-specific.
 };
 
 /// Convert an architecture name into ELF's e_machine value.
 uint16_t convertArchNameToEMachine(StringRef Arch);
 
 /// Convert an ELF's e_machine value into an architecture name.
 StringRef convertEMachineToArchName(uint16_t EMachine);
 
 } // end namespace ELF
 } // end namespace llvm
 
 #endif // LLVM_BINARYFORMAT_ELF_H

llvm/include/llvm/MC/MCTargetOptions.h

enum class ExceptionHandling {
   Wasm,     ///< WebAssembly Exception Handling
   AIX,      ///< AIX Exception Handling
 };
 
 enum class DebugCompressionType {
   None, ///< No compression
   GNU,  ///< zlib-gnu style compression
   Z,    ///< zlib style complession
+  Zstd, ///< Zstandard
 };
 
 enum class EmitDwarfUnwindType {
   Always,          // Always emit dwarf unwind
   NoCompactUnwind, // Only emit if compact unwind isn't available
   Default,         // Default behavior is based on the target
 };
 

llvm/include/llvm/Object/Decompressor.h

 //===-- Decompressor.h ------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===/
 
 #ifndef LLVM_OBJECT_DECOMPRESSOR_H
 #define LLVM_OBJECT_DECOMPRESSOR_H
 
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/StringRef.h"
+#include "llvm/MC/MCTargetOptions.h"
 #include "llvm/Support/Error.h"
 
 namespace llvm {
 namespace object {
 
 class SectionRef;
 
 /// Decompressor helps to handle decompression of compressed sections.
public:
 
   /// Return true if section is a ELF compressed one.
   static bool isCompressedELFSection(uint64_t Flags, StringRef Name);
 
   /// Return true if section name matches gnu style compressed one.
   static bool isGnuStyle(StringRef Name);
 
 private:
-  Decompressor(StringRef Data);
+  Decompressor(StringRef Data, DebugCompressionType CType);
 
   Error consumeCompressedGnuHeader();
   Error consumeCompressedZLibHeader(bool Is64Bit, bool IsLittleEndian);
+  Error consumeCompressedZStdHeader(bool Is64Bit, bool IsLittleEndian);
 
   StringRef SectionData;
+  DebugCompressionType CType;
   uint64_t DecompressedSize;
 };
 
 } // end namespace object
 } // end namespace llvm
 
 #endif // LLVM_OBJECT_DECOMPRESSOR_H

llvm/lib/MC/ELFObjectWriter.cpp

   // TargetObjectWriter wrappers.
   bool is64Bit() const;
   bool usesRela(const MCSectionELF &Sec) const;
 
   uint64_t align(unsigned Alignment);
 
   bool maybeWriteCompression(uint64_t Size,
                              SmallVectorImpl<char> &CompressedContents,
-                             bool ZLibStyle, unsigned Alignment);
+                             DebugCompressionType CompressionStyle,
+                             unsigned Alignment);
 
 public:
   ELFWriter(ELFObjectWriter &OWriter, raw_pwrite_stream &OS,
             bool IsLittleEndian, DwoMode Mode)
       : OWriter(OWriter),
         W(OS, IsLittleEndian ? support::little : support::big), Mode(Mode) {}
 
   void WriteWord(uint64_t Word) {

   MCSectionELF *RelaSection = Ctx.createELFRelSection(
       RelaSectionName, Rela ? ELF::SHT_RELA : ELF::SHT_REL, Flags, EntrySize,
       Sec.getGroup(), &Sec);
   RelaSection->setAlignment(is64Bit() ? Align(8) : Align(4));
   return RelaSection;
 }
 
 // Include the debug info compression header.
-bool ELFWriter::maybeWriteCompression(
-    uint64_t Size, SmallVectorImpl<char> &CompressedContents, bool ZLibStyle,
-    unsigned Alignment) {
-  if (ZLibStyle) {
+bool ELFWriter::maybeWriteCompression(uint64_t Size,
+                                      SmallVectorImpl<char> &CompressedContents,
+                                      DebugCompressionType CompressionStyle,
+                                      unsigned Alignment) {
+  if (CompressionStyle == DebugCompressionType::Zstd ||
+      CompressionStyle == DebugCompressionType::Z) {
     uint64_t HdrSize =
         is64Bit() ? sizeof(ELF::Elf32_Chdr) : sizeof(ELF::Elf64_Chdr);
     if (Size <= HdrSize + CompressedContents.size())
       return false;
     // Platform specific header is followed by compressed data.
     if (is64Bit()) {
       // Write Elf64_Chdr header.
-      write(static_cast<ELF::Elf64_Word>(ELF::ELFCOMPRESS_ZLIB));
+      write(static_cast<ELF::Elf64_Word>(CompressionStyle ==
+                                                 DebugCompressionType::Zstd
+                                             ? ELF::ELFCOMPRESS_ZSTD
+                                             : ELF::ELFCOMPRESS_ZLIB));
       write(static_cast<ELF::Elf64_Word>(0)); // ch_reserved field.
       write(static_cast<ELF::Elf64_Xword>(Size));
       write(static_cast<ELF::Elf64_Xword>(Alignment));
     } else {
       // Write Elf32_Chdr header otherwise.
-      write(static_cast<ELF::Elf32_Word>(ELF::ELFCOMPRESS_ZLIB));
+      write(static_cast<ELF::Elf32_Word>(CompressionStyle ==
+                                                 DebugCompressionType::Zstd
+                                             ? ELF::ELFCOMPRESS_ZSTD
+                                             : ELF::ELFCOMPRESS_ZLIB));
       write(static_cast<ELF::Elf32_Word>(Size));
       write(static_cast<ELF::Elf32_Word>(Alignment));
     }
     return true;
   }
 
   // "ZLIB" followed by 8 bytes representing the uncompressed size of the section,
   // useful for consumers to preallocate a buffer to decompress into.

   bool CompressionEnabled =
       MAI->compressDebugSections() != DebugCompressionType::None;
   if (!CompressionEnabled || !SectionName.startswith(".debug_")) {
     Asm.writeSectionData(W.OS, &Section, Layout);
     return;
   }
 
   assert((MAI->compressDebugSections() == DebugCompressionType::Z ||
-          MAI->compressDebugSections() == DebugCompressionType::GNU) &&
-         "expected zlib or zlib-gnu style compression");
+          MAI->compressDebugSections() == DebugCompressionType::GNU ||
+          MAI->compressDebugSections() == DebugCompressionType::Zstd) &&
+         "expected zlib or zlib-gnu or zstd style compression");
 
   SmallVector<char, 128> UncompressedData;
   raw_svector_ostream VecOS(UncompressedData);
   Asm.writeSectionData(VecOS, &Section, Layout);
 
   SmallVector<char, 128> CompressedContents;
-  compression::elf::compress(
-      StringRef(UncompressedData.data(), UncompressedData.size()),
-      CompressedContents);
 
-  bool ZlibStyle = MAI->compressDebugSections() == DebugCompressionType::Z;
+  DebugCompressionType CompressionStyle = MAI->compressDebugSections();
+  if (CompressionStyle == DebugCompressionType::Zstd) {
+    compression::zstd::compress(
+        StringRef(UncompressedData.data(), UncompressedData.size()),
+        CompressedContents);
+  } else {
+    compression::zlib::compress(
+        StringRef(UncompressedData.data(), UncompressedData.size()),
+        CompressedContents);
+  }
   if (!maybeWriteCompression(UncompressedData.size(), CompressedContents,
-                             ZlibStyle, Sec.getAlignment())) {
+                             MAI->compressDebugSections(),
+                             Sec.getAlignment())) {
     W.OS << UncompressedData;
     return;
   }
 
-  if (ZlibStyle) {
-    // Set the compressed flag. That is zlib style.
+  if (CompressionStyle == DebugCompressionType::Zstd ||
+      CompressionStyle == DebugCompressionType::Z) {
+    // Set the compressed flag. That is zlib or zstd style.
     Section.setFlags(Section.getFlags() | ELF::SHF_COMPRESSED);
     // Alignment field should reflect the requirements of
     // the compressed section header.
     Section.setAlignment(is64Bit() ? Align(8) : Align(4));
   } else {
     // Add "z" prefix to section name. This is zlib-gnu style.
     MC.renameELFSection(&Section, (".z" + SectionName.drop_front(1)).str());
   }

llvm/lib/ObjCopy/ELF/ELFObject.h

   Error accept(SectionVisitor &Sec) const override;
   Error accept(MutableSectionVisitor &Visitor) override;
   bool hasContents() const override { return true; }
 };
 
 class CompressedSection : public SectionBase {
   MAKE_SEC_WRITER_FRIEND
 
-  DebugCompressionType CompressionType;
   uint64_t DecompressedSize;
   uint64_t DecompressedAlign;
   SmallVector<char, 128> CompressedData;
 
 public:
+  DebugCompressionType CompressionType;
   CompressedSection(const SectionBase &Sec,
                     DebugCompressionType CompressionType);
   CompressedSection(ArrayRef<uint8_t> CompressedData, uint64_t DecompressedSize,
                     uint64_t DecompressedAlign);
 
   uint64_t getDecompressedSize() const { return DecompressedSize; }
   uint64_t getDecompressedAlign() const { return DecompressedAlign; }
 
   Error accept(SectionVisitor &Visitor) const override;
   Error accept(MutableSectionVisitor &Visitor) override;
 
   static bool classof(const SectionBase *S) {
     return (S->OriginalFlags & ELF::SHF_COMPRESSED) ||
            (StringRef(S->Name).startswith(".zdebug"));
   }
 };
 
 class DecompressedSection : public SectionBase {
   MAKE_SEC_WRITER_FRIEND
 
 public:
+  DebugCompressionType OriginalCompressionType;
   explicit DecompressedSection(const CompressedSection &Sec)
       : SectionBase(Sec) {
+    OriginalCompressionType = Sec.CompressionType;
     Size = Sec.getDecompressedSize();
     Align = Sec.getDecompressedAlign();
     Flags = OriginalFlags = (Flags & ~ELF::SHF_COMPRESSED);
     if (StringRef(Name).startswith(".zdebug"))
       Name = "." + Name.substr(2);
   }
 
   Error accept(SectionVisitor &Visitor) const override;

llvm/lib/ObjCopy/ELF/ELFObject.cpp

                                 ? (ZlibGnuMagic.size() + sizeof(Sec.Size))
                                 : sizeof(Elf_Chdr_Impl<ELFT>);
 
   StringRef CompressedContent(
       reinterpret_cast<const char *>(Sec.OriginalData.data()) + DataOffset,
       Sec.OriginalData.size() - DataOffset);
 
   SmallVector<char, 128> DecompressedContent;
-  if (Error Err =
-          compression::elf::uncompress(CompressedContent, DecompressedContent,
-                                       static_cast<size_t>(Sec.Size)))
-    return createStringError(errc::invalid_argument,
-                             "'" + Sec.Name + "': " + toString(std::move(Err)));
+  if (Sec.OriginalCompressionType == DebugCompressionType::Zstd) {
+    if (Error Err = compression::zstd::uncompress(
+            CompressedContent, DecompressedContent,
+            static_cast<size_t>(Sec.Size)))
+      return createStringError(errc::invalid_argument,
+                               "'" + Sec.Name +
+                                   "': " + toString(std::move(Err)));
+  } else {
+    if (Error Err = compression::zlib::uncompress(
+            CompressedContent, DecompressedContent,
+            static_cast<size_t>(Sec.Size)))
+      return createStringError(errc::invalid_argument,
+                               "'" + Sec.Name +
+                                   "': " + toString(std::move(Err)));
+  }
 
   uint8_t *Buf = reinterpret_cast<uint8_t *>(Out.getBufferStart()) + Sec.Offset;
   std::copy(DecompressedContent.begin(), DecompressedContent.end(), Buf);
 
   return Error::success();
 }
 
 Error BinarySectionWriter::visit(const DecompressedSection &Sec) {

     memcpy(Buf, Magic, strlen(Magic));
     Buf += strlen(Magic);
     const uint64_t DecompressedSize =
         support::endian::read64be(&Sec.DecompressedSize);
     memcpy(Buf, &DecompressedSize, sizeof(DecompressedSize));
     Buf += sizeof(DecompressedSize);
   } else {
     Elf_Chdr_Impl<ELFT> Chdr;
-    Chdr.ch_type = ELF::ELFCOMPRESS_ZLIB;
+    Chdr.ch_type = Sec.CompressionType == DebugCompressionType::Zstd
+                       ? ELF::ELFCOMPRESS_ZSTD
+                       : ELF::ELFCOMPRESS_ZLIB;
     Chdr.ch_size = Sec.DecompressedSize;
     Chdr.ch_addralign = Sec.DecompressedAlign;
     memcpy(Buf, &Chdr, sizeof(Chdr));
     Buf += sizeof(Chdr);
   }
 
   std::copy(Sec.CompressedData.begin(), Sec.CompressedData.end(), Buf);
   return Error::success();
 }
 
 CompressedSection::CompressedSection(const SectionBase &Sec,
                                      DebugCompressionType CompressionType)
-    : SectionBase(Sec), CompressionType(CompressionType),
-      DecompressedSize(Sec.OriginalData.size()), DecompressedAlign(Sec.Align) {
-  compression::elf::compress(
-      StringRef(reinterpret_cast<const char *>(OriginalData.data()),
-                OriginalData.size()),
-      CompressedData);
+    : SectionBase(Sec), DecompressedSize(Sec.OriginalData.size()),
+      DecompressedAlign(Sec.Align), CompressionType(CompressionType) {
+  if (CompressionType == DebugCompressionType::Zstd) {
+    compression::zstd::compress(
+        StringRef(reinterpret_cast<const char *>(OriginalData.data()),
+                  OriginalData.size()),
+        CompressedData);
+  } else {
+    compression::zlib::compress(
+        StringRef(reinterpret_cast<const char *>(OriginalData.data()),
+                  OriginalData.size()),
+        CompressedData);
+  }
 
   size_t ChdrSize;
   if (CompressionType == DebugCompressionType::GNU) {
     Name = ".z" + Sec.Name.substr(1);
     ChdrSize = sizeof("ZLIB") - 1 + sizeof(uint64_t);
   } else {
     Flags |= ELF::SHF_COMPRESSED;
     ChdrSize =
         std::max(std::max(sizeof(object::Elf_Chdr_Impl<object::ELF64LE>),
                           sizeof(object::Elf_Chdr_Impl<object::ELF64BE>)),
                  std::max(sizeof(object::Elf_Chdr_Impl<object::ELF32LE>),
                           sizeof(object::Elf_Chdr_Impl<object::ELF32BE>)));
   }
   Size = ChdrSize + CompressedData.size();
   Align = 8;
 }
 
 CompressedSection::CompressedSection(ArrayRef<uint8_t> CompressedData,
                                      uint64_t DecompressedSize,
                                      uint64_t DecompressedAlign)
-    : CompressionType(DebugCompressionType::None),
-      DecompressedSize(DecompressedSize), DecompressedAlign(DecompressedAlign) {
+    : DecompressedSize(DecompressedSize), DecompressedAlign(DecompressedAlign),
+      CompressionType(DebugCompressionType::None) {
   OriginalData = CompressedData;
 }
 
 Error CompressedSection::accept(SectionVisitor &Visitor) const {
   return Visitor.visit(*this);
 }
 
 Error CompressedSection::accept(MutableSectionVisitor &Visitor) {

llvm/lib/Object/Decompressor.cpp

 //===-- Decompressor.cpp --------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Object/Decompressor.h"
 #include "llvm/BinaryFormat/ELF.h"
+#include "llvm/MC/MCTargetOptions.h"
 #include "llvm/Object/ObjectFile.h"
 #include "llvm/Support/Compression.h"
 #include "llvm/Support/DataExtractor.h"
 #include "llvm/Support/Endian.h"
 
 using namespace llvm;
 using namespace llvm::support::endian;
 using namespace object;
 
 Expected<Decompressor> Decompressor::create(StringRef Name, StringRef Data,
                                             bool IsLE, bool Is64Bit) {
-  if (!compression::elf::isAvailable())
-    return createError("zlib is not available");
+  DebugCompressionType CType = DebugCompressionType::Z;
+  if (isGnuStyle(Name)) {
+    CType = DebugCompressionType::GNU;
+  } else {
+    if (Is64Bit) {
+      auto *chdr =
+          reinterpret_cast<const llvm::ELF::Elf64_Chdr *>(Data.bytes_begin());
+      if (chdr->ch_type == ELF::ELFCOMPRESS_ZLIB) {
+        CType = DebugCompressionType::Z;
+      } else if (chdr->ch_type == ELF::ELFCOMPRESS_ZSTD) {
+        CType = DebugCompressionType::Zstd;
+      } else {
+        return createError(
+            "unknown elf compress type in compressed section header");
+      }
+    } else {
+      auto *chdr =
+          reinterpret_cast<const llvm::ELF::Elf32_Chdr *>(Data.bytes_begin());
+      if (chdr->ch_type == ELF::ELFCOMPRESS_ZLIB) {
+        CType = DebugCompressionType::Z;
+      } else if (chdr->ch_type == ELF::ELFCOMPRESS_ZSTD) {
+        CType = DebugCompressionType::Zstd;
+      } else {
+        return createError(
+            "unknown elf compress type in compressed section header");
+      }
+    }
+  }
 
-  Decompressor D(Data);
-  Error Err = isGnuStyle(Name) ? D.consumeCompressedGnuHeader()
-                               : D.consumeCompressedZLibHeader(Is64Bit, IsLE);
+  if ((CType == DebugCompressionType::Z ||
+       CType == DebugCompressionType::GNU) &&
+      !compression::zlib::isAvailable())
+    return createError("zlib is not available");
+  Decompressor D(Data, CType);
+  Error Err = CType == DebugCompressionType::GNU
+                  ? D.consumeCompressedGnuHeader()
+              : CType == DebugCompressionType::Z
+                  ? D.consumeCompressedZLibHeader(Is64Bit, IsLE)
+                  : D.consumeCompressedZStdHeader(Is64Bit, IsLE);
   if (Err)
     return std::move(Err);
   return D;
 }
 
-Decompressor::Decompressor(StringRef Data)
-    : SectionData(Data), DecompressedSize(0) {}
+Decompressor::Decompressor(StringRef Data, DebugCompressionType CType)
+    : SectionData(Data), CType(CType), DecompressedSize(0) {}
 
 Error Decompressor::consumeCompressedGnuHeader() {
   if (!SectionData.startswith("ZLIB"))
     return createError("corrupted compressed section header");
 
   SectionData = SectionData.substr(4);
 
   // Consume uncompressed section size (big-endian 8 bytes).
if (Is64Bit)
     Offset += sizeof(Elf64_Word);
 
   DecompressedSize = Extractor.getUnsigned(
       &Offset, Is64Bit ? sizeof(Elf64_Xword) : sizeof(Elf32_Word));
   SectionData = SectionData.substr(HdrSize);
   return Error::success();
 }
 
+Error Decompressor::consumeCompressedZStdHeader(bool Is64Bit,
+                                                bool IsLittleEndian) {
+  using namespace ELF;
+  uint64_t HdrSize = Is64Bit ? sizeof(Elf64_Chdr) : sizeof(Elf32_Chdr);
+  if (SectionData.size() < HdrSize)
+    return createError("corrupted compressed section header");
+
+  DataExtractor Extractor(SectionData, IsLittleEndian, 0);
+  uint64_t Offset = 0;
+  if (Extractor.getUnsigned(&Offset, Is64Bit ? sizeof(Elf64_Word)
+                                             : sizeof(Elf32_Word)) !=
+      ELFCOMPRESS_ZSTD)
+    return createError("unsupported compression type");
+
+  // Skip Elf64_Chdr::ch_reserved field.
+  if (Is64Bit)
+    Offset += sizeof(Elf64_Word);
+
+  DecompressedSize = Extractor.getUnsigned(
+      &Offset, Is64Bit ? sizeof(Elf64_Xword) : sizeof(Elf32_Word));
+  SectionData = SectionData.substr(HdrSize);
+  return Error::success();
+}
+
 bool Decompressor::isGnuStyle(StringRef Name) {
   return Name.startswith(".zdebug");
 }
 
 bool Decompressor::isCompressed(const object::SectionRef &Section) {
   if (Section.isCompressed())
     return true;
 
   Expected<StringRef> SecNameOrErr = Section.getName();
   if (SecNameOrErr)
     return isGnuStyle(*SecNameOrErr);
 
   consumeError(SecNameOrErr.takeError());
   return false;
 }
 
 bool Decompressor::isCompressedELFSection(uint64_t Flags, StringRef Name) {
   return (Flags & ELF::SHF_COMPRESSED) || isGnuStyle(Name);
 }
 
 Error Decompressor::decompress(MutableArrayRef<char> Buffer) {
   size_t Size = Buffer.size();
-  return compression::elf::uncompress(SectionData, Buffer.data(), Size);
+  return CType == DebugCompressionType::Zstd
+             ? compression::zstd::uncompress(SectionData, Buffer.data(), Size)
+             : compression::zlib::uncompress(SectionData, Buffer.data(), Size);
 }

llvm/tools/llvm-objcopy/ObjcopyOptions.cpp

     Config.CompressionType = DebugCompressionType::Z;
 
     if (Arg->getOption().getID() == OBJCOPY_compress_debug_sections_eq) {
       Config.CompressionType =
           StringSwitch<DebugCompressionType>(
               InputArgs.getLastArgValue(OBJCOPY_compress_debug_sections_eq))
               .Case("zlib-gnu", DebugCompressionType::GNU)
               .Case("zlib", DebugCompressionType::Z)
+              .Case("zstd", DebugCompressionType::Zstd)
               .Default(DebugCompressionType::None);
       if (Config.CompressionType == DebugCompressionType::None)
         return createStringError(
             errc::invalid_argument,
             "invalid or unsupported --compress-debug-sections format: %s",
             InputArgs.getLastArgValue(OBJCOPY_compress_debug_sections_eq)
                 .str()
                 .c_str());
     }
-    if (!compression::elf::isAvailable())
-      return createStringError(
-          errc::invalid_argument,
-          "LLVM was not compiled with LLVM_ENABLE_ZLIB: can not compress");
+    if (Config.CompressionType == DebugCompressionType::Zstd) {
+      if (!compression::zstd::isAvailable())
+        return createStringError(
+            errc::invalid_argument,
+            "LLVM was not compiled with LLVM_ENABLE_ZSTD: can not compress");
+    } else {
+      if (!compression::zlib::isAvailable())
+        return createStringError(
+            errc::invalid_argument,
+            "LLVM was not compiled with LLVM_ENABLE_ZLIB: can not compress");
+    }
   }
 
   Config.AddGnuDebugLink = InputArgs.getLastArgValue(OBJCOPY_add_gnu_debuglink);
   // The gnu_debuglink's target is expected to not change or else its CRC would
   // become invalidated and get rejected. We can avoid recalculating the
   // checksum for every target file inside an archive by precomputing the CRC
   // here. This prevents a significant amount of I/O.
   if (!Config.AddGnuDebugLink.empty()) {

 
   if (Config.DecompressDebugSections &&
       Config.CompressionType != DebugCompressionType::None) {
     return createStringError(
         errc::invalid_argument,
         "cannot specify both --compress-debug-sections and "
         "--decompress-debug-sections");
   }
-
-  if (Config.DecompressDebugSections && !compression::elf::isAvailable())
+  if (Config.CompressionType != DebugCompressionType::Zstd) {
+    if (Config.DecompressDebugSections && !compression::zlib::isAvailable())
       return createStringError(
           errc::invalid_argument,
           "LLVM was not compiled with LLVM_ENABLE_ZLIB: cannot decompress");
+  } else {
+    if (Config.DecompressDebugSections && !compression::zstd::isAvailable())
+      return createStringError(
+          errc::invalid_argument,
+          "LLVM was not compiled with LLVM_ENABLE_ZSTD: cannot decompress");
+  }
 
   if (Config.ExtractPartition && Config.ExtractMainPartition)
     return createStringError(errc::invalid_argument,
                              "cannot specify --extract-partition together with "
                              "--extract-main-partition");
 
   DC.CopyConfigs.push_back(std::move(ConfigMgr));
   return std::move(DC);