[llvm-objcopy][NFC] Refactor output target parsing

[seiya]

Summary

Use an enum instead of string to hold the output file format in Config.InputFormat and Config.OutputFormat. It's essential to support other output file formats other than ELF.

[compnerd]

llvm/tools/llvm-objcopy/CopyConfig.cpp

279 ↗

(On Diff #204398)

I think it would be nicer to use a tuple with an enumeration to give the values names rather than "first" and "second"

[seiya]

llvm/tools/llvm-objcopy/CopyConfig.cpp

279 ↗

(On Diff #204398)

What is an elegant way to get an element from tuple using enum? We need to cast the enum values so it should be std::get<static_cast<std::size_t>(Foo::Bar)>(Tuple). It looks verbose a bit to me.

[jhenderson]

Could you explain a bit more the motivation behind this change, please? If I'm not mistaken, it's to allow using input and output format with non-ELF (e.g. COFF and Mach-O) targets?

llvm/tools/llvm-objcopy/CopyConfig.cpp

279 ↗

(On Diff #204398)

I think it would be nicer to use a tuple with an enumeration to give the values names rather than "first" and "second"

Shouldn't it should just be a struct then?

struct {
  FileFormat Format;
  MachineInfo Machine;
};

llvm/tools/llvm-objcopy/llvm-objcopy.cpp

32 ↗

(On Diff #204398)

Why has this been added?

135 ↗

(On Diff #204398)

Refer -> See

[seiya]

In D63239#1541208, @jhenderson wrote:

Could you explain a bit more the motivation behind this change, please? If I'm not mistaken, it's to allow using input and output format with non-ELF (e.g. COFF and Mach-O) targets?

Yes, that's correct. I'd like to use FileFormat instead of a target string for Config.OutputFormat to allow calling non-ELF implementations in executeObjcopyOnRawBinary.

[seiya]

llvm/tools/llvm-objcopy/CopyConfig.cpp

279 ↗

(On Diff #204398)

struct sounds good to me. I'll use it here.

[seiya]

• Replaced std::pair with a struct.
• Refer -> See

llvm/tools/llvm-objcopy/llvm-objcopy.cpp

32 ↗

(On Diff #204398)

Oh I forgot to remove this include. Thanks.

[jhenderson]

llvm/tools/llvm-objcopy/CopyConfig.h

53 ↗

(On Diff #204475)

As this is only used in the .cpp, it can be moved in there.

llvm/tools/llvm-objcopy/llvm-objcopy.cpp

134 ↗

(On Diff #204475)

Missing trailing full stop.

[seiya]

• Addressed review comments.
• Resolved conflicts with origin/master.

[seiya]

• Removed a temprary file accidentally included in the patch.

[jhenderson]

LGTM, with one nit. Also, if I understand it correctly, this change should have no functional change, right? In that case, please tag the summary with [NFC] (i.e. [llvm-objcopy][NFC] Refactor output target parsing) when you commit.

llvm/tools/llvm-objcopy/CopyConfig.cpp

324 ↗

(On Diff #204665)

I don't think you need to use auto here.

[seiya]

• Explicate the type instead of auto.

[seiya]

In D63239#1543266, @jhenderson wrote:

LGTM, with one nit. Also, if I understand it correctly, this change should have no functional change, right? In that case, please tag the summary with [NFC] (i.e. [llvm-objcopy][NFC] Refactor output target parsing) when you commit.

Yes it is NFC. Thanks.

[rupprecht]

llvm/tools/llvm-objcopy/CopyConfig.cpp

288 ↗

(On Diff #204734)

I haven't looked at the full list of bfd targets (for macho/coff in particular), but do we need the extra field, or can we infer it from starting with "elf"?

457–462 ↗

(On Diff #204734)

This should use a StringSwitch

(I suppose adding an Unrecognized file format type to handle the 'else' part?)

473–475 ↗

(On Diff #204734)

I'm not sure this is really an error. Behold:

$ echo 'int x=0;' | ~/dev/clang -x c -c - -o /tmp/bss.o
$ objcopy -Ielf32-x86-64 /tmp/bss.o /tmp/bss2.o
$ file /tmp/bss.o /tmp/bss2.o
/tmp/bss.o:  ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped
/tmp/bss2.o: ELF 32-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped

[seiya]

llvm/tools/llvm-objcopy/CopyConfig.cpp

288 ↗

(On Diff #204734)

I'm not quite sure but according to my GNU objcopy build (with --enable-targets=all) you're right: we can infer the file type from the prefix.

473–475 ↗

(On Diff #204734)

Nice catch! Will fix this.

[seiya]

• Addressed some review comments.

[seiya]

• Refactored a bit.

llvm/tools/llvm-objcopy/CopyConfig.cpp

457–462 ↗

(On Diff #204734)

I think we don't need Unrecognized for now because currently we simply ignore unrecognized input formats and guess the file format by createBinary.

[jhenderson]

llvm/tools/llvm-objcopy/CopyConfig.cpp

471 ↗

(On Diff #205235)

ignores -> ignore

I'm not really sure what this is trying to point out though. Is it an existing bug in llvm-objcopy or one you are introducing?

[rupprecht]

llvm/tools/llvm-objcopy/CopyConfig.cpp

471 ↗

(On Diff #205235)

It's an existing bug; we handle "-I binary" and "-I ihex", but specifying "-Ielf32..." is the same as not specifying it at all.

Maybe a clearer message would be "ignore the target for non-binary/ihex formats"

[seiya]

• Updated a comment.

[jhenderson]

LGTM, with one comment fix.

llvm/tools/llvm-objcopy/CopyConfig.cpp

471 ↗

(On Diff #205489)

Nit: We -> we

[seiya]

• We -> we

[jakehehrlich]

General idea looks good to me!

llvm/trunk/tools/llvm-objcopy/CopyConfig.h

 #include "llvm/Support/Regex.h"
 // Necessary for llvm::DebugCompressionType::None
 #include "llvm/Target/TargetOptions.h"
 #include <vector>
 
 namespace llvm {
 namespace objcopy {
 
+enum class FileFormat {
+  Unspecified,
+  ELF,
+  Binary,
+  IHex,
+};
+
 // This type keeps track of the machine info for various architectures. This
 // lets us map architecture names to ELF types and the e_machine value of the
 // ELF file.
 struct MachineInfo {
   MachineInfo(uint16_t EM, uint8_t ABI, bool Is64, bool IsLittle)
       : EMachine(EM), OSABI(ABI), Is64Bit(Is64), IsLittleEndian(IsLittle) {}
   // Alternative constructor that defaults to NONE for OSABI.
   MachineInfo(uint16_t EM, bool Is64, bool IsLittle)
struct NewSymbolInfo {
   uint8_t Bind = ELF::STB_GLOBAL;
   uint8_t Visibility = ELF::STV_DEFAULT;
 };
 
 // Configuration for copying/stripping a single file.
 struct CopyConfig {
   // Main input/output options
   StringRef InputFilename;
-  StringRef InputFormat;
+  FileFormat InputFormat;
   StringRef OutputFilename;
-  StringRef OutputFormat;
+  FileFormat OutputFormat;
 
   // Only applicable for --input-format=binary
   MachineInfo BinaryArch;
   // Only applicable when --output-format!=binary (e.g. elf64-x86-64).
   Optional<MachineInfo> OutputArch;
 
   // Advanced options
   StringRef AddGnuDebugLink;

llvm/trunk/tools/llvm-objcopy/CopyConfig.cpp

 static Expected<const MachineInfo &> getMachineInfo(StringRef Arch) {
   auto Iter = ArchMap.find(Arch);
   if (Iter == std::end(ArchMap))
     return createStringError(errc::invalid_argument,
                              "invalid architecture: '%s'", Arch.str().c_str());
   return Iter->getValue();
 }
 
+struct TargetInfo {
+  FileFormat Format;
+  MachineInfo Machine;
+};
+
 // FIXME: consolidate with the bfd parsing used by lld.
-static const StringMap<MachineInfo> OutputFormatMap{
+static const StringMap<MachineInfo> TargetMap{
     // Name, {EMachine, 64bit, LittleEndian}
     // x86
     {"elf32-i386", {ELF::EM_386, false, true}},
     {"elf32-x86-64", {ELF::EM_X86_64, false, true}},
     {"elf64-x86-64", {ELF::EM_X86_64, true, true}},
     // Intel MCU
     {"elf32-iamcu", {ELF::EM_IAMCU, false, true}},
     // ARM
static const StringMap<MachineInfo> TargetMap{
     {"elf32-tradlittlemips", {ELF::EM_MIPS, false, true}},
     {"elf64-tradbigmips", {ELF::EM_MIPS, true, false}},
     {"elf64-tradlittlemips", {ELF::EM_MIPS, true, true}},
     // SPARC
     {"elf32-sparc", {ELF::EM_SPARC, false, false}},
     {"elf32-sparcel", {ELF::EM_SPARC, false, true}},
 };
 
-static Expected<MachineInfo> getOutputFormatMachineInfo(StringRef Format) {
-  StringRef OriginalFormat = Format;
-  bool IsFreeBSD = Format.consume_back("-freebsd");
-  auto Iter = OutputFormatMap.find(Format);
-  if (Iter == std::end(OutputFormatMap))
+static Expected<TargetInfo>
+getOutputTargetInfoByTargetName(StringRef TargetName) {
+  StringRef OriginalTargetName = TargetName;
+  bool IsFreeBSD = TargetName.consume_back("-freebsd");
+  auto Iter = TargetMap.find(TargetName);
+  if (Iter == std::end(TargetMap))
     return createStringError(errc::invalid_argument,
                              "invalid output format: '%s'",
-                             OriginalFormat.str().c_str());
+                             OriginalTargetName.str().c_str());
   MachineInfo MI = Iter->getValue();
   if (IsFreeBSD)
     MI.OSABI = ELF::ELFOSABI_FREEBSD;
-  return {MI};
+
+  FileFormat Format;
+  if (TargetName.startswith("elf"))
+    Format = FileFormat::ELF;
+  else
+    // This should never happen because `TargetName` is valid (it certainly
+    // exists in the TargetMap).
+    llvm_unreachable("unknown target prefix");
+
+  return {TargetInfo{Format, MI}};
 }
 
 static Error addSymbolsFromFile(std::vector<NameOrRegex> &Symbols,
                                 BumpPtrAllocator &Alloc, StringRef Filename,
                                 bool UseRegex) {
   StringSaver Saver(Alloc);
   SmallVector<StringRef, 16> Lines;
   auto BufOrErr = MemoryBuffer::getFile(Filename);
Expected<DriverConfig> parseObjcopyOptions(ArrayRef<const char *> ArgsArr) {
   if (InputArgs.hasArg(OBJCOPY_target) &&
       (InputArgs.hasArg(OBJCOPY_input_target) ||
        InputArgs.hasArg(OBJCOPY_output_target)))
     return createStringError(
         errc::invalid_argument,
         "--target cannot be used with --input-target or --output-target");
 
   bool UseRegex = InputArgs.hasArg(OBJCOPY_regex);
+  StringRef InputFormat, OutputFormat;
   if (InputArgs.hasArg(OBJCOPY_target)) {
-    Config.InputFormat = InputArgs.getLastArgValue(OBJCOPY_target);
-    Config.OutputFormat = InputArgs.getLastArgValue(OBJCOPY_target);
+    InputFormat = InputArgs.getLastArgValue(OBJCOPY_target);
+    OutputFormat = InputArgs.getLastArgValue(OBJCOPY_target);
   } else {
-    Config.InputFormat = InputArgs.getLastArgValue(OBJCOPY_input_target);
-    Config.OutputFormat = InputArgs.getLastArgValue(OBJCOPY_output_target);
+    InputFormat = InputArgs.getLastArgValue(OBJCOPY_input_target);
+    OutputFormat = InputArgs.getLastArgValue(OBJCOPY_output_target);
   }
-  if (Config.InputFormat == "binary") {
+
+  // FIXME:  Currently, we ignore the target for non-binary/ihex formats
+  // explicitly specified by -I option (e.g. -Ielf32-x86-64) and guess the
+  // format by llvm::object::createBinary regardless of the option value.
+  Config.InputFormat = StringSwitch<FileFormat>(InputFormat)
+                           .Case("binary", FileFormat::Binary)
+                           .Case("ihex", FileFormat::IHex)
+                           .Default(FileFormat::Unspecified);
+  if (Config.InputFormat == FileFormat::Binary) {
     auto BinaryArch = InputArgs.getLastArgValue(OBJCOPY_binary_architecture);
     if (BinaryArch.empty())
       return createStringError(
           errc::invalid_argument,
           "specified binary input without specifiying an architecture");
     Expected<const MachineInfo &> MI = getMachineInfo(BinaryArch);
     if (!MI)
       return MI.takeError();
     Config.BinaryArch = *MI;
   }
-  if (!Config.OutputFormat.empty() && Config.OutputFormat != "binary" &&
-      Config.OutputFormat != "ihex") {
-    Expected<MachineInfo> MI = getOutputFormatMachineInfo(Config.OutputFormat);
-    if (!MI)
-      return MI.takeError();
-    Config.OutputArch = *MI;
+
+  Config.OutputFormat = StringSwitch<FileFormat>(OutputFormat)
+                            .Case("binary", FileFormat::Binary)
+                            .Case("ihex", FileFormat::IHex)
+                            .Default(FileFormat::Unspecified);
+  if (Config.OutputFormat == FileFormat::Unspecified && !OutputFormat.empty()) {
+    Expected<TargetInfo> Target = getOutputTargetInfoByTargetName(OutputFormat);
+    if (!Target)
+      return Target.takeError();
+    Config.OutputFormat = Target->Format;
+    Config.OutputArch = Target->Machine;
   }
 
   if (auto Arg = InputArgs.getLastArg(OBJCOPY_compress_debug_sections,
                                       OBJCOPY_compress_debug_sections_eq)) {
     Config.CompressionType = DebugCompressionType::Z;
 
     if (Arg->getOption().getID() == OBJCOPY_compress_debug_sections_eq) {
       Config.CompressionType =

llvm/trunk/tools/llvm-objcopy/ELF/ELFObjcopy.cpp

return llvm::make_unique<ELFWriter<ELF64BE>>(Obj, Buf,
                                                  !Config.StripSections);
   }
   llvm_unreachable("Invalid output format");
 }
 
 static std::unique_ptr<Writer> createWriter(const CopyConfig &Config,
                                             Object &Obj, Buffer &Buf,
                                             ElfType OutputElfType) {
-  using Functor = std::function<std::unique_ptr<Writer>()>;
-  return StringSwitch<Functor>(Config.OutputFormat)
-      .Case("binary", [&] { return llvm::make_unique<BinaryWriter>(Obj, Buf); })
-      .Case("ihex", [&] { return llvm::make_unique<IHexWriter>(Obj, Buf); })
-      .Default(
-          [&] { return createELFWriter(Config, Obj, Buf, OutputElfType); })();
+  switch (Config.OutputFormat) {
+  case FileFormat::Binary:
+    return llvm::make_unique<BinaryWriter>(Obj, Buf);
+  case FileFormat::IHex:
+    return llvm::make_unique<IHexWriter>(Obj, Buf);
+  default:
+    return createELFWriter(Config, Obj, Buf, OutputElfType);
+  }
 }
 
 template <class ELFT>
 static Expected<ArrayRef<uint8_t>>
 findBuildID(const CopyConfig &Config, const object::ELFFile<ELFT> &In) {
   auto PhdrsOrErr = In.program_headers();
   if (auto Err = PhdrsOrErr.takeError())
     return createFileError(Config.InputFilename, std::move(Err));

llvm/trunk/tools/llvm-objcopy/llvm-objcopy.cpp

static Error executeObjcopyOnIHex(const CopyConfig &Config, MemoryBuffer &In,
   // TODO: support output formats other than ELF.
   return elf::executeObjcopyOnIHex(Config, In, Out);
 }
 
 /// The function executeObjcopyOnRawBinary does the dispatch based on the format
 /// of the output specified by the command line options.
 static Error executeObjcopyOnRawBinary(const CopyConfig &Config,
                                        MemoryBuffer &In, Buffer &Out) {
-  // TODO: llvm-objcopy should parse CopyConfig.OutputFormat to recognize
-  // formats other than ELF / "binary" and invoke
-  // elf::executeObjcopyOnRawBinary, macho::executeObjcopyOnRawBinary or
-  // coff::executeObjcopyOnRawBinary accordingly.
+  switch (Config.OutputFormat) {
+  case FileFormat::ELF:
+  // FIXME: Currently, we call elf::executeObjcopyOnRawBinary even if the
+  // output format is binary/ihex or it's not given. This behavior differs from
+  // GNU objcopy. See https://bugs.llvm.org/show_bug.cgi?id=42171 for details.
+  case FileFormat::Binary:
+  case FileFormat::IHex:
+  case FileFormat::Unspecified:
     return elf::executeObjcopyOnRawBinary(Config, In, Out);
   }
+}
 
 /// The function executeObjcopyOnBinary does the dispatch based on the format
 /// of the input binary (ELF, MachO or COFF).
 static Error executeObjcopyOnBinary(const CopyConfig &Config,
                                     object::Binary &In, Buffer &Out) {
   if (auto *ELFBinary = dyn_cast<object::ELFObjectFileBase>(&In))
     return elf::executeObjcopyOnBinary(Config, *ELFBinary, Out);
   else if (auto *COFFBinary = dyn_cast<object::COFFObjectFile>(&In))

 /// format-agnostic modifications, i.e. preserving dates.
 static Error executeObjcopy(const CopyConfig &Config) {
   sys::fs::file_status Stat;
   if (Config.PreserveDates)
     if (auto EC = sys::fs::status(Config.InputFilename, Stat))
       return createFileError(Config.InputFilename, EC);
 
   typedef Error (*ProcessRawFn)(const CopyConfig &, MemoryBuffer &, Buffer &);
-  auto ProcessRaw = StringSwitch<ProcessRawFn>(Config.InputFormat)
-                        .Case("binary", executeObjcopyOnRawBinary)
-                        .Case("ihex", executeObjcopyOnIHex)
-                        .Default(nullptr);
+  ProcessRawFn ProcessRaw;
+  switch (Config.InputFormat) {
+  case FileFormat::Binary:
+    ProcessRaw = executeObjcopyOnRawBinary;
+    break;
+  case FileFormat::IHex:
+    ProcessRaw = executeObjcopyOnIHex;
+    break;
+  default:
+    ProcessRaw = nullptr;
+  }
 
   if (ProcessRaw) {
     auto BufOrErr = MemoryBuffer::getFileOrSTDIN(Config.InputFilename);
     if (!BufOrErr)
       return createFileError(Config.InputFilename, BufOrErr.getError());
     FileBuffer FB(Config.OutputFilename);
     if (Error E = ProcessRaw(Config, *BufOrErr->get(), FB))
       return E;