Add support for -fno-builtin to LTO and ThinLTO on Darwin

[tejohnson]

It looks like this is part of the required fix for PR30403? I guess the other part is to add the module flag to be set during the FE, and use that to set this flag?

It would be good to add support in the new LTO API too.

In the discussion on PR30403 it sounds like the desired behavior was closer to -ffreestanding than -fno-builtin, so should the variable names reflect that?

llvm/include/llvm/LTO/legacy/LTOCodeGenerator.h

146 ↗

(On Diff #91215)

Why not instead add this flag as a member of LTOCodeGenerator, similar to what you do for the ThinLTOCodeGenerator?

llvm/include/llvm/LTO/legacy/ThinLTOCodeGenerator.h

209 ↗

(On Diff #91215)

document

llvm/test/ThinLTO/X86/tli-nobuiltin.ll

38 ↗

(On Diff #91215)

s/fputs/fwrite/ per earlier checks?

[mehdi_amini]

In D30791#697755, @tejohnson wrote:

It looks like this is part of the required fix for PR30403? I guess the other part is to add the module flag to be set during the FE, and use that to set this flag?

This is a lame but pragmatic way of going around the issue and allow our kernel to be built with ThinLTO ;)
The plan for the "clean solution" in the future is to embed the information in function-level attributes. But it'll take too much time for me to accomplish right now (and in the 5.0 timeframe FWIW).

It would be good to add support in the new LTO API too.

In the discussion on PR30403 it sounds like the desired behavior was closer to -ffreestanding than -fno-builtin, so should the variable names reflect that?

The discussion on clang-dev blurred the lines between -ffreestanding and -fno-builtin, to the point where I gave up trying to differentiate them.

llvm/include/llvm/LTO/legacy/LTOCodeGenerator.h

146 ↗

(On Diff #91215)

I just tried to fit into the existing by mimicking DisableVectorization. That said I think I can do differently (see below).

llvm/include/llvm/LTO/legacy/ThinLTOCodeGenerator.h

209 ↗

(On Diff #91215)

Will do!

llvm/test/ThinLTO/X86/tli-nobuiltin.ll

38 ↗

(On Diff #91215)

Oh right!

llvm/tools/lto/lto.cpp

165 ↗

(On Diff #91215)

I likely can set the DisableLTOBuiltins on the CodeGenerator here.

[pcc]

If this is just a temporary solution for one specific application, does it really need first class support in the clang driver? Would it not be sufficient for you to pass -Wl,-mllvm,-lto-nobuiltin at link time?

I also have a mild preference to use the term "freestanding" for this.

[mehdi_amini]

In D30791#699844, @pcc wrote:

If this is just a temporary solution for one specific application, does it really need first class support in the clang driver? Would it not be sufficient for you to pass -Wl,-mllvm,-lto-nobuiltin at link time?

Sure. I'm fine dropping the clang part.

I also have a mild preference to use the term "freestanding" for this.

I don't mind.

[mehdi_amini]

Address comments:

• set Freestanding as a member of LTOCodegenerator
• document the setter
• drop the clang driver changes

[mehdi_amini]

clang-format

[pcc]

You may want to include the freestanding flag in the code generation options hash.

Aside from that, LGTM.

[mehdi_amini]

In D30791#712296, @pcc wrote:

You may want to include the freestanding flag in the code generation options hash.

done before committing

llvm/trunk/include/llvm/LTO/legacy/LTOCodeGenerator.h

struct LTOCodeGenerator {
   /// representing a linkable partition of the module. If out contains more
   /// than one element, code generation is done in parallel with out.size()
   /// threads.  Output files will be written to members of out. Returns true on
   /// success.
   ///
   /// Calls \a verifyMergedModuleOnce().
   bool compileOptimized(ArrayRef<raw_pwrite_stream *> Out);
 
+  /// Enable the Freestanding mode: indicate that the optimizer should not
+  /// assume builtins are present on the target.
+  void setFreestanding(bool Enabled) { Freestanding = Enabled; }
+
   void setDiagnosticHandler(lto_diagnostic_handler_t, void *);
 
   LLVMContext &getContext() { return Context; }
 
   void resetMergedModule() { MergedModule.reset(); }
 
 private:
   void initializeLTOPasses();
private:
   unsigned OptLevel = 2;
   lto_diagnostic_handler_t DiagHandler = nullptr;
   void *DiagContext = nullptr;
   bool ShouldInternalize = true;
   bool ShouldEmbedUselists = false;
   bool ShouldRestoreGlobalsLinkage = false;
   TargetMachine::CodeGenFileType FileType = TargetMachine::CGFT_ObjectFile;
   std::unique_ptr<tool_output_file> DiagnosticOutputFile;
+  bool Freestanding = false;
 };
 }
 #endif

llvm/trunk/include/llvm/LTO/legacy/ThinLTOCodeGenerator.h

public:
   /// Subtarget attributes
   void setAttr(std::string MAttr) { TMBuilder.MAttr = std::move(MAttr); }
 
   /// TargetMachine options
   void setTargetOptions(TargetOptions Options) {
     TMBuilder.Options = std::move(Options);
   }
 
+  /// Enable the Freestanding mode: indicate that the optimizer should not
+  /// assume builtins are present on the target.
+  void setFreestanding(bool Enabled) { Freestanding = Enabled; }
+
   /// CodeModel
   void setCodePICModel(Optional<Reloc::Model> Model) {
     TMBuilder.RelocModel = Model;
   }
 
   /// CodeGen optimization level
   void setCodeGenOptLevel(CodeGenOpt::Level CGOptLevel) {
     TMBuilder.CGOptLevel = CGOptLevel;
private:
   /// Flag to enable/disable CodeGen. When set to true, the process stops after
   /// optimizations and a bitcode is produced.
   bool DisableCodeGen = false;
 
   /// Flag to indicate that only the CodeGen will be performed, no cross-module
   /// importing or optimization.
   bool CodeGenOnly = false;
 
+  /// Flag to indicate that the optimizer should not assume builtins are present
+  /// on the target.
+  bool Freestanding = false;
+
   /// IR Optimization Level [0-3].
   unsigned OptLevel = 3;
 };
 }
 #endif

llvm/trunk/lib/LTO/LTOCodeGenerator.cpp

bool LTOCodeGenerator::optimize(bool DisableVerify, bool DisableInline,
   Triple TargetTriple(TargetMach->getTargetTriple());
   PassManagerBuilder PMB;
   PMB.DisableGVNLoadPRE = DisableGVNLoadPRE;
   PMB.LoopVectorize = !DisableVectorization;
   PMB.SLPVectorize = !DisableVectorization;
   if (!DisableInline)
     PMB.Inliner = createFunctionInliningPass();
   PMB.LibraryInfo = new TargetLibraryInfoImpl(TargetTriple);
+  if (Freestanding)
+    PMB.LibraryInfo->disableAllFunctions();
   PMB.OptLevel = OptLevel;
   PMB.VerifyInput = !DisableVerify;
   PMB.VerifyOutput = !DisableVerify;
 
   PMB.populateLTOPassManager(passes);
 
   // Run our queue of passes all at once now, efficiently.
   passes.run(*MergedModule);

llvm/trunk/lib/LTO/ThinLTOCodeGenerator.cpp

handleAllErrors(Result.takeError(), [&](ErrorInfoBase &EIB) {
                                       SourceMgr::DK_Error, EIB.message());
       Err.print("ThinLTO", errs());
     });
     report_fatal_error("importFunctions failed");
   }
 }
 
 static void optimizeModule(Module &TheModule, TargetMachine &TM,
-                           unsigned OptLevel) {
+                           unsigned OptLevel, bool Freestanding) {
   // Populate the PassManager
   PassManagerBuilder PMB;
   PMB.LibraryInfo = new TargetLibraryInfoImpl(TM.getTargetTriple());
+  if (Freestanding)
+    PMB.LibraryInfo->disableAllFunctions();
   PMB.Inliner = createFunctionInliningPass();
   // FIXME: should get it from the bitcode?
   PMB.OptLevel = OptLevel;
   PMB.LoopVectorize = true;
   PMB.SLPVectorize = true;
   PMB.VerifyInput = true;
   PMB.VerifyOutput = false;
 
public:
   // access the content in the cache.
   ModuleCacheEntry(
       StringRef CachePath, const ModuleSummaryIndex &Index, StringRef ModuleID,
       const FunctionImporter::ImportMapTy &ImportList,
       const FunctionImporter::ExportSetTy &ExportList,
       const std::map<GlobalValue::GUID, GlobalValue::LinkageTypes> &ResolvedODR,
       const GVSummaryMapTy &DefinedFunctions,
       const DenseSet<GlobalValue::GUID> &PreservedSymbols, unsigned OptLevel,
-      const TargetMachineBuilder &TMBuilder) {
+      bool Freestanding, const TargetMachineBuilder &TMBuilder) {
     if (CachePath.empty())
       return;
 
     if (!Index.modulePaths().count(ModuleID))
       // The module does not have an entry, it can't have a hash at all
       return;
 
     // Compute the unique hash for this entry
#endif
     AddUnsigned(TMBuilder.Options.FunctionSections);
     AddUnsigned(TMBuilder.Options.DataSections);
     AddUnsigned((unsigned)TMBuilder.Options.DebuggerTuning);
     AddString(TMBuilder.MAttr);
     if (TMBuilder.RelocModel)
       AddUnsigned(*TMBuilder.RelocModel);
     AddUnsigned(TMBuilder.CGOptLevel);
     AddUnsigned(OptLevel);
+    AddUnsigned(Freestanding);
 
     Hasher.update(ArrayRef<uint8_t>((uint8_t *)&ModHash[0], sizeof(ModHash)));
     for (auto F : ExportList)
       // The export list can impact the internalization, be conservative here
       Hasher.update(ArrayRef<uint8_t>((uint8_t *)&F, sizeof(F)));
 
     // Include the hash for every module we import functions from
     for (auto &Entry : ImportList) {

 ProcessThinLTOModule(Module &TheModule, ModuleSummaryIndex &Index,
                      StringMap<MemoryBufferRef> &ModuleMap, TargetMachine &TM,
                      const FunctionImporter::ImportMapTy &ImportList,
                      const FunctionImporter::ExportSetTy &ExportList,
                      const DenseSet<GlobalValue::GUID> &GUIDPreservedSymbols,
                      const GVSummaryMapTy &DefinedGlobals,
                      const ThinLTOCodeGenerator::CachingOptions &CacheOptions,
                      bool DisableCodeGen, StringRef SaveTempsDir,
-                     unsigned OptLevel, unsigned count) {
+                     bool Freestanding, unsigned OptLevel, unsigned count) {
 
   // "Benchmark"-like optimization: single-source case
   bool SingleModule = (ModuleMap.size() == 1);
 
   if (!SingleModule) {
     promoteModule(TheModule, Index);
 
     // Apply summary-based LinkOnce/Weak resolution decisions.
ProcessThinLTOModule(Module &TheModule, ModuleSummaryIndex &Index,
 
   if (!SingleModule) {
     crossImportIntoModule(TheModule, Index, ModuleMap, ImportList);
 
     // Save temps: after cross-module import.
     saveTempBitcode(TheModule, SaveTempsDir, count, ".3.imported.bc");
   }
 
-  optimizeModule(TheModule, TM, OptLevel);
+  optimizeModule(TheModule, TM, OptLevel, Freestanding);
 
   saveTempBitcode(TheModule, SaveTempsDir, count, ".4.opt.bc");
 
   if (DisableCodeGen) {
     // Configured to stop before CodeGen, serialize the bitcode and return.
     SmallVector<char, 128> OutputBuffer;
     {
       raw_svector_ostream OS(OutputBuffer);

 
 /**
  * Perform post-importing ThinLTO optimizations.
  */
 void ThinLTOCodeGenerator::optimize(Module &TheModule) {
   initTMBuilder(TMBuilder, Triple(TheModule.getTargetTriple()));
 
   // Optimize now
-  optimizeModule(TheModule, *TMBuilder.create(), OptLevel);
+  optimizeModule(TheModule, *TMBuilder.create(), OptLevel, Freestanding);
 }
 
 /**
  * Perform ThinLTO CodeGen.
  */
 std::unique_ptr<MemoryBuffer> ThinLTOCodeGenerator::codegen(Module &TheModule) {
   initTMBuilder(TMBuilder, Triple(TheModule.getTargetTriple()));
   return codegenModule(TheModule, *TMBuilder.create());
for (auto IndexCount : ModulesOrdering) {
 
         auto &DefinedFunctions = ModuleToDefinedGVSummaries[ModuleIdentifier];
 
         // The module may be cached, this helps handling it.
         ModuleCacheEntry CacheEntry(CacheOptions.Path, *Index, ModuleIdentifier,
                                     ImportLists[ModuleIdentifier], ExportList,
                                     ResolvedODR[ModuleIdentifier],
                                     DefinedFunctions, GUIDPreservedSymbols,
-                                    OptLevel, TMBuilder);
+                                    OptLevel, Freestanding, TMBuilder);
         auto CacheEntryPath = CacheEntry.getEntryPath();
 
         {
           auto ErrOrBuffer = CacheEntry.tryLoadingBuffer();
           DEBUG(dbgs() << "Cache " << (ErrOrBuffer ? "hit" : "miss") << " '"
                        << CacheEntryPath << "' for buffer " << count << " "
                        << ModuleIdentifier << "\n");
 
for (auto IndexCount : ModulesOrdering) {
         saveTempBitcode(*TheModule, SaveTempsDir, count, ".0.original.bc");
 
         auto &ImportList = ImportLists[ModuleIdentifier];
         // Run the main process now, and generates a binary
         auto OutputBuffer = ProcessThinLTOModule(
             *TheModule, *Index, ModuleMap, *TMBuilder.create(), ImportList,
             ExportList, GUIDPreservedSymbols,
             ModuleToDefinedGVSummaries[ModuleIdentifier], CacheOptions,
-            DisableCodeGen, SaveTempsDir, OptLevel, count);
+            DisableCodeGen, SaveTempsDir, Freestanding, OptLevel, count);
 
         // Commit to the cache (if enabled)
         CacheEntry.write(*OutputBuffer);
 
         if (SavedObjectsDirectoryPath.empty()) {
           // We need to generated a memory buffer for the linker.
           if (!CacheEntryPath.empty()) {
             // Cache is enabled, reload from the cache

llvm/trunk/test/ThinLTO/X86/tli-nobuiltin.ll

+; Test -lto-freestanding option for libLTO.
+; RUN: llvm-as < %s > %t.bc
+
+; Regular run: expects fprintf to be turned into fwrite
+; RUN: llvm-lto %t.bc -exported-symbol=_foo -o %t.o
+; RUN: llvm-nm %t.o | FileCheck %s --check-prefix=LTO
+; LTO: fwrite
+
+; Freestanding run: expects fprintf to NOT be turned into fwrite
+; RUN: llvm-lto %t.bc -lto-freestanding -exported-symbol=_foo -o %t.o
+; RUN: llvm-nm %t.o | FileCheck %s --check-prefix=LTO-FREESTANDING
+; LTO-FREESTANDING: fprintf
+
+; Same with ThinLTO now.
+; RUN: opt -module-hash -module-summary %s -o %t.bc
+
+; Regular run: expects fprintf to be turned into fwrite
+; RUN: llvm-lto -exported-symbol=_foo -thinlto-action=run %t.bc
+; RUN: llvm-nm %t.bc.thinlto.o | FileCheck %s --check-prefix=ThinLTO
+; ThinLTO: fwrite
+
+; Freestanding run: expects fprintf to NOT be turned into fwrite
+; RUN: llvm-lto -lto-freestanding -exported-symbol=_foo -thinlto-action=run %t.bc
+; RUN: llvm-nm %t.bc.thinlto.o | FileCheck %s --check-prefix=ThinLTO-FREESTANDING
+; ThinLTO-FREESTANDING: fprintf
+
+
+target datalayout = "e-m:o-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-macosx10.11.0"
+
+declare i32 @fprintf(%FILE*, i8*, ...)
+
+%FILE = type { }
+
+@hello_world = constant [13 x i8] c"hello world\0A\00"
+@percent_s = constant [3 x i8] c"%s\00"
+
+; Check fprintf(fp, "%s", str) -> fwrite(str, fp) only when builtins are enabled
+
+define void @foo(%FILE* %fp) {
+  %fmt = getelementptr [3 x i8], [3 x i8]* @percent_s, i32 0, i32 0
+  %str = getelementptr [13 x i8], [13 x i8]* @hello_world, i32 0, i32 0
+  call i32 (%FILE*, i8*, ...) @fprintf(%FILE* %fp, i8* %fmt, i8* %str)
+  ret void
+}
+

llvm/trunk/tools/llvm-lto/llvm-lto.cpp

 static cl::opt<bool>
     DisableGVNLoadPRE("disable-gvn-loadpre", cl::init(false),
                       cl::desc("Do not run the GVN load PRE pass"));
 
 static cl::opt<bool> DisableLTOVectorization(
     "disable-lto-vectorization", cl::init(false),
     cl::desc("Do not run loop or slp vectorization during LTO"));
 
+static cl::opt<bool> EnableFreestanding(
+    "lto-freestanding", cl::init(false),
+    cl::desc("Enable Freestanding (disable builtins / TLI) during LTO"));
+
 static cl::opt<bool> UseDiagnosticHandler(
     "use-diagnostic-handler", cl::init(false),
     cl::desc("Use a diagnostic handler to test the handler interface"));
 
 static cl::opt<bool>
     ThinLTO("thinlto", cl::init(false),
             cl::desc("Only write combined global index for ThinLTO backends"));
 

 class ThinLTOProcessing {
 public:
   ThinLTOCodeGenerator ThinGenerator;
 
   ThinLTOProcessing(const TargetOptions &Options) {
     ThinGenerator.setCodePICModel(getRelocModel());
     ThinGenerator.setTargetOptions(Options);
     ThinGenerator.setCacheDir(ThinLTOCacheDir);
+    ThinGenerator.setFreestanding(EnableFreestanding);
 
     // Add all the exported symbols to the table of symbols to preserve.
     for (unsigned i = 0; i < ExportedSymbols.size(); ++i)
       ThinGenerator.preserveSymbol(ExportedSymbols[i]);
   }
 
   void run() {
     switch (ThinLTOMode) {
int main(int argc, char **argv) {
   Context.setDiagnosticHandler(diagnosticHandler, nullptr, true);
 
   LTOCodeGenerator CodeGen(Context);
 
   if (UseDiagnosticHandler)
     CodeGen.setDiagnosticHandler(handleDiagnostics, nullptr);
 
   CodeGen.setCodePICModel(getRelocModel());
+  CodeGen.setFreestanding(EnableFreestanding);
 
   CodeGen.setDebugInfo(LTO_DEBUG_MODEL_DWARF);
   CodeGen.setTargetOptions(Options);
   CodeGen.setShouldRestoreGlobalsLinkage(RestoreGlobalsLinkage);
 
   llvm::StringSet<llvm::MallocAllocator> DSOSymbolsSet;
   for (unsigned i = 0; i < DSOSymbols.size(); ++i)
     DSOSymbolsSet.insert(DSOSymbols[i]);

llvm/trunk/tools/lto/lto.cpp

 static cl::opt<bool>
 DisableInline("disable-inlining", cl::init(false),
   cl::desc("Do not run the inliner pass"));
 
 static cl::opt<bool>
 DisableGVNLoadPRE("disable-gvn-loadpre", cl::init(false),
   cl::desc("Do not run the GVN load PRE pass"));
 
-static cl::opt<bool>
-DisableLTOVectorization("disable-lto-vectorization", cl::init(false),
+static cl::opt<bool> DisableLTOVectorization(
+    "disable-lto-vectorization", cl::init(false),
     cl::desc("Do not run loop or slp vectorization during LTO"));
 
+static cl::opt<bool> EnableFreestanding(
+    "lto-freestanding", cl::init(false),
+    cl::desc("Enable Freestanding (disable builtins / TLI) during LTO"));
+
 #ifdef NDEBUG
 static bool VerifyByDefault = false;
 #else
 static bool VerifyByDefault = true;
 #endif
 
 static cl::opt<bool> DisableVerify(
     "disable-llvm-verifier", cl::init(!VerifyByDefault),
if (MAttrs.size()) {
     }
 
     CG->setAttr(attrs);
   }
 
   if (OptLevel < '0' || OptLevel > '3')
     report_fatal_error("Optimization level must be between 0 and 3");
   CG->setOptLevel(OptLevel - '0');
+  CG->setFreestanding(EnableFreestanding);
 }
 
 extern const char* lto_get_version() {
   return LTOCodeGenerator::getVersionString();
 }
 
 const char* lto_get_error_message() {
   return sLastErrorString.c_str();

 }
 
 // ThinLTO API below
 
 thinlto_code_gen_t thinlto_create_codegen(void) {
   lto_initialize();
   ThinLTOCodeGenerator *CodeGen = new ThinLTOCodeGenerator();
   CodeGen->setTargetOptions(InitTargetOptionsFromCodeGenFlags());
+  CodeGen->setFreestanding(EnableFreestanding);
 
   if (OptLevel.getNumOccurrences()) {
     if (OptLevel < '0' || OptLevel > '3')
       report_fatal_error("Optimization level must be between 0 and 3");
     CodeGen->setOptLevel(OptLevel - '0');
     switch (OptLevel) {
     case '0':
       CodeGen->setCodeGenOptLevel(CodeGenOpt::None);