[mlgo] Dependency-free training mode logger

[mtrofin]

Summary

This is the next step in dropping the dependency on protobuf.

The simple logger produces an output consisting of lines of json
strings. Tensor values - which should constitute the bulk of the data -
are serialized as raw byte buffers. This allows for light-weight reading
of the values.

The next step is to switch the training logic to the new logging format,
following which the protobuf-based logger will be dropped, together with
the training dependency on protobuf.

Subsequent changes will also stop buffering and stream, instead - the
buffering model is just as a convenient point-in-time.

[jacobhegna]

llvm/lib/Analysis/TrainingLogger.cpp

112

where in the header is the length of the tensor buffer stored? (to deal with newline bytes in the tensor buffer), in the value of the features key?

[mtrofin]

llvm/lib/Analysis/TrainingLogger.cpp

112

we have the shape and the type of the scalar value in the buffer, so we get the size from there. See for example how we write (L 168 this file) and then how we read (line 63 in log_reader.py)

[jacobhegna]

just some small python style nits

llvm/lib/Analysis/models/log_reader.py

2

nit: docstring for file

30

nit: could make this a @classmethod

51

nit: return type annotations? same for next two methods

63

nit: annotate fs?

[mtrofin]

python nits

[jacobhegna]

llvm/lib/Analysis/models/log_reader.py

35

this won't work, no? if it's a classmethod the first argument could be cls, and the point is you could return cls(name=...) instead of TensorSpec(name=...), it's essentially the same but follows the convention I see people using for factory methods in python

[mtrofin]

llvm/lib/Analysis/models/log_reader.py

35

(sorry, went to fast over it)

what does it give us, though, compared to the @staticmethod way?

51

getitem is a bit trickier I think because it's the value of the underlying type. doesn't seem pytype lets us specify self._spec.type

[jacobhegna]

llvm/lib/Analysis/models/log_reader.py

35

in this case, basically nothing 😊 (that's why it was a nit)

in this case, it's just a stylistic choice: I tend to see people use @classmethod for factory-style methods, and @staticmethod for methods that don't return an instance of the type. it's not that important, though

[mtrofin]

back to @staticmethod

[thakis]

This doesn't build with some compilers: http://45.33.8.238/macm1/50259/step_4.txt

Please take a look and revert for now if it takes a while to fix.

[fhahn]

I went ahead and reverted the patch as it breaks the build on macOS to unblock bots. More details about the failure in the commit message of the revert.

.style.yapf

+[style]
+based_on_style = yapf
+indent_width = 2

llvm/include/llvm/Analysis/TensorSpec.h

#define SUPPORTED_TENSOR_TYPES(M)                                              \
   M(int64_t, Int64)                                                            \
   M(uint64_t, UInt64)
 
 enum class TensorType {
   Invalid,
 #define _TENSOR_TYPE_ENUM_MEMBERS(_, Name) Name,
   SUPPORTED_TENSOR_TYPES(_TENSOR_TYPE_ENUM_MEMBERS)
 #undef _TENSOR_TYPE_ENUM_MEMBERS
+      Total
 };
 
 class TensorSpec final {
 public:
   template <typename T>
   static TensorSpec createSpec(const std::string &Name,
                                const std::vector<int64_t> &Shape,
                                int Port = 0) {
public:
   template <typename T> bool isElementType() const {
     return getDataType<T>() == Type;
   }
 
   TensorSpec(const std::string &NewName, const TensorSpec &Other)
       : TensorSpec(NewName, Other.Port, Other.Type, Other.ElementSize,
                    Other.Shape) {}
 
+  void toJSON(json::OStream &OS) const;
+
 private:
   TensorSpec(const std::string &Name, int Port, TensorType Type,
              size_t ElementSize, const std::vector<int64_t> &Shape);
 
   template <typename T> static TensorType getDataType();
 
   std::string Name;
   int Port = 0;

llvm/lib/Analysis/TensorSpec.cpp

 
 #include "llvm/ADT/Twine.h"
 #include "llvm/Analysis/TensorSpec.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/JSON.h"
 #include "llvm/Support/ManagedStatic.h"
 #include "llvm/Support/raw_ostream.h"
+#include <array>
 #include <cassert>
 #include <numeric>
 
 using namespace llvm;
 
 namespace llvm {
 
 #define TFUTILS_GETDATATYPE_IMPL(T, E)                                         \
   template <> TensorType TensorSpec::getDataType<T>() { return TensorType::E; }
 
 SUPPORTED_TENSOR_TYPES(TFUTILS_GETDATATYPE_IMPL)
 
 #undef TFUTILS_GETDATATYPE_IMPL
 
+static std::array<std::string, static_cast<size_t>(TensorType::Total)>
+    TensorTypeNames{"INVALID",
+#define TFUTILS_GETNAME_IMPL(T, _) #T,
+                    SUPPORTED_TENSOR_TYPES(TFUTILS_GETNAME_IMPL)
+#undef TFUTILS_GETNAME_IMPL
+    };
+
+StringRef toString(TensorType TT) {
+  return TensorTypeNames[static_cast<size_t>(TT)];
+}
+
+void TensorSpec::toJSON(json::OStream &OS) const {
+  OS.object([&]() {
+    OS.attribute("name", name());
+    OS.attribute("type", toString(type()));
+    OS.attribute("port", port());
+    OS.attributeArray("shape", [&]() {
+      for (size_t D : shape())
+        OS.value(D);
+    });
+  });
+}
+
 TensorSpec::TensorSpec(const std::string &Name, int Port, TensorType Type,
                        size_t ElementSize, const std::vector<int64_t> &Shape)
     : Name(Name), Port(Port), Type(Type), Shape(Shape),
       ElementCount(std::accumulate(Shape.begin(), Shape.end(), 1,
                                    std::multiplies<int64_t>())),
       ElementSize(ElementSize) {}
 
 Optional<TensorSpec> getTensorSpecFromJSON(LLVMContext &Ctx,

llvm/lib/Analysis/TrainingLogger.cpp

 //===- TrainingLogger.cpp - mlgo feature/reward logging -------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements logging infrastructure for extracting features and
 // rewards for mlgo policy training.
 //
 //===----------------------------------------------------------------------===//
+#include "llvm/Analysis/TensorSpec.h"
 #include "llvm/Config/config.h"
 #if defined(LLVM_HAVE_TF_API)
 
 #include "llvm/ADT/Twine.h"
 #include "llvm/Analysis/Utils/TrainingLogger.h"
 #include "llvm/Support/Base64.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Debug.h"

 
 using google::protobuf::Message;
 using google::protobuf::TextFormat;
 
 static cl::opt<bool>
     ProtobufTextMode("tfutils-text-log", cl::init(false), cl::Hidden,
                      cl::desc("Output textual (human-readable) protobuf."));
 
+static cl::opt<bool>
+    UseSimpleLogger("tfutils-use-simplelogger", cl::init(false), cl::Hidden,
+                    cl::desc("Output simple (non-protobuf) log."));
+
 namespace {
 
 void serialize(const Message &SE, std::string *OutStr) {
   if (ProtobufTextMode) {
     TextFormat::PrintToString(SE, OutStr);
   } else {
     *OutStr = SE.SerializeAsString();
   }
public:
   virtual size_t getNrRecords() const = 0;
   virtual ~LoggerDataImpl() = default;
 
   template <typename T> void logReward(T Value) {
     logRewardImpl(reinterpret_cast<const char *>(&Value), sizeof(T));
   }
 };
 
+// The design goals of the simple logger are:
+// - no dependencies that llvm doesn't already have.
+// - support streaming, so that we don't need to buffer data during compilation
+// - 0-decoding tensor values. Tensor values are potentially very large buffers
+// of scalars. Because of their potentially large size, avoiding
+// serialization/deserialization overhead is preferred.
+//
+// The simple logger produces an output of the form (each line item on its line)
+// - header: a json object describing the data that will follow.
+// - context: e.g. function name, for regalloc, or "default" for module-wide
+// optimizations like the inliner. This is the context to which the subsequent
+// data corresponds.
+// - observation number.
+// - tensor values - raw bytes of the tensors, in the order given in the header.
+// The values are in succession, i.e. no separator is found between successive
+// tensor values. At the end, there is a new line character.
+// - [score] - this is optional, and is present if it was present in the header.
+// Currently, for final rewards, we output "0" scores after each observation,
+// except for the last one.
+// <repeat>
+// The file should be read as binary, but the reason we use newlines is mostly
+// ease of debugging: the log can be opened in a text editor and, while tensor
+// values are inscrutable, at least the sequence of data can be easily observed.
+// Of course, the buffer of tensor values could contain '\n' bytes. A reader
+// should use the header information to know how much data to read for the
+// tensor values, and not use line information for that.
+//
+// An example reader, used for test, is available at
+// Analysis/models/log_reader.py
+//
+// Example:
+// {"features":[list of TensorSpecs], "score":<a tensor spec>}
+// {"context": "aFunction"}
+// {"observation": 0}
+// <bytes>
+// {"outcome": 0}
+// <bytes for the tensor corresponding to the "score" spec in the header>
+// {"observation": 1}
+// ...
+// {"context": "anotherFunction"}
+// {"observation": 0}
+// ...
+//
+class SimpleLoggerDataImpl : public LoggerDataImpl {
+  std::vector<std::unique_ptr<char[]>> FeatureStorage;
+  std::vector<std::unique_ptr<char[]>> RewardStorage;
+
+  raw_ostream &dumpHeader(raw_ostream &OS) const {
+    json::OStream JOS(OS);
+    JOS.object([&]() {
+      JOS.attributeArray("features", [&]() {
+        for (const auto &TS : LoggedFeatureSpecs)
+          TS.toJSON(JOS);
+      });
+      if (IncludeReward) {
+        JOS.attributeBegin("score");
+        RewardSpec.toJSON(JOS);
+        JOS.attributeEnd();
+      }
+    });
+    OS << "\n";
+    return OS;
+  }
+
+  raw_ostream &startContext(raw_ostream &OS, StringRef Name) const {
+    json::OStream JOS(OS);
+    JOS.object([&]() { JOS.attribute("context", Name); });
+    OS << "\n";
+    return OS;
+  }
+
+  raw_ostream &startObservation(raw_ostream &OS, size_t Nr) const {
+    json::OStream JOS(OS);
+    JOS.object([&]() { JOS.attribute("observation", Nr); });
+    OS << "\n";
+    return OS;
+  }
+
+  raw_ostream &writeOutcome(raw_ostream &OS,
+                            size_t CurrentObservationID) const {
+    if (IncludeReward) {
+      OS << "\n";
+      json::OStream JOS(OS);
+      JOS.object([&]() { JOS.attribute("outcome", CurrentObservationID); });
+      OS << "\n";
+      OS.write(RewardStorage[CurrentObservationID].get(),
+               RewardSpec.getTotalTensorBufferSize());
+    }
+    OS << "\n";
+    return OS;
+  }
+  void flush(std::string *Str) override {
+    llvm_unreachable("Use the ostream implementation");
+  }
+
+  char *addNewTensor(size_t FeatureID) override {
+    return FeatureStorage
+        .emplace_back(
+            new char[LoggedFeatureSpecs[FeatureID].getTotalTensorBufferSize()])
+        .get();
+  }
+
+  size_t getNrRecords() const override {
+    assert(FeatureStorage.size() % LoggedFeatureSpecs.size() == 0);
+    return FeatureStorage.size() / LoggedFeatureSpecs.size();
+  }
+
+  void logRewardImpl(const char *Value, size_t Size) override {
+    std::memcpy(RewardStorage.emplace_back(new char[Size]).get(), Value, Size);
+  }
+
+public:
+  SimpleLoggerDataImpl(const std::vector<TensorSpec> &LoggedSpecs,
+                       const TensorSpec &RewardSpec, bool IncludeReward)
+      : LoggerDataImpl(LoggedSpecs, RewardSpec, IncludeReward) {}
+
+  raw_ostream &flush(raw_ostream &OS, bool WithHeader = true,
+                     StringRef Context = "default") const {
+    if (WithHeader)
+      dumpHeader(OS);
+    startContext(OS, Context);
+    size_t CurrentObservationID = 0;
+    for (size_t I = 0; I < FeatureStorage.size(); ++I) {
+      size_t TensorID = I % LoggedFeatureSpecs.size();
+      if (TensorID == 0) {
+        CurrentObservationID = I / LoggedFeatureSpecs.size();
+        startObservation(OS, CurrentObservationID);
+      }
+      OS.write(FeatureStorage[I].get(),
+               LoggedFeatureSpecs[TensorID].getTotalTensorBufferSize());
+      if (TensorID == LoggedFeatureSpecs.size() - 1) {
+        writeOutcome(OS, CurrentObservationID);
+      }
+    }
+    return OS;
+  }
+};
+
 class TFSequenceExampleLoggerDataImpl : public LoggerDataImpl {
   std::vector<tensorflow::FeatureList> FeatureLists;
   tensorflow::FeatureList Reward;
 
   bool isSelfConsistent(const tensorflow::SequenceExample &SE,
                         size_t NrRecords) const {
     bool Ret = true;
     for (const auto &TSpecs : LoggedFeatureSpecs) {
size_t getNrRecords() const override {
     return FeatureLists.empty() ? 0 : FeatureLists[0].feature().size();
   }
 };
 } // namespace llvm
 
 Logger::Logger(const std::vector<TensorSpec> &FeatureSpecs,
                const TensorSpec &RewardSpec, bool IncludeReward)
     : FeatureSpecs(FeatureSpecs), RewardSpec(RewardSpec),
-      IncludeReward(IncludeReward),
-      LoggerData(std::make_unique<TFSequenceExampleLoggerDataImpl>(
-          FeatureSpecs, RewardSpec, IncludeReward)) {}
+      IncludeReward(IncludeReward) {
+  if (UseSimpleLogger)
+    LoggerData = std::make_unique<SimpleLoggerDataImpl>(
+        FeatureSpecs, RewardSpec, IncludeReward);
+  else
+    LoggerData = std::make_unique<TFSequenceExampleLoggerDataImpl>(
+        FeatureSpecs, RewardSpec, IncludeReward);
+}
 
 Logger::~Logger() {}
 
 #define LOG_REWARD(NAME, TYPE)                                                 \
   void Logger::log##NAME##Reward(TYPE Value) {                                 \
     assert(IncludeReward);                                                     \
     LoggerData->logReward(Value);                                              \
   }

 
 char *Logger::addEntryAndGetFloatOrInt64Buffer(size_t FeatureID) {
   return reinterpret_cast<char *>(LoggerData->addNewTensor(FeatureID));
 }
 
 void Logger::flush(std::string *Str) { LoggerData->flush(Str); }
 
 void Logger::flush(raw_ostream &OS) {
+  if (UseSimpleLogger) {
+    reinterpret_cast<SimpleLoggerDataImpl *>(LoggerData.get())->flush(OS);
+  } else {
     std::string Buff;
     LoggerData->flush(&Buff);
     OS << Buff;
   }
+}
 
 void Logger::flushLogs(raw_ostream &OS,
                        const StringMap<std::unique_ptr<Logger>> &Loggers) {
+  if (UseSimpleLogger) {
+    bool IsFirst = true;
+    for (const auto &NamedLogger : Loggers) {
+      auto *Impl = NamedLogger.second->LoggerData.get();
+      reinterpret_cast<const SimpleLoggerDataImpl *>(Impl)->flush(
+          OS, IsFirst, NamedLogger.first());
+      IsFirst = false;
+    }
+  } else {
     google::protobuf::Struct Msg;
     for (const auto &NamedLogger : Loggers) {
       tensorflow::SequenceExample SE;
       const auto &Logger = NamedLogger.second;
       std::string Unencoded;
       if (Logger->LoggerData->getNrRecords() > 0)
         Logger->flush(&Unencoded);
 
       (*Msg.mutable_fields())[NamedLogger.first().str()]
           .mutable_string_value()
           ->append(ProtobufTextMode ? Unencoded : encodeBase64(Unencoded));
     }
 
     std::string OutStr;
     serialize(Msg, &OutStr);
     OS << OutStr;
   }
+}
 #endif // defined(LLVM_HAVE_TF_API)

llvm/lib/Analysis/models/log_reader.py

+"""Reader for training log.
+
+See lib/Analysis/TrainingLogger.cpp for a description of the format.
+"""
+import ctypes
+import dataclasses
+import json
+import math
+import sys
+import typing
+
+_element_types = {
+    'float': ctypes.c_float,
+    'double': ctypes.c_double,
+    'int8_t': ctypes.c_int8,
+    'uint8_t': ctypes.c_uint8,
+    'int16_t': ctypes.c_int16,
+    'uint16_t': ctypes.c_uint16,
+    'int32_t': ctypes.c_int32,
+    'uint32_t': ctypes.c_uint32,
+    'int64_t': ctypes.c_int64,
+    'uint64_t': ctypes.c_uint64
+}
+
+
[e-mail](frozen=True)
+class TensorSpec:
+  name: str
+  port: int
+  shape: list[int]
+  element_type: type
+
+  @staticmethod
+  def from_dict(d: dict):
+    name = d['name']
+    port = d['port']
+    shape = [int(e) for e in d['shape']]
+    element_type_str = d['type']
+    if element_type_str not in _element_types:
+      raise ValueError(f'uknown type: {element_type_str}')
+    return TensorSpec(
+        name=name,
+        port=port,
+        shape=shape,
+        element_type=_element_types[element_type_str])
+
+
+class TensorValue:
+
+  def __init__(self, spec: TensorSpec, buffer: bytes):
+    self._spec = spec
+    self._buffer = buffer
+    self._view = ctypes.cast(self._buffer,
+                             ctypes.POINTER(self._spec.element_type))
+    self._len = math.prod(self._spec.shape)
+
+  def spec(self) -> TensorSpec:
+    return self._spec
+
+  def __len__(self) -> int:
+    return self._len
+
+  def __getitem__(self, index):
+    if index < 0 or index >= self._len:
+      raise IndexError(f'Index {index} out of range [0..{self._len})')
+    return self._view[index]
+
+
+def read_tensor(fs: typing.BinaryIO, ts: TensorSpec) -> TensorValue:
+  size = math.prod(ts.shape) * ctypes.sizeof(ts.element_type)
+  data = fs.read(size)
+  return TensorValue(ts, data)
+
+
+def pretty_print_tensor_value(tv: TensorValue):
+  print(f'{tv.spec().name}: {",".join([str(v) for v in tv])}')
+
+
+def read_stream(fname: str):
+  with open(fname, 'rb') as f:
+    header = json.loads(f.readline())
+    tensor_specs = [TensorSpec.from_dict(ts) for ts in header['features']]
+    score_spec = TensorSpec.from_dict(
+        header['score']) if 'score' in header else None
+    context = None
+    while event_str := f.readline():
+      event = json.loads(event_str)
+      if 'context' in event:
+        context = event['context']
+        continue
+      observation_id = int(event['observation'])
+      features = []
+      for ts in tensor_specs:
+        features.append(read_tensor(f, ts))
+      f.readline()
+      score = None
+      if score_spec is not None:
+        score_header = json.loads(f.readline())
+        assert int(score_header['outcome']) == observation_id
+        score = read_tensor(f, score_spec)
+        f.readline()
+      yield context, observation_id, features, score
+
+
+def main(args):
+  last_context = None
+  for ctx, obs_id, features, score in read_stream(args[1]):
+    if last_context != ctx:
+      print(f'context: {ctx}')
+      last_context = ctx
+    print(f'observation: {obs_id}')
+    for fv in features:
+      pretty_print_tensor_value(fv)
+    if score:
+      pretty_print_tensor_value(score)
+
+
+if __name__ == '__main__':
+  main(sys.argv)

llvm/test/CodeGen/MLRegalloc/Inputs/two-large-fcts.ll

+; This is a copy of test/CodeGen/X86/ragreedy-hoist-spill.ll. It generates
+; sufficiently interesting differences between the default eviction heuristic
+; and the test ML policy: different eviction choices, and different reward.
+;
+;
+%struct.TMP.1 = type { %struct.TMP.2*, %struct.TMP.2*, [1024 x i8] }
+%struct.TMP.2 = type { i8*, i32, i32, i16, i16, %struct.TMP.3, i32, i8*, i32 (i8*)*, i32 (i8*, i8*, i32)*, i64 (i8*, i64, i32)*, i32 (i8*, i8*, i32)*, %struct.TMP.3, %struct.TMP.4*, i32, [3 x i8], [1 x i8], %struct.TMP.3, i32, i64 }
+%struct.TMP.4 = type opaque
+%struct.TMP.3 = type { i8*, i32 }
+
+@syBuf = external global [16 x %struct.TMP.1], align 16
+@syHistory = external global [8192 x i8], align 16
[e-mail] = external global [512 x i8], align 16
+@syCTRO = external global i32, align 4
+
+define i8* @SyFgets(i8* %line, i64 %length, i64 %fid) {
+entry:
+  %sub.ptr.rhs.cast646 = ptrtoint i8* %line to i64
+  %old = alloca [512 x i8], align 16
+  %0 = getelementptr inbounds [512 x i8], [512 x i8]* %old, i64 0, i64 0
+  switch i64 %fid, label %if.then [
+    i64 2, label %if.end
+    i64 0, label %if.end
+  ]
+
+if.then:
+  br label %cleanup
+
+if.end:
+  switch i64 undef, label %if.end25 [
+    i64 0, label %if.then4
+    i64 1, label %if.end25
+  ]
+
+if.then4:
+  br i1 undef, label %SyTime.exit, label %if.then.i
+
+if.then.i:
+  unreachable
+
+SyTime.exit:
+  br i1 undef, label %SyTime.exit2681, label %if.then.i2673
+
+if.then.i2673:
+  unreachable
+
+SyTime.exit2681:
+  br label %cleanup
+
+land.lhs.true14:
+  unreachable
+
+if.end25:
+  br i1 undef, label %SyTime.exit2720, label %if.then.i2712
+
+if.then.i2712:
+  unreachable
+
+SyTime.exit2720:
+  %add.ptr = getelementptr [512 x i8], [512 x i8]* %old, i64 0, i64 512
+  %cmp293427 = icmp ult i8* %0, %add.ptr
+  br i1 %cmp293427, label %for.body.lr.ph, label %while.body.preheader
+
+for.body.lr.ph:
+  call void @llvm.memset.p0i8.i64(i8* align 16 undef, i8 32, i64 512, i1 false)
+  br label %while.body.preheader
+
+while.body.preheader:
+  %add.ptr1603 = getelementptr [512 x i8], [512 x i8]* null, i64 0, i64 512
+  %echo.i3101 = getelementptr [16 x %struct.TMP.1], [16 x %struct.TMP.1]* @syBuf, i64 0, i64 %fid, i32 1
+  %1 = xor i64 %sub.ptr.rhs.cast646, -1
+  br label %do.body
+
+do.body:
+  %ch2.0 = phi i32 [ 0, %while.body.preheader ], [ %ch.12.ch2.12, %do.body ]
+  %rep.0 = phi i32 [ 1, %while.body.preheader ], [ %rep.6, %do.body ]
+  store i32 0, i32* @syCTRO, align 4, !tbaa !1
+  %ch.0.ch2.0 = select i1 undef, i32 14, i32 %ch2.0
+  %ch2.2 = select i1 undef, i32 0, i32 %ch.0.ch2.0
+  %ch.2.ch2.2 = select i1 undef, i32 0, i32 %ch2.2
+  %ch2.4 = select i1 undef, i32 278, i32 %ch.2.ch2.2
+  %ch2.5 = select i1 undef, i32 0, i32 %ch2.4
+  %rep.2 = select i1 undef, i32 undef, i32 %rep.0
+  %ch.5.ch2.5 = select i1 undef, i32 undef, i32 %ch2.5
+  %ch2.7 = select i1 undef, i32 0, i32 %ch.5.ch2.5
+  %rep.3 = select i1 undef, i32 undef, i32 %rep.2
+  %ch.7.ch2.7 = select i1 false, i32 0, i32 %ch2.7
+  %mul98.rep.3 = select i1 false, i32 0, i32 %rep.3
+  %ch2.9 = select i1 undef, i32 undef, i32 %ch.7.ch2.7
+  %rep.5 = select i1 undef, i32 undef, i32 %mul98.rep.3
+  %ch2.10 = select i1 false, i32 undef, i32 %ch2.9
+  %rep.6 = select i1 false, i32 undef, i32 %rep.5
+  %isdigittmp = add i32 %ch2.10, -48
+  %isdigit = icmp ult i32 %isdigittmp, 10
+  %cmp119 = icmp eq i32 undef, 22
+  %or.cond1875 = and i1 %isdigit, %cmp119
+  %ch.10.ch2.10 = select i1 %or.cond1875, i32 undef, i32 %ch2.10
+  %.ch.10 = select i1 %or.cond1875, i32 0, i32 undef
+  %ch2.12 = select i1 undef, i32 %.ch.10, i32 %ch.10.ch2.10
+  %ch.12 = select i1 undef, i32 0, i32 %.ch.10
+  %ch.12.ch2.12 = select i1 false, i32 %ch.12, i32 %ch2.12
+  %.ch.12 = select i1 false, i32 0, i32 %ch.12
+  %cmp147 = icmp eq i32 %.ch.12, 0
+  br i1 %cmp147, label %do.body, label %do.end
+
+do.end:
+  %cmp164 = icmp eq i32 %ch.12.ch2.12, 21
+  %mul167 = shl i32 %rep.6, 2
+  %rep.8 = select i1 %cmp164, i32 %mul167, i32 %rep.6
+  %..ch.19 = select i1 false, i32 2, i32 0
+  br i1 undef, label %while.body200, label %while.end1465
+
+while.body200:
+  %dec3386.in = phi i32 [ %dec3386, %while.cond197.backedge ], [ %rep.8, %do.end ]
+  %oldc.13384 = phi i32 [ %oldc.1.be, %while.cond197.backedge ], [ 0, %do.end ]
+  %ch.213379 = phi i32 [ %last.1.be, %while.cond197.backedge ], [ %..ch.19, %do.end ]
+  %last.13371 = phi i32 [ %last.1.be, %while.cond197.backedge ], [ 0, %do.end ]
+  %dec3386 = add i32 %dec3386.in, -1
+  switch i32 %ch.213379, label %sw.default [
+    i32 1, label %while.cond201.preheader
+    i32 322, label %sw.bb206
+    i32 354, label %sw.bb206
+    i32 2, label %sw.bb243
+    i32 364, label %sw.bb1077
+    i32 326, label %sw.bb256
+    i32 358, label %sw.bb256
+    i32 341, label %sw.bb979
+    i32 323, label %while.cond1037.preheader
+    i32 373, label %sw.bb979
+    i32 4, label %if.then1477
+    i32 332, label %sw.bb1077
+    i32 11, label %for.cond357
+    i32 355, label %while.cond1037.preheader
+    i32 324, label %sw.bb474
+    i32 356, label %sw.bb474
+    i32 20, label %sw.bb566
+    i32 -1, label %while.cond197.backedge
+    i32 268, label %sw.bb1134
+    i32 16, label %while.cond635.preheader
+    i32 18, label %sw.bb956
+    i32 316, label %while.cond864
+  ]
+
+while.cond1037.preheader:
+  %cmp10393273 = icmp eq i8 undef, 0
+  br i1 %cmp10393273, label %if.end1070, label %land.rhs1041
+
+while.cond635.preheader:
+  br i1 undef, label %for.body643.us, label %while.cond661
+
+for.body643.us:
+  br label %for.body643.us
+
+while.cond201.preheader:
+  %umax = select i1 false, i64 undef, i64 %1
+  %2 = xor i64 %umax, -1
+  %3 = inttoptr i64 %2 to i8*
+  br label %while.cond197.backedge
+
+sw.bb206:
+  br label %while.cond197.backedge
+
+sw.bb243:
+  br label %while.cond197.backedge
+
+sw.bb256:
+  br label %while.cond197.backedge
+
+while.cond197.backedge:
+  %last.1.be = phi i32 [ %ch.213379, %sw.default ], [ -1, %while.body200 ], [ %ch.213379, %sw.bb1077 ], [ %ch.213379, %sw.bb979 ], [ 18, %sw.bb956 ], [ 20, %sw.bb566 ], [ %ch.213379, %for.end552 ], [ %ch.213379, %sw.bb256 ], [ 2, %sw.bb243 ], [ 1, %while.cond201.preheader ], [ 268, %for.cond1145.preheader ], [ %ch.213379, %sw.bb206 ]
+  %oldc.1.be = phi i32 [ %oldc.13384, %sw.default ], [ %oldc.13384, %while.body200 ], [ %oldc.13384, %sw.bb1077 ], [ %oldc.13384, %sw.bb979 ], [ %oldc.13384, %sw.bb956 ], [ %oldc.13384, %sw.bb566 ], [ %oldc.13384, %for.end552 ], [ %oldc.13384, %sw.bb256 ], [ %oldc.13384, %sw.bb243 ], [ %oldc.13384, %while.cond201.preheader ], [ 0, %for.cond1145.preheader ], [ %oldc.13384, %sw.bb206 ]
+  %cmp198 = icmp sgt i32 %dec3386, 0
+  br i1 %cmp198, label %while.body200, label %while.end1465
+
+for.cond357:
+  br label %for.cond357
+
+sw.bb474:
+  ; spill is hoisted here. Although loop depth1 is even hotter than loop depth2, sw.bb474 is still cold.
+  %cmp476 = icmp eq i8 undef, 0
+  br i1 %cmp476, label %if.end517, label %do.body479.preheader
+
+do.body479.preheader:
+  %cmp4833314 = icmp eq i8 undef, 0
+  br i1 %cmp4833314, label %if.end517, label %land.rhs485
+
+land.rhs485:
+  %incdec.ptr4803316 = phi i8* [ %incdec.ptr480, %do.body479.backedge.land.rhs485_crit_edge ], [ undef, %do.body479.preheader ]
+  %isascii.i.i27763151 = icmp sgt i8 undef, -1
+  br i1 %isascii.i.i27763151, label %cond.true.i.i2780, label %cond.false.i.i2782
+
+cond.true.i.i2780:
+  br i1 undef, label %land.lhs.true490, label %lor.rhs500
+
+cond.false.i.i2782:
+  unreachable
+
+land.lhs.true490:
+  br i1 false, label %lor.rhs500, label %do.body479.backedge
+
+lor.rhs500:
+  ; Make sure spill is hoisted to a cold preheader in outside loop.
+  %call3.i.i2792 = call i32 @__maskrune(i32 undef, i64 256)
+  br i1 undef, label %land.lhs.true504, label %do.body479.backedge
+
+land.lhs.true504:
+  br i1 undef, label %do.body479.backedge, label %if.end517
+
+do.body479.backedge:
+  %incdec.ptr480 = getelementptr i8, i8* %incdec.ptr4803316, i64 1
+  %cmp483 = icmp eq i8 undef, 0
+  br i1 %cmp483, label %if.end517, label %do.body479.backedge.land.rhs485_crit_edge
+
+do.body479.backedge.land.rhs485_crit_edge:
+  br label %land.rhs485
+
+if.end517:
+  %q.4 = phi i8* [ undef, %sw.bb474 ], [ undef, %do.body479.preheader ], [ %incdec.ptr480, %do.body479.backedge ], [ %incdec.ptr4803316, %land.lhs.true504 ]
+  switch i32 %last.13371, label %if.then532 [
+    i32 383, label %for.cond534
+    i32 356, label %for.cond534
+    i32 324, label %for.cond534
+    i32 24, label %for.cond534
+    i32 11, label %for.cond534
+  ]
+
+if.then532:
+  store i8 0, i8* getelementptr inbounds ([512 x i8], [512 x i8]* @SyFgets.yank, i64 0, i64 0), align 16, !tbaa !5
+  br label %for.cond534
+
+for.cond534:
+  %cmp536 = icmp eq i8 undef, 0
+  br i1 %cmp536, label %for.cond542.preheader, label %for.cond534
+
+for.cond542.preheader:
+  br i1 undef, label %for.body545, label %for.end552
+
+for.body545:
+  br i1 undef, label %for.end552, label %for.body545
+
+for.end552:
+  %s.2.lcssa = phi i8* [ undef, %for.cond542.preheader ], [ %q.4, %for.body545 ]
+  %sub.ptr.lhs.cast553 = ptrtoint i8* %s.2.lcssa to i64
+  %sub.ptr.sub555 = sub i64 %sub.ptr.lhs.cast553, 0
+  %arrayidx556 = getelementptr i8, i8* null, i64 %sub.ptr.sub555
+  store i8 0, i8* %arrayidx556, align 1, !tbaa !5
+  br label %while.cond197.backedge
+
+sw.bb566:
+  br label %while.cond197.backedge
+
+while.cond661:
+  br label %while.cond661
+
+while.cond864:
+  br label %while.cond864
+
+sw.bb956:
+  br i1 undef, label %if.then959, label %while.cond197.backedge
+
+if.then959:
+  br label %while.cond962
+
+while.cond962:
+  br label %while.cond962
+
+sw.bb979:
+  br label %while.cond197.backedge
+
+land.rhs1041:
+  unreachable
+
+if.end1070:
+  br label %sw.bb1077
+
+sw.bb1077:
+  br label %while.cond197.backedge
+
+sw.bb1134:
+  br i1 false, label %for.body1139, label %for.cond1145.preheader
+
+for.cond1145.preheader:
+  br i1 %cmp293427, label %for.body1150.lr.ph, label %while.cond197.backedge
+
+for.body1150.lr.ph:
+  unreachable
+
+for.body1139:
+  unreachable
+
+sw.default:
+  br label %while.cond197.backedge
+
+while.end1465:
+  %oldc.1.lcssa = phi i32 [ 0, %do.end ], [ %oldc.1.be, %while.cond197.backedge ]
+  %ch.21.lcssa = phi i32 [ %..ch.19, %do.end ], [ %last.1.be, %while.cond197.backedge ]
+  switch i32 %ch.21.lcssa, label %for.cond1480.preheader [
+    i32 -1, label %if.then1477
+    i32 15, label %if.then1477
+    i32 13, label %if.then1477
+    i32 10, label %if.then1477
+  ]
+
+for.cond1480.preheader:
+  br i1 undef, label %for.body1606.lr.ph, label %for.end1609
+
+if.then1477:
+  %p.1.lcssa3539 = phi i8* [ null, %while.end1465 ], [ null, %while.end1465 ], [ null, %while.end1465 ], [ null, %while.end1465 ], [ %line, %while.body200 ]
+  %call1.i3057 = call i64 @"\01_write"(i32 undef, i8* undef, i64 1)
+  %sub.ptr.lhs.cast1717 = ptrtoint i8* %p.1.lcssa3539 to i64
+  %sub.ptr.sub1719 = sub i64 %sub.ptr.lhs.cast1717, %sub.ptr.rhs.cast646
+  %idx.neg1727 = sub i64 0, %sub.ptr.sub1719
+  br label %for.body1723
+
+for.body1606.lr.ph:
+  br label %for.end1609
+
+for.end1609:
+  br i1 undef, label %for.cond1659.preheader, label %land.lhs.true1614
+
+land.lhs.true1614:
+  br label %for.cond1659.preheader
+
+for.cond1659.preheader:
+  %cmp16623414 = icmp ult i8* undef, %add.ptr1603
+  br i1 %cmp16623414, label %for.body1664.lr.ph, label %while.body1703.lr.ph
+
+for.body1664.lr.ph:
+  %cmp16773405 = icmp slt i64 undef, undef
+  br i1 %cmp16773405, label %while.body1679, label %while.cond1683.preheader
+
+while.body1703.lr.ph:
+  unreachable
+
+while.cond1683.preheader:
+  br i1 undef, label %while.body1691, label %while.end1693
+
+while.body1679:
+  %oldc.43406 = phi i32 [ %inc, %syEchoch.exit3070 ], [ %oldc.1.lcssa, %for.body1664.lr.ph ]
+  %4 = load %struct.TMP.2*, %struct.TMP.2** %echo.i3101, align 8, !tbaa !6
+  %call.i3062 = call i32 @fileno(%struct.TMP.2* %4)
+  br i1 undef, label %if.then.i3069, label %syEchoch.exit3070
+
+if.then.i3069:
+  br label %syEchoch.exit3070
+
+syEchoch.exit3070:
+  %inc = add i32 %oldc.43406, 1
+  %conv1672 = sext i32 %inc to i64
+  %cmp1677 = icmp slt i64 %conv1672, undef
+  br i1 %cmp1677, label %while.body1679, label %while.cond1683.preheader
+
+while.body1691:
+  unreachable
+
+while.end1693:
+  unreachable
+
+for.body1723:
+  %q.303203 = phi i8* [ getelementptr inbounds ([8192 x i8], [8192 x i8]* @syHistory, i64 0, i64 8189), %if.then1477 ], [ %incdec.ptr1730, %for.body1723 ]
+  %add.ptr1728 = getelementptr i8, i8* %q.303203, i64 %idx.neg1727
+  %5 = load i8, i8* %add.ptr1728, align 1, !tbaa !5
+  %incdec.ptr1730 = getelementptr i8, i8* %q.303203, i64 -1
+  br label %for.body1723
+
+cleanup:
+  ret i8* undef
+}
+
+
+define i8* @SyFgetsCopy(i8* %line, i64 %length, i64 %fid) {
+entry:
+  %sub.ptr.rhs.cast646 = ptrtoint i8* %line to i64
+  %old = alloca [512 x i8], align 16
+  %0 = getelementptr inbounds [512 x i8], [512 x i8]* %old, i64 0, i64 0
+  switch i64 %fid, label %if.then [
+    i64 2, label %if.end
+    i64 0, label %if.end
+  ]
+
+if.then:
+  br label %cleanup
+
+if.end:
+  switch i64 undef, label %if.end25 [
+    i64 0, label %if.then4
+    i64 1, label %if.end25
+  ]
+
+if.then4:
+  br i1 undef, label %SyTime.exit, label %if.then.i
+
+if.then.i:
+  unreachable
+
+SyTime.exit:
+  br i1 undef, label %SyTime.exit2681, label %if.then.i2673
+
+if.then.i2673:
+  unreachable
+
+SyTime.exit2681:
+  br label %cleanup
+
+land.lhs.true14:
+  unreachable
+
+if.end25:
+  br i1 undef, label %SyTime.exit2720, label %if.then.i2712
+
+if.then.i2712:
+  unreachable
+
+SyTime.exit2720:
+  %add.ptr = getelementptr [512 x i8], [512 x i8]* %old, i64 0, i64 512
+  %cmp293427 = icmp ult i8* %0, %add.ptr
+  br i1 %cmp293427, label %for.body.lr.ph, label %while.body.preheader
+
+for.body.lr.ph:
+  call void @llvm.memset.p0i8.i64(i8* align 16 undef, i8 32, i64 512, i1 false)
+  br label %while.body.preheader
+
+while.body.preheader:
+  %add.ptr1603 = getelementptr [512 x i8], [512 x i8]* null, i64 0, i64 512
+  %echo.i3101 = getelementptr [16 x %struct.TMP.1], [16 x %struct.TMP.1]* @syBuf, i64 0, i64 %fid, i32 1
+  %1 = xor i64 %sub.ptr.rhs.cast646, -1
+  br label %do.body
+
+do.body:
+  %ch2.0 = phi i32 [ 0, %while.body.preheader ], [ %ch.12.ch2.12, %do.body ]
+  %rep.0 = phi i32 [ 1, %while.body.preheader ], [ %rep.6, %do.body ]
+  store i32 0, i32* @syCTRO, align 4, !tbaa !1
+  %ch.0.ch2.0 = select i1 undef, i32 14, i32 %ch2.0
+  %ch2.2 = select i1 undef, i32 0, i32 %ch.0.ch2.0
+  %ch.2.ch2.2 = select i1 undef, i32 0, i32 %ch2.2
+  %ch2.4 = select i1 undef, i32 278, i32 %ch.2.ch2.2
+  %ch2.5 = select i1 undef, i32 0, i32 %ch2.4
+  %rep.2 = select i1 undef, i32 undef, i32 %rep.0
+  %ch.5.ch2.5 = select i1 undef, i32 undef, i32 %ch2.5
+  %ch2.7 = select i1 undef, i32 0, i32 %ch.5.ch2.5
+  %rep.3 = select i1 undef, i32 undef, i32 %rep.2
+  %ch.7.ch2.7 = select i1 false, i32 0, i32 %ch2.7
+  %mul98.rep.3 = select i1 false, i32 0, i32 %rep.3
+  %ch2.9 = select i1 undef, i32 undef, i32 %ch.7.ch2.7
+  %rep.5 = select i1 undef, i32 undef, i32 %mul98.rep.3
+  %ch2.10 = select i1 false, i32 undef, i32 %ch2.9
+  %rep.6 = select i1 false, i32 undef, i32 %rep.5
+  %isdigittmp = add i32 %ch2.10, -48
+  %isdigit = icmp ult i32 %isdigittmp, 10
+  %cmp119 = icmp eq i32 undef, 22
+  %or.cond1875 = and i1 %isdigit, %cmp119
+  %ch.10.ch2.10 = select i1 %or.cond1875, i32 undef, i32 %ch2.10
+  %.ch.10 = select i1 %or.cond1875, i32 0, i32 undef
+  %ch2.12 = select i1 undef, i32 %.ch.10, i32 %ch.10.ch2.10
+  %ch.12 = select i1 undef, i32 0, i32 %.ch.10
+  %ch.12.ch2.12 = select i1 false, i32 %ch.12, i32 %ch2.12
+  %.ch.12 = select i1 false, i32 0, i32 %ch.12
+  %cmp147 = icmp eq i32 %.ch.12, 0
+  br i1 %cmp147, label %do.body, label %do.end
+
+do.end:
+  %cmp164 = icmp eq i32 %ch.12.ch2.12, 21
+  %mul167 = shl i32 %rep.6, 2
+  %rep.8 = select i1 %cmp164, i32 %mul167, i32 %rep.6
+  %..ch.19 = select i1 false, i32 2, i32 0
+  br i1 undef, label %while.body200, label %while.end1465
+
+while.body200:
+  %dec3386.in = phi i32 [ %dec3386, %while.cond197.backedge ], [ %rep.8, %do.end ]
+  %oldc.13384 = phi i32 [ %oldc.1.be, %while.cond197.backedge ], [ 0, %do.end ]
+  %ch.213379 = phi i32 [ %last.1.be, %while.cond197.backedge ], [ %..ch.19, %do.end ]
+  %last.13371 = phi i32 [ %last.1.be, %while.cond197.backedge ], [ 0, %do.end ]
+  %dec3386 = add i32 %dec3386.in, -1
+  switch i32 %ch.213379, label %sw.default [
+    i32 1, label %while.cond201.preheader
+    i32 322, label %sw.bb206
+    i32 354, label %sw.bb206
+    i32 2, label %sw.bb243
+    i32 364, label %sw.bb1077
+    i32 326, label %sw.bb256
+    i32 358, label %sw.bb256
+    i32 341, label %sw.bb979
+    i32 323, label %while.cond1037.preheader
+    i32 373, label %sw.bb979
+    i32 4, label %if.then1477
+    i32 332, label %sw.bb1077
+    i32 11, label %for.cond357
+    i32 355, label %while.cond1037.preheader
+    i32 324, label %sw.bb474
+    i32 356, label %sw.bb474
+    i32 20, label %sw.bb566
+    i32 -1, label %while.cond197.backedge
+    i32 268, label %sw.bb1134
+    i32 16, label %while.cond635.preheader
+    i32 18, label %sw.bb956
+    i32 316, label %while.cond864
+  ]
+
+while.cond1037.preheader:
+  %cmp10393273 = icmp eq i8 undef, 0
+  br i1 %cmp10393273, label %if.end1070, label %land.rhs1041
+
+while.cond635.preheader:
+  br i1 undef, label %for.body643.us, label %while.cond661
+
+for.body643.us:
+  br label %for.body643.us
+
+while.cond201.preheader:
+  %umax = select i1 false, i64 undef, i64 %1
+  %2 = xor i64 %umax, -1
+  %3 = inttoptr i64 %2 to i8*
+  br label %while.cond197.backedge
+
+sw.bb206:
+  br label %while.cond197.backedge
+
+sw.bb243:
+  br label %while.cond197.backedge
+
+sw.bb256:
+  br label %while.cond197.backedge
+
+while.cond197.backedge:
+  %last.1.be = phi i32 [ %ch.213379, %sw.default ], [ -1, %while.body200 ], [ %ch.213379, %sw.bb1077 ], [ %ch.213379, %sw.bb979 ], [ 18, %sw.bb956 ], [ 20, %sw.bb566 ], [ %ch.213379, %for.end552 ], [ %ch.213379, %sw.bb256 ], [ 2, %sw.bb243 ], [ 1, %while.cond201.preheader ], [ 268, %for.cond1145.preheader ], [ %ch.213379, %sw.bb206 ]
+  %oldc.1.be = phi i32 [ %oldc.13384, %sw.default ], [ %oldc.13384, %while.body200 ], [ %oldc.13384, %sw.bb1077 ], [ %oldc.13384, %sw.bb979 ], [ %oldc.13384, %sw.bb956 ], [ %oldc.13384, %sw.bb566 ], [ %oldc.13384, %for.end552 ], [ %oldc.13384, %sw.bb256 ], [ %oldc.13384, %sw.bb243 ], [ %oldc.13384, %while.cond201.preheader ], [ 0, %for.cond1145.preheader ], [ %oldc.13384, %sw.bb206 ]
+  %cmp198 = icmp sgt i32 %dec3386, 0
+  br i1 %cmp198, label %while.body200, label %while.end1465
+
+for.cond357:
+  br label %for.cond357
+
+sw.bb474:
+  ; spill is hoisted here. Although loop depth1 is even hotter than loop depth2, sw.bb474 is still cold.
+  %cmp476 = icmp eq i8 undef, 0
+  br i1 %cmp476, label %if.end517, label %do.body479.preheader
+
+do.body479.preheader:
+  %cmp4833314 = icmp eq i8 undef, 0
+  br i1 %cmp4833314, label %if.end517, label %land.rhs485
+
+land.rhs485:
+  %incdec.ptr4803316 = phi i8* [ %incdec.ptr480, %do.body479.backedge.land.rhs485_crit_edge ], [ undef, %do.body479.preheader ]
+  %isascii.i.i27763151 = icmp sgt i8 undef, -1
+  br i1 %isascii.i.i27763151, label %cond.true.i.i2780, label %cond.false.i.i2782
+
+cond.true.i.i2780:
+  br i1 undef, label %land.lhs.true490, label %lor.rhs500
+
+cond.false.i.i2782:
+  unreachable
+
+land.lhs.true490:
+  br i1 false, label %lor.rhs500, label %do.body479.backedge
+
+lor.rhs500:
+  ; Make sure spill is hoisted to a cold preheader in outside loop.
+  %call3.i.i2792 = call i32 @__maskrune(i32 undef, i64 256)
+  br i1 undef, label %land.lhs.true504, label %do.body479.backedge
+
+land.lhs.true504:
+  br i1 undef, label %do.body479.backedge, label %if.end517
+
+do.body479.backedge:
+  %incdec.ptr480 = getelementptr i8, i8* %incdec.ptr4803316, i64 1
+  %cmp483 = icmp eq i8 undef, 0
+  br i1 %cmp483, label %if.end517, label %do.body479.backedge.land.rhs485_crit_edge
+
+do.body479.backedge.land.rhs485_crit_edge:
+  br label %land.rhs485
+
+if.end517:
+  %q.4 = phi i8* [ undef, %sw.bb474 ], [ undef, %do.body479.preheader ], [ %incdec.ptr480, %do.body479.backedge ], [ %incdec.ptr4803316, %land.lhs.true504 ]
+  switch i32 %last.13371, label %if.then532 [
+    i32 383, label %for.cond534
+    i32 356, label %for.cond534
+    i32 324, label %for.cond534
+    i32 24, label %for.cond534
+    i32 11, label %for.cond534
+  ]
+
+if.then532:
+  store i8 0, i8* getelementptr inbounds ([512 x i8], [512 x i8]* @SyFgets.yank, i64 0, i64 0), align 16, !tbaa !5
+  br label %for.cond534
+
+for.cond534:
+  %cmp536 = icmp eq i8 undef, 0
+  br i1 %cmp536, label %for.cond542.preheader, label %for.cond534
+
+for.cond542.preheader:
+  br i1 undef, label %for.body545, label %for.end552
+
+for.body545:
+  br i1 undef, label %for.end552, label %for.body545
+
+for.end552:
+  %s.2.lcssa = phi i8* [ undef, %for.cond542.preheader ], [ %q.4, %for.body545 ]
+  %sub.ptr.lhs.cast553 = ptrtoint i8* %s.2.lcssa to i64
+  %sub.ptr.sub555 = sub i64 %sub.ptr.lhs.cast553, 0
+  %arrayidx556 = getelementptr i8, i8* null, i64 %sub.ptr.sub555
+  store i8 0, i8* %arrayidx556, align 1, !tbaa !5
+  br label %while.cond197.backedge
+
+sw.bb566:
+  br label %while.cond197.backedge
+
+while.cond661:
+  br label %while.cond661
+
+while.cond864:
+  br label %while.cond864
+
+sw.bb956:
+  br i1 undef, label %if.then959, label %while.cond197.backedge
+
+if.then959:
+  br label %while.cond962
+
+while.cond962:
+  br label %while.cond962
+
+sw.bb979:
+  br label %while.cond197.backedge
+
+land.rhs1041:
+  unreachable
+
+if.end1070:
+  br label %sw.bb1077
+
+sw.bb1077:
+  br label %while.cond197.backedge
+
+sw.bb1134:
+  br i1 false, label %for.body1139, label %for.cond1145.preheader
+
+for.cond1145.preheader:
+  br i1 %cmp293427, label %for.body1150.lr.ph, label %while.cond197.backedge
+
+for.body1150.lr.ph:
+  unreachable
+
+for.body1139:
+  unreachable
+
+sw.default:
+  br label %while.cond197.backedge
+
+while.end1465:
+  %oldc.1.lcssa = phi i32 [ 0, %do.end ], [ %oldc.1.be, %while.cond197.backedge ]
+  %ch.21.lcssa = phi i32 [ %..ch.19, %do.end ], [ %last.1.be, %while.cond197.backedge ]
+  switch i32 %ch.21.lcssa, label %for.cond1480.preheader [
+    i32 -1, label %if.then1477
+    i32 15, label %if.then1477
+    i32 13, label %if.then1477
+    i32 10, label %if.then1477
+  ]
+
+for.cond1480.preheader:
+  br i1 undef, label %for.body1606.lr.ph, label %for.end1609
+
+if.then1477:
+  %p.1.lcssa3539 = phi i8* [ null, %while.end1465 ], [ null, %while.end1465 ], [ null, %while.end1465 ], [ null, %while.end1465 ], [ %line, %while.body200 ]
+  %call1.i3057 = call i64 @"\01_write"(i32 undef, i8* undef, i64 1)
+  %sub.ptr.lhs.cast1717 = ptrtoint i8* %p.1.lcssa3539 to i64
+  %sub.ptr.sub1719 = sub i64 %sub.ptr.lhs.cast1717, %sub.ptr.rhs.cast646
+  %idx.neg1727 = sub i64 0, %sub.ptr.sub1719
+  br label %for.body1723
+
+for.body1606.lr.ph:
+  br label %for.end1609
+
+for.end1609:
+  br i1 undef, label %for.cond1659.preheader, label %land.lhs.true1614
+
+land.lhs.true1614:
+  br label %for.cond1659.preheader
+
+for.cond1659.preheader:
+  %cmp16623414 = icmp ult i8* undef, %add.ptr1603
+  br i1 %cmp16623414, label %for.body1664.lr.ph, label %while.body1703.lr.ph
+
+for.body1664.lr.ph:
+  %cmp16773405 = icmp slt i64 undef, undef
+  br i1 %cmp16773405, label %while.body1679, label %while.cond1683.preheader
+
+while.body1703.lr.ph:
+  unreachable
+
+while.cond1683.preheader:
+  br i1 undef, label %while.body1691, label %while.end1693
+
+while.body1679:
+  %oldc.43406 = phi i32 [ %inc, %syEchoch.exit3070 ], [ %oldc.1.lcssa, %for.body1664.lr.ph ]
+  %4 = load %struct.TMP.2*, %struct.TMP.2** %echo.i3101, align 8, !tbaa !6
+  %call.i3062 = call i32 @fileno(%struct.TMP.2* %4)
+  br i1 undef, label %if.then.i3069, label %syEchoch.exit3070
+
+if.then.i3069:
+  br label %syEchoch.exit3070
+
+syEchoch.exit3070:
+  %inc = add i32 %oldc.43406, 1
+  %conv1672 = sext i32 %inc to i64
+  %cmp1677 = icmp slt i64 %conv1672, undef
+  br i1 %cmp1677, label %while.body1679, label %while.cond1683.preheader
+
+while.body1691:
+  unreachable
+
+while.end1693:
+  unreachable
+
+for.body1723:
+  %q.303203 = phi i8* [ getelementptr inbounds ([8192 x i8], [8192 x i8]* @syHistory, i64 0, i64 8189), %if.then1477 ], [ %incdec.ptr1730, %for.body1723 ]
+  %add.ptr1728 = getelementptr i8, i8* %q.303203, i64 %idx.neg1727
+  %5 = load i8, i8* %add.ptr1728, align 1, !tbaa !5
+  %incdec.ptr1730 = getelementptr i8, i8* %q.303203, i64 -1
+  br label %for.body1723
+
+cleanup:
+  ret i8* undef
+}
+
+declare i32 @fileno(%struct.TMP.2* nocapture)
+declare i64 @"\01_write"(i32, i8*, i64)
+declare i32 @__maskrune(i32, i64)
+declare void @llvm.memset.p0i8.i64(i8* nocapture, i8, i64, i1)
+
+!llvm.ident = !{!0}
+
+!0 = !{!"clang version 3.5.0 (trunk 204257)"}
+!1 = !{!2, !2, i64 0}
+!2 = !{!"int", !3, i64 0}
+!3 = !{!"omnipotent char", !4, i64 0}
+!4 = !{!"Simple C/C++ TBAA"}
+!5 = !{!3, !3, i64 0}
+!6 = !{!7, !8, i64 8}
+!7 = !{!"", !8, i64 0, !8, i64 8, !3, i64 16}
+!8 = !{!"any pointer", !3, i64 0}

llvm/test/CodeGen/MLRegalloc/dev-mode-logging.ll

 ; RUN:   -regalloc-enable-advisor=development -regalloc-training-log=%t2 \
 ; RUN:   -tfutils-text-log -regalloc-model=%t < %S/Inputs/input.ll
 ; RUN: sed -i 's/ \+/ /g' %t2
 ; RUN: sed -i 's/\\n key:/\n key:/g' %t2
 ; RUN: sed -i 's/\\n feature/\n feature/g' %t2
 ; RUN: sed -i 's/\\n/ /g' %t2
 ; RUN: FileCheck --input-file %t2 %s --check-prefixes=CHECK,ML
 
+; RUN: llc -o /dev/null -mtriple=x86_64-linux-unknown -regalloc=greedy \
+; RUN:   -regalloc-enable-advisor=development -regalloc-training-log=%t3.log \
+; RUN:   -tfutils-use-simplelogger < %S/Inputs/two-large-fcts.ll
+; RUN: %python %S/../../../lib/Analysis/models/log_reader.py %t3.log | FileCheck %s --check-prefixes=CHECK-LOG
+
 ; CHECK-NOT: nan
 ; CHECK-LABEL: key: \"index_to_evict\"
 ; ML-NEXT:    value: 9
 ; NOML-NEXT:  value: 12
 ; CHECK-LABEL: key: \"reward\"
 ; ML:   value: 37.06
 ; NOML: value: 36.64
 ; CHECK-NEXT: feature_list
 ; CHECK-NEXT: key: \"start_bb_freq_by_max\"
+
+; CHECK-LOG: context: SyFgetsCopy
+; CHECK-LOG-NEXT: observation: 0
+; CHECK-LOG-NEXT: mask: 0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1
+; CHECK-LOG: index_to_evict: 12
+; CHECK-LOG: observation: 16
+; CHECK-LOG: reward: 36.64
+; CHECK-LOG: context: SyFgets
+; CHECK-LOG-NEXT: observation: 0
+; CHECK-LOG-NEXT: mask: 0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1
+; CHECK-LOG: index_to_evict: 12
+; CHECK-LOG: observation: 16
+; CHECK-LOG: reward: 36.64

llvm/test/Transforms/Inline/ML/ml-test-development-mode.ll

 ; The default inliner doesn't elide @adder, it believes it's too costly to inline 
 ; adder into switcher. The ML inliner carries out that inlining, resulting in
 ; a smaller result (part of it is that adder gets elided).
 ;
 ; This test uses Inputs/test-module.ll, as it shares it with a similar test
 ; for the 'release' mode.
 ;
 ; REQUIRES: have_tf_api
 ; RUN: rm -rf %t
 ; RUN: rm -rf %t_savedmodel
 ; RUN: %python %S/../../../../lib/Analysis/models/gen-inline-oz-test-model.py %t_savedmodel
 ; RUN: %python %S/../../../../lib/Analysis/models/saved-model-to-tflite.py %t_savedmodel %t
 ; RUN: opt -passes=scc-oz-module-inliner -enable-ml-inliner=default -S < %S/Inputs/test-module.ll 2>&1 | FileCheck %S/Inputs/test-module.ll --check-prefix=DEFAULT
 ; RUN: opt -passes=scc-oz-module-inliner -enable-ml-inliner=development -ml-inliner-model-under-training=%t -S < %S/Inputs/test-module.ll 2>&1 | FileCheck %S/Inputs/test-module.ll --check-prefix=CHECK
+; RUN: opt -passes=scc-oz-module-inliner -enable-ml-inliner=development -training-log=%t.log -tfutils-use-simplelogger -S < %S/Inputs/test-module.ll 2>&1
+; RUN: %python %S/../../../../lib/Analysis/models/log_reader.py %t.log | FileCheck %s --check-prefix=CHECK-LOG
+
+CHECK-LOG: observation: 0
+CHECK-LOG-NEXT: {{^sroa_savings:}} 0
+CHECK-LOG: {{^cost_estimate:}} -30
+CHECK-LOG: {{^inlining_decision:}} 1
+CHECK-LOG-NEXT: observation: 1
+CHECK-LOG: observation: 6