[LoopStrengthReduce] Don't use a DenseSet<int64_t> when we might add any valid int64_t to the set.

[jlebar]

Summary

SmallSetVector uses DenseSet, but that means we need to reserve some
values for the empty and tombstone keys.

It seems to me we should have a general way to let us store full-range
ints inside of DenseSets, and furthermore that we probably shouldn't
silently let you add ints into DenseSets without explicitly promising
that they're in range. But that's a battle for another day; for now,
just fix this code, since we currently do something Very Bad when
compiling ffmpeg.

Fixes PR30914.

[hfinkel]

LGTM

We should probably also have some template specialization in the DenseMap implementation that causes us to have an assert that will catch this situation more-directly (i.e. with an easy-to-understand assertion message).

llvm/trunk/lib/Transforms/Scalar/LoopStrengthReduce.cpp

class LSRInstance {
 
   /// This is the insert position that the current loop's induction variable
   /// increment should be placed. In simple loops, this is the latch block's
   /// terminator. But in more complicated cases, this is a position which will
   /// dominate all the in-loop post-increment users.
   Instruction *IVIncInsertPos;
 
   /// Interesting factors between use strides.
-  SmallSetVector<int64_t, 8> Factors;
+  ///
+  /// We explicitly use a SetVector which contains a SmallSet, instead of the
+  /// default, a SmallDenseSet, because we need to use the full range of
+  /// int64_ts, and there's currently no good way of doing that with
+  /// SmallDenseSet.
+  SetVector<int64_t, SmallVector<int64_t, 8>, SmallSet<int64_t, 8>> Factors;
 
   /// Interesting use types, to facilitate truncation reuse.
   SmallSetVector<Type *, 4> Types;
 
   /// The list of interesting uses.
   SmallVector<LSRUse, 16> Uses;
 
   /// Track which uses use which register candidates.

llvm/trunk/test/CodeGen/X86/loop-strength-reduce-crash.ll

+; RUN: llc < %s -o /dev/null
+
+; Check that this doesn't crash (by virtue of using INT64_MAX as a constant in
+; the loop).
+
+target datalayout = "e-m:o-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-macosx10.12.0"
+
+define void @foo() {
+entry:
+  br label %for
+
+for: 
+  %0 = phi i64 [ %add, %for ], [ undef, %entry ]
+  %next = phi i32 [ %inc, %for ], [ undef, %entry ]
+  store i32 %next, i32* undef, align 4
+  %add = add i64 %0, 9223372036854775807
+  %inc = add nsw i32 %next, 1
+  br i1 undef, label %exit, label %for
+
+exit:
+  store i64 %add, i64* undef
+  ret void
+}