This is an archive of the discontinued LLVM Phabricator instance.

[randstruct] Randomize all elements of a record
ClosedPublic

Authored by void on Apr 18 2022, 1:25 PM.

Details

Summary

A record may have more than just FieldDecls in it. If so, then we're
likely to drop them if we only randomize the FieldDecls.

We need to be careful about anonymous structs/unions. Their fields are
made available in the RecordDecl as IndirectFieldDecls, which are listed
after the anonymous struct/union. The ordering doesn't appear to be
super important, however we place them unrandomized at the end of the
RecordDecl just in case. There's also the possiblity of
StaticAssertDecls. We also want those at the end.

All other non-FieldDecls we place at the top, just in case we get
something like:

struct foo {
  enum e { BORK };
  enum e a;
};

Link: https://github.com/KSPP/linux/issues/185

Diff Detail

Event Timeline

void created this revision.Apr 18 2022, 1:25 PM
Herald added a project: Restricted Project. · View Herald TranscriptApr 18 2022, 1:25 PM
Herald added a subscriber: StephenFan. · View Herald Transcript
void requested review of this revision.Apr 18 2022, 1:25 PM
Herald added a project: Restricted Project. · View Herald TranscriptApr 18 2022, 1:25 PM
Herald added a subscriber: cfe-commits. · View Herald Transcript

I think you'll need a more targeted approach than assuming the only kinds of declarations in a struct are field-like in C.

It seems that the issue you've got is with anonymous objects in a structure where the inner fields are available for lookup in the outer structure. One question I have is: what's the expectation for the user? There's two ways to look at this. 1) The anonymous object is a single field; that its members can be found in the outer object is not relevant. 2) The fields of the anonymous object should also be randomized. The same is true for any inner structure, not just anonymous ones.

I had assumed that any structure not marked for randomization would not be randomized. Based on that, I don't think inner structure objects (anonymous or otherwise) should automatically randomize their fields. WDYT?

clang/lib/Sema/SemaDecl.cpp
18063–18064

This (and everything else about fields) is now misnamed.

clang/lib/Sema/SemaInit.cpp
2126–2128

This is now misnamed.

2191

This is no longer testing the correct thing. e.g.,

struct S {
  int f;
  _Static_assert(sizeof(int) == 4, "oh no!");
};

This has two decls but only one field:

TranslationUnitDecl
`-RecordDecl <line:1:1, line:4:1> line:1:8 struct S definition
  |-FieldDecl <line:2:3, col:7> col:7 f 'int'
  `-StaticAssertDecl <line:3:3, col:44> col:3
    |-ImplicitCastExpr <col:18, col:33> '_Bool' <IntegralToBoolean>
    | `-BinaryOperator <col:18, col:33> 'int' '=='
    |   |-UnaryExprOrTypeTraitExpr <col:18, col:28> 'unsigned long' sizeof 'int'
    |   `-ImplicitCastExpr <col:33> 'unsigned long' <IntegralCast>
    |     `-IntegerLiteral <col:33> 'int' 4
    `-StringLiteral <col:36> 'char[7]' lvalue "oh no!"

Note, when you randomize that structure, we drop the _Static_assert declaration, so there is definitely a bug to be fixed here: https://godbolt.org/z/n9YE63rno

kees added a comment.Apr 19 2022, 11:02 AM

I had assumed that any structure not marked for randomization would not be randomized. Based on that, I don't think inner structure objects (anonymous or otherwise) should automatically randomize their fields. WDYT?

Correct, inner structs should be evaluated without looking at the state of the outer randomization.

struct mixed {
    int a;
    float b;
} __attribute__((randomize_layout));

struct ordered {
    int foo;
    char bar[8];
};

struct composite {
    int one;
    struct mixed two;
    struct ordered three;
    struct {
        unsigned long am;
        double ordered;
    };
} __attribute__((randomize_layout));
  • Each of the member's relative positions of struct composite should be randomized: one, two, three, anon struct.
  • The members of two should be randomized.
  • The members of three should not be randomized.
  • The members of the anon struct should not be randomized.
void added a comment.Apr 19 2022, 11:06 PM

I think you'll need a more targeted approach than assuming the only kinds of declarations in a struct are field-like in C.

It seems that the issue you've got is with anonymous objects in a structure where the inner fields are available for lookup in the outer structure. One question I have is: what's the expectation for the user? There's two ways to look at this. 1) The anonymous object is a single field; that its members can be found in the outer object is not relevant. 2) The fields of the anonymous object should also be randomized. The same is true for any inner structure, not just anonymous ones.

I had assumed that any structure not marked for randomization would not be randomized. Based on that, I don't think inner structure objects (anonymous or otherwise) should automatically randomize their fields. WDYT?

We don't randomize inner structures unless they have the randomize_layout flag. That's always been the case, and this patch doesn't change that.

The issue is that we were dropping the inner structures/unions because they aren't FieldDecls, but RecordDecls, with an IndirectFieldDecl if the inner structure is anonymous. The IndirectFieldDecl bits appear after the RecordDecl they're attached to, but doesn't seem like the ordering is important. The IndirectFieldDecl thing is there so that the anonymous fields are available in the outer structure.

I think you'll need a more targeted approach than assuming the only kinds of declarations in a struct are field-like in C.

It seems that the issue you've got is with anonymous objects in a structure where the inner fields are available for lookup in the outer structure. One question I have is: what's the expectation for the user? There's two ways to look at this. 1) The anonymous object is a single field; that its members can be found in the outer object is not relevant. 2) The fields of the anonymous object should also be randomized. The same is true for any inner structure, not just anonymous ones.

I had assumed that any structure not marked for randomization would not be randomized. Based on that, I don't think inner structure objects (anonymous or otherwise) should automatically randomize their fields. WDYT?

We don't randomize inner structures unless they have the randomize_layout flag. That's always been the case, and this patch doesn't change that.

That's good, I misunderstood originally, so thanks!

The issue is that we were dropping the inner structures/unions because they aren't FieldDecls, but RecordDecls, with an IndirectFieldDecl if the inner structure is anonymous. The IndirectFieldDecl bits appear after the RecordDecl they're attached to, but doesn't seem like the ordering is important. The IndirectFieldDecl thing is there so that the anonymous fields are available in the outer structure.

Agreed that we need to fix that behavior, but I think it's pretty fragile to assume every declaration in a struct can be reordered. I mentioned static assert decls above as one obvious example, but also consider things like:

struct S {
  enum E {
    One,
  };

  enum E whatever;
} __attribute__((randomize_layout));

We currently drop the enumeration declaration entirely (oops), but if we rearrange all declaration orders, then the declaration of whatever may suddenly look to come BEFORE we know what the type of enum E will be.

void added a comment.Apr 20 2022, 12:37 PM

I think you'll need a more targeted approach than assuming the only kinds of declarations in a struct are field-like in C.

It seems that the issue you've got is with anonymous objects in a structure where the inner fields are available for lookup in the outer structure. One question I have is: what's the expectation for the user? There's two ways to look at this. 1) The anonymous object is a single field; that its members can be found in the outer object is not relevant. 2) The fields of the anonymous object should also be randomized. The same is true for any inner structure, not just anonymous ones.

I had assumed that any structure not marked for randomization would not be randomized. Based on that, I don't think inner structure objects (anonymous or otherwise) should automatically randomize their fields. WDYT?

We don't randomize inner structures unless they have the randomize_layout flag. That's always been the case, and this patch doesn't change that.

That's good, I misunderstood originally, so thanks!

The issue is that we were dropping the inner structures/unions because they aren't FieldDecls, but RecordDecls, with an IndirectFieldDecl if the inner structure is anonymous. The IndirectFieldDecl bits appear after the RecordDecl they're attached to, but doesn't seem like the ordering is important. The IndirectFieldDecl thing is there so that the anonymous fields are available in the outer structure.

Agreed that we need to fix that behavior, but I think it's pretty fragile to assume every declaration in a struct can be reordered. I mentioned static assert decls above as one obvious example, but also consider things like:

struct S {
  enum E {
    One,
  };

  enum E whatever;
} __attribute__((randomize_layout));

*sigh* Gotta love C.

We currently drop the enumeration declaration entirely (oops), but if we rearrange all declaration orders, then the declaration of whatever may suddenly look to come BEFORE we know what the type of enum E will be.

Maybe the best way we can handle these things is to reorder only FieldDecls and RecordDecls. All of the other Decl types will be smooshed towards the top of the RecordDecl. Again, I'm making an assumption that the ordering of the IndirectFieldDecls and others aren't going to be an issue once the parsing is finished. Alternatively, we could move things like the IndirectFieldDecls and StaticAsserts to the end of the RecordDecl (but before the flexible array decl).

If my assumption about the IndirectFieldDecls is wrong, and the ordering is important, then first, "Wha?! It's such a fragile model!!" Second, I'll have to come up with a way to encapsulate the inner RecordDecl and associated IndirectFieldDecls so that they can be moved around as a single unit.

void updated this revision to Diff 424034.Apr 20 2022, 2:30 PM

Correctly handle all types of Decls that can be found in a RecordDecl.

void edited the summary of this revision. (Show Details)Apr 20 2022, 2:32 PM
void added a comment.Apr 26 2022, 10:15 PM

Gentle ping.

Can you also add a test specifically for an enumeration declaration, as that was something we found bugs with?

clang/lib/AST/Randstruct.cpp
200

CA->isIncompleteArrayType() looks to be at the wrong spot. An incomplete array type is one of type IncompleteArrayType, which a ConstantArrayType is not. Are we missing test coverage, or does the hasFlexibleArrayMember() make it so that we just need to remove this check entirely?

clang/unittests/AST/RandstructTest.cpp
420

Can you add a test where the last field is char name[1]; and another one for char name[]; so that we have full coverage there?

void updated this revision to Diff 425593.Apr 27 2022, 1:06 PM
void marked 2 inline comments as done.

Add a few more tests and remove a dead check.

clang/lib/AST/Randstruct.cpp
200

It looks like if it's an IncompleteArrayType, then it will be captured by hasFlexibleArrayMember. I think we can do without the check here. (It doesn't hurt, because it should always return false.)

clang/unittests/AST/RandstructTest.cpp
420

The name[] is above this test. The name[1] is due to a copy-paste error (oops). Fixed.

void updated this revision to Diff 425595.Apr 27 2022, 1:15 PM

Fix test.

aaron.ballman accepted this revision.Apr 28 2022, 6:17 AM

LGTM!

clang/unittests/AST/RandstructTest.cpp
420

Ah, good call on name[], I missed that one.

This revision is now accepted and ready to land.Apr 28 2022, 6:17 AM
This revision was landed with ongoing or failed builds.Apr 28 2022, 12:01 PM
This revision was automatically updated to reflect the committed changes.