This is an archive of the discontinued LLVM Phabricator instance.

Differential D158414

[LexerTest] Use LexTokensUntilEOF() in StringifyArgs
Needs ReviewPublic

Authored by Hahnfeld on Aug 21 2023, 5:08 AM.

Details

Reviewers
erichkeane
Summary

Rely on Preprocessor::LexTokensUntilEOF() and just replace tok::comma by tok::eof and push a tok::eof as the last element.

Diff Detail

Event Timeline

Hahnfeld created this revision.Aug 21 2023, 5:08 AM
Herald added a project: Restricted Project. · View Herald TranscriptAug 21 2023, 5:09 AM
Hahnfeld requested review of this revision.Aug 21 2023, 5:09 AM
Herald added a project: Restricted Project. · View Herald TranscriptAug 21 2023, 5:09 AM
Herald added a subscriber: cfe-commits. · View Herald Transcript
Hahnfeld added a parent revision: D158413: [Lex] Introduce Preprocessor::LexTokensUntilEOF().Aug 21 2023, 5:09 AM
Harbormaster completed remote builds in B253825: Diff 551991.Aug 21 2023, 5:58 AM
Hahnfeld added a comment.Sep 4 2023, 2:57 AM

ping :)

Hahnfeld added a comment.Sep 14 2023, 1:53 AM

ping!

Hahnfeld updated this revision to Diff 556848.Sep 15 2023, 5:33 AM
Hahnfeld retitled this revision from [LexerTest] Use LexAll() in StringifyArgs to [LexerTest] Use LexTokensUntilEOF() in StringifyArgs.
Hahnfeld edited the summary of this revision. (Show Details)
Harbormaster completed remote builds in B257272: Diff 556848.Sep 15 2023, 6:34 AM
Hahnfeld added a comment.Oct 3 2023, 12:45 AM

ping

Hahnfeld added a comment.Oct 16 2023, 11:57 PM

ping!

Revision Contents

PathSize
clang/
unittests/
Lex/
17 lines

Diff 556848

clang/unittests/Lex/LexerTest.cpp

Show First 20 Lines • Show All 445 Lines • ▼ Show 20 LinesTEST_F(LexerTest, DontOverallocateStringifyArgs) {
MI->setIsFunctionLike(); MI->setIsFunctionLike();
MI->setParameterList(ParamList, Allocator); MI->setParameterList(ParamList, Allocator);
EXPECT_EQ(3u, MI->getNumParams()); EXPECT_EQ(3u, MI->getNumParams());
EXPECT_TRUE(MI->isFunctionLike()); EXPECT_TRUE(MI->isFunctionLike());
Token Eof; Token Eof;
Eof.setKind(tok::eof); Eof.setKind(tok::eof);
std::vector<Token> ArgTokens; std::vector<Token> ArgTokens;
while (1) { PP->LexTokensUntilEOF(&ArgTokens);
Token tok; // Replace all tok::comma with tok::eof for stringification.
PP->Lex(tok); for (auto &tok : ArgTokens) {
if (tok.is(tok::eof)) { if (tok.is(tok::comma)) {
ArgTokens.push_back(Eof); tok = Eof;
break;
} }
if (tok.is(tok::comma))
ArgTokens.push_back(Eof);
else
ArgTokens.push_back(tok);
} }
// Push a tok::eof as the last element.
ArgTokens.push_back(Eof);
auto MacroArgsDeleter = [&PP](MacroArgs *M) { M->destroy(*PP); }; auto MacroArgsDeleter = [&PP](MacroArgs *M) { M->destroy(*PP); };
std::unique_ptr<MacroArgs, decltype(MacroArgsDeleter)> MA( std::unique_ptr<MacroArgs, decltype(MacroArgsDeleter)> MA(
MacroArgs::create(MI, ArgTokens, false, *PP), MacroArgsDeleter); MacroArgs::create(MI, ArgTokens, false, *PP), MacroArgsDeleter);
auto StringifyArg = [&](int ArgNo) { auto StringifyArg = [&](int ArgNo) {
return MA->StringifyArgument(MA->getUnexpArgument(ArgNo), *PP, return MA->StringifyArgument(MA->getUnexpArgument(ArgNo), *PP,
/*Charify=*/false, {}, {}); /*Charify=*/false, {}, {});
}; };
▲ Show 20 Lines • Show All 213 Lines • Show Last 20 Lines