This is an archive of the discontinued LLVM Phabricator instance.

[LNT] Fix profile assigning to tests
ClosedPublic

Authored by kpdev42 on Dec 13 2021, 10:55 PM.

Details

Summary

report.json may contain the following content:

“Tests”: [
{“Data”: [ 123 ], “Name”: “nts.SomeTest.test:subTest1.exec”},
{“Data”: [ 456 ], “Name”: “nts.SomeTest.test:subTest2.exec”},
{“Data”: [ “base64_encoded_profile_data” ], “Name”: “nts.SomeTest.profile”}
]

This report.json contains 2 tests (execution_time) and 1 profile data which must be assigned to both tests.
Currently LNT creates 3 tests. First 2 tests do not have a profile data. 3rd test does not have any metric, but has the profile data.

This patch implements the correct processing such reports.

OS Laboratory. Huawei Russian Research Institute. Saint-Petersburg

Diff Detail

Repository
rLNT LNT

Event Timeline

kpdev42 created this revision.Dec 13 2021, 10:55 PM
kpdev42 requested review of this revision.Dec 13 2021, 10:55 PM
thopre added inline comments.Jan 17 2022, 2:29 AM
lnt/server/db/testsuitedb.py
1045

If a profile is determined by having the name ends in .profile I'd prefer to check that explicitely (i.e. test_data['name'].endswith('profie'))

1078

And mirror it here, perhaps introduce a "is_profile" lambda.

slydiman added inline comments.Jan 18 2022, 7:52 AM
lnt/server/db/testsuitedb.py
1045

It will not work because __init__.py (line 593) removes '.profile' from the name during preprocessing.

thopre added inline comments.Jan 18 2022, 7:59 AM
lnt/server/db/testsuitedb.py
1045

One more reason to rewrite this test.

kpdev42 updated this revision to Diff 401108.Jan 18 2022, 11:18 PM

Address review commentaries. Thank you

kpdev42 marked 4 inline comments as done.Jan 18 2022, 11:19 PM
slydiman added inline comments.Jan 19 2022, 1:00 AM
lnt/server/db/testsuitedb.py
1045

What test do you mean? This code works just fine.

You suggested

If a profile is determined by having the name ends in .profile I'd prefer to check that explicitely (i.e. test_data['name'].endswith('profie'))

But it will not work because lnt/testing/__init__.py (line 593) converts JSON data v1 to v2:

{“Data”: [ “base64_encoded_profile_data” ], “Name”: “nts.SomeTest.profile”}

to

{“profile”: [ “base64_encoded_profile_data” ], “name”: “nts.SomeTest”}
thopre accepted this revision.Jan 19 2022, 1:51 AM
thopre added inline comments.
lnt/server/db/testsuitedb.py
1045

Ah my bad, hadn't realized it gets transformed to a dictionary with a "profile" key. I thought it was looking for "profile" in one of the value. LGTM then.

This revision is now accepted and ready to land.Jan 19 2022, 1:51 AM
This revision was automatically updated to reflect the committed changes.