[LNT] Fix profile assigning to tests

[kpdev42]

Summary

report.json may contain the following content:

“Tests”: [
{“Data”: [ 123 ], “Name”: “nts.SomeTest.test:subTest1.exec”},
{“Data”: [ 456 ], “Name”: “nts.SomeTest.test:subTest2.exec”},
{“Data”: [ “base64_encoded_profile_data” ], “Name”: “nts.SomeTest.profile”}
]

This report.json contains 2 tests (execution_time) and 1 profile data which must be assigned to both tests.
Currently LNT creates 3 tests. First 2 tests do not have a profile data. 3rd test does not have any metric, but has the profile data.

This patch implements the correct processing such reports.

OS Laboratory. Huawei Russian Research Institute. Saint-Petersburg

[kpdev42]

Ping

[thopre]

lnt/server/db/testsuitedb.py

1045

If a profile is determined by having the name ends in .profile I'd prefer to check that explicitely (i.e. test_data['name'].endswith('profie'))

1078

And mirror it here, perhaps introduce a "is_profile" lambda.

[slydiman]

lnt/server/db/testsuitedb.py

1045

It will not work because __init__.py (line 593) removes '.profile' from the name during preprocessing.

[thopre]

lnt/server/db/testsuitedb.py

1045

One more reason to rewrite this test.

[kpdev42]

Address review commentaries. Thank you

[slydiman]

lnt/server/db/testsuitedb.py

1045

What test do you mean? This code works just fine.

You suggested

If a profile is determined by having the name ends in .profile I'd prefer to check that explicitely (i.e. test_data['name'].endswith('profie'))

But it will not work because lnt/testing/__init__.py (line 593) converts JSON data v1 to v2:

{“Data”: [ “base64_encoded_profile_data” ], “Name”: “nts.SomeTest.profile”}

to

{“profile”: [ “base64_encoded_profile_data” ], “name”: “nts.SomeTest”}

[thopre]

lnt/server/db/testsuitedb.py

1045

Ah my bad, hadn't realized it gets transformed to a dictionary with a "profile" key. I thought it was looking for "profile" in one of the value. LGTM then.

lnt/server/db/testsuitedb.py

class TestSuiteDB(object):
 
     def _importSampleValues(self, session, tests_data, run, config):
         # Load a map of all the tests, which we will extend when we find tests
         # that need to be added.
         # Downcast to str, so we match on MySQL.
         test_cache = dict((test.name, test)
                           for test in session.query(self.Test))
 
-        profiles = dict()
         field_dict = dict([(f.name, f) for f in self.sample_fields])
         all_samples_to_add = []
+        is_profile_only = lambda td : len(td) == 2 and 'profile' in td
         for test_data in tests_data:
+            if is_profile_only(test_data):
+                # Ignore for now profile data without other metrics
+                continue
+
             name = test_data['name']
             test = test_cache.get(name)
             if test is None:
                 test = self.Test(test_data['name'])
                 test_cache[name] = test
                 session.add(test)
 
             samples = []
             for key, values in test_data.items():
                 if key == 'name' or key == "id" or key.endswith("_id"):
                     continue
                 field = field_dict.get(key)
                 if field is None and key != 'profile':
                     raise ValueError("test %s: Metric '%s' unknown in suite " %
                                      (name, key))
 
                 if not isinstance(values, list):
                     values = [values]
                 while len(samples) < len(values):
                     sample = self.Sample(run, test)
                     samples.append(sample)
                     all_samples_to_add.append(sample)
                 for sample, value in zip(samples, values):
                     if key == 'profile':
-                        profile = self.Profile(value, config, name)
-                        sample.profile = profiles.get(hash(value), profile)
+                        sample.profile = self.Profile(value, config, name)
                     else:
                         sample.set_field(field, value)
+
+        for test_data in tests_data:
+            if not is_profile_only(test_data):
+                continue
+            name = test_data['name']
+            test = test_cache.get(name)
+            tests = [test_cache[test_name] for test_name in test_cache \
+                                           if test_name.startswith(name + '.test:')]
+            if test is not None:
+                tests.append(test)
+
+            value = test_data['profile']
+            new_profile = self.Profile(value, config, name)
+            count = 0
+            for test in tests:
+                sample_exist = False
+                for sample in all_samples_to_add:
+                    if sample.test == test:
+                        if sample.profile is None:
+                            sample.profile = new_profile
+                            count += 1
+                            sample_exist = True
+                        else:
+                            logger.warning('Test %s already contains the profile data. ' \
+                                'Profile %s was ignored.', test.name, name)
+                if not sample_exist:
+                    logger.warning('The test %s is invalid. It contains the profile, ' \
+                                   'but no any samples. Consider removing it.', test.name)
+            if count == 0:
+                logger.warning('Cannot find test(s) for the profile %s', name)
+            else:
+                logger.info('The profile %s was added to %d test(s).', name, count)
+
         session.add_all(all_samples_to_add)
 
     def importDataFromDict(self, session, data, config, select_machine,
                            merge_run):
         """
         importDataFromDict(session, data, config, select_machine, merge_run)
             -> Run  (or throws ValueError exception)
 

lnt/testing/__init__.py

metric_rename={
         '.compile': 'compile_time',
         '.compile.status': 'compile_status',
         '.exec': 'execution_time',
         '.exec.status': 'execution_status',
         '.hash': 'hash',
         '.hash.status': 'hash_status',
         '.mem': 'mem_bytes',
         '.score': 'score',
+        '.profile': 'profile',
     }, machine_param_rename={
         'name': 'hostname',  # Avoid name clash with actual machine name.
     }, run_param_rename={
         'run_order': 'llvm_project_revision',
     }
 )
 _compile_upgrade = _UpgradeSchema(
     metric_rename={