Rework machine creation strategy

[MatzeB]

Summary

Currently when submitting a run and the machine data does not match the
previous data, LNT creates a new machine with the same name (but
different id). This was often very confusing to users.

This changes the strategy to reject the submission if the data does not
match the previous data unless the --update-machine flag (or the
update_machine post parameters, etc.) is set in which case the new data
overrides the previous machine data.

Adding previously unset keys to the machine will not lead to a rejection
either way. Leaving out previously set keys is fine (but will not remove
the keys).

This new strategy will result in machine names being unique (except for
the case of older entries in the database before this change).

[MatzeB]

ping

[grosser]

Hi Matze,

I did not check the implementation in detail, but this makes total sense to me. From my perspective this is a clear improvement and should go in.

[jgreenhalgh]

In D35598#819677, @grosser wrote:

Hi Matze,

I did not check the implementation in detail, but this makes total sense to me. From my perspective this is a clear improvement and should go in.

A more useful flag for our use would restore the previous behavior, rather than always update the machine. We have a lot of historical data, crossing a number of kernel versions and other machine characteristics to import to LNT. The old behavior was very convenient for this data set - we automatically ended up with new machines after each system configuration change (actually, this automatic disambiguation of machines with variations was a key benefit of LNT for us, and caught a number of bugs in our test infrastructure). We don't want to always update the machine, that would "poison" the quality of the historical data.

Inventing unique names for each of the variants would be difficult but possible. It feels like it somewhat defeats the point of the machine field - I would be re-encoding the same information in to a machine name (for example something like gcc7-cortex-a57-ubuntu-14.04-linux-4.13-64k-pages ). That makes all other interactions with the system (e.g. choosing runs for comparison) very cumbersome.

I agree that the old behavior could be confusing, but I don't really know how to sensibly interact with the new design in a way that preserves data quality without needing an explosion in naming complexity. For me, this is not a clear improvement.

[MatzeB]

Hi James,

this is very interesting to hear as I would not have expected the previous behavior to be desirable. Just to explain some more where I am coming from:

• LNT Submissions are typically performed by CI jobs which for us are required to have a unique name, so it is natural to use the same LNT machine name as the CI jobs name.
• When selecting a machine in LNT the only thing to go on is the machine name. If for example I have 7 different machines named "gcc7" (with some of the other fields differing), I would need to click 7 times today, to figure out which is the machine that I want.
• Similarily when connecting 3rd party visualisation/analysis to LNT it is convenient to have unique machine names that you can reference. Machine id numbers are only valid within one LNT database, and are also not predictable.
• Looking at lnt runtest test-suite mode, nobody even bothered filling out the machine fields and to my knowledge nobody complained to this day.

To your points:

In D35598#838818, @jgreenhalgh wrote:

In D35598#819677, @grosser wrote:

Hi Matze,

I did not check the implementation in detail, but this makes total sense to me. From my perspective this is a clear improvement and should go in.

A more useful flag for our use would restore the previous behavior, rather than always update the machine. We have a lot of historical data, crossing a number of kernel versions and other machine characteristics to import to LNT. The old behavior was very convenient for this data set - we automatically ended up with new machines after each system configuration change (actually, this automatic disambiguation of machines with variations was a key benefit of LNT for us, and caught a number of bugs in our test infrastructure). We don't want to always update the machine, that would "poison" the quality of the historical data.

• Note that submission are rejected if the machine data doesn't match the previous data, so bugs are catched and incompatible/uncomparable data is avoided. (the --update-machine flag is not intended for the regular CI job, but rather to be used manually after updateing a machine in a way that changes the data but is believed to not change performance/keep the data comparable).
• If after changing a machine the new data is not comparable to the historical data I would expect the user to choose a new machine name (which is also nice as it makes the fact of the changed configuration more obvious).
• I also created the lnt admin subcommands to enable ways to rename, merge, delete machines to allow cleanup/reorganisation of the data.

Inventing unique names for each of the variants would be difficult but possible. It feels like it somewhat defeats the point of the machine field - I would be re-encoding the same information in to a machine name (for example something like gcc7-cortex-a57-ubuntu-14.04-linux-4.13-64k-pages ). That makes all other interactions with the system (e.g. choosing runs for comparison) very cumbersome.

I agree that the old behavior could be confusing, but I don't really know how to sensibly interact with the new design in a way that preserves data quality without needing an explosion in naming complexity. For me, this is not a clear improvement.

So I am not completely convinced the automatic machine name creation is a desirable behavior. I can see the convenience of machines getting created automatically at the cost of the machine names becoming less meaningful.

Having said all that I'd be fine to add a flag supporting a variation of the previous behavior where we create new machines if the machine data doesn't match (however I'd slightly change the behavior to append a number to the new machines name to maintain the property that machine names are unique). Would that be fine with you?

[jgreenhalgh]

In D35598#839397, @MatzeB wrote:

Hi James,

Hi,

So I am not completely convinced the automatic machine name creation is a desirable behavior. I can see the convenience of machines getting created automatically at the cost of the machine names becoming less meaningful.

Having said all that I'd be fine to add a flag supporting a variation of the previous behavior where we create new machines if the machine data doesn't match (however I'd slightly change the behavior to append a number to the new machines name to maintain the property that machine names are unique). Would that be fine with you?

Before I go in to more detail about how we're testing (for background, and your interest) - that sounds like a very helpful solution, thanks!

this is very interesting to hear as I would not have expected the previous behavior to be desirable. Just to explain some more where I am coming from:

• LNT Submissions are typically performed by CI jobs which for us are required to have a unique name, so it is natural to use the same LNT machine name as the CI jobs name.

This is also true for us, however we are building nightly using 30 groupings of "machines" (which are themselves pools of real machines driven by buildbot), and with historical data going back 4 years. Each of these groupings of machines use names automatically derived from the key aspects of their hardware and release branch they track, and we're diligent at recording more subtle machine differences in the "machine" field.

• When selecting a machine in LNT the only thing to go on is the machine name. If for example I have 7 different machines named "gcc7" (with some of the other fields differing), I would need to click 7 times today, to figure out which is the machine that I want.

That is probably where the difference in perspectives comes from. We would have 7 machines producing results, which we expect to have identical configuration, and which we would group under the name "gcc7.$target_board.$target_cpu". In normal use, we would not expect 7 machines named "gcc7" to produce a result in one night, we would expect there to be one "active" gcc7 at a time (measured in months), and so choosing the right machine would be a matter of picking the one which has been building most recently. Occasionally due to sysadmin/user error, one of the machines in the pool might malfunction and end up in an inappropriate configuration. As an example from today, one rogue machine in the pool was accidentally patched up to a newer kernel version. When we import a run from that machine, the old LNT behaviour would create a separate machine, ensuring that the data integrity was maintained with the new machine isolated (which it wouldn't be if we forced --update-machine) but that we still had data in the system that we could compare (which we wouldn't get with the new error behaviour). This becomes important to us when importing historical data, as we really do want a new machine every time configuration changes, but we don't want to have to encode that in the machine name. Put another way, when we migrate a grouping of boards to a more recent kernel version, we want that change to make the data sets disjoint, but without us having to invent a new name for the machine pool.

Automatically appending a number to the machine name would therefore work well for our use case.

• Similarily when connecting 3rd party visualisation/analysis to LNT it is convenient to have unique machine names that you can reference. Machine id numbers are only valid within one LNT database, and are also not predictable.

We don't do this, but I can see why this would be useful to you.

• Looking at lnt runtest test-suite mode, nobody even bothered filling out the machine fields and to my knowledge nobody complained to this day.

We're a somewhat unique consumer of LNT, in that we have a completely separate infrastructure for running and recording test results, we generate JSON from this infrastructure which is suitable for import to LNT for visualisation. We make heavy use of the machine fields.

I'm very grateful for your help in resolving this, I appreciate we're running a non-standard configuration over here!

James

[MatzeB]

In D35598#840451, @jgreenhalgh wrote:

In D35598#839397, @MatzeB wrote:

Hi James,

Hi,

So I am not completely convinced the automatic machine name creation is a desirable behavior. I can see the convenience of machines getting created automatically at the cost of the machine names becoming less meaningful.

Having said all that I'd be fine to add a flag supporting a variation of the previous behavior where we create new machines if the machine data doesn't match (however I'd slightly change the behavior to append a number to the new machines name to maintain the property that machine names are unique). Would that be fine with you?

Before I go in to more detail about how we're testing (for background, and your interest) - that sounds like a very helpful solution, thanks!

this is very interesting to hear as I would not have expected the previous behavior to be desirable. Just to explain some more where I am coming from:

• LNT Submissions are typically performed by CI jobs which for us are required to have a unique name, so it is natural to use the same LNT machine name as the CI jobs name.

This is also true for us, however we are building nightly using 30 groupings of "machines" (which are themselves pools of real machines driven by buildbot), and with historical data going back 4 years. Each of these groupings of machines use names automatically derived from the key aspects of their hardware and release branch they track, and we're diligent at recording more subtle machine differences in the "machine" field.

• When selecting a machine in LNT the only thing to go on is the machine name. If for example I have 7 different machines named "gcc7" (with some of the other fields differing), I would need to click 7 times today, to figure out which is the machine that I want.

That is probably where the difference in perspectives comes from. We would have 7 machines producing results, which we expect to have identical configuration, and which we would group under the name "gcc7.$target_board.$target_cpu". In normal use, we would not expect 7 machines named "gcc7" to produce a result in one night, we would expect there to be one "active" gcc7 at a time (measured in months), and so choosing the right machine would be a matter of picking the one which has been building most recently. Occasionally due to sysadmin/user error, one of the machines in the pool might malfunction and end up in an inappropriate configuration. As an example from today, one rogue machine in the pool was accidentally patched up to a newer kernel version. When we import a run from that machine, the old LNT behaviour would create a separate machine, ensuring that the data integrity was maintained with the new machine isolated (which it wouldn't be if we forced --update-machine) but that we still had data in the system that we could compare (which we wouldn't get with the new error behaviour). This becomes important to us when importing historical data, as we really do want a new machine every time configuration changes, but we don't want to have to encode that in the machine name. Put another way, when we migrate a grouping of boards to a more recent kernel version, we want that change to make the data sets disjoint, but without us having to invent a new name for the machine pool.

Automatically appending a number to the machine name would therefore work well for our use case.

• Similarily when connecting 3rd party visualisation/analysis to LNT it is convenient to have unique machine names that you can reference. Machine id numbers are only valid within one LNT database, and are also not predictable.

We don't do this, but I can see why this would be useful to you.

• Looking at lnt runtest test-suite mode, nobody even bothered filling out the machine fields and to my knowledge nobody complained to this day.

We're a somewhat unique consumer of LNT, in that we have a completely separate infrastructure for running and recording test results, we generate JSON from this infrastructure which is suitable for import to LNT for visualisation. We make heavy use of the machine fields.

I'm very grateful for your help in resolving this, I appreciate we're running a non-standard configuration over here!

James

FYI: I'm still working on a fix for this. But I am currently occupied with other things, I hopefully have something next week.

lnt/trunk/lnt/lnttool/import_data.py

 @click.option("--show-sample-count", is_flag=True)
 @click.option("--show-raw-result", is_flag=True)
 @click.option("--testsuite", "-s", default='nts')
 @click.option("--verbose", "-v", is_flag=True,
               help="show verbose test results")
 @click.option("--quiet", "-q", is_flag=True, help="don't show test results")
 @click.option("--no-email", is_flag=True, help="don't send e-mail")
 @click.option("--no-report", is_flag=True, help="don't generate report")
[e-mail]("--update-machine", is_flag=True, help="Update machine fields")
 def action_import(instance_path, files, database, output_format, commit,
                   show_sql, show_sample_count, show_raw_result, testsuite,
-                  verbose, quiet, no_email, no_report):
+                  verbose, quiet, no_email, no_report, update_machine):
     """import test data into a database"""
     import contextlib
     import lnt.server.instance
     import lnt.util.ImportData
     import pprint
     import sys
 
     # Load the LNT instance.
     instance = lnt.server.instance.Instance.frompath(instance_path)
     config = instance.config
 
     # Get the database.
     with contextlib.closing(config.get_database(database,
                                                 echo=show_sql)) as db:
         # Load the database.
         success = True
         for file_name in files:
             result = lnt.util.ImportData.import_and_report(
                 config, database, db, file_name,
                 output_format, testsuite, commit, show_sample_count,
-                no_email, no_report)
+                no_email, no_report, updateMachine=update_machine)
 
             success &= result.get('success', False)
             if quiet:
                 continue
 
             if show_raw_result:
                 pprint.pprint(result)
             else:
                 lnt.util.ImportData.print_report_result(result, sys.stdout,
                                                         sys.stderr,
                                                         verbose)
 
         if not success:
             raise SystemExit(1)

lnt/trunk/lnt/lnttool/main.py

for name in test_names:
         description = inspect.cleandoc(test_module.__doc__)
         print '  %-*s - %s' % (max_name, name, description)
 
 
 @click.command("submit")
 @click.argument("url")
 @click.argument("files", nargs=-1, type=click.Path(exists=True), required=True)
 @click.option("--commit", is_flag=True, help="actually commit the data")
[e-mail]("--update-machine", is_flag=True, help="Update machine fields")
 @click.option("--verbose", "-v", is_flag=True,
               help="show verbose test results")
-def action_submit(url, files, commit, verbose):
+def action_submit(url, files, commit, update_machine, verbose):
     """submit a test report to the server"""
     from lnt.util import ServerUtil
     import lnt.util.ImportData
 
     if commit:
         commit = True
     else:
         commit = False
         logger.warning("submit called without --commit, " +
                        "your results will not be saved at the server.")
 
-    files = ServerUtil.submitFiles(url, files, commit, verbose)
+    files = ServerUtil.submitFiles(url, files, commit, verbose,
+                                   updateMachine=update_machine)
     for submitted_file in files:
         if verbose:
             lnt.util.ImportData.print_report_result(
                 submitted_file, sys.stdout, sys.stderr, True)
         _print_result_url(submitted_file, verbose)
 
 
 @click.command("update")

lnt/trunk/lnt/lnttool/viewcomparison.py

try:
         instance = lnt.server.instance.Instance(None, config)
 
         # Create the database.
         lnt.server.db.migrate.update_path(db_path)
 
         # Import the two reports.
         with contextlib.closing(config.get_database('default')) as db:
             r = import_and_report(config, 'default', db, report_a, '<auto>',
-                                  testsuite, commit=True)
+                                  testsuite, commit=True, updateMachine=True)
             import_and_report(config, 'default', db, report_b, '<auto>',
-                              testsuite, commit=True)
+                              testsuite, commit=True, updateMachine=True)
 
             # Dispatch another thread to start the webbrowser.
             comparison_url = '%s/v4/nts/2?compare_to=1' % (url,)
             logger.info("opening comparison view: %s" % (comparison_url,))
 
             if not dry_run:
                 thread.start_new_thread(_start_browser, (comparison_url, True))
 

lnt/trunk/lnt/server/db/testsuitedb.py

def get_users_baseline(self):
             # Sometimes this is called from outside the app context.
             # In that case, don't get the user's session baseline.
             return None
         if session_baseline:
             return self.query(self.Baseline).get(session_baseline)
 
         return None
 
-    def _getOrCreateMachine(self, machine_data):
+    def _getOrCreateMachine(self, machine_data, forceUpdate):
         """
-        _getOrCreateMachine(data) -> Machine, bool
+        _getOrCreateMachine(data, forceUpdate) -> Machine
 
         Add or create (and insert) a Machine record from the given machine data
         (as recorded by the test interchange format).
-
-        The boolean result indicates whether the returned record was
-        constructed or not.
         """
 
-        # Convert the machine data into a machine record. We construct the
-        # query to look for any existing machine at the same time as we build
-        # up the record to possibly add.
-        name = machine_data['name']
-        query = self.query(self.Machine).filter(self.Machine.name == name)
-        machine = self.Machine(name)
+        # Convert the machine data into a machine record.
         machine_parameters = machine_data.copy()
-        machine_parameters.pop('name')
-        # Ignore incoming ids; we will create our own.
-        # TODO: Add some API/result so we can send a warning back to the user
-        # that we ignore the id.
+        name = machine_parameters.pop('name')
+        machine = self.Machine(name)
         machine_parameters.pop('id', None)
-
-        # First, extract all of the specified machine fields.
         for item in self.machine_fields:
             value = machine_parameters.pop(item.name, None)
-            query = query.filter(item.column == value)
             machine.set_field(item, value)
-
-        # Convert any remaining machine_parameters into a JSON encoded blob. We
-        # encode this as an array to avoid a potential ambiguity on the key
-        # ordering.
         machine.parameters = machine_parameters
-        query = query.filter(self.Machine.parameters_data ==
-                             machine.parameters_data)
 
-        # Execute the query to see if we already have this machine.
-        existing_machine = query.first()
-        if existing_machine is not None:
-            return existing_machine, False
-        else:
+        # Look for an existing machine.
+        existing_machines = self.query(self.Machine) \
+            .filter(self.Machine.name == name) \
+            .order_by(self.Machine.id.desc()) \
+            .all()
+        if len(existing_machines) == 0:
             self.add(machine)
-            return machine, True
+            return machine
+
+        existing = existing_machines[0]
+
+        # Unfortunately previous LNT versions allowed multiple machines
+        # with the same name to exist, so we should choose the one that
+        # matches best.
+        if len(existing_machines) > 1:
+            for m in existing_machines:
+                if m.parameters == machine.parameters:
+                    existing = m
+                    break
+
+        # Check and potentially update existing machine.
+        # Parameters that were previously unset are added. If a parameter
+        # changed then we update or abort depending on `forceUpdate`.
+        for field in self.machine_fields:
+            existing_value = existing.get_field(field)
+            new_value = machine.get_field(field)
+            if existing_value is None:
+                existing.set_field(field, new_value)
+            elif existing_value != new_value:
+                if not forceUpdate:
+                    raise ValueError("'%s' on machine '%s' changed." %
+                                     (field.name, name))
+                else:
+                    existing.set_field(field, new_value)
+        existing_parameters = existing.parameters
+        for key, value in machine.parameters.items():
+            existing_value = existing_parameters.get(key, None)
+            if existing_value is None:
+                existing_parameters[key] = value
+            elif existing_value != value:
+                if not forceUpdate:
+                    raise ValueError("'%s' on machine '%s' changed." %
+                                     (key, name))
+                else:
+                    existing_parameters[key] = value
+        existing.parameters = existing_parameters
+        return existing
 
     def _getOrCreateOrder(self, run_parameters):
         """
         _getOrCreateOrder(data) -> Order, bool
 
         Add or create (and insert) an Order record based on the given run
         parameters (as recorded by the test interchange format).
 
def _importSampleValues(self, tests_data, run, commit, config):
                     samples.append(sample)
                 for sample, value in zip(samples, values):
                     if key == 'profile':
                         profile = self.Profile(value, config, name)
                         sample.profile = profiles.get(hash(value), profile)
                     else:
                         sample.set_field(field, value)
 
-    def importDataFromDict(self, data, commit, config=None):
+    def importDataFromDict(self, data, commit, config=None,
+                           updateMachine=False):
         """
         importDataFromDict(data) -> bool, Run
 
         Import a new run from the provided test interchange data, and return
         the constructed Run record.
 
         The boolean result indicates whether the returned record was
         constructed or not (i.e., whether the data was a duplicate submission).
         """
-
-        # Construct the machine entry.
-        machine, inserted = self._getOrCreateMachine(data['machine'])
+        machine = self._getOrCreateMachine(data['machine'], updateMachine)
 
         # Construct the run entry.
         run, inserted = self._getOrCreateRun(data['run'], machine)
 
         # If we didn't construct a new run, this is a duplicate
         # submission. Return the prior Run.
         if not inserted:
             return False, run

lnt/trunk/lnt/server/ui/api.py

class Runs(Resource):
     method_decorators = [in_db]
 
     @staticmethod
     @requires_auth_token
     def post():
         """Add a new run into the lnt database"""
         db = request.get_db()
         data = request.data
+        updateMachine = request.values.get('update_machine', False)
         result = lnt.util.ImportData.import_from_string(
-            current_app.old_config, g.db_name, db, g.testsuite_name, data)
+            current_app.old_config, g.db_name, db, g.testsuite_name, data,
+            updateMachine=updateMachine)
 
         new_url = ('%sapi/db_%s/v4/%s/runs/%s' %
                    (request.url_root, g.db_name, g.testsuite_name,
                     result['run_id']))
         result['result_url'] = new_url
         response = jsonify(result)
         response.status = '301'
         response.headers.add('Location', new_url)

lnt/trunk/lnt/server/ui/templates/submit_run.html

 <textarea name="input_data"></textarea>
 
 <p><b>Commit*:</b><br/>
 <select name="commit">
 <option selected="selected" value="0">0</option>
 <option value="1">1</option>
 </select><br/>
 
+<p><b>Update Machine:</b><br/>
+<select name="update_machine">
+<option selected="selected" value="0">0</option>
+<option value="1">1</option>
+</select><br/>
+
 <p><input type="submit" name="submit" value="Submit">
 </form>
 
 {% endblock %}

lnt/trunk/lnt/server/ui/views.py

 def _do_submit():
     if request.method == 'GET':
         return render_template("submit_run.html")
 
     assert request.method == 'POST'
     input_file = request.files.get('file')
     input_data = request.form.get('input_data')
     commit = int(request.form.get('commit', 0)) != 0
+    updateMachine = int(request.form.get('update_machine', 0)) != 0
 
     if input_file and not input_file.content_length:
         input_file = None
 
     if not input_file and not input_data:
         return render_template(
             "submit_run.html", error="must provide input file or data")
     if input_file and input_data:
def _do_submit():
     if g.testsuite_name is None:
         g.testsuite_name = 'nts'
 
     # Get a DB connection.
     db = request.get_db()
 
     result = lnt.util.ImportData.import_from_string(
         current_app.old_config, g.db_name, db, g.testsuite_name, data_value,
-        commit=commit)
+        commit=commit, updateMachine=updateMachine)
 
     # It is nice to have a full URL to the run, so fixup the request URL
     # here were we know more about the flask instance.
     if result.get('result_url'):
         result['result_url'] = request.url_root + result['result_url']
 
     response = flask.jsonify(**result)
     if result['error'] is not None:

lnt/trunk/lnt/util/ImportData.py

 from lnt.util import NTEmailReport
 from lnt.util import async_ops
 from lnt.util import logger
 import collections
 import datetime
 import lnt.formats
 import lnt.server.reporting.analysis
 import lnt.testing
 import os
 import re
 import tempfile
 import time
 
 def import_and_report(config, db_name, db, file, format, ts_name,
                       commit=False, show_sample_count=False,
-                      disable_email=False, disable_report=False):
+                      disable_email=False, disable_report=False,
+                      updateMachine=False):
     """
     import_and_report(config, db_name, db, file, format, ts_name,
                       [commit], [show_sample_count],
                       [disable_email]) -> ... object ...
 
     Import a test data file into an LNT server and generate a test report. On
     success, run is the newly imported run. Note that success is uneffected by
     the value of commit, this merely changes whether the run (on success) is
def import_and_report(config, db_name, db, file, format, ts_name,
     importStartTime = time.time()
     try:
         data_schema = data.get('schema')
         if data_schema is not None and data_schema != ts_name:
             result['error'] = ("Importing '%s' data into test suite '%s'" %
                                (data_schema, ts_name))
             return result
 
-        success, run = ts.importDataFromDict(data, commit, config=db_config)
+        success, run = ts.importDataFromDict(data, commit, config=db_config,
+                                             updateMachine=updateMachine)
     except KeyboardInterrupt:
         raise
     except Exception as e:
         import traceback
         result['error'] = "import failure: %s" % e.message
         result['message'] = traceback.format_exc()
         return result
 
if config and config.databases[db_name].shadow_import:
             if shadow_db is None:
                 raise ValueError, ("invalid configuration, shadow import "
                                    "database %r does not exist") % shadow_name
 
             # Perform the shadow import.
             shadow_result = import_and_report(config, shadow_name,
                                               shadow_db, file, format, ts_name,
                                               commit, show_sample_count,
-                                              disable_email, disable_report)
+                                              disable_email, disable_report,
+                                              updateMachine)
 
             # Append the shadow result to the result.
             result['shadow_result'] = shadow_result
 
     result['success'] = True
     return result
 
 
if total_added:
             print >>out, "Added Samples : %d" % result['added_samples']
         print >>out
     print >>out, "Results"
     print >>out, "----------------"
     for kind, count in result_kinds.items():
         print >>out, kind, ":", count
 
 
-def import_from_string(config, db_name, db, ts_name, data, commit=True):
+def import_from_string(config, db_name, db, ts_name, data, commit=True,
+                       updateMachine=False):
     # Stash a copy of the raw submission.
     #
     # To keep the temporary directory organized, we keep files in
     # subdirectories organized by (database, year-month).
     utcnow = datetime.datetime.utcnow()
     tmpdir = os.path.join(config.tempDir, db_name,
                           "%04d-%02d" % (utcnow.year, utcnow.month))
     try:
def import_from_string(config, db_name, db, ts_name, data, commit=True,
     os.close(fd)
 
     # Import the data.
     #
     # FIXME: Gracefully handle formats failures and DOS attempts. We
     # should at least reject overly large inputs.
 
     result = lnt.util.ImportData.import_and_report(config, db_name, db,
-            path, '<auto>', ts_name, commit)
+            path, '<auto>', ts_name, commit, updateMachine=updateMachine)
     return result

lnt/trunk/lnt/util/ServerUtil.py

def _show_json_error(reply):
     except ValueError:
         print "error: {}".format(reply)
         return
     sys.stderr.write("lnt server error: {}\n".format(error.get('error')))
     message = error.get('message', '')
     if message:
         sys.stderr.write(message + '\n')
 
-def submitFileToServer(url, file, commit):
+def submitFileToServer(url, file, commit, updateMachine):
     with open(file, 'rb') as f:
-        values = { 'input_data' : f.read(),
-                   'commit' : ("0","1")[not not commit] }
+        values = {'input_data' : f.read(),
+                  'commit' : "1" if commit else "0",
+                  'update_machine': "1" if updateMachine else "0"}
     headers = {'Accept': 'application/json'}
     data = urllib.urlencode(values)
     try:
         response = urllib2.urlopen(urllib2.Request(url, data, headers=headers))
     except urllib2.HTTPError as e:
         _show_json_error(e.read())
         return
     result_data = response.read()
except:
         print
         print "Result:"
         print "error:", result_data
         return
 
     return reply
 
 
-def submitFileToInstance(path, file, commit):
+def submitFileToInstance(path, file, commit, updateMachine=False):
     # Otherwise, assume it is a local url and submit to the default database
     # in the instance.
     instance = lnt.server.instance.Instance.frompath(path)
     config = instance.config
     db_name = 'default'
     with contextlib.closing(config.get_database(db_name)) as db:
         if db is None:
             raise ValueError("no default database in instance: %r" % (path,))
         return lnt.util.ImportData.import_and_report(
             config, db_name, db, file, format='<auto>', ts_name='nts',
-            commit=commit)
+            commit=commit, updateMachine=updateMachine)
 
 
-def submitFile(url, file, commit, verbose):
+def submitFile(url, file, commit, verbose, updateMachine=False):
     # If this is a real url, submit it using urllib.
     if '://' in url:
-        result = submitFileToServer(url, file, commit)
+        result = submitFileToServer(url, file, commit, updateMachine)
         if result is None:
             return
     else:
-        result = submitFileToInstance(url, file, commit)
+        result = submitFileToInstance(url, file, commit, updateMachine)
     return result
 
 
-def submitFiles(url, files, commit, verbose):
+def submitFiles(url, files, commit, verbose, updateMachine=False):
     results = []
     for file in files:
-        result = submitFile(url, file, commit, verbose)
+        result = submitFile(url, file, commit, verbose, updateMachine)
         if result:
             results.append(result)
     return results

lnt/trunk/tests/lnttool/Inputs/compile_submission_machine_diff_fine.json

+{
+  "Machine": {
+    "Info": {
+      "hw.activecpu": "4",
+      "hostname": "test.local"
+    },
+    "Name": "some-compile-suite-machine"
+  },
+  "Run": {
+    "End Time": "2017-07-06 15:37:08",
+    "Start Time": "2017-07-06 15:05:23",
+    "Info": {
+      "__report_version__": "1",
+      "run_order": "663360",
+      "tag": "compile"
+    }
+  },
+  "Tests": [
+        {
+            "Data": [
+                11.601326,
+                11.411566,
+                11.490528
+            ],
+            "Info": {},
+            "Name": "compile.build/Adium-1.5.7(config='Debug',j=1).user"
+        }
+  ]
+}

lnt/trunk/tests/lnttool/Inputs/compile_submission_machine_diff_reject.json

+{
+  "Machine": {
+    "Info": {
+      "hw.activecpu": "1",
+      "machdep.cpu.vendor": "GenuineIntel"
+    },
+    "Name": "some-compile-suite-machine"
+  },
+  "Run": {
+    "End Time": "2017-07-06 15:37:08",
+    "Start Time": "2017-07-06 15:05:23",
+    "Info": {
+      "__report_version__": "1",
+      "run_order": "663400",
+      "tag": "compile"
+    }
+  },
+  "Tests": [
+        {
+            "Data": [
+                13.601326,
+                13.411566,
+                13.490528
+            ],
+            "Info": {},
+            "Name": "compile.build/Adium-1.5.7(config='Debug',j=1).user"
+        }
+  ]
+}

lnt/trunk/tests/lnttool/submit.shtest

 lnt submit "http://localhost:9091/db_default/v4/compile/submitRun" --commit "${INPUTS}/invalid_submission0.json" >> "${OUTPUT_DIR}/submit_errors.txt" 2>&1
 # CHECK-ERRORS: lnt server error: could not parse input format
 # ...
 # CHECK-ERRORS: SystemExit: unable to guess input format for
 lnt submit "http://localhost:9091/db_default/v4/compile/submitRun" --commit "${INPUTS}/invalid_submission1.json" >> "${OUTPUT_DIR}/submit_errors.txt" 2>&1
 # CHECK-ERRORS: lnt server error: import failure: machine
 # ...
 # CHECK-ERRORS: KeyError: 'machine'
+lnt submit "http://localhost:9091/db_default/v4/compile/submitRun" --commit "${INPUTS}/compile_submission_machine_diff_reject.json" >> "${OUTPUT_DIR}/submit_errors.txt" 2>&1
+# CHECK-ERRORS: lnt server error: import failure: 'hw.activecpu' on machine 'some-compile-suite-machine' changed.
+# ...
+# ValueError: 'hw.activecpu' on machine 'some-compile-suite-machine' changed.
+
+
+
+# Adding extra fields to the machine in a submission is fine.
+lnt submit "http://localhost:9091/db_default/v4/compile/submitRun" --commit "${INPUTS}/compile_submission_machine_diff_fine.json" -v > "${OUTPUT_DIR}/submit_compile_machine_diff.txt"
+# RUN: FileCheck %s --check-prefix=CHECK-MACHINEDIFF < %T/submit_compile_machine_diff.txt
+#
+# CHECK-MACHINEDIFF: Imported Data
+# CHECK-MACHINEDIFF: -------------
+# CHECK-MACHINEDIFF-NOT: Added Machines
+# CHECK-MACHINEDIFF: Added Runs    : 1
+# CHECK-MACHINEDIFF-NOT: Added Machines
+#
+# CHECK-MACHINEDIFF: Results
+# CHECK-MACHINEDIFF: ----------------
+# CHECK-MACHINEDIFF: PASS : 9
+# CHECK-MACHINEDIFF: Results available at: http://localhost:9091/db_default/v4/compile/6
+
+# Test updating existing machine
+lnt submit "http://localhost:9091/db_default/v4/compile/submitRun" --commit "${INPUTS}/compile_submission_machine_diff_reject.json" --update-machine -v > "${OUTPUT_DIR}/submit_compile_machine_update.txt"
+# RUN: FileCheck %s --check-prefix=CHECK-UPDATEMACHINE < %T/submit_compile_machine_update.txt
+#
+# CHECK-UPDATEMACHINE: Imported Data
+# CHECK-UPDATEMACHINE: -------------
+# CHECK-UPDATEMACHINE-NOT: Added Machines
+# CHECK-UPDATEMACHINE: Added Runs    : 1
+# CHECK-UPDATEMACHINE-NOT: Added Machines
+#
+# CHECK-UPDATEMACHINE: Results
+# CHECK-UPDATEMACHINE: ----------------
+# CHECK-UPDATEMACHINE: PASS : 9
+# CHECK-UPDATEMACHINE: Results available at: http://localhost:9091/db_default/v4/compile/7