[LNT] Adding initial support for javascript unit testing

[kristof.beyls]

Summary

This adds initial support for unit testing the javascript
functionality, by using the Jasmine javascript testing framework
(http://jasmine.github.io/) and the Jasmine-py
(https://github.com/jasmine/jasmine-py/) package to drive the
regression tests from python.

In this change:

• I added 2 simple javascript tests, see tests/spec/javascripts/profile_lnt_spec.js.
• Added integration for running these tests with lit, see tests/javascript_tests.py and tests/lit.cfg.
• Adapted an interface in lnt_profile.js, to make it easier to unit test (It also makes the interface cleaner, independent of unit testing needs).
• Unfortunately, there seems to be a problem with the jasmine pypi package: it installs fine with "pip install jasmine", but I get an error when trying to install it automatically when running "python ../lnt/setup.py develop": ValueError: bad marshal data (unknown type code)

The main reasons for picking Jasmine, out of quite a few different alternatives for javascript testing frameworks are:

• It seems to be one of the more popular and well-supported frameworks.
• It can easily be installed using "pip install".
• The jasmine-py package makes running the tests from python (e.g. lit) easy.

The tests are being run in a browser. When no browser is specified, a default one is used.
Probably, the most comfortable way to run these tests is to use the phantomjs head-less browser (http://phantomjs.org/).
The lit command line to run all tests (python + javascript ones) then looks like:

llvm-lit -sv -Dbrowser=phantomjs lnt/tests

As is, this isn't ready to be committed. First, the problem with installing the jasmine pypi
package as part of 'python ../lnt/setup.py develop' needs to be solved. I'm hoping to get
some review feedback on whether this looks like the right/a good direction for integrating
javascript testing into LNT's regression tests.

[cmatthews]

I think this is a good approach.

I do get the ValueError: bad marshal data (unknown type code) on install as well.

I also note I don't get the error when using ipython as the interpreter. ipython is python 2.7.10 vs the 2.7.6 that is on my system.

[kristof.beyls]

The issue with jasmine-py (bad marshall data) has been fixed in the latest 2.5.0 release, after having raised a bug for it: https://github.com/jasmine/jasmine-py/issues/30.

Unfortunately, I'm encountering the following 2 defects in Selenium and phantomjs as installed on Ubuntu 16.04:

• Selenium (used by jasmine-py to invoke browsers), doesn't seem to work with Firefox 47+, see #30 (comment)
• phantomjs on Ubuntu 16.04: Unable to load Atom 'find_elements' on Ubuntu 16.04 https://bugs.launchpad.net/ubuntu/+source/phantomjs/+bug/1578444

I'll need time to investigate these further to come to a conclusion on whether using jasmine-py, and it's dependencies like Selenium is a good route, or whether an approach where we rely on fewer third part libraries would be better.

lnt/server/ui/static/lnt_profile.js

 };
 
 function CFGBasicBlock (instructions, fallThruInstruction, cfg) {
     this.cfg = cfg;
     this.instructionParser = this.cfg.instructionParser;
     this.instructions = instructions;
     this.address = instructions[0].address;
     var gjt = this.instructionParser
-        .getJumpTargets(instructions[instructions.length-1], fallThruInstruction, this.cfg);
+        .getJumpTargets(instructions[instructions.length-1], fallThruInstruction);
     this.targets = gjt[1];
     this.noFallThru = gjt[0];
     this.weight = this.instructions
         .reduce(function (a,b) {
             var weight_a = (a.weight === undefined)?a:a.weight;
             var weight_b = (b.weight === undefined)?b:b.weight;
             return weight_a+weight_b;
         }, 0);
AArch32T32JumpTargetRegexps: [
         [new RegExp("^\\s*b(?:(?:ne)|(?:eq)|(?:cs)|(?:cc)|(?:mi)|(?:pl)|(?:vs)|(?:vc)|(?:hi)|(?:ls)|(?:ge)|(?:lt)|(?:gt)|(?:le))(?:\\.[nw])?\\s+([^\\s]+)"), false],
         // branch unconditional:
         [new RegExp("^\\s*b(?:\\.[nw])?\\s+([^\\s]+)"), true],
         // cb(n)z
         [new RegExp("^\\s*cbn?z\\s+[^,]+,\\s*([^\\s]+)"), false]
         // TODO: add all control-flow-changing instructions.
     ],
 
-    getJumpTargets: function(instruction, nextInstruction, cfg) {
+    // The following method will have different implementations depending
+    // on the profiler, or kind of profiling input.
+    convertToAddress: function (addressString) {
+      return parseInt(addressString, 16);
+    },
+
+    getJumpTargets: function(instruction, nextInstruction) {
         for(var i=0; i<this.jumpTargetRegexps.length; ++i) {
             var regexp = this.jumpTargetRegexps[i][0];
             var noFallThru = this.jumpTargetRegexps[i][1];
             var match = instruction.text.match(regexp);
             if (match) {
                 var targets = [];
                 if (!noFallThru && nextInstruction)
                     targets.push(nextInstruction.address);
                 if (match.length > 1)
-                    targets.push(cfg.convertToAddress(match[1]));
+                    targets.push(this.convertToAddress(match[1]));
                 return [noFallThru, targets];
             }
         }
         return [false, []];
     }
 };
 
 
 CFG.prototype = {
-    // The following method will have different implementations depending
-    // on the profiler, or kind of profiling input.
-    convertToAddress: function (addressString) {
-      return parseInt(addressString, 16);
-    },
-
     parseDisassembly: function() {
         var instructions = [];
         for (var i=0; i<this.disassembly.length; ++i) {
             var profiled_instruction = this.disassembly[i];
             var counter2weight = profiled_instruction[0];
             var address = profiled_instruction[1];
             var text = profiled_instruction[2];
             // FIXME: strip leading white space from text?
             var cfgInstruction =
                 new CFGInstruction(counter2weight['cycles'],
                                    address,
                                    text);
             instructions.push(cfgInstruction);
         }
         return instructions;
     },
 
     createBasicBlocks: function(instructions) {
         var bbstarts = {};
         for (var i=0; i<instructions.length; ++i) {
             var instruction = instructions[i];
             var nextInstruction = (i+1<instructions.length)?instructions[i+1]:null;
             var gjt = this.instructionParser
-                .getJumpTargets(instruction, nextInstruction, this);
+                .getJumpTargets(instruction, nextInstruction);
             var jumpTargets=gjt[1], noFallThru=gjt[0];
             if (jumpTargets.length > 0) {
                 for(var j=0; j<jumpTargets.length; ++j) {
                     var jumpTarget = jumpTargets[j];
                     bbstarts[jumpTarget] = true;
                 }
                 if (nextInstruction) {
                     bbstarts[nextInstruction.address] = true;

requirements.txt

 python-dateutil==1.5
 python-gnupg==0.3.7
 pytz==2015.4
 requests==2.3.0
 six==1.9.0
 wsgiref==0.1.2
 WTForms==2.0.2
 Flask-WTF==0.12
+jasmine==2.4.0

setup.py

entry_points = {
         'console_scripts': [
             'lnt = lnt.lnttool:main',
             ],
         },
     install_requires=['WTForms',
                       'Flask-WTF',
                       'Flask-RESTful',
                       'SQLAlchemy',
-                      'Flask'],
+                      'Flask',
+                      'jasmine'],
 
     ext_modules = [cPerf],
 )

tests/javascript_tests.py

+# Run the jasmine-based javascript unit tests
+#
+# RUN: python %s %{browser}
+
+import os
+import os.path
+import subprocess
+import sys
+
+basedir, browser = sys.argv[0], sys.argv[1]
+# the jasmine-ci framework assume the test description to be in a fixed
+# location, so we need to run this from the correct directory.
+os.chdir(os.path.dirname(basedir))
+cmd = 'jasmine-ci'
+if browser != '%{browser}':
+    cmd += ' --browser %s' % browser
+sys.exit(subprocess.check_call(cmd, shell=True))

tests/lit.cfg

 
 if lit_config.params.get('postgres', None):
     config.available_features.add('postgres')
     config.environment['LNT_POSTGRES_DB_URI'] = \
         lit_config.params.get('postgres')
     config.substitutions.append(('%{postgres_db_uri}',
                                  lit_config.params.get('postgres')))
 
+if lit_config.params.get('browser', None):
+    config.substitutions.append(('%{browser}',
+                                 lit_config.params.get('browser')))
+
+
 config.available_features.add(platform.system())
 
 # Enable coverage.py reporting, assuming the coverage module has been installed
 # and sitecustomize.py in the virtualenv has been modified appropriately.
 if lit_config.params.get('check-coverage', None):
     config.environment['COVERAGE_PROCESS_START'] = os.path.join(
         os.path.dirname(__file__), ".coveragerc")
-    

tests/spec/javascripts/profile_lnt_spec.js

+describe("An InstructionSetParser", function() {
+
+  describe("for AArch64", function() {
+    var instructionSetParser;
+    beforeEach(function() {
+      instructionSetParser = new InstructionSetParser(
+        InstructionSetParser.prototype.AArch64JumpTargetRegexps);
+    });
+
+    it("recognizes 'b.' as a conditional branch", function() {
+      var i = new CFGInstruction("0.0%", "4b09d8", "b.ne  4b09fc <f+0x9c>");
+      var jt = instructionSetParser.getJumpTargets(i, null);
+      var noFallThru=jt[0];
+      expect(noFallThru).toBe(false);
+    });
+    it("extracts the jump target for 'b.'", function() {
+      var i = new CFGInstruction("0.0%", "4b09d8", "b.ne  4b09fc <f+0x9c>");
+      var jt = instructionSetParser.getJumpTargets(i, null);
+      var jumpTargets=jt[1];
+      expect(jumpTargets).toEqual([0x4b09fc]);
+    });
+  });
+});

tests/spec/javascripts/support/jasmine.yml

+
+# src_files
+#
+# Return an array of filepaths relative to src_dir to include before jasmine specs.
+# Default: []
+#
+# EXAMPLE:
+#
+# src_files:
+#   - lib/source1.js
+#   - lib/source2.js
+#   - dist/**/*.js
+#
+src_files:
+  - lnt_profile.js
+
+# stylesheets
+#
+# Return an array of stylesheet filepaths relative to src_dir to include before jasmine specs.
+# Default: []
+#
+# EXAMPLE:
+#
+# stylesheets:
+#   - css/style.css
+#   - stylesheets/*.css
+#
+stylesheets:
+  - lnt_profile.css
+
+# helpers
+#
+# Return an array of filepaths relative to spec_dir to include before jasmine specs.
+# Default: ["helpers/**/*.js"]
+#
+# EXAMPLE:
+#
+# helpers:
+#   - helpers/**/*.js
+#
+helpers:
+  - "helpers/**/*.js"
+
+# spec_files
+#
+# Return an array of filepaths relative to spec_dir to include.
+# Default: ["**/*[sS]pec.js"]
+#
+# EXAMPLE:
+#
+# spec_files:
+#   - **/*[sS]pec.js
+#
+spec_files:
+  - "**/*[Ss]pec.js"
+
+# src_dir
+#
+# Source directory path. Your src_files must be returned relative to this path. Will use root if left blank.
+# Default: project root
+#
+# EXAMPLE:
+#
+# src_dir: public
+#
+src_dir: ../lnt/server/ui/static
+
+# spec_dir
+#
+# Spec directory path. Your spec_files must be returned relative to this path.
+# Default: spec/javascripts
+#
+# EXAMPLE:
+#
+# spec_dir: spec/javascripts
+#
+spec_dir: spec/javascripts
+
+# stop_spec_on_expectation_failure
+#
+# Stop executing each spec on the first expectation failure.
+# Default: false
+#
+# EXAMPLE:
+#
+# stop_spec_on_expectation_failure: true
+#
+stop_spec_on_expectation_failure:
+
+# random
+#
+# Run specs in semi-random order.
+# Default: false
+#
+# EXAMPLE:
+#
+# random: true
+#
+random: