This is an archive of the discontinued LLVM Phabricator instance.

Differential D57827

[opt-viewer] Add --filter option to select remarks for displaying.
ClosedPublic

Authored by fhahn on Feb 6 2019, 9:30 AM.

Details

Reviewers
hfinkel
anemet
thegameg
serge-sans-paille
Commits
rG169f64238fe9: [opt-viewer] Add --filter option to select remarks for displaying.
rL353322: [opt-viewer] Add --filter option to select remarks for displaying.
Summary

This allows limiting the displayed remarks to the ones with names
matching the filter (regular) expression.

Generating html pages for a larger project with optimization remarks can
result in a huge HTML documents and using --filter allows to focus on a
set of interesting remarks.

Diff Detail

Repository
rL LLVM

Event Timeline

fhahn created this revision.Feb 6 2019, 9:30 AM
Herald added a reviewer: serge-sans-paille. · View Herald TranscriptFeb 6 2019, 9:30 AM
Herald added a project: Restricted Project. · View Herald Transcript
Herald added a subscriber: arphaman. · View Herald Transcript
Harbormaster completed remote builds in B27828: Diff 185581.Feb 6 2019, 9:30 AM
lebedev.ri added a subscriber: lebedev.ri.Feb 6 2019, 9:36 AM
anemet accepted this revision.Feb 6 2019, 9:44 AM

LGTM with the comment.

llvm/tools/opt-viewer/opt-viewer.py
347–350 ↗(On Diff #185581)

Mention that it's the pass name that is matched.

This revision is now accepted and ready to land.Feb 6 2019, 9:44 AM
Closed by commit rL353322: [opt-viewer] Add --filter option to select remarks for displaying. (authored by fhahn). · Explain WhyFeb 6 2019, 10:44 AM
This revision was automatically updated to reflect the committed changes.
fhahn added a comment.Feb 6 2019, 10:45 AM

Thanks Adam. Changed help text to ''Only display remarks from passes matching filter expression' in the committed version.

Revision Contents

PathSize
llvm/
trunk/
test/
tools/
opt-viewer/
Outputs/
filter/
217 lines
260 lines
124 lines
8 lines
tools/
opt-viewer/
9 lines
8 lines
14 lines

Diff 185590

llvm/trunk/test/tools/opt-viewer/Outputs/filter/basic_or.h.html

<html>
<title>or.h</title>
<meta charset="utf-8" />
<head>
<link rel='stylesheet' type='text/css' href='style.css'>
<script type="text/javascript">
/* Simple helper to show/hide the expanded message of a remark. */
function toggleExpandedMessage(e) {
var FullTextElems = e.parentElement.parentElement.getElementsByClassName("full-info");
if (!FullTextElems || FullTextElems.length < 1) {
return false;
}
var FullText = FullTextElems[0];
if (FullText.style.display == 'none') {
e.innerHTML = '-';
FullText.style.display = 'block';
} else {
e.innerHTML = '+';
FullText.style.display = 'none';
}
}
</script>
</head>
<body>
<div class="centered">
<table class="source">
<thead>
<tr>
<th style="width: 2%">Line</td>
<th style="width: 3%">Hotness</td>
<th style="width: 10%">Optimization</td>
<th style="width: 70%">Source</td>
<th style="width: 15%">Inline Context</td>
</tr>
</thead>
<tbody>
<tr>
<td><a name="L1">1</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre>void TestH(int *res, int *c, int *d, int *p, int n) {</pre></div></td>
</tr>
<tr>
<td><a name="L2">2</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre> int i;</pre></div></td>
</tr>
<tr>
<td><a name="L3">3</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre></pre></div></td>
</tr>
<tr>
<td><a name="L4">4</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre>#pragma clang loop vectorize(assume_safety)</pre></div></td>
</tr>
<tr>
<td><a name="L5">5</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre> for (i = 0; i < 1600; i++) {</pre></div></td>
</tr>
<tr>
<td><a name="L6">6</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre> res[i] = (p[i] == 0) ? res[i] : res[i] + d[i];</pre></div></td>
</tr>
<tr>
<td><a name="L7">7</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre> }</pre></div></td>
</tr>
<tr>
<td><a name="L8">8</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre></pre></div></td>
</tr>
<tr>
<td><a name="L9">9</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre> for (i = 0; i < 16; i++) {</pre></div></td>
</tr>
<tr>
<td><a name="L10">10</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre> res[i] = (p[i] == 0) ? res[i] : res[i] + d[i];</pre></div></td>
</tr>
<tr>
<td><a name="L11">11</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre> }</pre></div></td>
</tr>
<tr>
<td><a name="L12">12</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre></pre></div></td>
</tr>
<tr>
<td><a name="L13">13</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre> foo();</pre></div></td>
</tr>
<tr>
<td></td>
<td></td>
<td class="column-entry-white">inline</td>
<td><pre style="display:inline"> </pre><span class="column-entry-yellow"> foo can be inlined into TestH with cost=30 (threshold=412)&nbsp;</span></td>
<td class="column-entry-yellow">TestH</td>
</tr>
<tr>
<td></td>
<td></td>
<td class="column-entry-green">inline</td>
<td><pre style="display:inline"> </pre><span class="column-entry-yellow"> foo inlined into TestH&nbsp;</span></td>
<td class="column-entry-yellow">TestH</td>
</tr>
<tr>
<td><a name="L14">14</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre></pre></div></td>
</tr>
<tr>
<td><a name="L15">15</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre> foo(); bar(); foo();</pre></div></td>
</tr>
<tr>
<td></td>
<td></td>
<td class="column-entry-red">inline</td>
<td><pre style="display:inline"> </pre><span class="column-entry-yellow"> bar will not be inlined into TestH because its definition is unavailable&nbsp;</span></td>
<td class="column-entry-yellow">TestH</td>
</tr>
<tr>
<td></td>
<td></td>
<td class="column-entry-white">inline</td>
<td><pre style="display:inline"> </pre><span class="column-entry-yellow"> foo can be inlined into TestH with cost=30 (threshold=412)&nbsp;</span></td>
<td class="column-entry-yellow">TestH</td>
</tr>
<tr>
<td></td>
<td></td>
<td class="column-entry-green">inline</td>
<td><pre style="display:inline"> </pre><span class="column-entry-yellow"> foo inlined into TestH&nbsp;</span></td>
<td class="column-entry-yellow">TestH</td>
</tr>
<tr>
<td></td>
<td></td>
<td class="column-entry-white">inline</td>
<td><pre style="display:inline"> </pre><span class="column-entry-yellow"> foo can be inlined into TestH with cost=30 (threshold=412)&nbsp;</span></td>
<td class="column-entry-yellow">TestH</td>
</tr>
<tr>
<td></td>
<td></td>
<td class="column-entry-green">inline</td>
<td><pre style="display:inline"> </pre><span class="column-entry-yellow"> foo inlined into TestH&nbsp;</span></td>
<td class="column-entry-yellow">TestH</td>
</tr>
<tr>
<td><a name="L16">16</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre>}</pre></div></td>
</tr>
<tr>
<td><a name="L17">17</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre></pre></div></td>
</tr>
</tbody>
</table>
</body>
</html>

llvm/trunk/test/tools/opt-viewer/Outputs/filter/basic_or.c.html

<html>
<title>or.c</title>
<meta charset="utf-8" />
<head>
<link rel='stylesheet' type='text/css' href='style.css'>
<script type="text/javascript">
/* Simple helper to show/hide the expanded message of a remark. */
function toggleExpandedMessage(e) {
var FullTextElems = e.parentElement.parentElement.getElementsByClassName("full-info");
if (!FullTextElems || FullTextElems.length < 1) {
return false;
}
var FullText = FullTextElems[0];
if (FullText.style.display == 'none') {
e.innerHTML = '-';
FullText.style.display = 'block';
} else {
e.innerHTML = '+';
FullText.style.display = 'none';
}
}
</script>
</head>
<body>
<div class="centered">
<table class="source">
<thead>
<tr>
<th style="width: 2%">Line</td>
<th style="width: 3%">Hotness</td>
<th style="width: 10%">Optimization</td>
<th style="width: 70%">Source</td>
<th style="width: 15%">Inline Context</td>
</tr>
</thead>
<tbody>
<tr>
<td><a name="L1">1</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre>void bar();</pre></div></td>
</tr>
<tr>
<td><a name="L2">2</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre>void foo() { bar(); }</pre></div></td>
</tr>
<tr>
<td></td>
<td></td>
<td class="column-entry-red">inline</td>
<td><pre style="display:inline"> </pre><span class="column-entry-yellow"> bar will not be inlined into foo because its definition is unavailable&nbsp;</span></td>
<td class="column-entry-yellow">foo</td>
</tr>
<tr>
<td><a name="L3">3</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre></pre></div></td>
</tr>
<tr>
<td><a name="L4">4</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre>#include "or.h"</pre></div></td>
</tr>
<tr>
<td><a name="L5">5</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre></pre></div></td>
</tr>
<tr>
<td><a name="L6">6</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre>void Test(int *res, int *c, int *d, int *p, int n) {</pre></div></td>
</tr>
<tr>
<td><a name="L7">7</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre> int i;</pre></div></td>
</tr>
<tr>
<td><a name="L8">8</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre></pre></div></td>
</tr>
<tr>
<td><a name="L9">9</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre>#pragma clang loop vectorize(assume_safety)</pre></div></td>
</tr>
<tr>
<td><a name="L10">10</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre> for (i = 0; i < 1600; i++) {</pre></div></td>
</tr>
<tr>
<td><a name="L11">11</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre> res[i] = (p[i] == 0) ? res[i] : res[i] + d[i];</pre></div></td>
</tr>
<tr>
<td><a name="L12">12</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre> }</pre></div></td>
</tr>
<tr>
<td><a name="L13">13</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre></pre></div></td>
</tr>
<tr>
<td><a name="L14">14</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre> for (i = 0; i < 16; i++) {</pre></div></td>
</tr>
<tr>
<td><a name="L15">15</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre> res[i] = (p[i] == 0) ? res[i] : res[i] + d[i];</pre></div></td>
</tr>
<tr>
<td><a name="L16">16</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre> }</pre></div></td>
</tr>
<tr>
<td><a name="L17">17</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre></pre></div></td>
</tr>
<tr>
<td><a name="L18">18</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre> foo();</pre></div></td>
</tr>
<tr>
<td></td>
<td></td>
<td class="column-entry-white">inline</td>
<td><pre style="display:inline"> </pre><span class="column-entry-yellow"> foo can be inlined into Test with cost=30 (threshold=412)&nbsp;</span></td>
<td class="column-entry-yellow">Test</td>
</tr>
<tr>
<td></td>
<td></td>
<td class="column-entry-green">inline</td>
<td><pre style="display:inline"> </pre><span class="column-entry-yellow"> foo inlined into Test&nbsp;</span></td>
<td class="column-entry-yellow">Test</td>
</tr>
<tr>
<td><a name="L19">19</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre></pre></div></td>
</tr>
<tr>
<td><a name="L20">20</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre> foo(); bar(); foo();</pre></div></td>
</tr>
<tr>
<td></td>
<td></td>
<td class="column-entry-red">inline</td>
<td><pre style="display:inline"> </pre><span class="column-entry-yellow"> bar will not be inlined into Test because its definition is unavailable&nbsp;</span></td>
<td class="column-entry-yellow">Test</td>
</tr>
<tr>
<td></td>
<td></td>
<td class="column-entry-white">inline</td>
<td><pre style="display:inline"> </pre><span class="column-entry-yellow"> foo can be inlined into Test with cost=30 (threshold=412)&nbsp;</span></td>
<td class="column-entry-yellow">Test</td>
</tr>
<tr>
<td></td>
<td></td>
<td class="column-entry-green">inline</td>
<td><pre style="display:inline"> </pre><span class="column-entry-yellow"> foo inlined into Test&nbsp;</span></td>
<td class="column-entry-yellow">Test</td>
</tr>
<tr>
<td></td>
<td></td>
<td class="column-entry-white">inline</td>
<td><pre style="display:inline"> </pre><span class="column-entry-yellow"> foo can be inlined into Test with cost=30 (threshold=412)&nbsp;</span></td>
<td class="column-entry-yellow">Test</td>
</tr>
<tr>
<td></td>
<td></td>
<td class="column-entry-green">inline</td>
<td><pre style="display:inline"> </pre><span class="column-entry-yellow"> foo inlined into Test&nbsp;</span></td>
<td class="column-entry-yellow">Test</td>
</tr>
<tr>
<td><a name="L21">21</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre>}</pre></div></td>
</tr>
<tr>
<td><a name="L22">22</a></td>
<td></td>
<td></td>
<td><div class="highlight"><pre></pre></div></td>
</tr>
</tbody>
</table>
</body>
</html>

llvm/trunk/test/tools/opt-viewer/Outputs/filter/index.html

<html>
<meta charset="utf-8" />
<head>
<link rel='stylesheet' type='text/css' href='style.css'>
</head>
<body>
<div class="centered">
<table>
<tr>
<td>Source Location</td>
<td>Hotness</td>
<td>Function</td>
<td>Pass</td>
</tr>
<tr>
<td class="column-entry-0"><a href="basic_or.c.html#L2">basic/or.c:2:14</a></td>
<td class="column-entry-0"></td>
<td class="column-entry-0">foo</td>
<td class="column-entry-red">inline</td>
</tr>
<tr>
<td class="column-entry-1"><a href="basic_or.c.html#L18">basic/or.c:18:3</a></td>
<td class="column-entry-1"></td>
<td class="column-entry-1">Test</td>
<td class="column-entry-white">inline</td>
</tr>
<tr>
<td class="column-entry-0"><a href="basic_or.c.html#L18">basic/or.c:18:3</a></td>
<td class="column-entry-0"></td>
<td class="column-entry-0">Test</td>
<td class="column-entry-green">inline</td>
</tr>
<tr>
<td class="column-entry-1"><a href="basic_or.c.html#L20">basic/or.c:20:3</a></td>
<td class="column-entry-1"></td>
<td class="column-entry-1">Test</td>
<td class="column-entry-white">inline</td>
</tr>
<tr>
<td class="column-entry-0"><a href="basic_or.c.html#L20">basic/or.c:20:3</a></td>
<td class="column-entry-0"></td>
<td class="column-entry-0">Test</td>
<td class="column-entry-green">inline</td>
</tr>
<tr>
<td class="column-entry-1"><a href="basic_or.c.html#L20">basic/or.c:20:10</a></td>
<td class="column-entry-1"></td>
<td class="column-entry-1">Test</td>
<td class="column-entry-red">inline</td>
</tr>
<tr>
<td class="column-entry-0"><a href="basic_or.c.html#L20">basic/or.c:20:17</a></td>
<td class="column-entry-0"></td>
<td class="column-entry-0">Test</td>
<td class="column-entry-white">inline</td>
</tr>
<tr>
<td class="column-entry-1"><a href="basic_or.c.html#L20">basic/or.c:20:17</a></td>
<td class="column-entry-1"></td>
<td class="column-entry-1">Test</td>
<td class="column-entry-green">inline</td>
</tr>
<tr>
<td class="column-entry-0"><a href="basic_or.h.html#L13">basic/or.h:13:3</a></td>
<td class="column-entry-0"></td>
<td class="column-entry-0">TestH</td>
<td class="column-entry-white">inline</td>
</tr>
<tr>
<td class="column-entry-1"><a href="basic_or.h.html#L13">basic/or.h:13:3</a></td>
<td class="column-entry-1"></td>
<td class="column-entry-1">TestH</td>
<td class="column-entry-green">inline</td>
</tr>
<tr>
<td class="column-entry-0"><a href="basic_or.h.html#L15">basic/or.h:15:3</a></td>
<td class="column-entry-0"></td>
<td class="column-entry-0">TestH</td>
<td class="column-entry-white">inline</td>
</tr>
<tr>
<td class="column-entry-1"><a href="basic_or.h.html#L15">basic/or.h:15:3</a></td>
<td class="column-entry-1"></td>
<td class="column-entry-1">TestH</td>
<td class="column-entry-green">inline</td>
</tr>
<tr>
<td class="column-entry-0"><a href="basic_or.h.html#L15">basic/or.h:15:10</a></td>
<td class="column-entry-0"></td>
<td class="column-entry-0">TestH</td>
<td class="column-entry-red">inline</td>
</tr>
<tr>
<td class="column-entry-1"><a href="basic_or.h.html#L15">basic/or.h:15:17</a></td>
<td class="column-entry-1"></td>
<td class="column-entry-1">TestH</td>
<td class="column-entry-white">inline</td>
</tr>
<tr>
<td class="column-entry-0"><a href="basic_or.h.html#L15">basic/or.h:15:17</a></td>
<td class="column-entry-0"></td>
<td class="column-entry-0">TestH</td>
<td class="column-entry-green">inline</td>
</tr>
</table>
</body>
</html>

llvm/trunk/test/tools/opt-viewer/filter.test

# Since we're performing a full compare of the generate HTML files disable
# syntax highlighting; pygments generates slightly different code with
# different versions.
RUN: %opt-viewer --filter inline -s %p/Inputs -o %t %p/Inputs/basic/or.yaml --no-highlight --demangler=llvm-cxxfilt
RUN: diff %p/Outputs/filter/index.html %t/index.html
RUN: diff %p/Outputs/filter/basic_or.h.html %t/basic_or.h.html
RUN: diff %p/Outputs/filter/basic_or.c.html %t/basic_or.c.html

llvm/trunk/tools/opt-viewer/opt-viewer.py

Show First 20 Lines • Show All 233 Lines • ▼ Show 20 Lines</tr>''', file=self.stream)
if not suppress(remark): if not suppress(remark):
self.render_entry(remark, i % 2) self.render_entry(remark, i % 2)
print(''' print('''
</table> </table>
</body> </body>
</html>''', file=self.stream) </html>''', file=self.stream)
def _render_file(source_dir, output_dir, ctx, no_highlight, entry): def _render_file(source_dir, output_dir, ctx, no_highlight, entry, filter_):
global context global context
context = ctx context = ctx
filename, remarks = entry filename, remarks = entry
SourceFileRenderer(source_dir, output_dir, filename, no_highlight).render(remarks) SourceFileRenderer(source_dir, output_dir, filename, no_highlight).render(remarks)
def map_remarks(all_remarks): def map_remarks(all_remarks):
# Set up a map between function names and their source location for # Set up a map between function names and their source location for
▲ Show 20 Lines • Show All 88 Lines • ▼ Show 20 Lines parser.add_argument(
'--no-highlight', '--no-highlight',
action='store_true', action='store_true',
default=False, default=False,
help='Do not use a syntax highlighter when rendering the source code') help='Do not use a syntax highlighter when rendering the source code')
parser.add_argument( parser.add_argument(
'--demangler', '--demangler',
help='Set the demangler to be used (defaults to %s)' % optrecord.Remark.default_demangler) help='Set the demangler to be used (defaults to %s)' % optrecord.Remark.default_demangler)
parser.add_argument(
'--filter',
default='',
help='Only display remarks from passes matching filter expression')
# Do not make this a global variable. Values needed to be propagated through # Do not make this a global variable. Values needed to be propagated through
# to individual classes and functions to be portable with multiprocessing across # to individual classes and functions to be portable with multiprocessing across
# Windows and non-Windows. # Windows and non-Windows.
args = parser.parse_args() args = parser.parse_args()
print_progress = not args.no_progress_indicator print_progress = not args.no_progress_indicator
if args.demangler: if args.demangler:
optrecord.Remark.set_demangler(args.demangler) optrecord.Remark.set_demangler(args.demangler)
files = optrecord.find_opt_files(*args.yaml_dirs_or_files) files = optrecord.find_opt_files(*args.yaml_dirs_or_files)
if not files: if not files:
parser.error("No *.opt.yaml files found") parser.error("No *.opt.yaml files found")
sys.exit(1) sys.exit(1)
all_remarks, file_remarks, should_display_hotness = \ all_remarks, file_remarks, should_display_hotness = \
optrecord.gather_results(files, args.jobs, print_progress) optrecord.gather_results(files, args.jobs, print_progress, args.filter)
map_remarks(all_remarks) map_remarks(all_remarks)
generate_report(all_remarks, generate_report(all_remarks,
file_remarks, file_remarks,
args.source_dir, args.source_dir,
args.output_dir, args.output_dir,
args.no_highlight, args.no_highlight,
should_display_hotness, should_display_hotness,
args.max_hottest_remarks_on_index, args.max_hottest_remarks_on_index,
args.jobs, args.jobs,
print_progress) print_progress)
if __name__ == '__main__': if __name__ == '__main__':
main() main()

llvm/trunk/tools/opt-viewer/optpmap.py

import sys import sys
import multiprocessing import multiprocessing
_current = None _current = None
_total = None _total = None
def _init(current, total): def _init(current, total):
global _current global _current
global _total global _total
_current = current _current = current
_total = total _total = total
def _wrapped_func(func_and_args): def _wrapped_func(func_and_args):
func, argument, should_print_progress = func_and_args func, argument, should_print_progress, filter_ = func_and_args
if should_print_progress: if should_print_progress:
with _current.get_lock(): with _current.get_lock():
_current.value += 1 _current.value += 1
sys.stdout.write('\r\t{} of {}'.format(_current.value, _total.value)) sys.stdout.write('\r\t{} of {}'.format(_current.value, _total.value))
sys.stdout.flush() sys.stdout.flush()
return func(argument) return func(argument, filter_)
def pmap(func, iterable, processes, should_print_progress, *args, **kwargs): def pmap(func, iterable, processes, should_print_progress, filter_=None, *args, **kwargs):
""" """
A parallel map function that reports on its progress. A parallel map function that reports on its progress.
Applies `func` to every item of `iterable` and return a list of the Applies `func` to every item of `iterable` and return a list of the
results. If `processes` is greater than one, a process pool is used to run results. If `processes` is greater than one, a process pool is used to run
the functions in parallel. `should_print_progress` is a boolean value that the functions in parallel. `should_print_progress` is a boolean value that
indicates whether a string 'N of M' should be printed to indicate how many indicates whether a string 'N of M' should be printed to indicate how many
of the functions have finished being run. of the functions have finished being run.
""" """
global _current global _current
global _total global _total
_current = multiprocessing.Value('i', 0) _current = multiprocessing.Value('i', 0)
_total = multiprocessing.Value('i', len(iterable)) _total = multiprocessing.Value('i', len(iterable))
func_and_args = [(func, arg, should_print_progress,) for arg in iterable] func_and_args = [(func, arg, should_print_progress, filter_) for arg in iterable]
if processes == 1: if processes == 1:
result = list(map(_wrapped_func, func_and_args, *args, **kwargs)) result = list(map(_wrapped_func, func_and_args, *args, **kwargs))
else: else:
pool = multiprocessing.Pool(initializer=_init, pool = multiprocessing.Pool(initializer=_init,
initargs=(_current, _total,), initargs=(_current, _total,),
processes=processes) processes=processes)
result = pool.map(_wrapped_func, func_and_args, *args, **kwargs) result = pool.map(_wrapped_func, func_and_args, *args, **kwargs)
pool.close() pool.close()
pool.join() pool.join()
if should_print_progress: if should_print_progress:
sys.stdout.write('\r') sys.stdout.write('\r')
return result return result

llvm/trunk/tools/opt-viewer/optrecord.py

Show All 18 Lines
import subprocess import subprocess
try: try:
# The previously builtin function `intern()` was moved # The previously builtin function `intern()` was moved
# to the `sys` module in Python 3. # to the `sys` module in Python 3.
from sys import intern from sys import intern
except: except:
pass pass
import re
import optpmap import optpmap
try: try:
dict.iteritems dict.iteritems
except AttributeError: except AttributeError:
# Python 3 # Python 3
def itervalues(d): def itervalues(d):
return iter(d.values()) return iter(d.values())
▲ Show 20 Lines • Show All 223 Lines • ▼ Show 20 Lines
class Missed(Remark): class Missed(Remark):
yaml_tag = '!Missed' yaml_tag = '!Missed'
@property @property
def color(self): def color(self):
return "red" return "red"
def get_remarks(input_file): def get_remarks(input_file, filter_):
max_hotness = 0 max_hotness = 0
all_remarks = dict() all_remarks = dict()
file_remarks = defaultdict(functools.partial(defaultdict, list)) file_remarks = defaultdict(functools.partial(defaultdict, list))
with open(input_file) as f: with open(input_file) as f:
docs = yaml.load_all(f, Loader=Loader) docs = yaml.load_all(f, Loader=Loader)
filter_e = re.compile(filter_)
for remark in docs: for remark in docs:
remark.canonicalize() remark.canonicalize()
# Avoid remarks withoug debug location or if they are duplicated # Avoid remarks withoug debug location or if they are duplicated
if not hasattr(remark, 'DebugLoc') or remark.key in all_remarks: if not hasattr(remark, 'DebugLoc') or remark.key in all_remarks:
continue continue
if filter_ and not filter_e.search(remark.Pass):
continue
all_remarks[remark.key] = remark all_remarks[remark.key] = remark
file_remarks[remark.File][remark.Line].append(remark) file_remarks[remark.File][remark.Line].append(remark)
# If we're reading a back a diff yaml file, max_hotness is already # If we're reading a back a diff yaml file, max_hotness is already
# captured which may actually be less than the max hotness found # captured which may actually be less than the max hotness found
# in the file. # in the file.
if hasattr(remark, 'max_hotness'): if hasattr(remark, 'max_hotness'):
max_hotness = remark.max_hotness max_hotness = remark.max_hotness
max_hotness = max(max_hotness, remark.Hotness) max_hotness = max(max_hotness, remark.Hotness)
return max_hotness, all_remarks, file_remarks return max_hotness, all_remarks, file_remarks
def gather_results(filenames, num_jobs, should_print_progress): def gather_results(filenames, num_jobs, should_print_progress, filter_):
if should_print_progress: if should_print_progress:
print('Reading YAML files...') print('Reading YAML files...')
if not Remark.demangler_proc: if not Remark.demangler_proc:
Remark.set_demangler(Remark.default_demangler) Remark.set_demangler(Remark.default_demangler)
remarks = optpmap.pmap( remarks = optpmap.pmap(
get_remarks, filenames, num_jobs, should_print_progress) get_remarks, filenames, num_jobs, should_print_progress, filter_)
max_hotness = max(entry[0] for entry in remarks) max_hotness = max(entry[0] for entry in remarks)
def merge_file_remarks(file_remarks_job, all_remarks, merged): def merge_file_remarks(file_remarks_job, all_remarks, merged):
for filename, d in iteritems(file_remarks_job): for filename, d in iteritems(file_remarks_job):
for line, remarks in iteritems(d): for line, remarks in iteritems(d):
for remark in remarks: for remark in remarks:
# Bring max_hotness into the remarks so that # Bring max_hotness into the remarks so that
# RelativeHotness does not depend on an external global. # RelativeHotness does not depend on an external global.
Show All 27 Lines