llvm.org GIT mirror lnt / 409eb43
[LNT] Wrap comments and docstrings at 72 chars As per PEP-008, wrap comments and docstrings in the test data creation library at 72 characters. Differential Revision: https://reviews.llvm.org/D65749 git-svn-id: https://llvm.org/svn/llvm-project/lnt/trunk@367923 91177308-0d34-0410-b5e6-96231b3b80d8 Thomas Preud'homme 2 months ago
1 changed file(s) with 50 addition(s) and 49 deletion(s). Raw diff Collapse all Expand all
0 """
11 Utilities for working with the LNT test format.
22
3 Clients can easily generate LNT test format data by creating Report objects for
4 the runs they wish to submit, and using Report.render to convert them to JSON
5 data suitable for submitting to the server.
3 Clients can easily generate LNT test format data by creating Report
4 objects for the runs they wish to submit, and using Report.render to
5 convert them to JSON data suitable for submitting to the server.
66 """
77
88 import datetime
1414 except ImportError:
1515 import simplejson as json
1616
17 # We define the following constants for use as sample values by convention.
17 # We define the following constants for use as sample values by
18 # convention.
1819 PASS = 0
1920 FAIL = 1
2021 XFAIL = 2
3132 class Report:
3233 """Information on a single testing run.
3334
34 In the LNT test model, every test run should define exactly one machine and
35 run, and any number of test samples.
35 In the LNT test model, every test run should define exactly one
36 machine and run, and any number of test samples.
3637 """
3738 def __init__(self, machine, run, tests):
3839 self.machine = machine
4849 assert isinstance(t, TestSamples)
4950
5051 def update_report(self, new_samples):
51 """Add extra samples to this report, and update
52 the end time.
53 """
52 """Add extra samples to this report, and update the end time."""
5453 self.check()
5554 self.tests.extend(new_samples)
5655 self.run.update_endtime()
5857
5958 def render(self, indent=4):
6059 # Note that we specifically override the encoding to avoid the
61 # possibility of encoding errors. Clients which care about the text
62 # encoding should supply unicode string objects.
60 # possibility of encoding errors. Clients which care about the
61 # text encoding should supply unicode string objects.
6362 return json.dumps({'Machine': self.machine.render(),
6463 'Run': self.run.render(),
6564 'Tests': [t.render() for t in self.tests]},
6968 class Machine:
7069 """Information on the machine the test was run on.
7170
72 The info dictionary can be used to describe additional information about
73 the machine, for example the hardware resources or the operating
74 environment.
75
76 Machines entries in the database are uniqued by their name and the entire
77 contents of the info dictionary.
71 The info dictionary can be used to describe additional information
72 about the machine, for example the hardware resources or the
73 operating environment.
74
75 Machines entries in the database are uniqued by their name and the
76 entire contents of the info dictionary.
7877 """
7978 def __init__(self, name, info={}):
8079 self.name = str(name)
8988 class Run:
9089 """Information on the particular test run.
9190
92 The start and end time should always be supplied with the run. Currently,
93 the server uses these to order runs. In the future we will support
94 additional ways to order runs (for example, by a source revision).
95
96 As with Machine, the info dictionary can be used to describe additional
97 information on the run. This dictionary should be used to describe
98 information on the software-under-test that is constant across the test
99 run, for example the revision number being tested. It can also be used to
100 describe information about the current state which could be useful in
101 analysis, for example the current machine load.
91 The start and end time should always be supplied with the run.
92 Currently, the server uses these to order runs. In the future we
93 will support additional ways to order runs (for example, by a source
94 revision).
95
96 As with Machine, the info dictionary can be used to describe
97 additional information on the run. This dictionary should be used to
98 describe information on the software-under-test that is constant
99 across the test run, for example the revision number being tested.
100 It can also be used to describe information about the current state
101 which could be useful in analysis, for example the current machine
102 load.
102103 """
103104 def __init__(self, start_time, end_time, info={}):
104105 if start_time is None:
134135 """Test sample data.
135136
136137 The test sample data defines both the tests that were run and their
137 values. The server automatically creates test database objects whenever a
138 new test name is seen.
139
140 Test names are intended to be a persistent, recognizable identifier for
141 what is being executed. Currently, most formats use some form of dotted
142 notation for the test name, and this may become enshrined in the format in
143 the future. In general, the test names should be independent of the
144 software-under-test and refer to some known quantity, for example the
145 software under test. For example, 'CINT2006.403_gcc' is a meaningful test
146 name.
147
148 The test info dictionary is intended to hold information on the particular
149 permutation of the test that was run. This might include variables specific
150 to the software-under-test . This could include, for example, the compile
151 flags the test was built with, or the runtime parameters that were used. As
152 a general rule, if two test samples are meaningfully and directly
153 comparable, then the should have the same test name but different info
154 paramaters.
155
156 The report may include an arbitrary number of samples for each test for
157 situations where the same test is run multiple times to gather statistical
158 data.
138 values. The server automatically creates test database objects
139 whenever a new test name is seen.
140
141 Test names are intended to be a persistent, recognizable identifier
142 for what is being executed. Currently, most formats use some form of
143 dotted notation for the test name, and this may become enshrined in
144 the format in the future. In general, the test names should be
145 independent of the software-under-test and refer to some known
146 quantity, for example the software under test. For example,
147 'CINT2006.403_gcc' is a meaningful test name.
148
149 The test info dictionary is intended to hold information on the
150 particular permutation of the test that was run. This might include
151 variables specific to the software-under-test . This could include,
152 for example, the compile flags the test was built with, or the
153 runtime parameters that were used. As a general rule, if two test
154 samples are meaningfully and directly comparable, then they should
155 have the same test name but different info paramaters.
156
157 The report may include an arbitrary number of samples for each test
158 for situations where the same test is run multiple times to gather
159 statistical data.
159160 """
160161
161162 def __init__(self, name, data, info={}, conv_f=float):