llvm.org GIT mirror llvm / 39f6fac
[test-suite/doc] Add list of programs we might add. Add a list of benchmarks, applications and algorithms which are under discussion to be added to the test-suite. The initial list includes the the benchmarks mentioned at https://llvm.org/PR34216, missing SPEC benchmarks, some image processing algorithms and a few others. The bug tracker only allows adding to the discussion, not removing, commenting, adding details to individual benchmarks. The first proposal was to add these benchmark into the test-suite repository, but after a discussion, adding it to llvm/docs/Proposals seem more appropriate. One advantage is that llvm.org will have a browsable web page with these suggestions. Suggested-by: Hal Finkel Differential Revision: https://reviews.llvm.org/D46714 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@345074 91177308-0d34-0410-b5e6-96231b3b80d8 Michael Kruse 11 months ago
2 changed file(s) with 314 addition(s) and 0 deletion(s). Raw diff Collapse all Expand all
0 =====================
1 Test-Suite Extentions
2 =====================
3
4 .. contents::
5 :depth: 1
6 :local:
7
8 Abstract
9 ========
10
11 These are ideas for additional programs, benchmarks, applications and
12 algorithms that could be added to the LLVM Test-Suite.
13 The test-suite could be much larger than it is now, which would help us
14 detecting compiler errors (crashes, miscompiles) during development.
15
16 Most probably, the reason why the programs below have not been added to
17 the test-suite yet is that nobody has found time to do it. But there
18 might be other issues as well, such as
19
20 * Licensing (Support can still be added as external module,
21 like for the SPEC benchmarks)
22
23 * Language (in particular, there is no official LLVM frontend
24 for FORTRAN yet)
25
26 * Parallelism (currently, all programs in test-suite use
27 one thread only)
28
29 Benchmarks
30 ==========
31
32 SPEC CPU 2017
33 -------------
34 https://www.spec.org/cpu2017/
35
36 The following have not been included yet because they contain Fortran
37 code.
38
39 In case of cactuBSSN only a small portion is Fortran. The hosts's
40 Fortran compiler could be used for these parts.
41
42 Note that CMake's Ninja generator has difficulties with Fortran. See the
43 `CMake documentation `_
44 for details.
45
46 * 503.bwaves_r/603.bwaves_s
47 * 507.cactuBSSN_r
48 * 521.wrf_r/621.wrf_s
49 * 527.cam4_r/627.cam4_s
50 * 628.pop2_s
51 * 548.exchange2_r/648.exchange2_s
52 * 549.fotonik3d_r/649.fotonik3d_s
53 * 554.roms_r/654.roms_s
54
55 SPEC OMP2012
56 ------------
57 https://www.spec.org/omp2012/
58
59 * 350.md
60 * 351.bwaves
61 * 352.nab
62 * 357.bt331
63 * 358.botsalgn
64 * 359.botsspar
65 * 360.ilbdc
66 * 362.fma3d
67 * 363.swim
68 * 367.imagick
69 * 370.mgrid331
70 * 371.applu331
71 * 372.smithwa
72 * 376.kdtree
73
74 OpenCV
75 ------
76 https://opencv.org/
77
78 OpenMP 4.x SIMD Benchmarks
79 --------------------------
80 https://github.com/flwende/simd_benchmarks
81
82 PWM-benchmarking
83 ----------------
84 https://github.com/tbepler/PWM-benchmarking
85
86 SLAMBench
87 ---------
88 https://github.com/pamela-project/slambench
89
90 FireHose
91 --------
92 http://firehose.sandia.gov/
93
94 A Benchmark for the C/C++ Standard Library
95 ------------------------------------------
96 https://github.com/hiraditya/std-benchmark
97
98 OpenBenchmarking.org CPU / Processor Suite
99 ------------------------------------------
100 https://openbenchmarking.org/suite/pts/cpu
101
102 This is a subset of the
103 `Phoronix Test Suite `_
104 and is itself a collection of benchmark suites
105
106 Parboil Benchmarks
107 ------------------
108 http://impact.crhc.illinois.edu/parboil/parboil.aspx
109
110 MachSuite
111 ---------
112 https://breagen.github.io/MachSuite/
113
114 Rodinia
115 -------
116 http://lava.cs.virginia.edu/Rodinia/download_links.htm
117
118 Rodinia has already been partially included in
119 MultiSource/Benchmarks/Rodinia. Benchmarks still missing are:
120
121 * streamcluster
122 * particlefilter
123 * nw
124 * nn
125 * myocyte
126 * mummergpu
127 * lud
128 * leukocyte
129 * lavaMD
130 * kmeans
131 * hotspot3D
132 * heartwall
133 * cfd
134 * bfs
135 * b+tree
136
137 vecmathlib tests harness
138 ------------------------
139 https://bitbucket.org/eschnett/vecmathlib/wiki/Home
140
141 PARSEC
142 ------
143 http://parsec.cs.princeton.edu/
144
145 Graph500 reference implementations
146 ----------------------------------
147 https://github.com/graph500/graph500/tree/v2-spec
148
149 NAS Parallel Benchmarks
150 -----------------------
151 https://www.nas.nasa.gov/publications/npb.html
152
153 The official benchmark is written in Fortran, but an unofficial
154 C-translation is available as well:
155 https://github.com/benchmark-subsetting/NPB3.0-omp-C
156
157 DARPA HPCS SSCA#2 C/OpenMP reference implementation
158 ---------------------------------------------------
159 http://www.highproductivity.org/SSCABmks.htm
160
161 This web site does not exist any more, but there seems to be a copy of
162 some of the benchmarks
163 https://github.com/gtcasl/hpc-benchmarks/tree/master/SSCA2v2.2
164
165 Kokkos
166 ------
167 https://github.com/kokkos/kokkos-kernels/tree/master/perf_test
168 https://github.com/kokkos/kokkos/tree/master/benchmarks
169
170 PolyMage
171 --------
172 https://github.com/bondhugula/polymage-benchmarks
173
174 PolyBench
175 ---------
176 https://sourceforge.net/projects/polybench/
177
178 A modified version of Polybench 3.2 is already presented in
179 SingleSource/Benchmarks/Polybench. A newer version 4.2.1 is available.
180
181 High Performance Geometric Multigrid
182 ------------------------------------
183 https://crd.lbl.gov/departments/computer-science/PAR/research/hpgmg/
184
185 RAJA Performance Suite
186 ----------------------
187 https://github.com/LLNL/RAJAPerf
188
189 CORAL-2 Benchmarks
190 ------------------
191 https://asc.llnl.gov/coral-2-benchmarks/
192
193 Many of its programs have already been integreated in
194 MultiSource/Benchmarks/DOE-ProxyApps-C and
195 MultiSource/Benchmarks/DOE-ProxyApps-C++.
196
197 * Nekbone
198 * QMCPack
199 * LAMMPS
200 * Kripke
201 * Quicksilver
202 * PENNANT
203 * Big Data Analytic Suite
204 * Deep Learning Suite
205 * Stream
206 * Stride
207 * ML/DL micro-benchmark
208 * Pynamic
209 * ACME
210 * VPIC
211 * Laghos
212 * Parallel Integer Sort
213 * Havoq
214
215 NWChem
216 ------
217 http://www.nwchem-sw.org/index.php/Benchmarks
218
219 TVM
220 ----
221 https://github.com/dmlc/tvm/tree/master/apps/benchmark
222
223 HydroBench
224 ----------
225 https://github.com/HydroBench/Hydro
226
227 ParRes
228 ------
229 https://github.com/ParRes/Kernels/tree/master/Cxx11
230
231 Applications/Libraries
232 ======================
233
234 GnuPG
235 -----
236 https://gnupg.org/
237
238 Blitz++
239 -------
240 https://sourceforge.net/projects/blitz/
241
242 FFmpeg
243 ------
244 https://ffmpeg.org/
245
246 FreePOOMA
247 ---------
248 http://www.nongnu.org/freepooma/
249
250 FTensors
251 --------
252 http://www.wlandry.net/Projects/FTensor
253
254 Generic Algorithms
255 ==================
256
257 Image processing
258 ----------------
259
260 Resampling
261 ``````````
262
263 * Bilinear
264 * Bicubic
265 * Lanczos
266
267 Dither
268 ``````
269
270 * Threshold
271 * Random
272 * Halftone
273 * Bayer
274 * Floyd-Steinberg
275 * Jarvis
276 * Stucki
277 * Burkes
278 * Sierra
279 * Atkinson
280 * Gradient-based
281
282 Feature detection
283 `````````````````
284
285 * Harris
286 * Histogram of Oriented Gradients
287
288 Color conversion
289 ````````````````
290
291 * RGB to grayscale
292 * HSL to RGB
293
294 Graph
295 -----
296
297 Search Algorithms
298 `````````````````
299
300 * Breadth-First-Search
301 * Depth-First-Search
302 * Dijkstra's algorithm
303 * A-Star
304
305 Spanning Tree
306 `````````````
307
308 * Kruskal's algorithm
309 * Prim's algorithm
558558
559559 CodeOfConduct
560560 Proposals/GitHubMove
561 Proposals/TestSuite
561562 Proposals/VectorizationPlan
562563
563564 :doc:`CodeOfConduct`
567568 :doc:`Proposals/GitHubMove`
568569 Proposal to move from SVN/Git to GitHub.
569570
571 :doc:`Proposals/TestSuite`
572 Proposals for additional benchmarks/programs for llvm's test-suite.
573
570574 :doc:`Proposals/VectorizationPlan`
571575 Proposal to model the process and upgrade the infrastructure of LLVM's Loop Vectorizer.
572576