llvm.org GIT mirror llvm / da51c4c
Formatting changes. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@42223 91177308-0d34-0410-b5e6-96231b3b80d8 Bill Wendling 13 years ago
1 changed file(s) with 150 addition(s) and 54 deletion(s). Raw diff Collapse all Expand all
115115 Alternatively, you can configure the test-suite module manually.

116116

To run all of the simple tests in LLVM using DejaGNU, use the master Makefile

117117 in the llvm/test directory:

118
119
118120

                  
                
119121 % gmake -C llvm/test
120122
121 or>
123 >
124
125

or

126
127
122128

                  
                
123129 % gmake check
124130
131
125132
126133

To run only a subdirectory of tests in llvm/test using DejaGNU (ie.

127134 Regression/Transforms), just set the TESTSUITE variable to the path of the
128135 subdirectory (relative to llvm/test):

136
137
129138

                  
                
130139 % gmake -C llvm/test TESTSUITE=Regression/Transforms
131140
141
132142
133143

Note: If you are running the tests with objdir != subdir, you

134144 must have run the complete testsuite before you can specify a
137147

To run the comprehensive test suite (tests that compile and execute whole

138148 programs), run the llvm-test tests:

139149
150
140151

                  
                
141152 % cd llvm/projects
142153 % svn co http://llvm.org/svn/llvm-project/test-suite/trunk llvm-test
144155 % ./configure --with-llvmsrc=$LLVM_SRC_ROOT --with-llvmobj=$LLVM_OBJ_ROOT
145156 % gmake
146157
158
147159
148160
149161
323335

324336
325337

Below is an example of legal RUN lines in a .ll file:

326

                  
                
327 ; RUN: llvm-as < %s | llvm-dis > %t1
328 ; RUN: llvm-dis < %s.bc-13 > %t2
329 ; RUN: diff %t1 %t2
330
338
339
340

                  
                
341 ; RUN: llvm-as < %s | llvm-dis > %t1
342 ; RUN: llvm-dis < %s.bc-13 > %t2
343 ; RUN: diff %t1 %t2
344
345
331346
332347

As with a Unix shell, the RUN: lines permit pipelines and I/O redirection

333348 to be used. However, the usage is slightly different than for Bash. To check
350365

There are some quoting rules that you must pay attention to when writing

351366 your RUN lines. In general nothing needs to be quoted. Tcl won't strip off any
352367 ' or " so they will get passed to the invoked program. For example:

353

                  
                
354 ... | grep 'find this string'
355
368
369
370

                  
                
371 ... | grep 'find this string'
372
373
374
356375

This will fail because the ' characters are passed to grep. This would

357376 instruction grep to look for 'find in the files this and
358377 string'. To avoid this use curly braces to tell Tcl that it should
359378 treat everything enclosed as one value. So our example would become:

360

                  
                
361 ... | grep {find this string}
362
379
380
381

                  
                
382 ... | grep {find this string}
383
384
385
363386

Additionally, the characters [ and ] are treated

364387 specially by Tcl. They tell Tcl to interpret the content as a command to
365388 execute. Since these characters are often used in regular expressions this can
366389 have disastrous results and cause the entire test run in a directory to fail.
367390 For example, a common idiom is to look for some basicblock number:

368

                  
                
369 ... | grep bb[2-8]
370
391
392
393

                  
                
394 ... | grep bb[2-8]
395
396
397
371398

This, however, will cause Tcl to fail because its going to try to execute

372399 a program named "2-8". Instead, what you want is this:

373

                  
                
374 ... | grep {bb\[2-8\]}
375
400
401
402

                  
                
403 ... | grep {bb\[2-8\]}
404
405
406
376407

Finally, if you need to pass the \ character down to a program,

377408 then it must be doubled. This is another Tcl special character. So, suppose
378409 you had:
379

                  
                
380 ... | grep 'i32\*'
381
410
411
412

                  
                
413 ... | grep 'i32\*'
414
415
416
382417

This will fail to match what you want (a pointer to i32). First, the

383418 ' do not get stripped off. Second, the \ gets stripped off
384419 by Tcl so what grep sees is: 'i32*'. That's not likely to match
385420 anything. To resolve this you must use \\ and the {}, like
386421 this:

387

                  
                
388 ... | grep {i32\\*}
389
422
423
424

                  
                
425 ... | grep {i32\\*}
426
427
390428
391429
392430
403441

404442 Here are the available variable names. The alternate syntax is listed in
405443 parentheses.

444
406445
407446
$test (%s)
408447
The full path to the test case's source. This is suitable for passing
409448 on the command line as the input to an llvm tool.
449
410450
$srcdir
411451
The source directory from where the "make check" was run.
452
412453
objdir
413454
The object directory that corresponds to the $srcdir.
455
414456
subdir
415457
A partial path from the test directory that contains the
416458 sub-directory that contains the test source being executed.
459
417460
srcroot
418461
The root directory of the LLVM src tree.
462
419463
objroot
420464
The root directory of the LLVM object tree. This could be the same
421465 as the srcroot.
466
422467
path
423468
The path to the directory that contains the test case source. This is
424469 for locating any supporting files that are not generated by the test, but
425470 used by the test.
471
426472
tmp
427473
The path to a temporary file name that could be used for this test case.
428474 The file name won't conflict with other test cases. You can append to it if
429475 you need multiple temporaries. This is useful as the destination of some
430476 redirected output.
477
431478
llvmlibsdir (%llvmlibsdir)
432479
The directory where the LLVM libraries are located.
480
433481
target_triplet (%target_triplet)
434482
The target triplet that corresponds to the current host machine (the one
435483 running the test cases). This should probably be called "host".
484
436485
prcontext (%prcontext)
437486
Path to the prcontext tcl script that prints some context around a
438487 line that matches a pattern. This isn't strictly necessary as the test suite
440489 the prcontext script is located. Note that this script is similar to
441490 grep -C but you should use the prcontext script because
442491 not all platforms support grep -C.
492
443493
llvmgcc (%llvmgcc)
444494
The full path to the llvm-gcc executable as specified in the
445495 configured LLVM environment
496
446497
llvmgxx (%llvmgxx)
447498
The full path to the llvm-gxx executable as specified in the
448499 configured LLVM environment
500
449501
llvmgcc_version (%llvmgcc_version)
450502
The full version number of the llvm-gcc executable.
503
451504
llvmgccmajvers (%llvmgccmajvers)
452505
The major version number of the llvm-gcc executable.
506
453507
gccpath
454508
The full path to the C compiler used to build LLVM. Note that
455509 this might not be gcc.
510
456511
gxxpath
457512
The full path to the C++ compiler used to build LLVM. Note that
458513 this might not be g++.
514
459515
compile_c (%compile_c)
460516
The full command line used to compile LLVM C source code. This has all
461517 the configured -I, -D and optimization options.
518
462519
compile_cxx (%compile_cxx)
463520
The full command used to compile LLVM C++ source code. This has
464521 all the configured -I, -D and optimization options.
522
465523
link (%link)
466524
This full link command used to link LLVM executables. This has all the
467525 configured -I, -L and -l options.
526
468527
shlibext (%shlibext)
469528
The suffix for the host platforms share library (dll) files. This
470529 includes the period as the first character.
490549 non-zero result will cause the test to fail. This script overcomes that
491550 issue and nicely documents that the test case is purposefully ignoring the
492551 result code of the tool
552
493553
not
494554
This script runs its arguments and then inverts the result code from
495555 it. Zero result codes become 1. Non-zero result codes become 0. This is
510570 succeed. To XFAIL everywhere just specify XFAIL: *. When matching
511571 the llvm-gcc version, you can specify the major (e.g. 3) or full version
512572 (i.e. 3.4) number. Here is an example of an XFAIL line:

513

                  
                
514 ; XFAIL: darwin,sun,llvmgcc4
515
573
574
575

                  
                
576 ; XFAIL: darwin,sun,llvmgcc4
577
578
516579
517580

To make the output more useful, the llvm_runtest function wil

518581 scan the lines of the test case for ones that contain a pattern that matches
572635 uses the default value
573636 /home/vadve/shared/benchmarks/speccpu2000/benchspec.
574637

638
575639
--enable-spec95
576640
--enable-spec95=<directory>
577641
578642 Enable the use of SPEC95 when testing LLVM. It is similar to the
579643 --enable-spec2000 option.
580644

645
581646
--enable-povray
582647
--enable-povray=<directory>
583648
597662 are not executed inside of the LLVM source tree. This is because the
598663 test suite creates temporary files during execution.

599664
600

The master Makefile in llvm/test is capable of running only the DejaGNU

601 driven tests. By default, it will run all of these tests.

665

The master Makefile in llvm/test is capable of running only the

666 DejaGNU driven tests. By default, it will run all of these tests.

602667
603668

To run only the DejaGNU driven tests, run gmake at the

604669 command line in llvm/test. To run a specific directory of tests, use
605 the TESTSUITE variable.
670 the TESTSUITE variable.
606671

607672
608673

For example, to run the Regression tests, type

612677 llvm/test/Regression. You must use DejaGNU from the llvm/test
613678 directory to run them.

614679
615

To run the llvm-test suite, you need to use the following steps:

616 </p>
680 <p>To run the llvm-test suite, you need to use the following steps:</p>
681
617682
618
  • cd into the llvm/projects directory
  • 619
  • check out the test-suite module with:
  • 620 svn co http://llvm.org/svn/llvm-project/test-suite/trunk llvm-test
    621 This will get the test suite into llvm/projects/llvm-test
    622
  • configure the test suite. You can do this one of two ways:
  • 683
  • cd into the llvm/projects directory
  • 684
    685
  • Check out the test-suite module with:

  • 686
    687
    688
    
                      
                    
    689 % svn co http://llvm.org/svn/llvm-project/test-suite/trunk llvm-test
    690
    691
    692
    693

    This will get the test suite into llvm/projects/llvm-test

    694
    695
  • Configure the test suite. You can do this one of two ways:

  • 696
    623697
    624
  • Use the regular llvm configure:
  • 625 cd $LLVM_OBJ_ROOT ; $LLVM_SRC_ROOT/configure
    626 This will ensure that the projects/llvm-test directory is also
    627 properly configured.
    628
  • Use the configure script found in the llvm-test source
  • 629 directory:
    630 $LLVM_SRC_ROOT/projects/llvm-test/configure
    631 --with-llvmsrc=$LLVM_SRC_ROOT --with-llvmobj=$LLVM_OBJ_ROOT>
    698
  • Use the regular llvm configure:

    >
  • 699
    700
    701
    
                      
                    
    702 % cd $LLVM_OBJ_ROOT ; $LLVM_SRC_ROOT/configure
    703
    704
    705
    706

    This will ensure that the projects/llvm-test directory is

    707 also properly configured.

    708
    709
  • Use the configure script found in the llvm-test

  • 710 source directory:

    711
    712
    713
    
                      
                    
    714 % $LLVM_SRC_ROOT/projects/llvm-test/configure \
    715 --with-llvmsrc=$LLVM_SRC_ROOT \
    716 --with-llvmobj=$LLVM_OBJ_ROOT
    717
    718
    632719
    633720
    634
  • gmake
  • 721
  • gmake
  • 635722
    636723

    Note that the second and third steps only need to be done once. After you

    637724 have the suite checked out and configured, you don't need to do it again (unless
    638725 the test code or configure script changes).

    639726
    640727

    To make a specialized test (use one of the

    641 llvm-test/TEST.<type>.Makefiles), just run:
    642 gmake TEST=<type> test
    For example, you could run the
    643 nightly tester tests using the following commands:

    644
    645
    
                      
                    
    646 % cd llvm/projects/llvm-test
    647 % gmake TEST=nightly test
    648 </pre>
    728 <tt>llvm-test/TEST.<type>.Makefiles), just run:>
    729
    730
    731
    
                      
                    
    732 % gmake TEST=<type> test
    733
    734
    735
    736

    For example, you could run the nightly tester tests using the following

    737 commands:

    738
    739
    740
    
                      
                    
    741 % cd llvm/projects/llvm-test
    742 % gmake TEST=nightly test
    743
    744
    649745
    650746

    Regardless of which test you're running, the results are printed on standard

    651747 output and standard error. You can redirect these results to a file if you