| 1 | # DataRaceBench 1.0.0
|
|---|
| 2 |
|
|---|
| 3 | DataRaceBench is a benchmark suite designed to systematically and
|
|---|
| 4 | quantitatively evaluate the effectiveness of data race detection
|
|---|
| 5 | tools. It includes a set of microbenchmarks with and without data
|
|---|
| 6 | races. Parallelism is represented by OpenMP directives. OpenMP is a
|
|---|
| 7 | popular parallel programming model for multi-threaded applications.
|
|---|
| 8 |
|
|---|
| 9 | Note that if you are using gcc for compiling the microbenchmarks, at
|
|---|
| 10 | least version 4.9 is required to have support for all used OpenMP
|
|---|
| 11 | directives.
|
|---|
| 12 |
|
|---|
| 13 | DataRaceBench also comes with an evaluation script
|
|---|
| 14 | (check-data-races.sh). The script can be used to evaluate the tools
|
|---|
| 15 | Helgrind, Archer, Thread Sanitizer, and Intel Inspector. In addition a
|
|---|
| 16 | parameterized test harness (scripts/test-harness.sh) is available
|
|---|
| 17 | which allows to provide a number of different parameters for the
|
|---|
| 18 | evaluation. The test harness is used by the evaluation script with
|
|---|
| 19 | some pre-defined values.
|
|---|
| 20 |
|
|---|
| 21 | ## Microbenchmark property labels (P-Labels)
|
|---|
| 22 |
|
|---|
| 23 | P-Label | Meaning (microbenchmarks with data races) | P-Label | Meaning (microbenchmarks without data races)
|
|---|
| 24 | ------|-----------------------------------|------|------------------------------
|
|---|
| 25 | Y1 | Unresolvable dependences | N1 | Embarrassingly parallel
|
|---|
| 26 | Y2 | Missing data sharing clauses | N2 | Use of data sharing clauses
|
|---|
| 27 | Y3 | Missing synchronization | N3 | Use of synchronization
|
|---|
| 28 | Y4 | SIMD data races | N4 | Use of SIMD directives
|
|---|
| 29 | Y5 | Accelerator data races | N5 | Use of accelerator directives
|
|---|
| 30 | Y6 | Undefined behavior | N6 | Use of special language features
|
|---|
| 31 | Y7 | Numerical kernel data races | N7 | Numerical kernels
|
|---|
| 32 |
|
|---|
| 33 |
|
|---|
| 34 | ## Microbenchmarks with known data races (some have a varying length version)
|
|---|
| 35 |
|
|---|
| 36 | ID | Microbenchmark |P-Label| Description | Source
|
|---|
| 37 | ----------|-----------------------------------------------|-------|------------------------------------------------------------------------------|----------
|
|---|
| 38 | 1|2 | antidep1-(orig|var)-yes.c |Y1 | Anti-dependence within a single loop | AutoPar
|
|---|
| 39 | 3|4 | antidep2-(orig|var)-yes.c |Y1 | Anti-dependence within a two-level loop nest | AutoPar
|
|---|
| 40 | 5 | indirectaccess1-orig-yes.c |Y7 | Indirect access with overlapped index array elements | LLNL App
|
|---|
| 41 | 6 | indirectaccess2-orig-yes.c |Y7 | Overlapping index array elements when 36 or more threads are used | LLNL App
|
|---|
| 42 | 7 | indirectaccess3-orig-yes.c |Y7 | Overlapping index array elements when 60 or more threads are used | LLNL App
|
|---|
| 43 | 8 | indirectaccess4-orig-yes.c |Y7 | Overlapping index array elements when 180 or more threads are used | LLNL App
|
|---|
| 44 | 9|10 | lastprivatemissing-(orig|var)-yes.c |Y2 | Data race due to a missing `lastprivate()` clause | AutoPar
|
|---|
| 45 | 11|12| minusminus-(orig|var)-yes.c |Y3 | Unprotected decrement operation `--` | AutoPar
|
|---|
| 46 | 13 | nowait-orig-yes.c |Y3 | Missing barrier due to a wrongfully used nowait | AutoPar
|
|---|
| 47 | 14|15| outofbounds-(orig|var)-yes.c |Y6 | Out of bound access of the 2nd dimension of array | AutoPar
|
|---|
| 48 | 16|17| outputdep-(orig|var)-yes.c |Y1 | Output dependence and true dependence within a loop | AutoPar
|
|---|
| 49 | 18|19| plusplus-(orig|var)-yes.c |Y1 | increment operation `++` on array index variable | AutoPar
|
|---|
| 50 | 20|21| privatemissing-(orig|var)-yes.c |Y2 | Missing `private()` for a temp variable | AutoPar
|
|---|
| 51 | 22|23| reductionmissing-(orig|var)-yes.c |Y2 | Missing `reduction()` for a variable | AutoPar
|
|---|
| 52 | 24 | sections1-orig-yes.c |Y3 | Unprotected data writes in parallel sections | New
|
|---|
| 53 | 25|26| simdtruedep-(orig|var)-yes.c |Y1,Y4 | SIMD instruction level data races | New
|
|---|
| 54 | 27 | targetparallelfor-orig-yes.c |Y1,Y5 | Data races in loops offloaded to accelerators | New
|
|---|
| 55 | 28 | taskdependmissing-orig-yes.c |Y3 | Unprotected data writes in two tasks | New
|
|---|
| 56 | 29|30| truedep1-(orig|var)-yes.c |Y1 | True data dependence among multiple array elements within a single level loop| AutoPar
|
|---|
| 57 | 31|32| truedepfirstdimension-(orig|var)-yes.c |Y1 | True data dependence of first dimension for a 2-D array accesses | AutoPar
|
|---|
| 58 | 33|34| truedeplinear-(orig|var)-yes.c |Y1 | Linear equation as array subscript | AutoPar
|
|---|
| 59 | 35|36| truedepscalar-(orig|var)-yes.c |Y1 | True data dependence due to scalar | AutoPar
|
|---|
| 60 | 37|38| truedepseconddimension-(orig|var)-yes.c |Y1 | True data dependence on 2nd dimension of a 2-D array accesses | AutoPar
|
|---|
| 61 | 39|40| truedepsingleelement-(orig|var)-yes.c |Y1 | True data dependence due to a single array element | AutoPar
|
|---|
| 62 |
|
|---|
| 63 |
|
|---|
| 64 | ## Microbenchmarks without known data races
|
|---|
| 65 |
|
|---|
| 66 | ID| Microbenchmark |P-Label| Description | Source
|
|---|
| 67 | --|-----------------------------------|-------|--------------------------------------------------------------------------------------|------------
|
|---|
| 68 | 41| 3mm-parallel-no.c |N2 | 3-step matrix-matrix multiplication, non-optimized version | Polyhedral
|
|---|
| 69 | 42| 3mm-tile-no.c |N2,N4 | 3-step matrix-matrix multiplication, with tiling and nested SIMD | Polyhedral
|
|---|
| 70 | 43| adi-parallel-no.c |N2 | Alternating Direction Implicit solver, non-optimized version | Polyhedral
|
|---|
| 71 | 44| adi-tile-no.c |N2,N4 | Alternating Direction Implicit solver, with tiling and nested SIMD | Polyhedral
|
|---|
| 72 | 45| doall1-orig-no.c |N1 | Classic DOAll loop operating on a one dimensional array | AutoPar
|
|---|
| 73 | 46| doall2-orig-no.c |N1 | Classic DOAll loop operating on a two dimensional array | AutoPar
|
|---|
| 74 | 47| doallchar-orig-no.c |N1 | Classic DOALL loop operating on a character array | New
|
|---|
| 75 | 48| firstprivate-orig-no.c |N2 | Example use of firstprivate | AutoPar
|
|---|
| 76 | 49| functionparameter-orig-no.c |N6 | Arrays passed as function parameters | LLNL App
|
|---|
| 77 | 50| fprintf-orig-no.c |N6 | Use of `fprintf()` | New
|
|---|
| 78 | 51| getthreadnum-orig-no.c |N2 | single thread execution using `if (omp_get_thread_num()==0)` | New
|
|---|
| 79 | 52| indirectaccesssharebase-orig-no.c |N7 | Indirect array accesses using index arrays without overlapping | LLNL App
|
|---|
| 80 | 53| inneronly1-orig-no.c |N1 | Two-level nested loops, inner level is parallelizable. True dependence on outer level| AutoPar
|
|---|
| 81 | 54| inneronly2-orig-no.c |N1 | Two-level nested loops, inner level is parallelizable. Anti dependence on outer level| AutoPar
|
|---|
| 82 | 55| jacobi2d-parallel-no.c |N7 | Jacobi with array copying, no reduction, non-optimized version | Polyhedral
|
|---|
| 83 | 56| jacobi2d-tile-no.c |N4,N7 | Jacobi with array copying, no reduction, with tiling and nested SIMD | Polyhedral
|
|---|
| 84 | 57| jacobiinitialize-orig-no.c |N7 | The array initialization parallel loop in Jacobi | AutoPar
|
|---|
| 85 | 58| jacobikernel-orig-no.c |N7 | Parallel Jacobi stencil computation kernel with array copying and reduction | AutoPar
|
|---|
| 86 | 59| lastprivate-orig-no.c |N2 | Example use of lastprivate | AutoPar
|
|---|
| 87 | 60| matrixmultiply-orig-no.c |N7 | Classic i-k-j order matrix multiplication using OpenMP | AutoPar
|
|---|
| 88 | 61| matrixvector1-orig-no.c |N7 | Matrix-vector multiplication parallelized at the outer level loop | AutoPar
|
|---|
| 89 | 62| matrixvector2-orig-no.c |N7 | Matrix-vector multiplication parallelized at the inner level loop with reduction | AutoPar
|
|---|
| 90 | 63| outeronly1-orig-no.c |N2 | Two-level nested loops, outer level is parallelizable. True dependence on inner level| AutoPar
|
|---|
| 91 | 64| outeronly2-orig-no.c |N2 | Two-level nested loops, outer level is parallelizable. Anti dependence on inner level| AutoPar
|
|---|
| 92 | 65| pireduction-orig-no.c |N7 | PI calculation using reduction | AutoPar
|
|---|
| 93 | 66| pointernoaliasing-orig-no.c |N6 | Pointers assigned by different malloc calls, without aliasing | LLNL App
|
|---|
| 94 | 67| restrictpointer1-orig-no.c |N6 | C99 restrict pointers used for array initialization, no aliasing | LLNL App
|
|---|
| 95 | 68| restrictpointer2-orig-no.c |N6 | C99 restrict pointers used for array computation, no aliasing | LLNL App
|
|---|
| 96 | 69| sectionslock1-orig-no.c |N3 | OpenMP parallel sections with a lock to protect shared data writes | New
|
|---|
| 97 | 70| simd1-orig-no.c |N1,N4 | OpenMP SIMD directive to indicate vectorization of a loop | New
|
|---|
| 98 | 71| targetparallelfor-orig-no.c |N1,N5 | data races in loops offloaded to accelerators | New
|
|---|
| 99 | 72| taskdep1-orig-no.c |N3 | OpenMP task with depend clauses to avoid data races | New
|
|---|
| 100 |
|
|---|
| 101 | ## Authors
|
|---|
| 102 |
|
|---|
| 103 | DataRaceBench was created by Chunhua Liao, Pei-Hung Lin, Joshua Asplund, Markus Schordan, and Ian Karlin.
|
|---|
| 104 |
|
|---|
| 105 | ## Release
|
|---|
| 106 |
|
|---|
| 107 | DataRaceBench is released under a BSD license. For more details see
|
|---|
| 108 | the file LICENSE.txt. The microbenchmarks marked 'Polyhedral' in above
|
|---|
| 109 | table were generated as optimization variants of benchmarks from the
|
|---|
| 110 | PolyOpt benchmark suite. For those benchmarks see the license file
|
|---|
| 111 | LICENSE.OSU.txt.
|
|---|
| 112 |
|
|---|
| 113 | `LLNL-CODE-732144`
|
|---|
