# DataRaceBench 1.0.0 DataRaceBench is a benchmark suite designed to systematically and quantitatively evaluate the effectiveness of data race detection tools. It includes a set of microbenchmarks with and without data races. Parallelism is represented by OpenMP directives. OpenMP is a popular parallel programming model for multi-threaded applications. Note that if you are using gcc for compiling the microbenchmarks, at least version 4.9 is required to have support for all used OpenMP directives. DataRaceBench also comes with an evaluation script (check-data-races.sh). The script can be used to evaluate the tools Helgrind, Archer, Thread Sanitizer, and Intel Inspector. In addition a parameterized test harness (scripts/test-harness.sh) is available which allows to provide a number of different parameters for the evaluation. The test harness is used by the evaluation script with some pre-defined values. ## Microbenchmark property labels (P-Labels) P-Label | Meaning (microbenchmarks with data races) | P-Label | Meaning (microbenchmarks without data races) ------|-----------------------------------|------|------------------------------ Y1 | Unresolvable dependences | N1 | Embarrassingly parallel Y2 | Missing data sharing clauses | N2 | Use of data sharing clauses Y3 | Missing synchronization | N3 | Use of synchronization Y4 | SIMD data races | N4 | Use of SIMD directives Y5 | Accelerator data races | N5 | Use of accelerator directives Y6 | Undefined behavior | N6 | Use of special language features Y7 | Numerical kernel data races | N7 | Numerical kernels ## Microbenchmarks with known data races (some have a varying length version) ID | Microbenchmark |P-Label| Description | Source ----------|-----------------------------------------------|-------|------------------------------------------------------------------------------|---------- 1|2 | antidep1-(orig|var)-yes.c |Y1 | Anti-dependence within a single loop | AutoPar 3|4 | antidep2-(orig|var)-yes.c |Y1 | Anti-dependence within a two-level loop nest | AutoPar 5 | indirectaccess1-orig-yes.c |Y7 | Indirect access with overlapped index array elements | LLNL App 6 | indirectaccess2-orig-yes.c |Y7 | Overlapping index array elements when 36 or more threads are used | LLNL App 7 | indirectaccess3-orig-yes.c |Y7 | Overlapping index array elements when 60 or more threads are used | LLNL App 8 | indirectaccess4-orig-yes.c |Y7 | Overlapping index array elements when 180 or more threads are used | LLNL App 9|10 | lastprivatemissing-(orig|var)-yes.c |Y2 | Data race due to a missing `lastprivate()` clause | AutoPar 11|12| minusminus-(orig|var)-yes.c |Y3 | Unprotected decrement operation `--` | AutoPar 13 | nowait-orig-yes.c |Y3 | Missing barrier due to a wrongfully used nowait | AutoPar 14|15| outofbounds-(orig|var)-yes.c |Y6 | Out of bound access of the 2nd dimension of array | AutoPar 16|17| outputdep-(orig|var)-yes.c |Y1 | Output dependence and true dependence within a loop | AutoPar 18|19| plusplus-(orig|var)-yes.c |Y1 | increment operation `++` on array index variable | AutoPar 20|21| privatemissing-(orig|var)-yes.c |Y2 | Missing `private()` for a temp variable | AutoPar 22|23| reductionmissing-(orig|var)-yes.c |Y2 | Missing `reduction()` for a variable | AutoPar 24 | sections1-orig-yes.c |Y3 | Unprotected data writes in parallel sections | New 25|26| simdtruedep-(orig|var)-yes.c |Y1,Y4 | SIMD instruction level data races | New 27 | targetparallelfor-orig-yes.c |Y1,Y5 | Data races in loops offloaded to accelerators | New 28 | taskdependmissing-orig-yes.c |Y3 | Unprotected data writes in two tasks | New 29|30| truedep1-(orig|var)-yes.c |Y1 | True data dependence among multiple array elements within a single level loop| AutoPar 31|32| truedepfirstdimension-(orig|var)-yes.c |Y1 | True data dependence of first dimension for a 2-D array accesses | AutoPar 33|34| truedeplinear-(orig|var)-yes.c |Y1 | Linear equation as array subscript | AutoPar 35|36| truedepscalar-(orig|var)-yes.c |Y1 | True data dependence due to scalar | AutoPar 37|38| truedepseconddimension-(orig|var)-yes.c |Y1 | True data dependence on 2nd dimension of a 2-D array accesses | AutoPar 39|40| truedepsingleelement-(orig|var)-yes.c |Y1 | True data dependence due to a single array element | AutoPar ## Microbenchmarks without known data races ID| Microbenchmark |P-Label| Description | Source --|-----------------------------------|-------|--------------------------------------------------------------------------------------|------------ 41| 3mm-parallel-no.c |N2 | 3-step matrix-matrix multiplication, non-optimized version | Polyhedral 42| 3mm-tile-no.c |N2,N4 | 3-step matrix-matrix multiplication, with tiling and nested SIMD | Polyhedral 43| adi-parallel-no.c |N2 | Alternating Direction Implicit solver, non-optimized version | Polyhedral 44| adi-tile-no.c |N2,N4 | Alternating Direction Implicit solver, with tiling and nested SIMD | Polyhedral 45| doall1-orig-no.c |N1 | Classic DOAll loop operating on a one dimensional array | AutoPar 46| doall2-orig-no.c |N1 | Classic DOAll loop operating on a two dimensional array | AutoPar 47| doallchar-orig-no.c |N1 | Classic DOALL loop operating on a character array | New 48| firstprivate-orig-no.c |N2 | Example use of firstprivate | AutoPar 49| functionparameter-orig-no.c |N6 | Arrays passed as function parameters | LLNL App 50| fprintf-orig-no.c |N6 | Use of `fprintf()` | New 51| getthreadnum-orig-no.c |N2 | single thread execution using `if (omp_get_thread_num()==0)` | New 52| indirectaccesssharebase-orig-no.c |N7 | Indirect array accesses using index arrays without overlapping | LLNL App 53| inneronly1-orig-no.c |N1 | Two-level nested loops, inner level is parallelizable. True dependence on outer level| AutoPar 54| inneronly2-orig-no.c |N1 | Two-level nested loops, inner level is parallelizable. Anti dependence on outer level| AutoPar 55| jacobi2d-parallel-no.c |N7 | Jacobi with array copying, no reduction, non-optimized version | Polyhedral 56| jacobi2d-tile-no.c |N4,N7 | Jacobi with array copying, no reduction, with tiling and nested SIMD | Polyhedral 57| jacobiinitialize-orig-no.c |N7 | The array initialization parallel loop in Jacobi | AutoPar 58| jacobikernel-orig-no.c |N7 | Parallel Jacobi stencil computation kernel with array copying and reduction | AutoPar 59| lastprivate-orig-no.c |N2 | Example use of lastprivate | AutoPar 60| matrixmultiply-orig-no.c |N7 | Classic i-k-j order matrix multiplication using OpenMP | AutoPar 61| matrixvector1-orig-no.c |N7 | Matrix-vector multiplication parallelized at the outer level loop | AutoPar 62| matrixvector2-orig-no.c |N7 | Matrix-vector multiplication parallelized at the inner level loop with reduction | AutoPar 63| outeronly1-orig-no.c |N2 | Two-level nested loops, outer level is parallelizable. True dependence on inner level| AutoPar 64| outeronly2-orig-no.c |N2 | Two-level nested loops, outer level is parallelizable. Anti dependence on inner level| AutoPar 65| pireduction-orig-no.c |N7 | PI calculation using reduction | AutoPar 66| pointernoaliasing-orig-no.c |N6 | Pointers assigned by different malloc calls, without aliasing | LLNL App 67| restrictpointer1-orig-no.c |N6 | C99 restrict pointers used for array initialization, no aliasing | LLNL App 68| restrictpointer2-orig-no.c |N6 | C99 restrict pointers used for array computation, no aliasing | LLNL App 69| sectionslock1-orig-no.c |N3 | OpenMP parallel sections with a lock to protect shared data writes | New 70| simd1-orig-no.c |N1,N4 | OpenMP SIMD directive to indicate vectorization of a loop | New 71| targetparallelfor-orig-no.c |N1,N5 | data races in loops offloaded to accelerators | New 72| taskdep1-orig-no.c |N3 | OpenMP task with depend clauses to avoid data races | New ## Authors DataRaceBench was created by Chunhua Liao, Pei-Hung Lin, Joshua Asplund, Markus Schordan, and Ian Karlin. ## Release DataRaceBench is released under a BSD license. For more details see the file LICENSE.txt. The microbenchmarks marked 'Polyhedral' in above table were generated as optimization variants of benchmarks from the PolyOpt benchmark suite. For those benchmarks see the license file LICENSE.OSU.txt. `LLNL-CODE-732144`