Results from Diehard tests on a binary file with 230=1073741824 Bytes generated by csprngAlvo
Algorithm. x0=0; b0=1 and c=0.5
NOTE: Most of the tests in DIEHARD
return a p-value, which
should be uniform on [0,1) if the input
file contains truly
independent random bits. Those p-values are obtained by
p=F(X), where F is the assumed
distribution of the sample
random variable X---often normal. But
that assumed F is just
an asymptotic approximation, for which
the fit will be worst
in the tails. Thus you should not be
surprised with
occasional p-values near 0 or 1, such as
.0012 or .9983.
When a bit stream really FAILS BIG, you
will get p's of 0 or
1 to six or more places. By all means, do not, as a
Statistician might, think that a p <
.025 or p> .975 means
that the RNG has "failed the test
at the .05 level". Such
p's happen among the hundreds that
DIEHARD produces, even
with good RNG's. So keep in mind that " p
happens".
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: This is the BIRTHDAY SPACINGS
TEST ::
:: Choose m birthdays in a year of n
days. List the spacings ::
:: between the birthdays. If j is the number of values that ::
:: occur more than once in that list, then
j is asymptotically ::
:: Poisson distributed with mean
m^3/(4n). Experience shows n ::
:: must be quite large, say n>=2^18,
for comparing the results ::
:: to the Poisson distribution with that
mean. This test uses ::
:: n=2^24 and m=2^9, so that the underlying distribution for
j ::
:: is taken to be Poisson with
lambda=2^27/(2^26)=2. A sample ::
:: of 500 j's is taken, and a chi-square
goodness of fit test ::
:: provides a p value. The first test uses bits 1-24 (counting ::
:: from the left) from integers in the
specified file. ::
::
Then the file is closed and reopened. Next, bits 2-25 are ::
:: used to provide birthdays, then 3-26
and so on to bits 9-32. ::
:: Each set of bits provides a p-value,
and the nine p-values ::
:: provide a sample for a KSTEST. ::
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
BIRTHDAY SPACINGS TEST, M= 512 N=2**24
LAMBDA= 2.0000
Results for rnd.txt
For a sample of size
500: mean
rnd.txt using bits 1 to 24
1.932
duplicate number number
spacings
observed expected
0 79. 67.668
1 123.
135.335
2 143. 135.335
3 91. 90.224
4 46. 45.112
5 11. 18.045
6 to INF 7. 8.282
Chisquare with
6 d.o.f. = 6.43 p-value= .623122
:::::::::::::::::::::::::::::::::::::::::
For a sample of size
500: mean
rnd.txt using bits 2 to 25
2.036
duplicate number number
spacings
observed expected
0 64. 67.668
1 128. 135.335
2 141. 135.335
3 97. 90.224
4 44. 45.112
5 18. 18.045
6 to INF 8. 8.282
Chisquare with
6 d.o.f. = 1.38 p-value= .032909
:::::::::::::::::::::::::::::::::::::::::
For a sample of size
500: mean
rnd.txt using bits 3 to 26
1.942
duplicate number number
spacings
observed expected
0
68. 67.668
1 140. 135.335
2 147. 135.335
3 83. 90.224
4 34. 45.112
5 20. 18.045
6 to INF 8. 8.282
Chisquare with
6 d.o.f. = 4.70 p-value= .417783
:::::::::::::::::::::::::::::::::::::::::
For a sample of size
500: mean
rnd.txt using bits 4 to 27
1.918
duplicate number number
spacings
observed expected
0 82. 67.668
1 129. 135.335
2 135. 135.335
3 91. 90.224
4 38. 45.112
5 19. 18.045
6 to INF
6. 8.282
Chisquare with
6 d.o.f. = 5.14 p-value= .474033
:::::::::::::::::::::::::::::::::::::::::
For a sample of size
500: mean
rnd.txt using bits 5 to 28
1.968
duplicate number number
spacings
observed expected
0 60. 67.668
1 145. 135.335
2 141. 135.335
3 90. 90.224
4 42. 45.112
5 16. 18.045
6 to INF 6. 8.282
Chisquare with
6 d.o.f. = 2.87 p-value= .175233
:::::::::::::::::::::::::::::::::::::::::
For a sample of size
500: mean
rnd.txt using bits 6 to 29
1.912
duplicate number number
spacings
observed expected
0 57. 67.668
1 155. 135.335
2 146. 135.335
3 86. 90.224
4 36. 45.112
5 15. 18.045
6 to INF 5. 8.282
Chisquare with
6 d.o.f. = 9.23 p-value= .839046
:::::::::::::::::::::::::::::::::::::::::
For a sample of size
500: mean
rnd.txt using bits 7 to 30
2.000
duplicate number number
spacings
observed expected
0 76. 67.668
1 125. 135.335
2 129. 135.335
3 99. 90.224
4 44. 45.112
5 21. 18.045
6 to INF 6. 8.282
Chisquare with
6 d.o.f. = 4.11 p-value= .337624
:::::::::::::::::::::::::::::::::::::::::
For a sample of size
500: mean
rnd.txt using bits 8 to 31
2.066
duplicate number number
spacings
observed expected
0 63. 67.668
1 121. 135.335
2 145. 135.335
3
98. 90.224
4 45. 45.112
5 21. 18.045
6 to INF 7. 8.282
Chisquare with
6 d.o.f. = 3.88 p-value= .307568
:::::::::::::::::::::::::::::::::::::::::
For a sample of size 500: mean
rnd.txt using bits 9 to 32
1.966
duplicate number number
spacings
observed expected
0 72. 67.668
1 125. 135.335
2
147. 135.335
3 92. 90.224
4 43. 45.112
5 13. 18.045
6 to INF 8. 8.282
Chisquare with
6 d.o.f. = 3.63 p-value= .272837
:::::::::::::::::::::::::::::::::::::::::
The 9 p-values were
.623122 .032909
.417783 .474033 .175233
.839046 .337624
.307568 .272837
A KSTEST for the 9 p-values yields .621526
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: THE OVERLAPPING 5-PERMUTATION
TEST ::
:: This is the OPERM5 test. It looks at a sequence of one mill- ::
:: ion 32-bit random integers. Each set of five consecutive ::
:: integers can be in one of 120 states,
for the 5! possible or- ::
:: derings of five numbers. Thus the 5th, 6th, 7th,...numbers ::
:: each provide a state. As many thousands
of state transitions ::
:: are observed, cumulative counts are made of the number
of ::
:: occurences of each state. Then the quadratic form in the ::
:: weak inverse of the 120x120 covariance
matrix yields a test ::
:: equivalent to the likelihood ratio test
that the 120 cell ::
:: counts came from the specified
(asymptotically) normal dis- ::
:: tribution with the specified 120x120
covariance matrix (with ::
:: rank 99). This version uses 1,000,000 integers,
twice. ::
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
OPERM5 test for file rnd.txt
For a sample of 1,000,000 consecutive
5-tuples,
chisquare for 99 degrees of freedom=101.323;
p-value= .583625
OPERM5 test for file rnd.txt
For a sample of 1,000,000 consecutive
5-tuples,
chisquare for 99 degrees of freedom= 99.349;
p-value= .528754
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: This is the BINARY RANK TEST for 31x31
matrices. The leftmost ::
:: 31 bits of 31 random integers from the
test sequence are used ::
:: to form a 31x31 binary matrix over the
field {0,1}. The rank ::
:: is determined. That rank can be from 0
to 31, but ranks< 28 ::
:: are rare, and their counts are pooled
with those for rank 28. ::
:: Ranks are found for 40,000 such random
matrices and a chisqua-::
:: re test is performed on counts for
ranks 31,30,29 and <=28. ::
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Binary rank test for rnd.txt
Rank test for 31x31 binary matrices:
rows from leftmost 31 bits of each
32-bit integer
rank
observed expected (o-e)^2/e sum
28
219 211.4 .271909
.272
29
5103 5134.0 .187307
.459
30
23277 23103.0 1.309770
1.769
31
11401 11551.5 1.961438
3.730
chisquare= 3.730 for 3 d. of f.; p-value=
.734632
--------------------------------------------------------------
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: This is the BINARY RANK TEST for 32x32
matrices. A random 32x ::
:: 32 binary matrix is formed, each row a
32-bit random integer. ::
:: The rank is determined. That rank can
be from 0 to 32, ranks ::
:: less than 29 are rare, and their counts
are pooled with those ::
:: for rank 29. Ranks are found for 40,000 such random
matrices ::
:: and a chisquare test is performed on
counts for ranks 32,31, ::
::
30 and <=29.
::
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Binary rank test for rnd.txt
Rank test for 32x32 binary matrices:
rows from leftmost 32 bits of each
32-bit integer
rank
observed expected (o-e)^2/e sum
29
186 211.4 3.055915
3.056
30
5207 5134.0 1.037688
4.094
31
23108 23103.0 .001062
4.095
32 11499
11551.5 .238827 4.333
chisquare= 4.333 for 3 d. of f.; p-value=
.791053
--------------------------------------------------------------
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: This is the BINARY RANK TEST for 6x8
matrices. From each of ::
:: six random 32-bit integers from the
generator under test, a ::
:: specified byte is chosen, and the
resulting six bytes form a ::
:: 6x8 binary matrix whose rank is
determined. That rank can be ::
:: from 0 to 6, but ranks 0,1,2,3 are
rare; their counts are ::
:: pooled with those for rank 4. Ranks are
found for 100,000 ::
:: random matrices, and a chi-square test
is performed on ::
:: counts for ranks 6,5 and <=4. ::
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Binary Rank Test for rnd.txt
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 1 to 8
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 903 944.3 1.806 1.806
r
=5 21794 21743.9 .115 1.922
r =6 77303 77311.8 .001 1.923
p=1-exp(-SUM/2)= .61765
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 2 to 9
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 954 944.3 .100 .100
r
=5 21707 21743.9 .063 .162
r =6 77339 77311.8 .010 .172
p=1-exp(-SUM/2)= .08231
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 3 to 10
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 958 944.3 .199 .199
r
=5 21914 21743.9 1.331 1.529
r =6 77128 77311.8 .437 1.966
p=1-exp(-SUM/2)= .62588
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 4 to 11
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 918 944.3 .733 .733
r
=5 21831 21743.9 .349 1.081
r =6 77251 77311.8 .048 1.129
p=1-exp(-SUM/2)= .43143
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 5 to 12
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 949 944.3 .023 .023
r
=5 21696 21743.9 .106 .129
r =6 77355 77311.8 .024 .153
p=1-exp(-SUM/2)= .07366
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 6 to 13
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 1006 944.3 4.031 4.031
r
=5 21847 21743.9 .489 4.520
r =6 77147 77311.8 .351 4.871
p=1-exp(-SUM/2)= .91246
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 7 to 14
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 995 944.3 2.722 2.722
r
=5 21839 21743.9 .416 3.138
r =6 77166 77311.8 .275 3.413
p=1-exp(-SUM/2)= .81849
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 8 to 15
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 945 944.3 .001 .001
r
=5 21658 21743.9 .339 .340
r =6 77397 77311.8 .094 .434
p=1-exp(-SUM/2)= .19497
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 9 to 16
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 962 944.3 .332 .332
r
=5 21668 21743.9 .265 .597
r =6 77370 77311.8 .044 .640
p=1-exp(-SUM/2)= .27402
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 10 to 17
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 979 944.3 1.275 1.275
r
=5 21742 21743.9 .000 1.275
r =6 77279 77311.8 .014 1.289
p=1-exp(-SUM/2)= .47510
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 11 to 18
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 983 944.3 1.586 1.586
r
=5 21622 21743.9 .683 2.269
r =6 77395 77311.8 .090 2.359
p=1-exp(-SUM/2)= .69254
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 12 to 19
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 935 944.3 .092 .092
r
=5 21698
21743.9 .097 .189
r =6 77367 77311.8 .039 .228
p=1-exp(-SUM/2)= .10770
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 13 to 20
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 972 944.3 .812 .812
r
=5 21673 21743.9 .231 1.044
r =6 77355 77311.8 .024 1.068
p=1-exp(-SUM/2)= .41368
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 14 to 21
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 973 944.3 .872 .872
r
=5 21661 21743.9 .316 1.188
r =6 77366 77311.8 .038 1.226
p=1-exp(-SUM/2)= .45835
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 15 to 22
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 906 944.3 1.554
1.554
r
=5 21671 21743.9 .244 1.798
r =6 77423 77311.8 .160 1.958
p=1-exp(-SUM/2)= .62429
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 16 to 23
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 975 944.3 .998 .998
r
=5 21849 21743.9 .508 1.506
r =6 77176 77311.8 .239 1.745
p=1-exp(-SUM/2)= .58200
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 17 to 24
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 979 944.3 1.275 1.275
r
=5 21937 21743.9 1.715 2.990
r =6 77084 77311.8 .671 3.661
p=1-exp(-SUM/2)= .83968
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 18 to 25
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 966 944.3 .499 .499
r
=5 21827 21743.9 .318 .816
r =6 77207 77311.8 .142 .958
p=1-exp(-SUM/2)= .38068
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 19 to 26
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 938 944.3 .042 .042
r
=5 21870 21743.9 .731 .773
r =6 77192 77311.8 .186 .959
p=1-exp(-SUM/2)= .38090
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 20 to 27
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 980 944.3 1.350 1.350
r
=5 21776 21743.9 .047 1.397
r =6 77244 77311.8 .059 1.456
p=1-exp(-SUM/2)= .51723
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 21 to 28
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 956 944.3 .145 .145
r
=5 21985 21743.9 2.673 2.818
r =6 77059 77311.8 .827 3.645
p=1-exp(-SUM/2)= .83837
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 22 to 29
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 898 944.3 2.270 2.270
r =5 21825 21743.9 .302 2.573
r =6 77277 77311.8 .016 2.588
p=1-exp(-SUM/2)= .72588
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 23 to 30
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 931 944.3 .187 .187
r
=5 21856 21743.9 .578 .765
r =6 77213 77311.8 .126 .892
p=1-exp(-SUM/2)= .35967
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 24 to 31
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 955 944.3 .121 .121
r
=5 21737 21743.9 .002 .123
r =6 77308 77311.8 .000 .124
p=1-exp(-SUM/2)= .05993
Rank of a 6x8 binary matrix,
rows formed from eight bits of the RNG
rnd.txt
b-rank test for bits 25 to 32
OBSERVED EXPECTED
(O-E)^2/E SUM
r<=4 917 944.3 .789 .789
r
=5 21726 21743.9 .015 .804
r =6 77357 77311.8 .026 .830
p=1-exp(-SUM/2)= .33982
TEST SUMMARY, 25 tests on 100,000 random 6x8
matrices
These should be 25 uniform [0,1] random
variables:
.617652
.082315 .625884 .431434
.073664
.912465
.818489 .194971 .274021
.475102
.692544
.107705 .413684 .458345
.624287
.582001
.839675 .380680
.380905 .517228
.838374
.725884 .359674 .059925
.339818
brank test summary for rnd.txt
The KS test for those 25 supposed UNI's
yields
KS p-value= .359077
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: THE BITSTREAM TEST ::
:: The file under test is viewed as a
stream of bits. Call them ::
:: b1,b2,... . Consider an alphabet with two
"letters", 0 and 1 ::
:: and think of the stream of bits as a
succession of 20-letter ::
:: "words", overlapping. Thus the first word is b1b2...b20, the
::
:: second is b2b3...b21, and so on. The bitstream test counts ::
:: the number of missing 20-letter
(20-bit) words in a string of ::
:: 2^21 overlapping 20-letter words. There are 2^20 possible 20 ::
:: letter words. For a truly random string of 2^21+19 bits,
the ::
:: number of missing words j should be
(very close to) normally ::
:: distributed with mean 141,909 and sigma
428. Thus ::
::
(j-141909)/428 should be a standard normal variate (z score) ::
:: that leads to a uniform [0,1) p
value. The test is repeated ::
:: twenty times.
::
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
THE OVERLAPPING 20-tuples BITSTREAM TEST, 20 BITS PER WORD, N words
This test uses N=2^21 and samples the
bitstream 20 times.
No. missing words should average 141909. with sigma=428.
---------------------------------------------------------
tst no
1: 141722 missing words, -.44 sigmas from mean, p-value= .33081
tst no
2: 142151 missing words, .56 sigmas from mean, p-value= .71385
tst no
3: 142561 missing words, 1.52 sigmas from mean, p-value= .93607
tst no
4: 141270 missing words, -1.49 sigmas from mean, p-value= .06762
tst no
5: 141841 missing words, -.16 sigmas from mean, p-value= .43658
tst no
6: 141976 missing words, .16 sigmas from mean, p-value= .56189
tst no
7: 140931 missing words, -2.29 sigmas from mean, p-value= .01113
tst no
8: 142613 missing words, 1.64 sigmas from mean, p-value= .94992
tst no
9: 141312 missing words, -1.40 sigmas from mean, p-value= .08141
tst no 10:
142065 missing words, .36 sigmas from mean, p-value= .64197
tst no 11:
142296 missing words, .90
sigmas from mean, p-value= .81685
tst no 12:
142221 missing words, .73
sigmas from mean, p-value= .76676
tst no 13:
141641 missing words, -.63
sigmas from mean, p-value= .26535
tst no 14:
141419 missing words, -1.15
sigmas from mean, p-value= .12597
tst no 15:
142296 missing words, .90
sigmas from mean, p-value= .81685
tst no 16:
141861 missing words, -.11
sigmas from mean, p-value= .45505
tst no 17:
141809 missing words, -.23
sigmas from mean, p-value= .40733
tst no 18:
142587 missing words, 1.58
sigmas from mean, p-value= .94333
tst no 19:
142030 missing words, .28
sigmas from mean, p-value= .61101
tst no 20:
142020 missing words, .26 sigmas from mean, p-value= .60202
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: The tests OPSO, OQSO and DNA ::
::
OPSO means Overlapping-Pairs-Sparse-Occupancy ::
:: The OPSO test considers 2-letter words
from an alphabet of ::
:: 1024 letters. Each letter is determined by a specified
ten ::
:: bits from a 32-bit integer in the
sequence to be tested. OPSO ::
:: generates 2^21 (overlapping) 2-letter words (from 2^21+1 ::
:: "keystrokes") and counts the number of missing
words---that ::
:: is 2-letter words which do not appear
in the entire sequence. ::
:: That count should be very close to
normally distributed with ::
:: mean 141,909, sigma 290. Thus
(missingwrds-141909)/290 should ::
:: be a standard normal variable. The OPSO
test takes 32 bits at ::
:: a time from the test file and uses a
designated set of ten ::
:: consecutive bits. It then restarts the
file for the next de- ::
:: signated 10 bits, and so on. ::
::
::
::
OQSO means Overlapping-Quadruples-Sparse-Occupancy ::
::
The test OQSO is similar, except that it considers 4-letter ::
:: words from an alphabet of 32 letters,
each letter determined ::
:: by a designated string of 5 consecutive
bits from the test ::
:: file, elements of which are assumed
32-bit random integers. ::
:: The mean number of missing words in a
sequence of 2^21 four- ::
:: letter words, (2^21+3 "keystrokes"), is again
141909, with ::
:: sigma = 295. The mean is based on theory; sigma comes
from ::
:: extensive simulation.
::
::
::
::
The DNA test considers an alphabet of 4 letters:: C,G,A,T,::
:: determined by two designated bits in
the sequence of random ::
:: integers being tested. It considers 10-letter words, so that ::
:: as in OPSO and OQSO, there are 2^20
possible words, and the ::
:: mean number of missing words from a
string of 2^21 (over- ::
:: lapping) 10-letter
words (2^21+9 "keystrokes") is 141909. ::
:: The standard deviation sigma=339 was
determined as for OQSO ::
:: by simulation. (Sigma for OPSO, 290, is the true value
(to ::
:: three places), not determined by
simulation. ::
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
OPSO test for generator rnd.txt
Output: No. missing words (mw), equiv normal
variate (z), p-value (p)
mw z p
OPSO for rnd.txt using bits 23 to 32 142342
1.492 .9321
OPSO for rnd.txt using bits 22 to 31 142060
.520 .6983
OPSO for rnd.txt using bits 21 to 30 142083
.599 .7254
OPSO for rnd.txt using bits 20 to 29 141647
-.905 .1828
OPSO for rnd.txt using bits 19 to 28 141377 -1.836 .0332
OPSO for rnd.txt using bits 18 to 27 142398
1.685 .9540
OPSO for rnd.txt using bits 17 to 26 142085
.606 .7277
OPSO for rnd.txt using bits 16 to 25 141998
.306 .6201
OPSO for rnd.txt using bits 15 to 24 142310
1.382 .9165
OPSO for rnd.txt using bits 14 to 23 142345
1.502 .9335
OPSO for rnd.txt using bits 13 to 22 141709
-.691 .2448
OPSO for rnd.txt using bits 12 to 21 141750
-.549 .2914
OPSO for rnd.txt using bits 11 to 20 142353
1.530 .9370
OPSO for rnd.txt using bits 10 to 19 142036
.437 .6689
OPSO for rnd.txt using bits 9 to 18
141792 -.405 .3429
OPSO for rnd.txt using bits 8 to 17
141979 .240 .5949
OPSO for rnd.txt using bits 7 to 16
141900 -.032 .4872
OPSO for rnd.txt using bits 6 to 15
142011 .351 .6371
OPSO for rnd.txt using bits 5 to 14
141767 -.491 .3118
OPSO for rnd.txt using bits 4 to 13
142002 .320 .6253
OPSO for rnd.txt using bits 3 to 12
141236 -2.322 .0101
OPSO for rnd.txt using bits 2 to 11
141670 -.825 .2046
OPSO for rnd.txt using bits 1 to 10
141827 -.284 .3882
OQSO test for generator rnd.txt
Output: No. missing words (mw), equiv normal
variate (z), p-value (p)
mw z
p
OQSO for rnd.txt using bits 28 to 32 142129
.745 .7718
OQSO for rnd.txt using bits 27 to 31 142235
1.104 .8652
OQSO for rnd.txt using bits 26 to 30 141740
-.574 .2830
OQSO for rnd.txt using bits 25 to 29 142191
.955 .8302
OQSO for rnd.txt using bits 24 to 28 141051 -2.910 .0018
OQSO for rnd.txt using bits 23 to 27 142622
2.416 .9922
OQSO for rnd.txt using bits 22 to 26 142285
1.273 .8986
OQSO for rnd.txt using bits 21 to 25 141995
.290 .6142
OQSO for rnd.txt using bits 20 to 24 142070
.545 .7070
OQSO for rnd.txt using bits 19 to 23 141844
-.221 .4124
OQSO for rnd.txt using bits 18 to 22 141796
-.384 .3504
OQSO for rnd.txt using bits 17 to 21 141469 -1.493 .0678
OQSO for rnd.txt using bits 16 to 20 141810 -.337
.3682
OQSO for rnd.txt using bits 15 to 19 141633
-.937 .1745
OQSO for rnd.txt using bits 14 to 18 141940
.104 .5414
OQSO for rnd.txt using bits 13 to 17 141705
-.693 .2443
OQSO for rnd.txt using bits 12 to 16 141700
-.710 .2390
OQSO for rnd.txt using bits 11 to 15 141721
-.638 .2616
OQSO for rnd.txt using bits 10 to 14 142076
.565 .7140
OQSO for rnd.txt using bits 9 to 13
141715 -.659 .2550
OQSO for rnd.txt using bits 8 to 12
142142 .789 .7849
OQSO for rnd.txt using bits 7 to 11
142262 1.195 .8841
OQSO for rnd.txt using bits 6 to 10
141984 .253 .5999
OQSO for rnd.txt using bits 5 to
9 142331 1.429
.9236
OQSO for rnd.txt using bits 4 to
8 141596 -1.062 .1441
OQSO for rnd.txt using bits 3 to
7 141548 -1.225 .1103
OQSO for rnd.txt using bits 2 to
6 142226 1.073
.8585
OQSO for rnd.txt using bits 1 to
5 141847 -.211
.4163
DNA test for generator rnd.txt
Output: No. missing words (mw), equiv normal
variate (z), p-value (p)
mw z p
DNA for rnd.txt using bits 31 to 32 142103
.571 .7161
DNA for rnd.txt using bits 30 to 31 142706
2.350 .9906
DNA for rnd.txt using bits 29 to 30 141566 -1.013 .1556
DNA for rnd.txt using bits 28 to 29 141827
-.243 .4041
DNA for rnd.txt using bits 27 to 28 141897
-.036 .4855
DNA for rnd.txt using bits 26 to 27 142208
.881 .8109
DNA for rnd.txt using bits 25 to 26 141913
.011 .5043
DNA for rnd.txt using bits 24 to 25 141773
-.402 .3438
DNA for rnd.txt using bits 23 to 24 141607
-.892 .1862
DNA for rnd.txt using bits 22 to 23 142140
.680 .7519
DNA for rnd.txt using bits 21 to 22 141634
-.812 .2083
DNA for rnd.txt using bits 20 to 21 141791
-.349 .3635
DNA for rnd.txt using bits 19 to 20 141827
-.243 .4041
DNA for rnd.txt using bits 18 to 19 142064
.456 .6759
DNA for rnd.txt using bits 17 to 18 141359 -1.623 .0523
DNA for rnd.txt using bits 16 to 17 142101
.565 .7141
DNA for rnd.txt using bits 15 to 16 141268 -1.892 .0293
DNA for rnd.txt using bits 14 to 15 142469
1.651 .9506
DNA for rnd.txt using bits 13 to 14 141710
-.588 .2783
DNA for rnd.txt using bits 12 to 13 141925
.046 .5184
DNA for rnd.txt using bits 11 to 12 141800
-.323 .3735
DNA for rnd.txt using bits 10 to 11 141841
-.202 .4201
DNA for rnd.txt using bits 9 to 10
141833 -.225 .4109
DNA for rnd.txt using bits 8 to
9 141485 -1.252 .1053
DNA for rnd.txt using bits 7 to
8 142971 3.132
.9991
DNA for rnd.txt using bits 6 to 7 142121
.624 .7338
DNA for rnd.txt using bits 5 to
6 142223 .925
.8226
DNA for rnd.txt using bits 4 to
5 141473 -1.287 .0990
DNA for rnd.txt using bits 3 to
4 141676 -.688
.2456
DNA for rnd.txt using bits 2 to
3 141902 -.022
.4914
DNA for rnd.txt using bits 1 to
2 141919 .029
.5114
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::
This is the COUNT-THE-1's TEST on a stream of bytes. ::
:: Consider the file under test as a
stream of bytes (four per ::
:: 32 bit integer). Each byte can contain from 0 to 8 1's, ::
:: with probabilities
1,8,28,56,70,56,28,8,1 over 256. Now
let ::
:: the stream of bytes provide a string of
overlapping 5-letter ::
:: words, each "letter" taking
values A,B,C,D,E. The letters are ::
:: determined by the number of 1's in a
byte:: 0,1,or 2 yield A,::
:: 3 yields B, 4 yields C, 5 yields D and
6,7 or 8 yield E. Thus ::
:: we have a monkey at a typewriter
hitting five keys with vari- ::
:: ous probabilities (37,56,70,56,37 over
256). There are 5^5 ::
:: possible 5-letter words, and from a
string of 256,000 (over- ::
:: lapping) 5-letter words, counts are
made on the frequencies ::
:: for each word. The quadratic form in the weak inverse
of ::
:: the covariance matrix of the cell
counts provides a chisquare ::
:: test::
Q5-Q4, the difference of the naive Pearson sums of ::
:: (OBS-EXP)^2/EXP on counts for 5- and
4-letter cell counts. ::
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Test results for rnd.txt
Chi-square with 5^5-5^4=2500 d.of f. for
sample size:2560000
chisquare equiv normal
p-value
Results fo COUNT-THE-1's in successive bytes:
byte stream for rnd.txt 2510.60 .150
.559565
byte stream for rnd.txt 2423.96 -1.075
.141105
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::
This is the COUNT-THE-1's TEST for specific bytes. ::
:: Consider the file under test as a
stream of 32-bit integers. ::
:: From each integer, a specific byte is
chosen , say the left- ::
:: most::
bits 1 to 8. Each byte can contain from 0 to 8 1's, ::
:: with probabilitie
1,8,28,56,70,56,28,8,1 over 256. Now
let ::
:: the specified bytes from successive
integers provide a string ::
:: of (overlapping) 5-letter words, each
"letter" taking values ::
:: A,B,C,D,E. The letters are
determined by the number of 1's, ::
:: in that byte:: 0,1,or 2 ---> A, 3 ---> B, 4 ---> C,
5 ---> D,::
:: and
6,7 or 8 ---> E. Thus we have
a monkey at a typewriter ::
:: hitting five keys with with various
probabilities:: 37,56,70,::
:: 56,37 over 256. There are 5^5 possible
5-letter words, and ::
:: from a string of 256,000 (overlapping)
5-letter words, counts ::
:: are made on the frequencies for each
word. The quadratic form ::
:: in the weak inverse of the covariance
matrix of the cell ::
:: counts provides a chisquare test:: Q5-Q4, the difference of ::
:: the naive Pearson sums of (OBS-EXP)^2/EXP on counts for 5- ::
:: and 4-letter cell counts. ::
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Chi-square with 5^5-5^4=2500 d.of f. for
sample size: 256000
chisquare equiv normal
p value
Results for COUNT-THE-1's in specified bytes:
bits
1 to 8 2592.28
1.305 .904063
bits
2 to 9 2385.97
-1.613 .053410
bits
3 to 10 2381.46 -1.676
.046836
bits
4 to 11 2513.47 .191
.575555
bits 5
to 12 2469.02 -.438
.330668
bits
6 to 13 2490.08 -.140
.444230
bits
7 to 14 2613.26 1.602
.945392
bits
8 to 15 2443.88 -.794
.213704
bits
9 to 16 2547.57 .673
.749460
bits 10 to 17 2425.99
-1.047 .147641
bits 11 to 18 2517.10
.242 .595527
bits 12 to 19 2524.45
.346 .635249
bits 13 to 20 2444.18
-.789 .214944
bits 14 to 21 2577.33
1.094 .862942
bits 15 to 22 2491.84
-.115 .454045
bits 16 to 23 2475.96
-.340 .366926
bits 17 to 24 2509.01
.127 .550699
bits 18 to 25 2528.35
.401 .655744
bits 19 to 26 2466.35
-.476 .317091
bits 20 to 27 2465.30
-.491 .311794
bits 21 to 28 2552.47
.742 .770969
bits 22 to 29 2457.93
-.595 .275916
bits 23 to 30 2510.86
.154 .561045
bits 24 to 31 2529.91
.423 .663826
bits 25 to 32 2544.70
.632 .736354
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: THIS IS A PARKING LOT TEST ::
:: In a square of side 100, randomly
"park" a car---a circle of
::
:: radius 1. Then try to park a 2nd, a 3rd, and so on,
each ::
:: time parking "by ear". That is, if an attempt to park a car ::
:: causes a crash with one already parked,
try again at a new ::
:: random location. (To avoid path problems,
consider parking ::
:: helicopters rather than cars.) Each attempt leads to either ::
:: a crash or a success, the latter
followed by an increment to ::
:: the list of cars already parked. If we
plot n: the number of ::
:: attempts, versus k:: the number successfully parked, we get
a::
:: curve that should be similar to those
provided by a perfect ::
:: random number generator. Theory for the behavior of such a ::
:: random curve seems beyond reach, and as
graphics displays are ::
:: not available for this battery of
tests, a simple characteriz ::
:: ation of the random experiment is used:
k, the number of cars ::
::
successfully parked after n=12,000 attempts. Simulation shows ::
:: that k should average 3523 with sigma
21.9 and is very close ::
:: to normally distributed. Thus (k-3523)/21.9 should be a st- ::
:: andard normal variable, which,
converted to a uniform varia- ::
:: ble, provides input to a KSTEST based
on a sample of 10. ::
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
CDPARK: result of ten tests on file
rnd.txt
Of 12,000 tries, the average no. of
successes
should be 3523 with sigma=21.9
Successes: 3519 z-score:
-.183 p-value: .427537
Successes: 3506 z-score:
-.776 p-value: .218799
Successes: 3534 z-score:
.502 p-value: .692266
Successes: 3535 z-score:
.548 p-value: .708135
Successes: 3556 z-score:
1.507 p-value: .934075
Successes: 3497 z-score: -1.187 p-value: .117571
Successes: 3536 z-score:
.594 p-value: .723613
Successes: 3538 z-score:
.685 p-value: .753306
Successes: 3548 z-score:
1.142 p-value: .873180
Successes: 3518 z-score:
-.228 p-value: .409702
square size avg. no.
parked sample sigma
100. 3528.700 17.467
KSTEST for the above 10: p= .438300
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: THE MINIMUM DISTANCE TEST ::
:: It does this 100 times:: choose n=8000 random points in a ::
:: square of side 10000. Find d, the minimum distance between ::
:: the (n^2-n)/2 pairs of points. If the points are truly inde- ::
:: pendent uniform, then d^2, the square
of the minimum distance ::
:: should be (very close to) exponentially
distributed with mean ::
:: .995 .
Thus 1-exp(-d^2/.995) should be uniform on [0,1) and ::
:: a KSTEST on the resulting 100 values
serves as a test of uni- ::
:: formity for random points in the
square. Test numbers=0 mod 5 ::
:: are printed but the KSTEST is based on
the full set of 100 ::
:: random choices of 8000 points in the
10000x10000 square. ::
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
This is the MINIMUM DISTANCE
test
for random integers in the file
rnd.txt
Sample no. d^2
avg equiv uni
5 .0874
.5851 .084103
10
.0902 .9722 .086626
15
.3670 1.2155 .308472
20
.5384 1.1275 .417923
25
.9195 1.0329 .603102
30
.8928 .9910 .592332
35
.0259 1.0519 .025697
40
4.3788 1.1148 .987732
45
.5976 1.0936 .451527
50
.1089 1.0156 .103663
55
1.3435 1.0464 .740822
60
.8943 .9859 .592945
65
.0116 1.0144 .011612
70
.4537 1.0299 .366150
75
.2247 .9702 .202182
80
.1575 .9635 .146420
85
.2743 .9818 .240940
90
.3900 .9534 .324299
95
.3696 .9363 .310284
100
.1236 .9498 .116854
MINIMUM DISTANCE TEST for rnd.txt
Result of KS test on 20 transformed
mindist^2's:
p-value=
.583890
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: THE 3DSPHERES TEST ::
:: Choose
4000 random points in a cube of edge 1000. At each
::
:: point, center a sphere large enough to
reach the next closest ::
:: point. Then the volume of the smallest
such sphere is (very ::
:: close to) exponentially distributed
with mean 120pi/3. Thus ::
:: the radius cubed is exponential with
mean 30. (The mean is ::
:: obtained by extensive simulation). The 3DSPHERES test gener- ::
:: ates 4000 such spheres 20 times. Each min radius cubed leads ::
:: to a uniform variable by means of
1-exp(-r^3/30.), then a ::
::
KSTEST is done on the 20 p-values. ::
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
The 3DSPHERES test for file
rnd.txt
sample no: 1 r^3=
53.024 p-value= .82923
sample no:
2 r^3= 57.427
p-value= .85255
sample no:
3 r^3= 55.101
p-value= .84066
sample no:
4 r^3= 13.891
p-value= .37062
sample no:
5 r^3= 45.576
p-value= .78112
sample no:
6 r^3= 46.709
p-value= .78922
sample no:
7 r^3= 38.730
p-value= .72500
sample no:
8 r^3= 31.979
p-value= .65560
sample no:
9 r^3= 34.304
p-value= .68129
sample no: 10 r^3=
24.730 p-value= .56147
sample no: 11 r^3=
35.756 p-value= .69635
sample no: 12 r^3=
27.478 p-value= .59986
sample no: 13 r^3=
25.481 p-value= .57231
sample no: 14 r^3=
21.136 p-value= .50567
sample no: 15 r^3=
76.281 p-value= .92135
sample no: 16 r^3=
.249 p-value= .00828
sample no: 17 r^3=
48.421 p-value= .80092
sample no: 18 r^3=
.699 p-value= .02302
sample no: 19 r^3=
29.867 p-value= .63049
sample no: 20 r^3=
9.605 p-value= .27398
A KS
test is applied to those 20 p-values.
---------------------------------------------------------
3DSPHERES test for file rnd.txt p-value= .937251
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::
This is the SQEEZE test ::
::
Random integers are floated to get uniforms on [0,1). Start- ::
::
ing with k=2^31=2147483647, the test finds j, the number of ::
::
iterations necessary to reduce k to 1, using the reduction ::
::
k=ceiling(k*U), with U provided by floating integers from ::
::
the file being tested. Such j's
are found 100,000 times, ::
::
then counts for the number of times j was <=6,7,...,47,>=48 ::
::
are used to provide a chi-square test for cell frequencies. ::
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
RESULTS OF SQUEEZE TEST FOR
rnd.txt