Stefan Pohl Computer Chess

private website for chessengine-tests


Latest Website-News (2017/05/19): The huge opening-book testrun (comparing my SALC V2 book, the new FEOBOS-book (v3.0 beta) by Frank Quisinsky and the 8moves openings, used in Stockfish-framework) is done (took around 12 days...). Take a look at the interesting results in the "Experiments"-section. And download the SALC-book and the FEOBOS-book in the "Downloads & Links"-section.

 

Next testrun is Komodo 11 (will start on Monday, when Komodo 11 is released). And after that testrun, the next Stockfish testrun will follow.

 

Stay tuned.


Stockfish testing

 

Playing conditions:

 

Hardware: i7-2630QM 2.0GHz Notebook, Windows 10 64bit, 4GB RAM

Fritzmark: singlecore: 3.97 / 1905 (all engines running on one core, only), average meganodes/s displayed by LittleBlitzerGUI: Houdini: 2.0 mn/s, Stockfish: 1.7 mn/s

Hash: 128MB per engine

GUI: LittleBlitzerGUI (draw at 120 moves, resign at 450cp (for 4 moves))

Tablebases: None

Openings: 10moves_SALC_500.epd (download the file at the "Download & Links"-section)

Ponder, Large Memory Pages & learning: Off

Thinking time: 70''+700ms per game/engine (average game-duration: 3.5 minutes)(standardized to the hardware-speed and the thinking time of the excellent FGRL Bullet-ratinglist). One 7000 games-testrun takes about 6 days (running on only 3 of 4 cores). The version-numbers of the Stockfish-development engines are the release-date, written backwards (year,month,day))(example: 141028 = October, 28, 2014), downloaded at chess.ultimaiq.net I always use the latest version of one day, if more than one version per day is released. And I use the version "for modern computers". (At the moment, the compiles for modern windows machines on abrok.eu are around 8% slower, so I dont use them anymore)

 

Each Stockfish-version plays 1000 games against Komodo 11, Houdini 5, Shredder 13, Fizbo 1.9, Gull 3, Fire 4, and Critter 1.6a.

 

Latest update: 2017/05/18: asmFish 170502

 

Download the individual statistics here

 

     Program                    Elo    +    -   Games   Score   Av.Op.  Draws

   1 BrainFish 161009 numa    : 3469    8    8  7000    85.0 %   3152   24.7 %
   2 BrainFish 161128 x64     : 3463    8    8  7000    81.3 %   3188   30.5 %
   3 BrainFish 170410 x64     : 3461    8    8  7000    80.0 %   3203   31.6 %
   4 asmFish 170426 x64       : 3442    7    7  7000    78.3 %   3203   32.3 %
   5 asmFish 170502 x64       : 3441    7    7  7000    78.2 %   3203   33.8 % (new)
   6 asmFish 170328 x64       : 3430    7    7  7000    77.1 %   3203   33.5 %
   7 asmFish 161207 x64       : 3426    7    7  7000    78.1 %   3188   33.6 %
   8 asmFish 170211 x64       : 3426    7    7  7000    76.9 %   3201   33.3 %
   9 asmFish 170202 x64       : 3426    7    7  7000    76.9 %   3201   33.6 %
  10 asmFish 170310 x64       : 3425    7    7  7000    76.8 %   3201   33.8 %
  11 asmFish 161217 x64       : 3425    7    7  7000    78.0 %   3188   32.9 %
  12 asmFish 170301 x64       : 3425    7    7  7000    76.8 %   3201   33.8 %
  13 asmFish 170109 x64       : 3424    7    7  7000    76.8 %   3201   33.9 %
  14 asmFish 161004 x64       : 3422    8    8  7000    81.5 %   3152   29.2 %
  15 asmFish 170122 x64       : 3421    7    7  7000    76.5 %   3201   33.6 %
  16 Stockfish 170423 x64     : 3417    7    7  7000    75.9 %   3203   34.6 %
  17 CFish 170408 x64         : 3411    7    7  7000    75.3 %   3203   35.8 %
  18 Stockfish 161212 x64     : 3407    7    7  7000    76.4 %   3188   35.8 %
  19 Stockfish 170417 x64     : 3407    7    7  7000    74.8 %   3203   35.2 %
  20 Stockfish 170503 x64     : 3405    7    7  7000    74.7 %   3203   36.3 %
  21 Stockfish 170305 x64     : 3405    7    7  7000    74.9 %   3201   35.5 %
  22 Stockfish 170402 x64     : 3405    7    7  7000    74.6 %   3203   34.9 %
  23 Stockfish 170214 x64     : 3403    7    7  7000    74.7 %   3201   35.2 %
  24 Stockfish 170105 x64     : 3403    7    7  7000    74.7 %   3201   36.0 %
  25 Stockfish 170113 x64     : 3402    7    7  7000    74.6 %   3201   35.7 %
  26 Stockfish 170318 x64     : 3402    7    7  7000    74.4 %   3203   36.3 %
  27 Stockfish 170223 x64     : 3402    7    7  7000    74.6 %   3201   35.4 %
  28 Stockfish 170129 x64     : 3400    6    6  7000    74.4 %   3201   36.6 %
  29 Stockfish 161120 x64     : 3394    7    7  7000    75.1 %   3188   35.7 %
  30 Stockfish 161127 x64     : 3393    7    7  7000    74.9 %   3188   36.2 %
  31 Stockfish 8 161101       : 3390    5    5 12000    73.0 %   3204   37.2 %
  32 Houdini 5 x64            : 3363    3    3 38000    49.1 %   3365   49.8 %
  33 Komodo 10.4 x64          : 3339    4    4 17000    50.0 %   3335   42.4 %
  34 Komodo 10.3 x64          : 3329    4    4 20000    49.7 %   3326   41.5 %
  35 Komodo 10.1 x64          : 3324    7    7  8000    65.0 %   3204   33.3 %
  36 Komodo 10.2 x64          : 3315    4    4 15000    54.2 %   3279   38.8 %
  37 Houdini 4 x64            : 3195    5    5 11000    44.7 %   3241   34.7 %
  38 Shredder 13 x64          : 3181    3    3 39000    27.0 %   3365   34.4 %
  39 Fizbo 1.9 x64            : 3176    3    3 30000    25.7 %   3371   29.0 %
  40 Gull 3 x64               : 3130    3    3 43000    22.4 %   3363   31.0 %
  41 Fire 4 x64               : 3120    3    3 43000    21.5 %   3363   30.6 %
  42 Critter 1.6a x64         : 3112    3    3 43000    20.8 %   3363   27.5 %
  43 Equinox 3.3 x64          : 3094    5    5 19000    24.5 %   3313   31.7 %
  44 Mars 3.41 x64            : 3093    6    6 10000    30.7 %   3257   34.5 %

Below you find a diagram of the progress of Stockfish in my tests since the end of 2016

And below that diagram, the older diagrams.

 

You can save the diagrams (as a JPG-picture (in originial size)) on your PC with mouseclick (right button) and then choose "save image"...

The Elo-ratings of older Stockfish dev-versions in the Ordo-calculation can be a little different to the Elo-"dots" in the diagram, because the results/games of new Stockfish dev-versions - when getting part of the Ordo-calculation - can change the Elo-ratings of the opponent engines and that can change the Elo-ratings of older Stockfish dev-versions (in the Ordo-calculation / ratinglist, but not in the diagram, where all Elo-"dots" are the rating of one Stockfish dev-version at the moment, when the testrun of that Stockfish dev-version was finished).


Sie sind Besucher Nr.