Analysing the 125,262 puzzles in the lichess puzzle set

Note: the data used when making this article is probably out of date by the time you are reading this, but the insights may still be useful.

You can see a larger version of charts below by clicking on them.

All code used to fetch, process and plot the puzzle set can be found on Gitlab.

Ratings

The lichess puzzleset is quite diverse, with ratings spanning from 767 (puzzle #11219) to 2926 (puzzle #122656). The median puzzle rating is 1410. 80% of puzzles have a rating between 1040 and 2160.

Player attempts are more biased towards the lower end of the rating range: 90% of attempted puzzles have a rating below 2000.

Puzzles generally receive positive votes by players. However, higher rated puzzles tend to receive more negative votes. Maybe players vent out their frustration at the more difficult puzzles they didn't get correctly by voting them down.

Puzzle rating dosen't seem to be strongly correlated with the rating of the players from the source game.

Depth

As expected, higher rated puzzles require more calculation on average, but the standard deviation is still very high: harder puzzles are all over the place, when it comes to calculation depth.

Time

Tactical opportunities seem to happen regularly throughout the game. Half of the puzzles happen before the 23rd move. 80% of puzzles take place between the 11th and the 42nd move.

The majority of puzzles come from either blitz or rapid games. Puzzles from bullet games are slightly in the lower end of the rating range, and correspondence games yield slightly more highly rated puzzles.

Tactical motifs

Note: the motifs were detected programatically using Ormulogun, which is far from perfect at identifying everything. Chess is hard.

Easier puzzles feature more forks, harder puzzles feature more discovered attacks and pins. The other motifs seem roughly consistent across the whole rating range.

Puzzles with forks tend to be the easiest, with a median rating of about 1450, while puzzles with discovered checks or pins tend to be the hardest, with a median rating of about 2200.

Puzzle types

Checkmate puzzles tend to be significantly easier than other kinds of puzzles, and get rarer as puzzle rating increases.

Endings

Ending type seems uncorrelated with puzzle difficulty. Rook endings are the most common by a fair margin.