The Touching Responses to “On Learning Chess”

On Sunday night I jotted down a short reflection on my journey learning chess over the last two years. I made a couple graphs to fill it out and submitted it to HackerNews. The response has been overwhelming – tens of thousands of readers, and hundreds of helpful, funny, and encouraging comments, through HackerNews, Reddit, email, and other sources – so I want to respond to everyone here, and highlight a few remarks that stood out to me.

I’ll start by answering the most frequently asked questions, and then move to the highlights.

FAQ

Does your chess.com blitz rating correspond to a standard FIDE rating?

The ratings don’t correspond – I don’t have a FIDE rating, and online blitz is a different game from classical OTB chess. With that said, you can compute some statistical correlations between the ratings among players who have both, and a 1750 chess.com Blitz rating (my personal best, achieved November 2020) roughly correlates to a 1665 FIDE classical rating, according to this analysis, or a 1745 USCF rating.

What is your chess.com username? I want to analyze your games.

While that’s very touching, I have deliberately withheld my chess.com username because of the privacy ramifications of connecting my real-life identity to my chess.com account.

Can you share your Anki cards?

My Anki cards would not be useful to others except as examples, because they’re based on the specific mistakes I make and off-beat openings I use. In 6 months, my current set of Anki cards will probably not even be particularly useful to me. But, just to expand on my method, my Anki cards have 4 fields – Front (a picture of the position, with me to play); Back (A picture of the position after the correct move), FEN (a unique text representation of the starting position, to help me avoid duplicates), and GameHits (to help me keep track of how often I see a card’s position in an actual game.)

How can you spend so much time on chess?

Like everyone, I make tradeoffs. Life is not a contest. I have a fairly demanding job and I’m in grad school, but I still have significant flexibility in how I spend my time. I even have other hobbies! And I do deal with burnout and stress. Chess has actually helped me with that, because I’m less tempted to stay up late solving tactics than I am to watch a show. The feeling of steady progress, and a reliable routine, can actually provide stability and calm that outweighs the time and effort one puts in.

Highlights from the Comments

This is EXACTLY what I needed to hear as an adult who is just now getting really into the game. What was your training routine like for 800-1200 ELO on chess.com?

sonicfood – Reddit

I’m so happy to hear this. I actually talk about exactly what I did in an earlier post, here (in the “Chess” section.) I would note that I didn’t do a lot of tactics training then, but I believe the process would have gone much faster if I had.

Hi, I would really encourage you to spend more time doing tactical problems and not spend as much with strategy concepts and openings. You can get to 2000 with minimal opening and strategy study, and you will get there faster. Until you have mastered tactics, you should have a basic understanding of opening concepts and strategy, no more than that. Have a look at Michael de la Maza’s book, or better yet, just read this article about it, it will save you some money: https://www.chess.com/article/view/the-michael-de-la-maza-story

P.S. I’m a chess teacher and I’ve switched to this approach a long time ago, based on research and results

P.S.2 There are some very good software for tactical training, I use the ones from Convekta, but there are others. Maybe check out Dan Heisman’s articles, I haven’t been up to date on him for a while but it was pretty solid advice.

Mesquita – WordPress

I definitely agree that most progress comes from solving tactics. Anki, though, is my silent chess coach, pushing me to analyze my games and explore the bad decisions I make. Chess is, after all, about making better choices. To those of you out there who already play slow time controls and analyze your games, Anki is probably superfluous. And thank you for the link about Michael de la Maza! His story shows us, I think, that for many of us the limiting factor is just how long we continue training. It’s hard work.

> on Friday I’m better than average,

I wonder if this is because the opponents might have been drinking.

For me at least, I know that even two beers has a noticeable effect when I do lichess puzzles.

emmelaich, HackerNews

I love this theory! Anecdotally, people do sometimes announce that they’re drinking in Friday games.

I did something very similar with an old strategy game called Age of empires II. Over the course of two years I went from a 1300 rating to 1850. I don’t play chess, so it’s hard for me to compare the scales, but I will say the highest players were rated around 2600. I think being 1850 put me around 90-95th percentile for active players– there were less than 50 players over 2300, and only a few hundred over 2000.

My only real commentary is that trying to follow a progression like this can be profoundly, profoundly frustrating. You need to really make sure you’re enjoying this hobby before you try to dedicate yourself to a trajectory like this. It can really make you feel like shit a lot of the time, as you find yourself continually repeating mistakes that other people breeze past without issue. It takes true love of the game and a great deal of patience and persistence to push through all the downspells.

Eventually I decided that I no longer wished to prioritize the game over other things. The enormous time investment was no longer a worthwhile trade-off for me, and I stopped playing almost all together. I haven’t regretted it since. There’s just so many things to do in life. I gave it thousands of hours of my time, and now the door is closed on that. Now I’m learning Ruby on rails!

aerovistae, HackerNews

You really speak to the psychology of training here. Sometimes it’s such a struggle. Some days you keep making mistakes, and everything is hard, and that feeling of “profound, profound frustration” just wells up. At those moments I try to fall back to having the clichéed “faith in the process” and grind out another puzzle. Fortunately, at so many moments, it’s fun and easy and flowing and you’ll see a beautiful tactic that wins your opponent’s queen. And you get more of those moments as you continue to grind.

Chess is pretty great.

I hope many of you are encouraged that you can improve with time and a good routine, as you have encouraged me to carry on. I particularly appreciate the kind words from much stronger players, who could easily have disregarded this result as unimportant in absolute terms. (But imagine Magnus Carlsen’s two year posts! Would we find them relatable? And so, I think, there is a use for celebrating small gains by ordinary people like me.)

I’ll end by signal-boosting some other interesting things I’ve done with chess.

I published an article in Math Horizons magazine earlier this year about a really fun puzzle that uses a chess board and chess pieces (but isn’t, strictly speaking, chess). (Spring 2020)

I really love 4-player chess and have spent a good deal of time working on a 4-player chess bot for chess.com (spring 2019).

Comments on Hacker News

On Learning Chess as an Adult – From 650 to 1750 in Two Years

I caught the chess bug as a college student, twenty-one years old, and it’s become my favorite way to unwind since then, as I’m now working at VisioStack and starting a master’s in CS at Clemson University.

Like all hobbies, chess is more fun when you’re making progress. And progress as an adult is certainly more difficult, as for instance recounted in this fine story by Tom Vanderbilt. It’s not so easy to rewire your brain.

I started out as a below-average-rated player, which was humbling and rather galling. My chess-playing friends IRL beat me easily. Today, I’m still a poor player – I frequently blunder pieces and make other mistakes – yet I’m much stronger; I beat those friends and am now around the 95th percentile at chess.com. My rating progress at blitz looks like this:

(I would note that in fall of 2018, I played a great deal of 4-player chess instead of standard chess; this improved my regular game indirectly, fortunately.)

How I Study

Much of this progress was haphazard: I played lots of games, read the occasional book, learned a few openings. Earlier this year I hit a road block around 1450; the following routine is what’s helped me continue to progress.

I spend about two hours per day on chess, broken into three periods. In the first period, I drill opening theory and positional concepts. I use the spaced-repetition app Anki for this. My cards are just positions drawn from computer analysis of my own games: I put blunders and mistakes from my games here (front side of the card is the position before my move, reverse shows the correct move.) I also put many positions from the openings that I play here. I add about 25 new cards a day, which means I have 100+ cards to go over on a given day. Reviewing Anki cards takes between 30 minutes and an hour.

In the second period, I solve puzzles on Chess.com for 30 minutes to an hour. I would note that this requires a chess.com membership, as does unlimited analysis of your games. I pay $99/year for a Diamond membership; there are other, cheaper levels as well. Still, for a serious hobby, it’s pretty cheap.

In the third period, I play blitz games (usually at the 3+0 time control). I will analyze many of these games – especially ones I lose – and put anything surprising into Anki. I might play for anywhere from 30 minutes to 3 hours.

Progress: Not a Straight Line

It’s fascinating to me how much chess performance can vary from day to day. Every two months or so, I’ll have an amazing chess day where my rating skyrockets by 150-200 points (and promptly returns to earth in the next few days.) More concretely: The median interval between my personal best ratings is exactly 60 days.

My rating varies by weekday. I have played over 10,000 blitz games in the last 2+ years. With the exception of a couple of multi-week breaks, I have played essentially every day. If, for each week, I find my median ELO and compare each day to that median (so, for example, that week’s median ELO is 1200, but on Wednesday is 1235, Wednesday gets a +35), and then take the median of all the deltas for each day, we get the following effect:

So on Thursday I’m about average, on Friday I’m better than average, and on other days I’m much worse. I assume this effect arises from playing fewer games on Monday and Tuesday, and so being a bit rusty, and then playing too many games on the weekend, ending with a lower rating. A win or loss is worth about 8 ELO, so on Tuesdays I lose about 9 more games than I win.

Time of day and number of games played also certainly affect rating, but I don’t have that data readily at hand. I do believe that it’s possible to determine via the extensive chess.com API (https://www.npmjs.com/package/chess-web-api), though.

Goals

I’d like to reach 2000 in 2021. My current study strategy seems to net me around 1.5 ELO/day, so this seems achievable. Of course I could hit a wall where I need to reinvent myself again; if I do, I’ll consider getting a chess coach, reading chess books, etc.

Some day, I want to become a titled player. A candidate master needs an ELO of 2200, but this is in over-the-board play, which probably corresponds to around 2300 in online blitz. I don’t have a timeline for this, but I imagine that if I do get there, it will happen with in the next five years.

Comments on Hacker News | Reddit | My response

Massive Bulk Update

First, a life update: I graduated from college, I’m working at VisioStack doing some interesting greenfield work, and I’m taking my first class from Harvard Extension School,  hopefully en route to a degree in data science.

Signs & Seasons

My friend Elliot Lovegrove and I got back together for an ambitious project: A 4′ x 4′ painting on plexiglass with a network of ~80 NeoPixel lights inside the frame, powered by an Arduino Mega. It premiered at the Greenville Center for the Creative Arts on 8/2/19. I wrote a program to convert patterns of effects that run on the lights into Arduino-flavored C. That way Elliot could design everything and I could avoid messing everything up.

Here’s how the patterns work. An effect is some action (fading, twinkling, etc.) that happens to a subset of the lights. There are lots of effects and lots of options for each effect, which you can get a sense of at the link above. A stream is a consecutive set of effects, which happen in order and on repeat, forever. A stream’s modularity tells how many cycles a stream should go through to be active (e.g. with a modularity of 5 and length of 10 seconds, a stream would fire at times 0, 50, 100, etc.). A stream can also have an offset x to determine how long to wait before it runs the first time.

The formal way to say this is that the times when stream i begins are congruent to its offset x_i modulo (its length l_i times its modularity k_i). So each stream has its own little modular equation, and you can calculate how long it will take for everything to repeat using the Chinese Remainder Theorem. It’s a textbook case!

All streams run consecutively all the time. Two streams could potentially have effects that modify the same lights at the same time. If this happens, the conflicting colors are blended, unless the streams are on different layers, in which case the higher layer overrides the lower layer.

I don’t have a video of the painting right now; I was sort of hoping the gallery would put up a video, but if I get one through some other channel, I can add it here.

Chess

I started getting into chess last summer at the Clemson REU. It’s definitely grown on me! Although I remain objectively bad at it, I have more appreciation for how many different levels of suck are possible. Generally an ELO difference of 100 means that the better player will win or draw the other player most of the time. Since human blitz ratings go from 0 to ~3000, that means there are about 30 different levels of enlightenment to reach. I started out at about 700 a year a go – a number that’s phenomenally and embarrassingly low – and I gradually increased to about 1300 today. (I almost hit 1400 in July, though!)

The journey looked something like this:

From 700 to 900: Just playing a few hundred games online. 2 months.

From 900 to 1000: This happened while I wasn’t actually playing chess, rather four player chess, which I’ll discuss more below. Basically I played only 4-player chess for a few months, and when I returned, I had marginally improved my regular chess. 3 months.

From 1000 to 1100: There are a few common openings and a few reasonable responses to them. Using the analysis board to learn the openings to a depth of, like, two made a huge difference. 3 months.

From 1100 to 1200: I learned some basics about positional chess, like what constitutes a good pawn structure, two bishops good, etc. I read some of a beginner’s book on chess. 1200 is generally considered ‘standard’ chess noobishness, I think, and it’s about the 50th percentile of players on Chess.com. 3 months.

From 1200 to 1300: I did some tactics training on Chess.com and try to look for options (and my opponent’s response) when I make a move. 1 month.

My best rating is 1390, from July 2019, and the highest-rated player I’ve beaten is 1490, from August 2019.

As I said, I took a break last fall to play a lot of 4 player chess. I actually was much better at it than I am at regular chess – reaching a rating of 1550, where the max was about 1800 – and I wrote a bot for it as a senior project at BJU during the spring semester. The bot used TamperMonkey to play games on the live chess server. It had a standard game tree / evaluate structure, and I tuned some parameters using a genetic algorithms idea on AWS EC2. I’m happy that I got it to work, though (TBH) I wish that I’d put more time into it to try out new ideas. I thought that by making a personal project a formal requirement I’d have more incentive to put time into the project, but I actually experienced the opposite. That’s psychology for you!

On a related note, I’ve found it much easier to invest in projects at my job than I generally did on school projects. In fact, work often feels like my most engaged moments as a hobbyist. I’m not sure exactly why this is, but I’m grateful for it.

Roundup

It’s been a while since the last update. Three notable miniatures, all from April:

‘BJU Blind Date’: With 24 hours to get a date to the last artist series, I launched a website to coordinate blind dates to the event. I got ~50 Facebook shares and 7 signups, including my date(s), which was cool.

Animal Moral Value: I interacted with this Slate Star Codex post as Tibbar. You can see what I did by checking the comments, but basically I try (and fail) reproducing the claims of the post by commissioning a couple large Mechanical Turk surveys. I spent about $200 and I surveyed ~500 respondents. My work generated not one but two followup posts from Scott, first a ‘partial retraction’ and then a retraction of the retraction when other people invalidated my study. It was pretty fun. Also, it was my first solo attempt at Mechanical Turk (perhaps explaining why my results were irreparably flawed!), which I learned about while tutoring UPenn’s Crowdsourcing Class – a steep learning experience, but very rewarding.

[Note – don’t visit the following link – it will lock up your computer.]

BJUCoin: For a senior math project, I created a simulation of cryptocurrency mining and had a live audience (~20 people) compete to create the longest fork and win the most mining rewards. Later my friend automated the mining process so there’s way too much data at that page (hence why I say don’t visit it). All the coding was done in 24 hours due to my procrastination, but it went off without a hitch!

What next?

I have a wishlist of ideas that I’d like to bring to life. It takes a lot of emotional currency to invest in a personal project. Most of the things I do happen over very short periods of time when boredom, time, and curiosity explode in a magical way. A review of my projects folder reveals that I’ve initiated over 50 projects since November 2017:

projects3.png

The x-axis gives folder creation date; the y-axis is in log-bytes, so a 3 is a kilobyte, a 6 is a megabyte, and a 9 is a gigabyte.

Caveats: This is not an exhaustive list of my projects (but almost so). For any given project, work might be missing (on another server, deleted, etc.). Also, for gigabyte- or even megabyte- sized projects, the material may be dominated by non-code (usually generated files, occasionally images.) Also, not all of these creation dates are accurate, unfortunately: I’m not sure why.

I would consider anything 5 and above to be a ‘meaty project’, with some 4’s possibly qualifying as well. Clearly it’s been a while since I got near an 8, but not all interesting projects need to be an 8.

Anyway, some upcoming ideas include:

  • Calculating news bias with sentiment analysis and clustering of news topics. (This has been on the radar for nearly a year.) I’ve done some initial work on this in the past.
  • Collecting data on my school’s graduates, allowing society rankings and interesting analyses on marriage rates, etc. I know that the results would be interesting, but there’s so much data to scrape. I’ve done some initial work on this as well (like 2%).
  • Get reputation on StackOverflow by automatically identifying new error messages in popular open source projects (likely on GitHub) and proactively asking about and answering them, hopefully creating canonical responses.
  • Baseball:
    • Can you influence what a pitcher throws by your pattern of swinging?
    • Do batters do better against pitchers when they’ve seen a similar pitcher recently? / Can you arrange orders of pitchers in a way that’s most disorienting to batters?
  • Chess – how does time/piece odds affect the relative ELO of players?

I’m leaning towards the baseball projects, largely because it’s been so long since I did one. I’d like to get something together before next summer’s conference cycle. I’m definitely still interested in breaking into that world.

This should cover things for the next few months!

 

Microthought 0

Many hotels ask you to provide your last name and room number to use the WiFi.

In the US, three percent of people are named Smith, Williams, Brown, Johnson, or Jones.

Therefore, in a hotel with 50 rooms, there is a 78% change that one could sign in by just trying these names on each room.

In fact, when I tested this last, using only “Smith”, it took only ten rooms.

Your results may vary.

Update 0

Life Updates

I am college student, although this semester I’m traveling with a team from my school, Bob Jones University. Grad school applications feel very close. I’m studying for the Math GRE with the Princeton Review book. Ok that’s enough about me, let’s get to the good stuff.

Project Updates

Chess Slides

Start date: June 15, 2018

[See website for background]

Earlier this week I finished generating the SCC polynomials; I used ~20 hours of AWS computing time to get up to the 16×16 case. Interestingly, the 14×14 case broke Mono (the C# runtime), and I had to rewrite that part of the project in C++. If you want the stack trace, let me know, it’s some wacky memory management thing; estimating that the application was using 25-30GB of memory at the time.

Was pleased to see that several of the polynomials match existing OEIS sequences 🙂 My favorite is that the number of size-60 components is the same as the number of possible knight moves on an (n – 2)x(n – 2) board (A035008).

Also, this yields a general formula for the size of the largest component:

{n^2 \choose 4}- 8036 + \frac{15334}{3}x - \frac{1735}{4}x^2 - \frac{454}{3}x^3 - \frac{611}{24}x^4 + 12x^5 + \frac{1}{4}x^6 - \frac{1}{24}x^7.

I’m working on publishing an article about all this with a math magazine. I’ve never done this before so I’m paranoid about it falling through somehow, but I will post appropriate linkage if/when it’s live.

Media Bias Dashboard

Start date: August 9, 2018 (new)

Goal: create a live console to track political events and track reporting bias by news source. Inspiration: Bot SentinelNick Diakopolous. Right now I’m just sorting out what my approach will be, but I’m very excited about https://newsapi.org‘s free API, and I’ll probably use SentiWordNet. I’ll try to automate the backend with AWS (I have a free EC2 instance just sitting there… :))

I’ll need to find all the articles about a given event in real time, across the major media sources. (For example today everyone has a token article about the “Space Force.”) I think I can do this by looking for unusual words that appear in multiple news sources and then using some kind of similarity score to find all the articles on the given topic. (Basically a probabilistic union-find algorithm.) I don’t expect too many false positives, and I probably don’t care about false negatives.

Then, we need to evaluate the slant – pro or con – of each article towards the subject matter. I’ll likely work with just the headlines and article summaries I can pull from News API. Rather than use machine learning or Mechanical Turk etc, I want to formalize “bias” using word sentiment, which side is getting quoted, whether the article is rebutting the subject’s statements, etc.

Also, I want this to be very user-friendly, probably with its own domain name.

I tentatively predict a beta release in October. Stay tuned! 🙂

Floating Aces

Start date: July 4, 2018

[If I can get a better solution, I’ll do a better writeup. Fair? :)]

Suppose you have a 4×4 grid of playing cards – Ace, 2, 3, and 4 of each suit. You can swap any two adjacent (row/column) cards of the same rank or suit, and you can always swap an ace with the card above. Can you get the four aces to the bottom row?

(Not every combination of cards is solvable, but the vast majority are.)

The number of states is quite large, and so I’ve tried a breadth-first search that uses heuristics to pick a subset of the adjacent states to try. We assign a score to a given arrangement, and only look at new arrangements that are at least as high-scoring as the best score we’ve seen (minus a small fudge factor).

I arbitrarily define success as finding a solution in the first 100,000 boards considered.

This algorithm, as described, was solving about half my 30 tests. So I introduced randomness and various little tweaks and optimizations, tried a zillion different parameter values, and now I can solve (on average) about 83% of the test arrangements. (Some arrangements are solved 100% of the time, one was only solved 40% of the time.)

For your edification, the magic parameters are –

AceVal = 24, AceMoveDown = 9, AceMoveOver = 8, Lower = 15, QueueThrottle = 99,  Luck = 0.896.

I want a better solution, but I’ll likely have to change my approach.