Back Case study 02 — Creative Technology

OTBSync

A computer-vision web app that turns any physical chess board into a chess computer. Move real pieces, and they play online.

Role: Creative Technologist & Designer
Tools: Claude · TypeScript · Homegrown computer vision · WebRTC · Lichess API
Where: otbsync.com, live in beta

This game: 30 moves at 84% auto-commit accuracy. 26 read straight off the board, 0 misread, 5 user assists (4 moves and 1 resync).

Physical board, digital play

Online chess means staring at a screen, but the game was made to be played over the board. OTBSync bridges the two: point your camera at any real chess board, and the app reads your moves as you make them, keeps the digital position in sync, and speaks your opponent's replies out loud. You play on wood; the internet keeps up.

A laptop camera can watch the board directly, or a phone can be paired as a wireless camera over WebRTC. From there, the same pipeline drives every mode: rated play on Lichess, direct games with a friend, bot games, and an in-house analysis board with engine evaluation, arrows, and PGN import and export.

Try it at otbsync.com

How it works

All of the vision runs in the browser. Calibration fits a grid to the board automatically, modeling both perspective and lens distortion, and tracks it if the camera drifts. Each square is then read through several kinds of evidence at once: color separation, texture energy, occupancy, motion, and a session-local visual classifier that learns your exact pieces.

Instead of trusting the camera blindly, the evidence is checked against the moves that are actually legal in the position. A conservative policy commits a move only when the evidence supports it, and asks, waits, or retries instead of guessing. That safety layer is what makes it reliable on real-world boards with bad lighting, odd angles, and hands in the frame.