How do bikes stay upright?

After my son asking about how bikes balance, I realised that I didn't really understand it myself. This being me, I pondered the question on and off for a few days until I felt like I understood the fundamentals. It is actually pretty interesting, and ties together quite a few concepts from physics and control theory, so I decided to write a little article about it.