To be clear, this is actually a "popular science" book. The fact that it was written by the creator of the theories of Relativity helps because Einstein himself tries to break it down into relatively (see what I did there) bite sized pieces.
To follow the book, you need to know some Math, but high school Algebra and a minute amount of Trigonometry are all that you need. No Calculus, no Riemannian Geometry.
If you are looking for an Advanced and Mathematically rigorous exposition of the topic, this is probably not the right book for you. However, if you are looking for a good introduction to the theories of Relativity, you can do a lot worse than this book.
That said, I am sure that even those used to reading more advanced work can benefit from reading this, (even if they decide it is light reading), as the writing is a great example of how to take a complex and really advanced topic and explain it in really simple terms.