First figure out what kind of game you are going to do. eg. Some 2D games may not need/involve any scrolling at all. If there's any elements that need scrolling then if you also care about very smooth/fluid scrolling at any speed, it's probably good idea to determine if whatever technology choices you are looking at are suitable for that. Smooth scrolling means graphics updates at a constant and high enough pace and if the game is done C#, that may turn out to be non-trivial after the game is otherwise finished if there were poor technology/algo choices somewhere along the line.
start with algebra, seriously
I have to say I've bought lots of books recommended and they all seem to require knowledge of any sort on programming. Even though people have said they've started with a certain book, they've always been at Uni or something first and had someone teaching them.
I know these books are looked down upon but I bought a Dummies book to C and it's amazing. I've been reading it on and off for a month, going over each page again and again, I'm only a quarter of the way through as there are exercises and I don't have unlimited time. However I must say it's the best out of all of them if you really have no clue, it really does teach you from the basics and helps you to understand C.
All the other books I've tried have been like French text books at school where they don't teach you to understand how to construct the language, they just give you phrases to repeat. Where as this one has really gotten me to understand, though I'm ashamed it took a Dummies book :\
It is very slow though, really moves at a snails pace, however maybe that's the key for me lol.