I would wait as well. Some people are odd about having others ask them out after a few days and might think you're trying to move too fast. Also, I'm a fan of giving it some time because then you can experience any quirks the person might have as you spend more time with them. Case in point - I once had someone ask me out, and I was so enthused that I agreed immediately. Said guy turned out to be a possessive asshole when he was alone with me. >_<

I'd give it some time. Spend time with the guy, but don't start dating just yet. Ask him to do things with you, but don't make it a date - just do stuff as friends. Go see a movie, go to an arcade, etc. Get to know his quirks, how he acts when you two are alone, and so on before you ask him out. :)