Party City is like a dollar store on steroids and, you guessed it, 100% PARTY! They have a whole aisle dedicated to paper plates and plastic cups in every colour. Yup, you can theme out your party with any colour of the rainbow, or, rainbow colour!
Most of the stuff is pretty cheap, meant to be used for one night only. But, it's really easy to find some special touches to spruce of your party for cheap.
The price range is a little higher than your typical Dollar Store, but I guess that's the price you pay for selection. Luckily, there's a dollar store right next door, so if some of the items you need to have to be extra specialized, there's a chance you can grab it at The Dollar Store on the same trip and maybe save a few buck.