Criminal records is tops for buying new release vinyl. There is no other place that I have found that has this many indie records all offered at very competitive prices. The staff is very friendly and knowledgeable, with just the right amount of record shop smugness. They have even started selling gear (turntables and such) although I would prefer a hi-fi shop for this kind of purchase. They will custom order just about anything, but will ask for a deposit as evidenced by an unclaimed Robin Thicke record.
Not much else to say really except that they also have a large selection of band merch if you are looking for the T of that band you couldn't get tickets for.