There's this used music shop back in my hometown in NY where I would go every couple of weeks to see what CDs they got in and what not. I haven't found a similar place until we walked into Lunchbox Records. Not only did they have cheaper CDs, but they actually had vinyl albums, something we rarely see outside of a garage sale.
The selection is decently organized alphabetically. Thankfully they have the new and used music separated and clearly labeled so you know exactly which section to search through. I tackled the used CDs while the miss' went after the albums and I was impressed with the prices of the used stuff. The CDs seem to go for an average of $4-5 and my wife picked up three vinyls for $5 each. The vinyls were all in great condition, scratch-free and played beautifully when we popped them in the record player at home.
The only reason I give Lunchbox four stars is due to the parking situation. There are no parking spaces in front and apparently the lots on the side of the building are vigilantly guarded by the other stores who will report squatters. We ended up parking across the street in the huge Family Dollar lot and then ran across Central Avenue. Not the most ideal situation, but totally worth it if you want to spend some time perusing for some cheap tunes.