Having frequented Trader Joe's on the west coast, I was elated when a TJ's opened in Madison. Trader Joes is great for wine, cheese, eggs, dairy, meats and fish, juices, and many other basics, with an eclectic variety of specialty foods, most of which is sold under the Trader Joe's label.
Some of the packaged and frozen items can be hit or miss, and you probably wouldn't want to do all your shopping at TJ's, but it's a great alternative to your standard supermarket and has more reasonable prices than Whole Foods (aka "Whole Paycheck"). Staff can be good sources for recommendations, and there's usually a free sample booth for featured products. Just be careful--it's all too easy to go in for just a few things and walk out with a lot more!