The Olbrich Botanical Gardens make for a great (free!) summer afternoon. We went during the butterfly bloom, well worth the $7, and enjoyed the surrounding grounds.
The gardens are easily walkable, there is also a small bus, and can be viewed in an hour or so. Plenty of spots to grab a seat or read in the quiet. The gardens are well kept and the occasional guide is sure to have an interesting tidbit.
The month-long butterfly bloom is awesome. The plant and bird life in the conservatory make for an interesting and eye-catching backdrop to the butterflies.