My girlfriend and I came to see the pandas in late May. Coming early in the season, and especially on a weekday, is great because the zoo caters to school groups at this time of the year. You're not going to get huge crowds and we almost felt like we had the place to ourselves.
The down side to visiting early in the season is that many things are closed. We tried a few different restaurants which were either closed or under renovation. we also couldn't win at the Coke machine either as it ate my $5 bill. We ended up at the restaurant (if you can call it that) at the front gates where the selection was miserable.
The best part of our visit was the panda exhibit. We were there right when the gates opened so we missed the swarms of school kids that would be coming later. There were only a handful of people there so we were able to get all the photos we wanted without being obstructed. The bonus here was that one of the pandas was quite active which is apparently a rare site to see.
There is certainly a wide variety of animals to be seen and you're going to do a lot of walking to see them all. The only exhibit I would skip is the Canadian Domain. There isn't anything you haven't seen here before and there is a HUGE hill that you have to walk back up after being disappointed by the area.