The historical Penn Station structure is magnificent and gorgeous, and it's worth going out of your way to visit. Too bad that's not where the "real" train station is nowadays, though.
"Dumpy" is a good word for the real waiting area. I've only been inside of three train stations (Pittsburgh, Philadelphia, and Providence) and Pittsburgh's was definitely the lousiest.
Regardless, taking the train to Philadelphia was a great experience. I had a ton of leg room (I'm 6'3") and a set of power outlets, and the ability to get up at any time to wander around or get a cup of coffee or snack was great. It's also been one of my little dreams to take a train around Horseshoe Curve, so I got to check that one off my list, too.
If your schedule can accommodate it, I'd definitely suggest taking the train instead of flying.