Freemont is worth going to at least once if you are visiting Vegas. There is plenty to look at and usually a random assortment of street performers can be seen. As a local I think going on the weekend is your best bet. While it is busier, with the area, that's a good thing.
There are a lot of homeless people that hang around the area so when you are out in the middle of the week at about 11pm it's..interesting.