Overall fitness is obviously important... just for one's general health. In most cases it probably does help with golf also.
There may, however, be SOME exercises and activities that could be detrimental to GOLFERS.
Some of the one's I've heard are bad for golf...
Swimming
Pushups
Weight lifting that greatly increases pec and shoulder girth
Forward lunges... lateral are OK
Bowling
Some yoga poses... may cause too much flexibility resulting in instability
What do you think?
So are these exercises bad for golf?
Swimming. I don't think so!
Pushups. I don't think so! (How many pushups do you need to do to get pecs half the size of Arnold's? Answer: you can't by doing pushups)
Weight lifting. I don't think so! (Try telling the world number one player. Oh, and number two. And number three...)
Bowling. I don't think so! Do you bowl 24/7? Probably not. If the ball is too heavy, then use a lighter one or get stronger.
Yoga and too much flexibility. I don't think so!
Obviously I'm assuming you're doing the above with proper technique.