Splitting objects into their most fundamental parts

Not sure if the title captures what I'm trying to say here.

When designing in OO should I be splitting my objects up into their most specific areas - so if I have a factory object that deals with creating objects but later on i come across a way of creating objects for another purpose even though they may be the same objects is it worth creating a seperate fcatory or just add to the exsiting.

My biggest worry is bulking up classes with tons of stuff, or splitting objects and diluting my projects into a sea of classes.

Any help?

EDIT:

I guess on a side note/sub topic part of me wants to find out the level of granularity you should use in a program. Kind of, how low should you go?