When to dispose and why?

I asked a question about this method:

// Save an object out to the disk
public static void SerializeObject(this T toSerialize, String filename)
{
    XmlSerializer xmlSerializer = new XmlSerializer(toSerialize.GetType());
    TextWriter textWriter = new StreamWriter(filename);

    xmlSerializer.Serialize(textWriter, toSerialize);
    textWriter.Close();
}

in the response I got this as an added remark:

Make sure you always dispose disposable resources such as streams and text readers and writers. This doesn't seem to be the case in your SerializeObject method.

So, I can tell that this is going to seem super lame for someone who has been coding C# for a year or two, but why do I have to dispose it?

Is see that testWriter has a dispose method, but shouldn't garbage collection take care of that? I came from Delphi to C#. In Delphi I had to clean up everything, so this is not a case of me wanting to be lazy. I just was told that if you force freeing up the memory that your objects take then it can cause bad stuff. I was told to "Just let the garbage collector do it".

1. So, why do I need to call dispose? (My guess is that it is because textWriter hits the disk.)
2. Is there a list of objects I need to be careful with? (Or an easy way to know when I need to call dispose?)