When / where should I use capacitors?

I appologize if this is a pretty basic question, I'm a software engineer by trade and do electronics stuff for fun.

I'm trying to figure out when, where, and more importantly why you would use caps a circuit?

I know what the function of a cap is for, to store electrons, or to filter out a particular frequency. But I'm somewhat lost as to when and why you would use them.

For example, in the Atmega schematics, they show two caps around the legs of the crystal (http://www.arduino.cc/en/Main/StandaloneAssembly). Why would you do that?

Any help, resources or rules of thumb you can pass on to me would be most appreciated!

-dan