Spirit was originally designed to be the mana and health regeneration stat, but has been in some ways rebuilt in the expansions. The health regeneration component is vastly ignored, since it does not function while in combat, and health is much faster regained out of combat using food. Spirit mana regeneration is active in combat, but only five seconds after the last spell has been cast (timer starts at the beginning of a spell cast). This is called the Five Second Rule, or FSR. You'll often come across the expression ooFSR meaning "out of" - the time while in combat that mana is being regenerated through Spirit.
In Patch 3.1 it was decided that the ooFSR mana regeneration was too powerful, primarily as regards to healers, and it was drastically reduced. The vast majority of classes considered Spirit to be a useless stat, though it was given additional use through a Warlock's Fel Armor and Demonic Aegis (30% of Spirit converted into Spellpower), and through a Mage's Molten Armor (35% of Spirit is converted into critical strike rating). Shadow Priests benefit from Spirit because of Improved Spirit Tap which allows them to regenerate additional mana upon critical strikes while casting (inside FSR). Spirit is ignored by Holy Paladins, who favor their mana regeneration to come from critical heals and MP5, and therefore stack Intellect. It is also much less desired by Restoration Shamans who tend to prefer MP5 as well. Holy/Discipline Priests and Restoration Druids are a different story.
Restoration Druids have the Intensity talent, which allows them to have a percentage of their ooFSR regeneration while iFSR. Resto Druids also have the Living Spirit talent that further increases their Spirit by 15%. The Improved Tree of Life talent gives the Druid additional spellpower as a percentage of their Spirit.
Holy Priests get a lot from Spirit as well. Both Holy and Discipline healing Priests take the Meditation talent in the Discipline tree, which allows them to have a percentage of their ooFSR regeneration while iFSR. Holy Priests also get the Spiritual Guidance talent which gives them 25% of their Spirit as spellpower. Additionally Holy Priests get the Holy Concentration talent which, when proc'd, gives them 50% of their ooFSR regen while iFSR (on top of the 50% from Meditation). Finally, the Spirit of Redemption talent grants an additional 5% Spirit.
Discipline Priests, however, only have two useful Spirit-related talents. The first is, of course, Meditation to help their iFSR regeneration. The second is Enlightenment, which increases their total Spirit by 6%. In general, Discipline healers will prefer gear with MP5, not spirit, but a balance is certainly required.
Spirit is found on Blue Gems.