As the name suggests, RAW is the raw data captured by the camera's sensor.
If you don't know what I mean by a Bayer Pattern sensor, read up on it a little (Wikipedia will do). Shooting RAW saves the data BEFORE the in-camera de-bayer, allowing you a much larger range of control.
Essentially, shooting RAW saves the ENTIRE range of dynamic range that the sensor is capable of exposing at once (in-camera conversion to JPG throws away dynamic range). It also, as rastasean says, saves the entire range of colour information, allowing you to fix White Balance at will. It allows you to use a different, more accurate de-bayering algorithm (this depends on the RAW decoding software you use) to get the absolute maximum amount of clean, non-aliased detail possible.
It also bypasses all of the crappy in-camera settings like sharpness, de-noising, contrast, saturation, etc. The more expensive the camera, the less important this is, but "bridge" style cameras with RAW shooting often have roots in mass consumer appeal - and mass consumer appeal usually involves "sharp, bright, striking" images... I.E., the in-cam JPG production of many cameras involves many random automated contrast, sharpness & saturation increases - which you have no control over, and all of which throw away image data that you might want. Shooting RAW bypasses everything and allows you to do it yourself on a computer afterward.
Advantages - better image quality, more control over the look of the image.
Disadvantages - larger file size, more work to process.
RAW is essentially the digital equivalent of a 35mm negative, where JPG is the equivalent of a standard 35mm print.
I never shoot anything other than RAW.