I've been running the betas on a spare drive, and so far I'm impressed enough to pay the upgrade fee. I was running the very first beta full time at one point, but the next one broke core audio so I had to go back to 10.2 so I could keep taping.
The only thing I don't like about it is the brushed metal finder. Apple's breaking their own design guidelines, and they're making it ugly when they do it.
I'm getting sick of brushed metal going into every application, they may as well make the entire OS metal (blech!).
But since Panther is so amazingly fast, I think I can deal with the nasty metal.
Edit - And yes, you can upgrade from any X install to 10.3. I'd recommend a format and a fresh install, though. Usually tends to work out better.