As you mentioned you had gone down the SLR route and found it didn't work for you, I would recommend one of the more advanced point & shoot cameras.
There is deffinetly a major jump in image quality between the entry level DSLRs and the High end point & shoots. But it's going to come down to your needs and your budget.
In the advanced Point & shoot market I would look at two camera's;
The Canon Powershot G12
http://www.dpreview.com/products/canon/compacts/canon_g12/review
and;
The Panasonic Lumix LX5
http://www.dpreview.com/products/panasonic/compacts/panasonic_dmclx5/review
The Canon is a bit bigger and heavier, but has a few more dials for easy and quick adjustments and has a large 2.8" flip out screen for easier viewing.
The Panasonic is a lighter and more compact body, but still has all the abilities that the larger G12 has, though with no flip out screen.
Both of these cameras do everything from full auto straight up to full manual. They also have the ability to shoot RAW (a type of file format best suited for post editing work).
(This gets long)
After these you could step up to a "Bridge Camera" like the Nikon V1, Panasonic GX1, or Sony Nex-5. But personally I don't recommend it.
The benefit of the Advanced Point & shoots is that they are extremely compact, with everything built in. Though they don't except additional accessories (besides flashes), most people using them aren't looking to build on their system.
Fulled sized and entry level DSLRs allow you to add lots of accessories and build your kit to exactly what you want. Though they have the disadvantage of being large and needing to carry more lenses as your kit and needs grow.
The Bridge Cameras carry the advantage of being small, but the disadvantage of you possibly needing a larger kit as you learn what you are doing, and then you just end up at the same point as a full sized DSLR, only with the issue of there not being as many lenses built specially for the cameras (yet).
I used to work in a camera shop (for 3 years) and saw way too many people getting a camera that they are only going to use 25% of. Most of the issue is that they will get something like the D3100 mentioned above, but then when it comes to going on a trip the owner will say, "well I just kind of want to saunter and don't really feel like carrying something that large around all day" and thus the camera gets left at home.
But you need to make decisions about whether or not a large camera will work with you and how you vacation and live. From there you can really narrow down what you need and want.
Just my two (hundred) cents.