The thing that bugs me is the "micro" caches. You can spend a half hour looking for some of those, and still walk away wondering if you didn't look hard enough, or if some donkey walked away with it or something like that.
It's a great way to get some exercise though, and a lot of times it can take you to interesting places that you wouldn't normally go. There are websites that list hundreds of thousands of them. When the wifey and I are travelling with the dogs we will sometimes load a few into the GPS that are on our route, and then make a few stops along the way. Makes a boring road trip a little more exciting.