Okay, so I have covered geocaching a couple times. Perhaps you followed the Wikipedia links…perhaps not. In either case, here is a good video about Geocaching.
Some videos such as the one below shows that it seems to be a largely group activity:
It seems lots of people go out in groups or teams.
A few notes. Some may think having a GPSr ruins the point, but it only gets you to 10 feet at best, and when you add that person who hid it had a receiver that was at best 10 feet off, you end up with a fairly large area to search, and will need to rely on the clues in the cache description to find it. Also, most parks have many trails, so finding which trail will take you closest to the cache is also a challenge.
The cache will have sometimes have some items in it. It is good practice that if you take a trinket you leave one in its place. You will almost always fill out the log included with it.
You should also practice cache in, trash out, where by you pickup trash on your way out of the location to clean up the park or whatever location you are at.