Sure there's tons of outdoor stuff to do. Here are a few links that will help out. There are outdoor clubs to join as well that I'd imagine know even more activities in the area. Here's what comes to mind at the moment:
New York State Parks :: State Parks
LocalHikes - Hikes and trails near U.S. Metropolitan Areas
Hudson-Mohawk Group Outings
Parks & Trails New York - working to expand, protect and promote a network of parks, trails and open spaces throughout our state for use and enjoyment by all
Some of my favorite hike/walks close by are:
Lisha Kill Nature Preserve, Niskayuna
Five Rivers, Delmar
Plotter Kill, Rotterdam
Indian Kill Nature Preserve, Glenville
Christman Sanctuary, Duansburg
Some a bit further away, more like an hour
Vroman's Nose, further away, can't remember the town
Buck Mtn
Hadley Hill
Swimming:
State Parks, Grafton, Moreau, & several others
SwimmingHoles.info Swimming Holes and Hot Springs rivers creek springs falls hiking camping outdoors skinny dipping
Metroland Online - Summer Guide 2003
Best Swimming Hole - According to metroland.com best of 2005
Metroland Online - Best of 2005 - Readers Poll
1. Grafton State Park
2. Sacandaga Lake
3. Poestenkill Gorge
There's also a pond in Scotia for swimming, for a fee.