Celebrate that special occasion in a romantic bed & breakfast like the White Lace Inn, The Barbican, Colonial Gardens, Scofield House all located close to downtown or the seclusion of Little Harbor Inn or Connemara B&B, or any of the other bed & breakfasts or inns located in and around Sturgeon Bay. Either that, or a waterfront resort with a beautiful view (Stone Harbor, Westwood Shores, Bay Shore Inn, Glidden Lodge, Cliff Dwellers).
If you're looking for a nice dinner to commemorate the event, the Sage Restaurant, the Vineyard or Stone Harbor's Restaurant is a perfect dining atmosphere for an evening out.  As for more casual dining,  make sure you check out the Inn at Cedar Crossing, The Neighborhood Pub & Grill, Bluefront Café, Galley Café or Scaturo's Restaurant.  Think about taking in a show at Third Avenue Playhouse.
If the two of you are nature lovers, take a hike in Whitefish Dunes State Park or Potowatomi State Park, which are 10 and 3 miles away from downtown.  Then there's also the Ahnapee County Bike Trail which is on an old railroad bed that takes you through many scenic locations.  With 9 different lighthouses in the county, four of which would make a good day trip, complete with picnic in the middle and maybe a stop or two at a local winery for tasting provides relaxing activity.  Perhaps a calming boat ride on Door County Cruises to see the shorelines of Sturgeon Bay aboard a retired Chicago Fire Boat. AND last but not least, remind your loved one of this occasion with a gift from one of the four fantastic jewelry stores in the area: Gold & Silver Creations, T. Simon, Draeb Jewelers, or Gwendolyn's.