The info that syncs from the Fitbit includes all moderate and vigorous activity. I find that it counts things like an easy walk around the neighborhood with my kids, which doesn't elevate my heart rate and isn't something I would consider to be "exercise". The only way to avoid that is to manually enter your own workouts based on the activities you're doing.
Hope that helps,
"You may have a fresh start any moment you choose, for this thing that we call "failure" is not the falling down but the staying down." Mary Pickford