I do layers.. I start with a light jersey short sleeved jersey, then sometimes, depending on weather, I then put on a snug campy jacket, a barrier jacket and then sometimes a fleece over that. I have winter cycling tights, and I double up on socks, plus you can get those booties to go over your cycling shoes. I also wear a cap to keep my ears warm, under my helmet. And gavia gloves.
Works pretty well. Heck on new years my boyfriend and I were riding around town in weather in the single digits, with no issues.
So yah... layers layers layers.
A ship is safe in harbor, but that's not what ships are for.