Find a coffee that you like. I know you're talking about making it at home, but I'll provide an example from my own life. When I go to Starbucks, I always get a flavoured latte. Vanilla, gingerbread, whatever. When I go to my neighbourhood coffee shop, I always get a "plain" latte because I actually LIKE the taste of their coffee (as opposed to the taste of their syrups, like at Starbucks). Experiment with different beans and different roasts until you find the one that you most enjoy drinking... and then you should be fine adding just your skim milk!