Most frustrations we feel while trying to lose weight, we bring upon ourselves: a bad day eating, a missed day of exercise - we think of it as having "blown it" and get frustrated. Which tends to lead to giving up and failure. I can't say it enough -we're human, we're expected to have bad days.
Go with the flow. As long as bad days are the exception and not the rule, you're still on the right path. So don't be frustrated. And moreover, don't let the frustration lead to giving up.