Sounds like you are doing lots of right things
Back in 2003, I was feeling much like you do now. In researching nutrition I was enlightened about the steroids, hormones, antibiotics and other things added to meat (including poultry). I had a notion that these additives some of which were used to fatten animals were fattening me. I became a vegetarian and steadily lost 1-2 pounds a week until I reached my goal weight. (50 pounds lost) Maybe you would find this works for you too. Bonus was feeling renewed energy and lightness of spirit. Several years ago I became a pescatarian, choosing sustainable fish to eat occasionally.
I wish you success with your goals.