Lots of slender people throughout the world eat plenty of rice. Why don't you have one serving of your favorite rice daily and one serving of brown rice? Just think of them as totally different foods. That works for my husband. White biscuits are his food of choice; he'll eat whole wheat ones, but he just thinks of them as different food. They don't take the place of his favorite.
Of course, watch the amounts of oil and increase the veggies. Best wishes!