Don't give up! If you saw a 23 pound loss in just 4 weeks, it is likely that some of that weight was water weight, and your gain may very well just be an increase in water retention. Any time you increase your activity levels, your muscles increase their glycogen stores. Each molecule of glycogen requires extra molecules of water that it is bonded to, so when you are rebuilding glycogen stores, water also increases.
Weight loss doesn't typically happen in a perfect, linear fashion. Sometimes you may be at the same wight for what feels like a long time, but then you begin losing again. Similarly, like what you experienced, you may have weeks where you exceed the recommended average of 1-2 pounds per week, and then you may feel like you are "stuck."
Stick with it :) A half-pound for one weigh-in is just a drop in the bucket and could easily be related to hormones, how much sodium you've had, etc. The changes on the scale will come! You might also want to try focusing on some non-scale ways to measure your progress. That can often help motivation when the scale doesn't seem to be moving: www.sparkpeople.com/resource/motivation_ar
Congratulations on your amazing success so far!