You are experiencing what is called Delay Onset of Muscles Soreness ("DOMS"). The exact cause of DOMS is not well understood. One school of thought attributes DOMS to lactic acid build up during exercise. Some think DOMs is caused by fluid build up due to micro-tears in the muscles caused by exercise. Yet another theory attributes DOMS to the body's repair of damaged muscle fibers.
As you exercise you are tearing muscle fibers. Your body responds by repairing the torn muscle fibers. The body also strengthens the muscle fibers to make them more resistant to future damage.
DOMS is most severe when you first start and exercise program. The only cure for DOMS is to rest for 3+ days. Taking an anti-inflammatory pain killer such as Advil can help with the discomfort.