Whether or not they know it as Newton's Third Law, most people have probably heard the statement "For every action there is an equal and opposite reaction." That expression is the principle behind rocketry. The action is the expulsion of gas through an opening; the reaction is the rocket's liftoff. It's not unlike what happens when you blow into a balloon, then release it. As air escapes (that's the action), it propels the balloon (that's the reaction), making the balloon zip through the room until its air is completely gone.
In order to create a forceful expulsion of gas from a rocket, fuel in a combustion chamber is ignited. The fuel can be in the form of solid or liquid substances; some rockets ("hybrid launch systems") may make use of both. These substances are the propellants that characterize rockets as either "solid-rocket motors" or "liquid-rocket engines." For the fuel to burn, oxygen or another oxidizing substance must be present. When the fuel burns, gases accumulate and pressure builds until the gases are expelled through an exhaust nozzle.
In solid-rocket motors, the fuel and oxidizing chemicals are suspended in a solid binder. Solid motors are used as boosters for launch vehicles (such as the space shuttle and the Delta series). They're very reliable, and they're simpler than liquid engines, but they're difficult to control. Once a solid rocket is ignited, all its fuel will be consumed, without any option for adjusting thrust (force).
Liquid-rocket engines are more powerful than solid-rocket motors—they can generate more thrust—but the price of their power is complexity, in the form of many pipes, pumps, valves, gauges, and other parts. Liquid rockets require attention to storage issues and oftentimes the need to maintain very cold temperatures. Their complexity affects vehicle reliability, if only because the introduction of more components means the introduction of more opportunities for problems to occur.