While environmentalists would argue you should never let your engine idle due to the unnecessary greenhouse gas emissions, such an edict may run thin in sub-zero temperatures. In fact, it can be unsafe—let alone uncomfortable—to drive a car without first warming up the engine.
You should warm up your vehicle if it is carbureted. If your car is fuel-injected, then it is a personal matter of how much you can tolerate the cold.
When you have an older car with a carburetor, the engine will run smoother if it has had a chance to warm up. Older vehicles also benefit from having time for the oil to heat up, thin out, and lubricate the engine. Newer cars that use fuel injection and computer controls are good to go without idling.