Multiprocessing

Multiprocessing is traditionally known as the use of multiple concurrent processes in a system as opposed to a single process at any one instant. Like multiprogramming, which allows multiple threads to be used for a single process, multiple processors may be used with more than one processors.

Multiprocessing can also be the use of more than one CPU in a computer system in order to improve performance. The term can also be used to describe a number of completely separate computers running together, but this is more typically referred to as clustering.

Multiprocessing for general tasks is often fairly difficult to achieve due to various programss holding internal data, known as state (or context). Essentially the programs are typically written in such a fashion that they assume their data is incorruptible. However if another copy of the program is running on another processor, the two copies can interfere with each other by both attempting to read and write their state at the same time. A variety of programming techniques are used to avoid this problem, including semaphores and other checks and blocks which allow only one copy of the program to change such values at a time.

Another problem is that processors often use a speed-increasing technique known as caching in which small pools of very fast memory are associated with each processor in order to allow them to work with temporary values very quickly. This can lead to a situation in which each processor is working in a separate cache, rather than in the shared memory; changes to a processor's local cache will not be communicated to other processors until the contents of the cache are written to shared memory. This cannot be helped via programming techniques because it is invisible to the programs themselves. In this case the problem requires additional hardware in order to make sure that all caches on the various processors are up to date, and synchronized with one another.

There are a number of ways to solve this latter problem, of varying complexity. The most common for smaller systems is symmetric multiprocessing (SMP) while larger systems use non-uniform memory access (NUMA) multiprocessing. Multiprocessors may be thought as subgenre of distrubuted shared memory system, namely hardware one.