Analysis of parallel algorithms

百度强化技术标准规范对安全和服务的保障和引领作用,以保障建设质量和安全运行为重点,进一步修订完善城市轨道交通工程建设标准体系。

In computer science, analysis of parallel algorithms is the process of finding the computational complexity of algorithms executed in parallel – the amount of time, storage, or other resources needed to execute them. In many respects, analysis of parallel algorithms is similar to the analysis of sequential algorithms, but is generally more involved because one must reason about the behavior of multiple cooperating threads of execution. One of the primary goals of parallel analysis is to understand how a parallel algorithm's use of resources (speed, space, etc.) changes as the number of processors is changed.

Background

A so-called work-time (WT) (sometimes called work-depth, or work-span) framework was originally introduced by Shiloach and Vishkin ^[1] for conceptualizing and describing parallel algorithms. In the WT framework, a parallel algorithm is first described in terms of parallel rounds. For each round, the operations to be performed are characterized, but several issues can be suppressed. For example, the number of operations at each round need not be clear, processors need not be mentioned and any information that may help with the assignment of processors to jobs need not be accounted for. Second, the suppressed information is provided. The inclusion of the suppressed information is guided by the proof of a scheduling theorem due to Brent,^[2] which is explained later in this article. The WT framework is useful since while it can greatly simplify the initial description of a parallel algorithm, inserting the details suppressed by that initial description is often not very difficult. For example, the WT framework was adopted as the basic presentation framework in the parallel algorithms books (for the parallel random-access machine PRAM model) ^[3] and, ^[4] as well as in the class notes .^[5] The overview below explains how the WT framework can be used for analyzing more general parallel algorithms, even when their description is not available within the WT framework.

Definitions

Suppose computations are executed on a machine that has $p$ processors. Let $T p$ denote the time that expires between the start of the computation and its end. Analysis of the computation's running time focuses on the following notions:

The work of a computation executed by $p$ processors is the total number of primitive operations that the processors perform.^[6] Ignoring communication overhead from synchronizing the processors, this is equal to the time used to run the computation on a single processor, denoted $T 1$ .
The depth or span is the length of the longest series of operations that have to be performed sequentially due to data dependencies (the critical path). The depth may also be called the critical path length of the computation.^[7] Minimizing the depth/span is important in designing parallel algorithms, because the depth/span determines the shortest possible execution time.^[8] Alternatively, the span can be defined as the time $T \infty$ spent computing using an idealized machine with an infinite number of processors.^[9]
The cost of the computation is the quantity $pT p$ . This expresses the total time spent, by all processors, in both computing and waiting.^[6]

Several useful results follow from the definitions of work, span and cost:

Work law. The cost is always at least the work: $pT p \geq T 1$ . This follows from the fact that $p$ processors can perform at most $p$ operations in parallel.^[6]^[9]
Span law. A finite number $p$ of processors cannot outperform an infinite number, so that $T p \geq T \infty$ .^[9]

Using these definitions and laws, the following measures of performance can be given:

Speedup is the gain in speed made by parallel execution compared to sequential execution: $S p = T 1 / T p$ . When the speedup is $Ω(p)$ for $p$ processors (using big O notation), the speedup is linear, which is optimal in simple models of computation because the work law implies that $T 1 / T p \leq p$ (super-linear speedup can occur in practice due to memory hierarchy effects). The situation $T 1 / T p = p$ is called perfect linear speedup.^[9] An algorithm that exhibits linear speedup is said to be scalable.^[6] Analytical expressions for the speedup of many important parallel algorithms are presented in this book.^[10]
Efficiency is the speedup per processor, $S p / p$ .^[6]
Parallelism is the ratio $T 1 / T \infty$ . It represents the maximum possible speedup on any number of processors. By the span law, the parallelism bounds the speedup: if $p > T 1 / T \infty$ , then:^[9] ${\frac {T_{1}}{T_{p}}}\leq {\frac {T_{1}}{T_{\infty }}}<p.$
The slackness is $T 1 / (pT \infty)$ . A slackness less than one implies (by the span law) that perfect linear speedup is impossible on $p$ processors.^[9]

Execution on a limited number of processors

Analysis of parallel algorithms is usually carried out under the assumption that an unbounded number of processors is available. This is unrealistic, but not a problem, since any computation that can run in parallel on $N$ processors can be executed on $p < N$ processors by letting each processor execute multiple units of work. A result called Brent's law states that one can perform such a "simulation" in time $T p$ , bounded by^[11]

T_{p}\leq T_{N}+{\frac {T_{1}-T_{N}}{p}},

or, less precisely,^[6]

T_{p}=O\left(T_{N}+{\frac {T_{1}}{p}}\right).

An alternative statement of the law bounds $T p$ above and below by

{\frac {T_{1}}{p}}\leq T_{p}\leq {\frac {T_{1}}{p}}+T_{\infty }

.

showing that the span (depth) $T \infty$ and the work $T 1$ together provide reasonable bounds on the computation time.^[2]

References

^ Shiloach, Yossi; Vishkin, Uzi (1982). "An O(n² log n) parallel max-flow algorithm". Journal of Algorithms. 3 (2): 128–146. doi:10.1016/0196-6774(82)90013-X.
^ ^a ^b Brent, Richard P. (2025-08-06). "The Parallel Evaluation of General Arithmetic Expressions". Journal of the ACM. 21 (2): 201–206. CiteSeerX 10.1.1.100.9361. doi:10.1145/321812.321815. ISSN 0004-5411. S2CID 16416106.
^ JaJa, Joseph (1992). An Introduction to Parallel Algorithms. Addison-Wesley. ISBN 978-0-201-54856-3.
^ Keller, Jorg; Kessler, Cristoph W.; Traeff, Jesper L. (2001). Practical PRAM Programming. Wiley-Interscience. ISBN 978-0-471-35351-5.
^ Vishkin, Uzi (2009). Thinking in Parallel: Some Basic Data-Parallel Algorithms and Techniques, 104 pages (PDF). Class notes of courses on parallel algorithms taught since 1992 at the University of Maryland, College Park, Tel Aviv University and the Technion.
^ ^a ^b ^c ^d ^e ^f Casanova, Henri; Legrand, Arnaud; Robert, Yves (2008). Parallel Algorithms. CRC Press. p. 10. CiteSeerX 10.1.1.466.8142.
^ Blelloch, Guy (1996). "Programming Parallel Algorithms" (PDF). Communications of the ACM. 39 (3): 85–97. CiteSeerX 10.1.1.141.5884. doi:10.1145/227234.227246. S2CID 12118850.
^ Michael McCool; James Reinders; Arch Robison (2013). Structured Parallel Programming: Patterns for Efficient Computation. Elsevier. pp. 4–5.
^ ^a ^b ^c ^d ^e ^f Cormen, Thomas H.; Leiserson, Charles E.; Rivest, Ronald L.; Stein, Clifford (2009) [1990]. Introduction to Algorithms (3rd ed.). MIT Press and McGraw-Hill. pp. 779–784. ISBN 0-262-03384-4.
^ Kurgalin, Sergei; Borzunov, Sergei (2020). The discrete math workbook: a companion manual using Python. Texts in Computer Science (2nd ed.). Cham, Switzerland: Springer Naturel. ISBN 978-3-030-42220-2.
^ Gustafson, John L. (2011). "Brent's Theorem". Encyclopedia of Parallel Computing. pp. 182–185. doi:10.1007/978-0-387-09766-4_80. ISBN 978-0-387-09765-7.

[shiloach-1] Shiloach, Yossi; Vishkin, Uzi (1982). "An O(n² log n) parallel max-flow algorithm". Journal of Algorithms. 3 (2): 128–146. doi:10.1016/0196-6774(82)90013-X.

[brent-2] Brent, Richard P. (2025-08-06). "The Parallel Evaluation of General Arithmetic Expressions". Journal of the ACM. 21 (2): 201–206. CiteSeerX 10.1.1.100.9361. doi:10.1145/321812.321815. ISSN 0004-5411. S2CID 16416106.

[jaja-3] JaJa, Joseph (1992). An Introduction to Parallel Algorithms. Addison-Wesley. ISBN 978-0-201-54856-3.

[kkt-4] Keller, Jorg; Kessler, Cristoph W.; Traeff, Jesper L. (2001). Practical PRAM Programming. Wiley-Interscience. ISBN 978-0-471-35351-5.

[uv-5] Vishkin, Uzi (2009). Thinking in Parallel: Some Basic Data-Parallel Algorithms and Techniques, 104 pages (PDF). Class notes of courses on parallel algorithms taught since 1992 at the University of Maryland, College Park, Tel Aviv University and the Technion.

[casanova-6] ^ ^a ^b ^c ^d ^e ^f Casanova, Henri; Legrand, Arnaud; Robert, Yves (2008). Parallel Algorithms. CRC Press. p. 10. CiteSeerX 10.1.1.466.8142.

[cacm-7] Blelloch, Guy (1996). "Programming Parallel Algorithms" (PDF). Communications of the ACM. 39 (3): 85–97. CiteSeerX 10.1.1.141.5884. doi:10.1145/227234.227246. S2CID 12118850.

[spp-8] Michael McCool; James Reinders; Arch Robison (2013). Structured Parallel Programming: Patterns for Efficient Computation. Elsevier. pp. 4–5.

[clrs-9] ^ ^a ^b ^c ^d ^e ^f Cormen, Thomas H.; Leiserson, Charles E.; Rivest, Ronald L.; Stein, Clifford (2009) [1990]. Introduction to Algorithms (3rd ed.). MIT Press and McGraw-Hill. pp. 779–784. ISBN 0-262-03384-4.

[10] Kurgalin, Sergei; Borzunov, Sergei (2020). The discrete math workbook: a companion manual using Python. Texts in Computer Science (2nd ed.). Cham, Switzerland: Springer Naturel. ISBN 978-3-030-42220-2.

[11] Gustafson, John L. (2011). "Brent's Theorem". Encyclopedia of Parallel Computing. pp. 182–185. doi:10.1007/978-0-387-09766-4_80. ISBN 978-0-387-09765-7.

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

v t e Parallel computing
General	Distributed computing Parallel computing Parallel algorithm Massively parallel Cloud computing High-performance computing Multiprocessing Manycore processor GPGPU Computer network Systolic array
Levels	Bit Instruction Thread Task Data Memory Loop Pipeline
Multithreading	Temporal Simultaneous (SMT) Simultaneous and heterogenous Speculative (SpMT) Preemptive Cooperative Clustered multi-thread (CMT) Hardware scout
Theory	PRAM model PEM model Analysis of parallel algorithms Amdahl's law Gustafson's law Cost efficiency Karp–Flatt metric Slowdown Speedup
Elements	Process Thread Fiber Instruction window Array
Coordination	Multiprocessing Memory coherence Cache coherence Cache invalidation Barrier Synchronization Application checkpointing
Programming	Stream processing Dataflow programming Models Implicit parallelism Explicit parallelism Concurrency Non-blocking algorithm
Hardware	Flynn's taxonomy SISD SIMD Array processing (SIMT) Pipelined processing Associative processing MISD MIMD Dataflow architecture Pipelined processor Superscalar processor Vector processor Multiprocessor symmetric asymmetric Memory shared distributed distributed shared UMA NUMA COMA Massively parallel computer Computer cluster Beowulf cluster Grid computer Hardware acceleration
APIs	Ateji PX Boost Chapel HPX Charm++ Cilk Coarray Fortran CUDA Dryad C++ AMP Global Arrays GPUOpen MPI OpenMP OpenCL OpenHMPP OpenACC Parallel Extensions PVM pthreads RaftLib ROCm UPC TBB ZPL
Problems	Automatic parallelization Deadlock Deterministic algorithm Embarrassingly parallel Parallel slowdown Race condition Software lockout Scalability Starvation
Category: Parallel computing

石花膏是什么做的	执行标准是什么意思	孕期血糖高有什么症状	第二视角是什么意思	阴虚吃什么药效果最好
梦到女朋友出轨是什么意思	凶狠的反义词是什么	酱油什么时候发明的	什么是毒龙	p2大于a2什么意思
狗懒子是什么意思	熟络是什么意思	流产后吃什么	咳白色泡沫痰是什么病	脾虚吃什么水果
鳄鱼的天敌是什么	嗓子疼吃什么	干疮是什么样子的图片	夏天吃姜有什么好处	为什么会流鼻涕

为什么拉屎会有血hcv8jop7ns4r.cn	弓箭是什么时候发明的hcv9jop3ns4r.cn	什么南瓜hcv7jop9ns5r.cn	麾下什么意思adwl56.com	什么男什么女的成语hcv9jop3ns2r.cn
每天吃葡萄有什么好处和坏处zhiyanzhang.com	2002年出生属什么hcv8jop4ns4r.cn	喝黑苦荞茶有什么好处和坏处hcv9jop2ns6r.cn	什么是健康证hcv9jop1ns6r.cn	一什么杨桃hcv8jop6ns7r.cn
风调雨顺的下联是什么hcv8jop3ns2r.cn	灶性肠化是什么意思cl108k.com	排卵试纸阴性是什么意思hanqikai.com	常熟有什么好玩的地方hcv8jop4ns2r.cn	石膏的主要成分是什么hcv9jop5ns1r.cn
0a是什么意思hcv9jop1ns0r.cn	月经不调吃什么luyiluode.com	脾五行属什么hcv9jop6ns8r.cn	小妾是什么意思hcv8jop4ns5r.cn	老炮是什么意思hcv8jop9ns7r.cn