High performance computing

• concetti fondamentali e tecniche di strutturazione e progetti di computazioni parallele, metodologie e paradigmi di parallelizzazione, modelli di programmazione parallela, loro implementazione e modelli di costo;
• architetture ad alte prestazioni: multiprocessor a memoria condivisa, multicomputer a memoria distribuita, cluster, ed altre.

This course deals with two interrelated issues in high-performance computing:

1.       fundamental concepts and techniques in parallel computation structuring and design, including parallelization methodologies and paradigms, parallel programming models, their implementation, and related cost models;

2.       architectures of high-performance computing systems, including shared memory multiprocessors, distributed memory multicomputers, clusters, and others.

Both issues are studied in terms of structural model, static and dynamic support to computation and programming models, performance evaluation, capability for building complex and heterogeneous applications and/or enabling platforms, also through examples of application cases. Technological features and trends are studied, in particular multi-/many-core technology and high-performance networks.

An initial part is dedicated to review basic concepts and techniques in structured computer architecture, in order to render the different backgrounds of students as uniform as possible.

Syllabus

1.       A structured computer architecture primer: review of level structuring, processing modules, firmware architecture, assembler machine, memory hierarchies and caching, process level and interprocess communication

2.       Methodology for structuring and programming high-performance parallel applications, basic cost models: metrics, elements of queueing theory and queueing networks, load balancing, static and dynamic optimizations

3.       Parallel paradigms: stream-parallel (pipeline, data-flow, farm, divide and conquer, functional partitioning), data-parallel (map, fixed and variable stencils, reduce, prefix), and their compositions

4.       Run-time supports of parallel programs and their optimization

5.       Shared memory multiprocessors: SMP and NUMA, cost models; interconnection networks and their evaluation: indirect and multistage networks, direct and cube networks, fat tree, on-chip networks

6.       Distributed memory architectures: multicomputers, clusters, distributed heterogeneous platforms, high-performance communication networks

7.       Advanced research and/or technological issues: multi-/many-core, multithreading simd/vectorization/gpu, pervasive high-performance computing.

1. Computing architecture primer
2. Metodologie per strutturare e programmare applicazioni parallele ad alte prestazioni; fondamenti di modelli di costo: metriche, elementi di teoria delle code e reti di code, bilanciamento del carico, ottimizzazioni statiche e dinamiche
3. Paradigmi di parallelizzazione: stream-parallel (pipeline, data-flow, farm, divide and conquer, partizionamento funzionale), data-parallel (map, stencil fissi e variabili, reduce, prefix), and loro composizioni
4. Meccanismi di cooperazione: comunicazioni dedicate e collettive, oggetti condivisi, strumenti di compilazione, supporti a tempo di esecuzione e loro ottimizzazioni
5. Multiprocessor a memoria condivisa. SMP, NUMA, modelli di costo; reti di interconnessione e loro valutazione: reti indirette e multistadio, reti diretti e cubi, fat tree, reti on-chip
6. Architetture a memoria distribuita: multicomputer, cluster, piattaforme distribuite eterogenee, reti di comunicazione ad alte prestazioni
7. Aspetti avanzati di ricerca e/o tecnologici: multi-/many-core, multithreading simd/vectorizzazione/gpu, calcolo pervasivo ad alte prestazioni.