1. Intel Many Integrated Cores Architecture
Martin Kruliš
by Martin Kruliš (v1.2)

2. Intel Many Integrated Cores
History
Original idea – there is a lot of silicon on a CPU die
Use many x86 cores to create a GPU
2006 Project Larabee
The GPU could not keep up with AMD and NVIDIA
2007 Teraflops Research Chip (96-bit VLIW arch)
2009 Single Chip Cloud Computer (48 cores)
2010 Knights Ferry (32 cores)
2011 Knights Corner (60 cores based on Pentium 1)
2013 Knights Landing (72 cores based on Atom)
Knights Hill announced
by Martin Kruliš (v1.2)

3. Intel MIC Architecture
The Xeon Phi Device
Many simpler (Pentium/Atom) cores
Each equipped with powerful 512bit vector engine
by Martin Kruliš (v1.2)

4. Intel MIC Architecture
Software Architecture
Xeon Phi is basically an independent Linux machine
Example
by Martin Kruliš (v1.2)

5. Usage Modes Modes of Xeon Phi Usage OpenCL device
Standalone computational device
Connected over TCP/IP (SSH, …)
Using low-level symmetric communications interface
MPI device
Communicating over TCP/IP or OFED
May be used in both ways (ranked from host or device)
Offload device
Explicit mode offloading
Implicit mode offloading
OFED = Open Fabric Enterprise Edition
by Martin Kruliš (v1.2)

6. OpenCL Xeon Phi as OpenCL Accelerator Card
Requires Intel OpenCL platform
HW to OpenCL mapping
Xeon Phi card = OCL compute device
Virtual core = OCL compute unit
Work items in work group are processed in a loop
Unrolled 16x and vectorized when possible
OCL manages thread pool
One thread per virtual core
Work groups are assigned to threads as tasks
Better to run more WGs and less WIs then on GPU
The Intel OpenCL library directory must be the first directory in the LD_LIBRARY_PATH list.
Example
by Martin Kruliš (v1.2)

7. Standalone Device Using Xeon Phi as Standalone Device
The device is autonomous linux machine
The code is cross-compiled on host and deployed on Xeon Phi (including libraries)
Complete freedom in parallelization techniques
OpenMP, Intel TBB, Intel CILK, pthreads, …
Communication has to be performed manually by
Symmetric communication interface (SCIF)
TCP/IP stack, which uses SCIF as data link layer
Useful for extending master-worker applications
Example
by Martin Kruliš (v1.2)

8. SCIF Symmetric Communications Interface
Socket-like interface that encapsulates PCI-Express data transfers
Message passing and RMA transfers
Memory mapping techniques
Device memory may be mapped to host address space
Host memory and memory of other devices can be mapped to address space of a device
Upper 512G (32x 16G pages)
Supports direct assignment virtualization model
All other communication methods are built on SCIF
by Martin Kruliš (v1.2)

9. Offload Model Offload Execution Model
Both host and device code is written together
Offload parts for the device are explicitly marked
The compiler performs the dual compilation, inserts the stubs, handles the data transfers, …
Explicit offload model (a.k.a. Pragma offload)
Everything is controlled by programmer
Only binary-safe data structures can be transferred
Implicit offload model (a.k.a. Shared VM model)
Data transfers are handled automatically
Complex data structures and pointers may be transferred
by Martin Kruliš (v1.2)

10. Offload Model Code Compilation Source Host MIC Compilation Stub
#pragma offload
Stub
_Cilk_offload
by Martin Kruliš (v1.2)

11. Explicit Offload Model
Pragma Offload
Functions and variables are declared with
__attribute__((target(mic)))
Offloaded code is invoked as
#pragma offload <clauses>
<offloaded_statement>
A clause may select target card target(mic[:id])
Or list data structures which are used for the offload
in(varlist) – copied to the device before the offload
out(varlist) – copied back to host after the offload
inout, nocopy, length, align
Allocation control alloc_if(), free_if()
by Martin Kruliš (v1.2)

12. Explicit Offload Model
Example
__attribute__((target(mic))) void preprocess(…) { … } ...
__attribute__((target(mic))) static float *X; // of N items
#pragma offload target(mic:0) \
in(X:length(N) alloc_if(1) free_if(0))
preprocess(X);
nocopy(X:length(N) alloc_if(0) free_if(0))
process(X);
out(X:length(N) alloc_if(0) free_if(1))
finalize(X);
by Martin Kruliš (v1.2)

13. Explicit Offload Model
Asynchronous Operations
Execution
char sigVar;
#pragma offload target(mic) signal(&sigVar)
long_lasting_func();
... concurrent CPU work ...
Data transfers
#pragma offload_transfer clauses signal(&sigVar)
Waiting, polling
#pragma offload_wait wait(&sigVar)
if (_Offload_signaled(micID, &sigVar)) ...
The wait(&sigVar) clause can be added to offload or offload_transfer pragmas to create dependencies.
by Martin Kruliš (v1.2)

14. Implicit Offload Model
Shared Virtual Memory Mode
Code and variables that are shared have attribute
_Cilk_shared (e.g., float _Cilk_shared *X)
Shared memory allocation must be performed via specialized functions
_Offload_shared_malloc, _Offload_shared_free
Offloaded code gets executed by
_Cilk_offload <statement>
_Cilk_offload_to(micId) <statement>
The data are transferred as compiler see fit
Example
by Martin Kruliš (v1.2)

15. Offload Model A Few More Things Compile time macros for MIC detection
__INTEL_OFFLOAD, __MIC__
Runtime MIC detection
_Offload_number_of_devices()
_Offload_get_device_number()
Querying device capabilities is more complicated
You can read /proc/cpuinfo on the device
Or use special MicAccessAPI
Asynchronous data transfers and execution
Using signal variables for synchronization
by Martin Kruliš (v1.2)

16. New Xeon Phi Xeon Phi (x200 family)
Available as a PCIe card and “regular” CPU
by Martin Kruliš (v1.2)

17. New Xeon Phi Xeon Phi (x200 family)
Up to 72 cores organized in a 2D mesh (8x9 cores)
by Martin Kruliš (v1.2)

18. New Xeon Phi Xeon Phi (x200 family)
Cores based on Atom (Airmont) architecture
More versatile than previous Pentium-based cores
Supports also SSE, AVX, AVX2, …
Two AVX512 units per core
Memory
16GB on chip MCDRAM (either used as transparent cache or as a separate NUMA node)
Up to 384 GB DDR4
Connectivity
Intel Omni-Path Fabric (via 2x16 PCIe links)
by Martin Kruliš (v1.2)

19. Discussion
by Martin Kruliš (v1.2)