1. Update on ALICE software status and ideas Ervin Dénes Wigner Research Center Hungarian Academy of Sciences

2. PC RORC How Does the RORC SW Work? We have DMA engines in the RORC cards (FW)that take data over optical links (DDL) to store it in RAM. E. Dénes, Wigner RC (Hu)2Cross LHC exp. DAQ meeting, 9 July 2014 Detector SIU We need a PCI driver for the RORC. The driver maps the RORC’s registers into memory, so that the SW can read/write The SW can give the addresses where to put the collected data and can assure the RORC always has addresses of free memory. DIU DDL

3. Physmem To support the DDL transfers we need a RAM area which is pinned (non-swapped), with known physical addresses, and which is accessible from user-level code. E. Dénes, Wigner RC (Hu)3Cross LHC exp. DAQ meeting, 9 July 2014 For that at boot time we limit the amount of memory used by Linux (grub parameter mem=...) e.g.: kernel /vmlinuz-2.6.32-358.23.2.el6.x86_64 ….. nopat mem=4G We have a small device driver that maps the memory outside the Linux area via remap_pfn_range() The driver gives to the user SW the size of the mapped area and its physical address Linux Physmem area #nPhysmem area 0 ………… 4G~2,5G RAM

4. Evolution of Physmem Driver Around year 2000 we started with Bigphys – Part of the Linux kernel – Not supported by IT --> for every new kernel version we had to recompile the kernel First Physmem from 2002 – Kernel modul: no need of kernel compilation Modifications for SLC3 and SLC4 (2006, 2008) Compatible 32 and 64 bit versions for SLC5 (2008) – Solved to skip the “hole” under 4G and 4G crosses Modifications for SLC6 (2012) – /proc/iomem changed, does not report above boot parameter -> we use BIOS map Kernel 2.6.32 uses new memory management – using PAT (Page Attribute Table) which is not correct for mapping big memory (> 3.8 GB): read and write speeds fall drastically. Solution: “nopat” in GRUB (2013) Modifications for Ubuntu (for C-RORC tests) (2013) E. Dénes, Wigner RC (Hu)Cross LHC exp. DAQ meeting, 9 July 20144 1 physmem area

5. Evolution of RORC Driver First driver for PCI32 RORC: 2001 2001 – 2012: – following the evolution of SLC2, 3, 4, 5, 6 kernels – following the evolution of RORC HW: 66MHz PCI, 100 MHz PC-X, PCI-Express Gen2 2013: – Common driver for D-RORC and C-RORC – modifications for Ubuntu Ubuntu is needed for HW tests of C-RORC only RUN 3: – Commercial 10/40 GbE or PCIe over fibre HW come with their own drivers – Performance should be checked E. Dénes, Wigner RC (Hu)Cross LHC exp. DAQ meeting, 9 July 20145

6. How to follow? Should we follow with Physmem? – Pros: no data move between kernel and user space no limit of memory size several process can reach data in the same time any part of the memory can be selected – Cons : Linux developers do not support What else could be used? – Scatter-Gather DMA is one option Pros : – No need to develop memory management driver Cons : – Not known if sgDMA has all features what Physmem has E. Dénes, Wigner RC (Hu)Cross LHC exp. DAQ meeting, 9 July 20146 e.g. use this

7. Next steps in common There is some opportunity for – Exchange of information – Common development E. Dénes, Wigner RC (Hu)Cross LHC exp. DAQ meeting, 9 July 20147