1. NC STATE UNIVERSITY Center for Embedded Systems Research (CESR) Electrical & Computer Engineering North Carolina State University Ali El-Haj-Mahmoud and Eric Rotenberg Safely Exploiting Multithreaded Processors to Tolerate Memory Latency in Real-Time Systems

2. NC STATE UNIVERSITY CASES 20042El-Haj-Mahmoud © 2004 Embedded Processor Trends More demanding applications and user expectations Higher frequency –ARM-11 (0.13µ): 500 MHz –ARM-11 (0.10µ): 1 GHz Processor-memory speed gap

3. NC STATE UNIVERSITY CASES 20043El-Haj-Mahmoud © 2004 Memory Wall How to capitalize on higher frequency?

4. NC STATE UNIVERSITY CASES 20044El-Haj-Mahmoud © 2004 Multithreading Switch-on-event coarse-grain multithreading –Multiple register contexts for fast switching –Switch to alternate task when current task accesses memory –Overlap memory accesses with computation But what about multithreading in hard-real-time? –Require analyzability –Cannot rely on dynamic schemes

5. NC STATE UNIVERSITY CASES 20045El-Haj-Mahmoud © 2004 Exploiting Multithreading in Hard-Real-Time Systems Safety –Guarantee all tasks meet deadlines (worst-case) –Statically bound overlap under all scenarios Tractability –Confirm/disconfirm schedulability mathematically using closed-form tests –Consider each task individually

6. NC STATE UNIVERSITY CASES 20046El-Haj-Mahmoud © 2004 Classic Real-Time Scheduling Utilization-based schedulability test No need to construct the schedule a priori Example: Earliest Deadline First (EDF) time Task A Task B EDF B1B2B3B4 A1A2 period A period B release A1 deadline A1 release A2 release B1 deadline B1 release B2 B1A1B2B3B4A2 EDF schedulability test

7. NC STATE UNIVERSITY CASES 20047El-Haj-Mahmoud © 2004 Performance vs. Tractability Performance: Memory overlap Tractability: Closed-form schedulability test Only basic parameters of tasks are known –WCET = C + M (from conventional WCET analysis) –Period = deadline A B M MM

8. NC STATE UNIVERSITY CASES 20048El-Haj-Mahmoud © 2004 EDF (Classic Real-Time) Low Performance but Tractable Memory overlap: No Closed-form test: Yes A B M MM MMM DEADLINE MISSED

9. NC STATE UNIVERSITY CASES 20049El-Haj-Mahmoud © 2004 A B M MM MMM EDF (Classic Real-Time) Low Performance but Tractable Memory overlap: No Closed-form test: Yes DEADLINE MISSED

10. NC STATE UNIVERSITY CASES 200410El-Haj-Mahmoud © 2004 A B M MM MMM EDF (Classic Real-Time) Low Performance but Tractable Memory overlap: No Closed-form test: Yes DEADLINE MISSED

11. NC STATE UNIVERSITY CASES 200411El-Haj-Mahmoud © 2004 A B M MM MMM EDF (Classic Real-Time) Low Performance but Tractable Memory overlap: No Closed-form test: Yes DEADLINE MISSED

12. NC STATE UNIVERSITY CASES 200412El-Haj-Mahmoud © 2004 A B MM MM M M Dynamic Switch on Memory Access Possibly High Performance but Intractable Memory overlap: Possible, unfair for low priority Closed-form test: No –Must examine memory positioning!

13. NC STATE UNIVERSITY CASES 200413El-Haj-Mahmoud © 2004 A B MM MM M M Dynamic Switch on Memory Access Possibly High Performance but Intractable Memory overlap: Possible, unfair for low priority Closed-form test: No –Must examine memory positioning!

14. NC STATE UNIVERSITY CASES 200414El-Haj-Mahmoud © 2004 A B MM MM M M Dynamic Switch on Memory Access Possibly High Performance but Intractable Memory overlap: Possible, unfair for low priority Closed-form test: No –Must examine memory positioning!

15. NC STATE UNIVERSITY CASES 200415El-Haj-Mahmoud © 2004 Dynamic Switch on Memory Access Possibly High Performance but Intractable Memory overlap: Possible, unfair for low priority Closed-form test: No –Must examine memory positioning! A B MM MM M M

16. NC STATE UNIVERSITY CASES 200416El-Haj-Mahmoud © 2004 A B MM MM M M Dynamic Switch on Memory Access Possibly High Performance but Intractable Memory overlap: Possible, unfair for low priority Closed-form test: No –Must examine memory positioning! DEADLINE MISSED

17. NC STATE UNIVERSITY CASES 200417El-Haj-Mahmoud © 2004 A B MM MM M M Dynamic Switch on Memory Access Possibly High Performance but Intractable Memory overlap: Possible, unfair for low priority Closed-form test: No –Must examine memory positioning! DEADLINE MISSED

18. NC STATE UNIVERSITY CASES 200418El-Haj-Mahmoud © 2004 A B M M M M M M Deterministic Switching High Performance and Tractable Memory overlap: Yes (fair and bounded) Closed-form test: Yes

19. NC STATE UNIVERSITY CASES 200419El-Haj-Mahmoud © 2004 A B M M M M M M Deterministic Switching High Performance and Tractable Memory overlap: Yes (fair and bounded) Closed-form test: Yes

20. NC STATE UNIVERSITY CASES 200420El-Haj-Mahmoud © 2004 Deterministic Switching High Performance and Tractable Memory overlap: Yes (fair and bounded) Closed-form test: Yes A B M M M M M M

21. NC STATE UNIVERSITY CASES 200421El-Haj-Mahmoud © 2004 Deterministic Switching High Performance and Tractable Memory overlap: Yes (fair and bounded) Closed-form test: Yes A B M M M M M M

22. NC STATE UNIVERSITY CASES 200422El-Haj-Mahmoud © 2004 A B M M M M M M Deterministic Switching High Performance and Tractable Memory overlap: Yes (fair and bounded) Closed-form test: Yes

23. NC STATE UNIVERSITY CASES 200423El-Haj-Mahmoud © 2004 Tractability through Deterministic Switching Fully decouple independent tasks by forcing periodic switches –Every task gets a chance to initiate/overlap memory accesses –No scheduling dependences –No specificity regarding memory positioning

24. NC STATE UNIVERSITY CASES 200424El-Haj-Mahmoud © 2004 Weighted-Round-Robin (WRR) Pipeline Virt. Proc. 1 Virt. Proc. 2 Virt. Proc. 3 Virt. Proc. 4 round = memory latency T1T2 T3 T4 time T1 T2 T3T1T2T4T3T1T2T3T4 T3 T1 T2 T4 T3 T1 T2 forced pre-emption dilate WCET T4 memory transfer operation Round iRound i+1Round i+2 duty cycle

25. NC STATE UNIVERSITY CASES 200425El-Haj-Mahmoud © 2004 Analytical Framework for WRR d: duty cycle 0 < d  1 P: period = deadline WCET: Worst-Case Execution Time WCET = C + M C: aggregate computation time M: aggregate memory time WCET’: dilated Worst-Case Execution Time WCET’ = (C/d) + M

26. NC STATE UNIVERSITY CASES 200426El-Haj-Mahmoud © 2004 Schedulability Test 1. Dilated task meets deadline on its virtual processor

27. NC STATE UNIVERSITY CASES 200427El-Haj-Mahmoud © 2004 Schedulability Test 1. Dilated task meets deadline on its virtual processor

28. NC STATE UNIVERSITY CASES 200428El-Haj-Mahmoud © 2004 Schedulability Test 1. Dilated task meets deadline on its virtual processor

29. NC STATE UNIVERSITY CASES 200429El-Haj-Mahmoud © 2004 Schedulability Test 1. Dilated task meets deadline on its virtual processor

30. NC STATE UNIVERSITY CASES 200430El-Haj-Mahmoud © 2004 Schedulability Test 2. Sum of all duty cycles less than or equal to 1 1. Dilated task meets deadline on its virtual processor

31. NC STATE UNIVERSITY CASES 200431El-Haj-Mahmoud © 2004 Schedulability Test 2. Sum of all duty cycles less than or equal to 1 1. Dilated task meets deadline on its virtual processor

32. NC STATE UNIVERSITY CASES 200432El-Haj-Mahmoud © 2004 Schedulability Test 2. Sum of all duty cycles less than or equal to 1 1. Dilated task meets deadline on its virtual processor

33. NC STATE UNIVERSITY CASES 200433El-Haj-Mahmoud © 2004 Generalized Analytical Framework Multiple tasks per VP:

34. NC STATE UNIVERSITY CASES 200434El-Haj-Mahmoud © 2004 Modeling Bus and Memory System Addressed in detail in paper Analysis accounts for: –Worst-case task serialization on memory bus –DRAM bank conflicts –Multiple VPs sharing single DRAM bank

35. NC STATE UNIVERSITY CASES 200435El-Haj-Mahmoud © 2004 Results

36. NC STATE UNIVERSITY CASES 200436El-Haj-Mahmoud © 2004 Results mem component

37. NC STATE UNIVERSITY CASES 200437El-Haj-Mahmoud © 2004 Results mem component

38. NC STATE UNIVERSITY CASES 200438El-Haj-Mahmoud © 2004 Results 50% 28% mem component

39. NC STATE UNIVERSITY CASES 200439El-Haj-Mahmoud © 2004 Novel Memory overlap Formalism (safety & tractability) Classic real-timeNoYes Classic multithreadingYesNo Our real-time multithreading framework Yes

40. NC STATE UNIVERSITY CASES 200440El-Haj-Mahmoud © 2004 Useful Fully capitalize on high-frequency embedded microprocessors Exceed schedulability limit of conventional real-time theory for uniprocessors by analytically bounding WCET overlap

41. NC STATE UNIVERSITY CASES 200441El-Haj-Mahmoud © 2004 Deployable Software-only solution –Use Ubicom IP3023 8-thread embedded microprocessor –Analytical framework + scheduling policy

42. NC STATE UNIVERSITY CASES 200442El-Haj-Mahmoud © 2004 Summary  Safely expose multithreading to hard-real-time schedulability analysis  Bound computation / memory overlap  Offline closed-form schedulability test  Safe  Tractable  Scale “ Memory Wall ” in embedded systems  Expose full benefits of high-frequency embedded processors

43. NC STATE UNIVERSITY CASES 200443El-Haj-Mahmoud © 2004 Questions?