1. Elastic Circuits blending synchronous and asynchronous technologies Jordi Cortadella Universitat Politècnica de Catalunya, Barcelona (joint work with M. Kishinevsky and M. Galceran-Oms) Collège de France May 21 st, 2013

2. Elastic circuitsCollège de France 20132

3. Synchronous circuit Elastic circuits Combinational Logic Flip Flops PLLPLL Collège de France 20133

4. Asynchronous circuit Elastic circuits Combinational Logic LL LL delay CC CC 4-phase Collège de France 20134

5. Asynchronous circuitCC ReqInReqOut AckIn AckOut CC CC CC David Mullers pipeline (late 50s) Sutherlands Micropipelines (Turing award, 1989) Elastic circuitsCollège de France 20135

6. Globally-asynchronous Locally-synchronous GALS

7. SoC design with GALS Most IPs are synchronous Different components may have different operating frequencies Some components have variable latencies (e.g., cache hit/miss latency) Multiple clock domains are essential Elastic circuits BridgeBridge CDCCDC DSPDSP P P Fast Bus Slow Bus BridgeBridge CDCCDCMemMem CLK2 CLK1 CLK3 Collège de France 20137

8. Multiple clock domains Elastic circuits CLK Single clock (mesochronous) f1/f0 f2/f0 f3/f0 CLK (f0) Rational clock frequencies CLK1 CLK2 CLK3 CLK0 Independent clocks (controllable skew) Collège de France 20138

9. Synchronous handshakes Elastic circuits CLK1CLK2 DataData SenderSenderReceiverReceiver Valid Ack The arrival of data is unpredictable Handshakes solve the problem Collège de France 20139

10. The problem: metastability Elastic circuits DQ CLK S DQ ? D Q CLK R setup hold Collège de France 201310

11. Metastability Elastic circuits Source: W. J. Dally, Lecture notes for EE108A, Lecture 13. Metastability and Synchronization Failure (or When Good Flip-Flops go Bad) 11/9/2005. Collège de France 201311

12. Metastability Elastic circuits logic 0 logic 1 metastable Collège de France 201312

13. Classical synchronous solution Elastic circuits DQDQDQDQ CLK T CLK R Mean Time Between Failures f Ф :frequency of the clock f D :frequency of the data t r :resolve time available W:metastability window :resolve time constant # FFs MTBF 1 FF 15 min 2 FF 9 days 3 FF 23 years Example Collège de France 201313

14. Handshake with synchronizers Elastic circuits CLK1CLK2 DataData SenderSenderReceiverReceiver Valid Ack Simple solution Throughput can be highly degraded: a long round trip for every transaction Collège de France 201314

15. Asynchronous FIFOs Elastic circuits Circular buffer Valid Ack Data Clk In Clk Out FIFO control Ack is issued as soon as data has been delivered No impact on throughput (1 token/cycle) Min latency determined by the internal synchronizers Some tricky structures for the FIFO pointers (e.g. Grey encoding) Collège de France 201315 1 cycle 3-4 cycles

16. SoC design with GALS Elastic circuits BridgeBridge CDCCDC DSPDSP P P Fast Bus Slow Bus BridgeBridge CDCCDCMemMem CLK2 CLK1 CLK3 Bridges for Clock Domain Crossing usually contain asynchronous FIFOs Latency cost only when interfacing with synchronous domains No latency penalty between asynchronous domains Collège de France 201316

17. Synchronous and Asynchronous meeting each other

18. AsynchroniaAsynchronia Collège de France 2013Elastic circuits Meanwhile, a small village of indomitable engineers was resisting the synchronous occupation … 18

19. Bill Grundmann (Intels director of CAD research, Technical director for CAD technology for the Alpha Microprocessor): The specification of a complex system is usually asynchronous (functional units, messages, queues, …), … however the clock appears when we move down to the implementation levels (in a technical discussion about system design with M. Kishinevsky and J. Cortadella, 2004) Elastic circuitsCollège de France 201319

20. Async and Sync meeting each other Elastic circuits Modular (time elasticity) But hard to analyze and synthesize Easy to analyze and synthesize Not modular (time rigid) J. OLeary and G. Brown, 1997 Synchronous emulation of asynchronous circuits A. Peeters and K. Van Berkel, 2001 Synchronous handshake circuits Elastic Circuits (Sync / Async) L. Carloni et al., 1999 A methodology for correct-by-construction latency-insensitive design Cortadella et al., Desynchronization, 2003 Collège de France 201320

21. Different flavors of elasticity Elastic circuits + 1 4 7 … 3 4 8 2 0 1 … …Rigid + a 4 8 … 1 4 7 … 2 0 1 … 3 Asynchronous + e … … Synchronous Elastic 843 741 12 0 … timetime Collège de France 201321

22. Why synchronous elasticity? Time is discrete (cycle based), but unpredictable (unknown number of cycles) Examples – Short/long integer addition (8 bits, 64 bits) – Floating-point units – Cache latency: fast hit(2), slow hit(3), miss(>20) – Bus arbitration – Latencies in Network-on-Chip – … and many others Elastic circuitsCollège de France 201322

23. … even at design time Elastic circuits ReceiverSender CLK Can we add a register without modifying the functionality of the system? Collège de France 201323

24. Many systems are already elastic AMBA AXI bus protocol Handshake signals Elastic circuitsCollège de France 201324

25. Designing with synchronous elasticity

26. Communication channel Elastic circuits receiversender Data Long wires: slow transmission Collège de France 201326

27. Pipelined communication Elastic circuits senderreceiver Data How about if the sender does not always send valid data? Collège de France 201327

28. Pipelined communication Elastic circuits senderreceiver Data Collège de France 201328

29. Pipelined communication Elastic circuits senderreceiver Data Collège de France 201329

30. Pipelined communication Elastic circuits senderreceiver Data Collège de France 201330

31. Pipelined communication Elastic circuits senderreceiver Data ??? Collège de France 201331

32. The Valid bit Elastic circuits senderreceiver Data Valid Collège de France 201332

33. The Valid bit Elastic circuits senderreceiver Data Valid Data Valid Collège de France 201333

34. The Valid bit Elastic circuits sender Data Valid receiver Data Valid Collège de France 201334

35. The Valid bit Elastic circuits sender Data Valid receiver Data Valid Collège de France 201335

36. The Valid bit Elastic circuits Data Valid senderreceiver Data Valid How about if the receiver is not always ready ? Collège de France 201336

37. The Stop bit Elastic circuits00000 sender Data Valid Stop receiver Data Valid Stop Collège de France 201337

38. The Stop bit Elastic circuits11000 sender Data Valid Stop receiver Data Valid Stop Collège de France 201338

39. The Stop bit Elastic circuits11100 sender Data Valid Stop receiver Data Valid Stop Collège de France 201339

40. The Stop bit Elastic circuits11111 sender Data Valid Stop receiver Data Valid Stop Back-pressure Collège de France 201340

41. The Stop bit Elastic circuits01111 sender Data Valid Stop receiver Data Valid Stop Collège de France 201341

42. The Stop bit Elastic circuits00000 sender Data Valid Stop receiver Data Valid Stop Collège de France 201342

43. The Stop bit Elastic circuits00000 sender Data Valid Stop receiver Data Valid Stop Collège de France 201343

44. The Stop bit Elastic circuits00000 sender Data Valid Stop receiver Data Valid Stop Collège de France 201344

45. The Stop bit Elastic circuits00000 sender Data Valid Stop receiver Data Valid Stop Collège de France 201345

46. The Stop bit Elastic circuits10000 sender Data Valid Stop receiver Data Valid Stop Long combinational path Collège de France 201346

47. Relay stations (Carloni, 1999) Elastic circuits shell pearl sender shell pearl receiver main aux Collège de France 201347

48. Relay stations (Carloni, 1999) Elastic circuits main aux shell pearl receiver shell pearl sender Collège de France 201348

49. Relay stations (Carloni, 1999) Elastic circuits main aux shell pearl receiver shell pearl sender Collège de France 201349

50. Relay stations (Carloni, 1999) Elastic circuits main aux shell pearl receiver shell pearl sender Collège de France 201350

51. Relay stations (Carloni, 1999) Elastic circuits main aux shell pearl receiver shell pearl sender Collège de France 201351

52. Relay stations (Carloni, 1999) Elastic circuits main aux shell pearl sender shell pearl receiver Collège de France 201352

53. Relay stations (Carloni, 1999) Elastic circuits main aux shell pearl sender shell pearl receiver Collège de France 201353

54. Relay stations (Carloni, 1999) Elastic circuits main aux shell pearl sender shell pearl receiver Collège de France 201354

55. Relay stations (Carloni, 1999) Elastic circuits main aux shell pearl sender shell pearl receiver Collège de France 201355

56. Relay stations (Carloni, 1999) Elastic circuits main aux shell pearl receiver shell pearl sender Handshakes with short wires Double storage required Collège de France 201356

57. Flip-flops vs. latches Elastic circuits senderreceiver 1 cycle FF Collège de France 201357

58. Flip-flops vs. latches Elastic circuits senderreceiver 1 cycle HLHL Collège de France 201358

59. Flip-flops vs. latches Elastic circuits senderreceiver 1 cycle HLHL Collège de France 201359

60. Flip-flops vs. latches Elastic circuits senderreceiver 1 cycle HLHL Collège de France 201360

61. Flip-flops vs. latches Elastic circuits senderreceiver 1 cycle HLHL Collège de France 201361

62. Flip-flops vs. latches Elastic circuits senderreceiver 1 cycle HLHL Collège de France 201362

63. Flip-flops vs. latches Elastic circuits senderreceiver 1 cycle HLHL Collège de France 201363

64. Flip-flops vs. latches Elastic circuits senderreceiver 1 cycle HLHL Flip-flops already have a double storage capability, but … Collège de France 201364

65. Flip-flops vs. latches Elastic circuits senderreceiver 1 cycle HLHL Not allowed in conventional FF-based design ! Collège de France 201365

66. Flip-flops vs. latches Elastic circuits senderreceiver 1 cycle HLLH Lets make the master/slave latches independent Collège de France 201366

67. Flip-flops vs. latches Elastic circuits senderreceiver HLHL ½ cycle Lets make the master/slave latches independent Only half of the latches (H or L) can move tokens Collège de France 201367

68. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop Data Valid Stop Collège de France 201368

69. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop Data Valid Stop 1 0 Collège de France 201369

70. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop Data Valid Stop 1 0 Collège de France 201370

71. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop Data Valid Stop 1 0 Collège de France 201371

72. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop Data Valid Stop 1 0 Collège de France 201372

73. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop Data Valid Stop 1 0 Collège de France 201373

74. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop Data Valid Stop 0 0 Collège de France 201374

75. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop Data Valid Stop 0 0 Collège de France 201375

76. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop Data Valid Stop 0 0 Collège de France 201376

77. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop Data Valid Stop 0 0 Collège de France 201377

78. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop Data Valid Stop 0 0 Collège de France 201378

79. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop Data Valid Stop 1 1 Collège de France 201379

80. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop 1 1 Data Valid Stop Collège de France 201380

81. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop 1 1 Data Valid Stop Collège de France 201381

82. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop 1 1 Data Valid Stop Collège de France 201382

83. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop 1 1 Data Valid Stop Collège de France 201383

84. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop 1 1 Data Valid Stop Collège de France 201384

85. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop 1 1 Data Valid Stop Collège de France 201385

86. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop 1 1 Data Valid Stop Collège de France 201386

87. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop 1 1 Data Valid Stop Collège de France 201387

88. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop 1 0 Data Valid Stop Collège de France 201388

89. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En 1 0 Data Valid Stop Data Valid Stop Collège de France 201389

90. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En 1 0 Data Valid Stop Data Valid Stop Collège de France 201390

91. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En 1 0 Data Valid Stop Data Valid Stop Collège de France 201391

92. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En 1 0 Data Valid Stop Data Valid Stop Collège de France 201392

93. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop Data Valid Stop 1 0 Collège de France 201393

94. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop Data Valid Stop 1 0 Collège de France 201394

95. Synchronous elasticity Elastic circuits senderreceiver V V V V S S S S En Data Valid Stop Data Valid Stop 1 0 Collège de France 201395

96. Basic VS block SiSiSiSi En i ViViViVi S i-1 V i-1 VSVS SiSiSiSi En i ViViViVi S i-1 V i-1 Elastic circuitsCollège de France 201396

97. Elastic netlistsVS Fork Join Join / Fork Enable signal to data latches Elastic circuits VS VS VS VSVS VSVS VSVS VSVS Collège de France 201397

98. JoinVSVS + V1V1 V2V2 S1S1 S2S2 V S VS VSVS VSVS Elastic circuitsCollège de France 201398

99. Lazy Fork V1V1 V2V2 S1S1 S2S2 V S Elastic circuitsCollège de France 201399

100. Eager Fork Elastic circuits V1V1 V2V2 S1S1 S2S2 ^ ^ V S Collège de France 2013100

101. Variable Latency Units Elastic circuits [0 - k] cycles done go clear V/S Collège de France 2013101

102. Design automation

103. Transforming sync into elastic Elastic circuitsCollège de France 2013103

104. Transforming sync into elastic Elastic circuitsCollège de France 2013104

105. Transforming sync into elastic Elastic circuits Behavioral equivalence is preserved Collège de France 2013105

106. Elastic Esterel module ABRO: input A,B,R; output O; loop [ await A || await B ]; emit O each R end module Elastic circuitsCollège de France 2013 Marc Galceran Oms, Master thesis, 2007 106

107. Elastic Esterel A B R O Boot PauseReg7 PauseReg11 Elastic circuitsCollège de France 2013107

108. Elastic Esterel A B R O Elastic Control Layer Valid_A Stop_A Valid_B Stop_B Valid_R Stop_R Valid_O Stop_O Boot PauseReg7 PauseReg11 Elastic circuitsCollège de France 2013108

109. Circuit vs. μarchitectural cycles Elastic circuitsCollège de France 2013109

110. Synchronous handshake circuits (Peeters, 2001) Collège de France 2013Elastic circuits int = type [0..255] & gcd: main proc (in? chan > & out! chan int) begin x, y: var int | forever do in? > ; do x <> y then if x < y then y:=y-x else x:=x-y fi od ; out!x od end Sources: J. Kessels and A. Peeters. DESCALE: A Design Experiment for a Smart Card Application Consuming Low Energy, in Principles of Asynchronous Circuit Design, A Systems Perspective, Eds., J. Sparso and S. Furber, Kluwer Academic Publishers, 2001. P.A.Beerel, R.O. Ozdag and M. Ferretti. A Designers Guide to Asynchronous VLSI, Cambridge University Press, 2010. 110

111. Generalization: bounded FIFOsIn Out B1 B3 B2 Bounded Dataflow Networks Elastic circuitsCollège de France 2013111

112. Behavioral equivalence Elastic circuits D: a b c d e f g h i j k … Synchronous: Elastic: D: a a b b b c d e e f g g h i i i j k … D: a a b b b c d e e f g g h i i i j k … V: 1 0 1 0 0 1 1 1 0 1 1 0 1 1 0 0 1 1 … V: 1 0 1 0 0 1 1 1 0 1 1 0 1 1 0 0 1 1 … Collège de France 2013112

113. Early evaluation

114. 52 3 x 15 Elastic circuitsCollège de France 2013114

115. Early evaluation 2 3 x 6 Elastic circuitsCollège de France 2013115

116. Early evaluation 0 x 0 8 Elastic circuitsCollège de France 2013116

117. PC+4 Branch target address Example: mux for next-PC calculation Take branch Only wait for required inputs Late arriving tokens are cancelled by anti-tokens No branch Early evaluation Collège de France 2013Elastic circuits117

118. How to implement anti-tokens ? Valid + Valid – Valid + Stop + Valid – Stop – + - Elastic circuitsCollège de France 2013118

119. Dual elastic controllers S + V + V - S - S + V + V - S - En Elastic circuitsCollège de France 2013119

120. Fork/join Dual fork/joinJoin with early evaluation Elastic circuitsCollège de France 2013120

121. Re-designing for average performanceFF Early evaluation slow / fast F slow F fast Elastic circuitsCollège de France 2013121

122. H.264 CABAC decoder Gotmanov, Kishinevsky and Galceran-Oms Evaluation of flexible latencies: designing synchronous elastic H.264 CABAC decoder Proc. Problems in design of micro- and nano-electronic systems Moscow, Oct. 2010 (in Russian) Collège de France 2013Elastic circuits122

123. Profiling Collège de France 2013Elastic circuits123

124. H.264 CABAC decoder Collège de France 2013Elastic circuits124

125. Area vs. PerformanceArea Effective Cycle Time Collège de France 2013Elastic circuits125

126. Conclusions Rigid systems preserve timing equivalence (data always valid at every cycle) Elastic systems waive timing equivalence to enable more concurrency (bubbles decrease throughput, but reduce cycle time) A new avenue of performance optimizations can emerge to build correct-by-construction pipelines ΘΘΘΘ Elastic circuitsCollège de France 2013126

127. Unifying sync/async elasticity J. Carmona, J. Cortadella, M. Kishinevsky and A. Taubin, Elastic Circuits, IEEE Trans. On CAD, Oct. 2009. Collège de France 2013Elastic circuits127

128. Collège de France 2013Elastic circuits128