1. Alessandro Leoni, Felix Siegle September, 2017
Possible requirements for the SpaceFibre network layer End-to-End Determinism, FDIR, Network Discovery, Configuration & Status
Alessandro Leoni, Felix Siegle
September, 2017

2. Outline OSRA-NET requirements
SpaceFibre determinism – Single master per Virtual Network
SpaceFibre determinism – Multiple masters per Virtual Network
Simulation results
Network discovery and configuration
Broadcast definitions
Open discussion…

3. OSRA-NET Requirements
OSRA-NET: set of high-level requirements for forthcoming satellite networks covering:
Traffic classes (Synchronous, Asynchronous, Best Effort)
FDIR
Quality of Service / Reliability (At least one, at most one, exactly one)
Redundancy …

4. SpaceFibre determinism – Single master
Straightforward solution : only one master in each Virtual Network *
Can exploit SpaceFibre built-in features
Virtual channels are (virtually) isolated
No contentions at the output ports of a switch
*S. Parkes, A. F. Florit, A. G. Villafranca, C. McClements and D. McLaren, "SpaceFibre network
and routing switch," 2017 IEEE Aerospace Conference, 2017.
A failing node doesn’t affect the rest of the network

5. Multiple masters on the same Virtual Network
Possible problems when using one master per Virtual Network:
The network becomes very big
>32 Virtual Networks sharing the same link
>64 Virtual Networks overall
The switch has limited capacities (output ports with few Virtual Channels)

6. Multiple masters on the same Virtual Network
Need to multiplex multiple input virtual channels into the same output buffer
Same problem of SpaceWire networks Possibly undefined blocking time
SpFi Router
VC 0
VC 0
VC 1
VC 1
VC 2
VC 2
VC 0
VC 1
VC 2
Now we have to find a rule to CUT OFF wormholes from failing nodes because we are SHARING the buffers

7. Multiple masters – Synchronous Time Critical Data
Prevent that a node transmits outside of its time-slot
Almost zero blocking time R0 B
Dest
TS=0
TS=1
TS=2 A B
A produces data
B produces data (faulty)
C produces data
Maximum estimated time
Actual transmission time

8. Multiple masters – Synchronous Time Critical Data
We want a Time-slots guardian with scheduling tables replicated inside the switch to:
Be able to detect when a fault happens (Fault Detection)
Prevent a faulty node from affecting other communications (Fault Isolation)
VC0 X X
VC1
VC0 X X
VC1
VC0 X X X X
VC1
VC0 X X
VC1
VC0 X X
VC1

9. Multiple masters – Asynchronous Time Critical Data
Wormhole timeout: close a wormhole when it is taking more time than allowed
Round Robin is used to arbitrate input port.
Worst-case blocking time
Dest E
Start timeout for D
Kill D / Start timeout for E
D produces data (faulty)
Timeout
Transmission time
E produces data

10. OMNeT++ Network Simulator
SpaceFibre/SpaceWire simulator
Developed in the framework of the NPI at ESTEC
Based on OMNeT++ (open-source, C++ library for discrete-event simulation)
Noticeable features:
Path and Logical Addressing
Multicast/Group Adaptive Routing
SpaceWire/SpaceFibre bridge for packets and time-codes (with some assumptions on the network level)
Hardware-in-the-loop support through Ethernet (at packet level)

11. OMNeT++ Network Simulator
Already existing nodes:
SpaceFibre Codec
SpaceWire Codec
Switch
RMAP Initiator/Target
General purpose application to simulate traffic
HIL Bridge
Useful for:
Extract network performance results
Test a new protocol

12. Asynchronous traffic analysis
atcNode0 and atcNode1 send asynchronous packets to spfiNode3
High priority
Packets with predefined max length
They share VN 1
Router4 uses wormhole timeout (8000ns) for them
spfiNode2 sends BE data continuously to spfiNode3
Uses VN 2
Low priority
VN2
VN1
VN1
Scenario 1:
atcNode0 and atcNode1 behave correctly.
atcNode1_max_latency = time_to_transmit_atcNode0_packet + delta
Scenario 2:
atcNode0 fails and sends infinite packet.
atcNode1_max_latency = atcNode0_timeout + delta

13. Router input/output ports utilization
Input port 2, vn 2 (spfiNode2, BE)
Input port 1, vn 1 (atcNode1)
Input port 0, vn 1 (atcNode0, failing)
Output port 3, all vn
8000 ns
Remaining data in the buffer

14. Synchronous traffic analysis
stcNode0 and stcNode1 send synchronous packets to spfiNode3
High priority
Packets with predefined max length
They share VN 1, but on different time slots
Router4 closes wormhole when timeslot changes
spfiNode2 sends BE data continuously to spfiNode3
Uses VN 2
Low priority
masterNode4 sends out broadcasts to advertise about the new timeslots
VN2
VN1
VN1
ts:0 stcNode0
ts:1 stcNode1
ts:0 stcNode0
ts:1 stcNode1
ts:n/a

15. Router input/output ports utilization
Broadcast msgs (timeslots)
ts0 (stcNode0)
ts1 (stcNode1)
Input port 2, vn 2 (SpfiNode2, BE)
Input port 1, vn 1 (stcNode1)
Input port 0, vn 1 (stcNode0, failing)
Output port 3, all vn
~2200 ns  latency depends only on the remaining data in the buffer

16. Required Routing Switch (and Node) Extensions: Network Discovery
Network Discovery can be kept simple by requiring any node and routing switch to have a dedicated RMAP target reachable via VN0.
Implementation complexity is not an argument, one could specify that only a very minimal set of RMAP functions must be supported (basically: 32-bit write and 32-bit read).
We need to specify a memory map that is valid for all nodes and routing switches.
A unique Device Class ID identifies the device.
The Network Manager can look up the Device Class ID and simply consult a corresponding Electronic Data Sheet for more information.
Address Range
0x00 – 0x3c
Network Discovery
Device Type, Device Class ID etc.
0x40 – 0x7c
Device Status
0x80 – 0xbc
Device Configuration
0x100 – 0x1fc
SpFi-Device
Specific Registers
0x200 – […]
User Registers

17. Required Routing Switch (and Node) Extensions: Status and Configuration
Similar to the SpaceFibre Port specification, we should also specify the basic status flags and configuration parameters for SpaceFibre routing switch implementations.
At least the essential features should work the same across several implementations.
Then, we should fix them to registers in the memory map, making it easy for the user to configure and monitor SpaceFibre nodes and routing switches without any knowledge about manufacturer-specific details!
Address Range
0x00 – 0x3c
Network Discovery
Device Type, Device Class ID etc.
0x40 – 0x7c
Device Status
0x80 – 0xbc
Device Configuration
0x100 – 0x1fc
SpFi-Device
Specific Registers
0x200 – […]
User Registers

18. Required Routing Switch (and Node) Extensions: Device-Specific Registers
The next memory block allows an end node or routing switch to implement manufacturer-specific configuration and status registers that extend the basic set of functionality.
The exact meaning of the bit fields is unknown. However, by reading the Device Class ID, the user can e.g. consult an Electronic Data Sheet that contains the required information.
Address Range
0x00 – 0x3c
Network Discovery
Device Type, Device Class ID etc.
0x40 – 0x7c
Device Status
0x80 – 0xbc
Device Configuration
0x100 – 0x1fc
SpFi-Device
Specific Registers
0x200 – […]
User Registers

19. Required Routing Switch (and Node) Extensions: User-Specific Registers
The last memory block can be used to make unit-specific status and configuration registers available via VN0.
For instance, if the SpaceFibre end point is within a payload instrument, the configuration of the instrument itself could be done through these registers.
Since these registers are not SpaceFibre-related, their meaning is only known to the user and does not need to be included in Electronic Datasheets.
(although one could create extended EDS that also include this information)
Address Range
0x00 – 0x3c
Network Discovery
Device Type, Device Class ID etc.
0x40 – 0x7c
Device Status
0x80 – 0xbc
Device Configuration
0x100 – 0x1fc
SpFi-Device
Specific Registers
0x200 – […]
User Registers

20. Required Routing Switch (and Node) Extensions: Definition of Broadcasts
So far, only the broadcast mechanism is specified but not the meaning of a broadcast!
We need to specify at least:
A broadcast to transfer SpW time-codes through a SpFi network.
A broadcast to update time and time-slot information throughout the network (redundant time masters?).
Broadcasts that are generated in case of errors and that include an error code. Like the memory map, we have to specify a list of error codes for all common errors that can occur in a SpaceFibre node and routing switch.
Broadcasts to execute basic FDIR actions in case the node or router is not reachable via RMAP  e.g. disable port, reset port, close wormhole…
In addition, we could specify a “universal weapon” broadcast that allows simple 32-bit read or write operations. Then, any possible configuration and status field would be accessible in case of failures.

21. Open discussion…
Is the “Single Master per Virtual Network” approach a real limitation (considering future architectures) ?
If yes, what about the two methods proposed:
Wormhole Timeout
Can deal with Asynchronous traffic
Worst-case assumption can lead to relatively high latency
Timeslot guardian
Very low latency (~0) even in worst-case assumption
Network Discovery and Configuration
Broadcasts definition
They provide:
- Fault Detection (Visibility)
- Fault Isolation