Download presentation
Presentation is loading. Please wait.
1
Scheduling Design and Verification for Open Soft Real-time Systems
Terry Tidwell, Robert Glaubius, Christopher Gill, and William D. Smart Washington University in St. Louis November 15, 2018 RTSS 2008
2
The One Slide Summary Our goal is to automatically generate scheduling policies Given a system model with discrete, stochastic elements and a set of system design criteria We introduce a technique for the design and verification of a scheduling policy given A system model with probabilistic models of thread execution times The ability to define a resource share per thread Study the characteristics of resource demand functions that can yield to this technique November 15, 2018 RTSS 2008
3
Motivating Example A mobile robot with multiple onboard sensors interacting with a dynamic unknown environment Onboard sensors have different but measurable characteristics Cameras - long, uncertain activation time Laser range finder - short, more predictable activation time Scheduling the activation of sensors is critical to timely and correct operation November 15, 2018 RTSS 2008
4
Open Soft Real-time Systems
Must respond adaptively to varying operating conditions Decisions in such systems must be made and enacted within specific timing constraints Systems should be designed to meet timing constraints to the extent possible Robot can pause but that is sub-optimal November 15, 2018 RTSS 2008
5
Goal and Approach The scheduling policies produced should be tailored, efficiently computable, and verifiable We propose Constructing and solving a Markov Decision Process (MDP) to derive scheduling policies Using Model Checking to perform verification of the system under those policies November 15, 2018 RTSS 2008
6
System Model Simple Thread Model
No preemption Thread execution time is non-deterministic with a prior known distribution Discrete run times Each invocation is independent Utilization goal assigned to each thread RLG: Each thread invocation is independent in two ways: independent of prior invocations of the same thread, and invocations of other threads. November 15, 2018 RTSS 2008
7
Setting up the Markov Decision Process
A utilization state x is a vector of threads’ CPU utilizations The utilization target is encoded as a ray in state space The goal is to develop a policy, (x), for which thread to dispatch at each utilization state x November 15, 2018 RTSS 2008
8
Policy Iteration Define a cost function r(x) penalizing deviation from target utilization Start with some initial policy 0 Repeat for t=0,1,2,… Compute the value Vt(x) -- the accumulated cost of following t -- for each state x. Obtain a new policy, t+1, by choosing the greedy action at each state. Guaranteed to converge to the optimal policy Requires storing Vt and t in lookup tables. November 15, 2018 RTSS 2008
9
Policy iteration limitations
State space is not finite Can not apply MDP solution techniques directly Need to reduce the size of the state space in order to solve for some policy. Our approach: reduce the state space to a collection of equivalence classes. RLG: The story arc here is to point at the image here (prev. seen on slide 5) and notice that the state space extends infinitely. Thus we can’t use policy iteration, since we can’t store the policies and value functions. We need a way to reduce the size of the problem. Our approach is to collapse the state space into a number of equivalent states. The next slide goes into more detail. November 15, 2018 RTSS 2008
10
State Value Equivalence
Cost function Any two states co-linear along the target utilization ray have the same cost Any two states have the same relative distribution over future states Any two states with the same cost have the same optimal value! RLG: Need to make sure to qualify statements on this slide -- the result that two states with the same cost have the same value is NOT generally true, and only works because of the self-similarity of transitions and costs across the entire state space. RLG: I’d tell the story as follows. We have a cost function that is based on the distance between x and the utilization ray (if questioned, the cost is actually the difference between x and the point where the utilization ray has the same cumulative utilization as x, ||x||). Under this definition of cost, states with displacement parallel to the utilization vector (colinear along the utilization ray) have equal cost [advance animation, point out two examples of pairs of states with equal cost]. Since threads in this model have independent invocations, every state has a similar distribution over future states as well. In the paper, we show that these two premises result in states with equal cost having equal value. November 15, 2018 RTSS 2008
11
State Wrapping We are able to collapse these equivalent states down into a set of exemplar states. We then bound the number of states by introducing “absorbing” states Greedy and optimal policies appear to agree at sufficient distance from target utilization Can then use policy iteration to obtain a policy. RLG: There are a couple of things that are important to stress here. One is that the wrapping action is not sufficient to make the set of states finite, but that there are only finitely many states with cost less than epsilon for any epsilon -- e.g., there are only finitely many states “close” to target utilization for any closeness threshold. For any state sufficiently far away from target utilization, the greedy policy and optimal policy are (I believe) the same. Therefore, we can represent all of the states at sufficient distance by absorbing states, and pass control to the greedy policy if the system ever actually ends up in them (which it shouldn’t if our prior knowledge is correct and the system follows the computed policy). November 15, 2018 RTSS 2008
12
Model Checking Given a model and a property decides whether the property holds in the model Builds transition system of reachable states Repeatedly applies the successor operator, succ(s,a), to compute the successor state(s) to all previously visited states s, with all possible actions a, until a fixed point is reached. November 15, 2018 RTSS 2008
13
Applying Model Checking
Some definitions Each verification state is a set of utilization states Actions are the same as in the MDP Initial verification state contains only the utilization state representing zero CPU usage for each thread We then define the successor function over state-action pairs November 15, 2018 RTSS 2008
14
The Successor Function
Successor function computation steps: Split on policy Simulate action Wrap Project it back into the wrapped state space using the operators November 15, 2018 RTSS 2008
15
The Successor Function (Illustrated!)
November 15, 2018 RTSS 2008
16
The Successor Function
November 15, 2018 RTSS 2008
17
Split on Policy (Red vs Blue)
November 15, 2018 RTSS 2008
18
Simulate Action November 15, 2018 RTSS 2008
19
Apply Wrapping November 15, 2018 RTSS 2008
20
Potential Improvements
The size of a verification state is proportional to the size of the wrapped utilization state space Still exponential in the number of threads The size of the verification state is also sensitive to the time scale used More densely sampled, the bigger the representation Ideally we would like a verification state representation that is less affected by these factors November 15, 2018 RTSS 2008
21
Difference Bound Matrices (DBM)
Compactly represent continuous regions in utilization space Each element represents an equation e.g. Quadratic in the number of threads Independent of the time scale Timed verification November 15, 2018 RTSS 2008
22
The Successor Function (Revisited)
Successor Function Again Computed in Steps Split on Policy Simulate Action Wrap This time we use three DBM operations Intersection -- canonical DBM operation Run and wrap -- novel extensions November 15, 2018 RTSS 2008
23
Split on Policy First, represent the decision boundary as a DBM
Then intersect the DBM representing the verification state with the DBM for the appropriate decision region November 15, 2018 RTSS 2008
24
Simulate Action Apply Run operator Takes a DBM as input
Creates a DBM representing the region that would be produced by running the appropriate thread November 15, 2018 RTSS 2008
25
Apply Wrapping Wrap operator takes a DBM and a non-negative integer
Invoke repeatedly for successively larger integers until it produces an empty DBM (fixed point) November 15, 2018 RTSS 2008
26
Apply Wrapping Confine the DBMs produced by the Wrap operator
Need a DBM per thread to represent the wrapped space November 15, 2018 RTSS 2008
27
Apply Wrapping Intersect the products of Wrap with the constraining DBMs Gives the set succ(s,a) November 15, 2018 RTSS 2008
28
Model Checking the Scheduling Policy
The successor function induces a transition system that models our scheduling policy Apply standard model checking techniques to verify properties with this model November 15, 2018 RTSS 2008
29
Conclusions Our goal is to design tailored scheduling policies for open soft real-time systems We use a Markov Decision Process to derive scheduling policies We developed a state wrapping scheme to allow policy iteration We introduce novel DBM operators to constrain verification complexity Allows property verification via Model Checking November 15, 2018 RTSS 2008
30
Questions? November 15, 2018 RTSS 2008
Similar presentations
