1. Learning to Satisfy Actuator Networks
Mark Coates
National Science and Engineering Research Council of Canada (NSERC)

2. A Journey
“And what is a journey?  Is it just… distance traveled?  Time spent?  No.  It's what happens on the way, the things that happen to you.  At the end of the journey you're not the same.”

3. Plan your Journey, Learn
“It's what happens on the way, it the things that happen to you.”
Sensor Networks SANETs
Local Actuation:
Control sensors, control objects.
Modify the environment.
Learn causal relationships between actuations and environmental (model) variables.
Plan behaviour to optimize performance

4. Sensor/Actuator Network
Set of actuators ( ) + associated sensors ( ).
Actuators perform a physical (modifying) action.
Sensors monitor the response of the system.
Quantify the net effect on the system (positive or negative)
Design actuation strategy to optimize response

5. Causal Analysis
How can we infer the impact of an actuation based on a set of observations?
In particular, how do we derive:
Manipulated Probability P(Y | X := x, Z=z)
From (observations based on)
Unmanipulated Probability P(Y | X = x, Z=z)

6. Example Problem
We wish to evaluate the average effectiveness of a fertilizer
Local background variables (for example soil moisture, temperature, salinity, weed density) affect:
The successful reception of the fertilizer
The impact of the fertilizer on the crop

7. Causal Graph
uj : local realizations of background variables with global distribution g
zj : action by actuator (0/1 = off/on) [known or measured]
dj : actuation received (0/1 = no/yes) [unobserved]
yj : response (0/1 = negative/positive) [unobserved]
xj , wj: observed measurements, dependent on dj and yj.

8. Average Causal Effect (ACE)
Expectation (over latent variables) of:
[ Prob. of positive response given fertilizer ― Prob. of positive response without fertilizer ]

9. Model the mapping not the variable
Problem:
Latent variables u can be high dimensional;
Probability distribution g(u) can have complex structure.
Approach:
We have binary variables Z, D, Y
We don’t care about the value of uj and how that directly influences dj
What we do care about is how u impacts the mappings ZD and DY

10. New Causal Graph cr: sixteen states c=
Much easier to estimate this distribution
0: inhibit
1: pass
2: flip
3: activate
crj zj dj xj yj wj

11. Evaluating ACE
Estimate ACE by applying distributed EM algorithm across the graphical model (model g(cr) as multinomial)
Locally maximize the likelihood function:
Expectation: calculate expected crj at each node
Maximization: average the expected cri to estimate g(cr).

12. Sensor Network Evaluation
Tree network topology: An efficient mechanism for data aggregation and dissemination.
Data aggregation (bottom-up)
Leaf nodes: Transmit E[crj] to parent node
Parent node: Performs aggregation and relays result to its own parent
Root node: Performs maximization
Result dissemination (top-down)
Each node broadcasts result to its children nodes

13. Influencing the Environment
Design an actuation strategy
Set of decision rules
Map from (current and past) sensor measurements to an actuation
Possible Objectives:
Maximize expected response of system
Provide probabilistic bounds on worst-case behaviour
Possible Scenarios
Accurate models of probability distributions  Bayesian networks
Uninformative models  Learning approaches

14. Problem Formulation Epoch of T discrete time intervals
At times t = 0,…,T, node i measures a set of environmental variables
Chooses an actuation belonging to a discrete set of actuations
At the end of the epoch, measure a local response variable

15. Maximization Approach
Single binary actuation decision without a good model of p(y|v,a)
Consider p(y|v,a) = f(y|v,a) + n(v,a)
We have a set of points (vi,ai,yi) and want to learn the best actuation strategy A(v), i.e., that which maximizes f(y|v,a).
Approach: regression + subsequent maximization

16. Robustness concerns Maximization amplifies regression errors.
Multi-stage planning implies repeated regression + maximization
Proliferation of error

17. Relaxed Problem
Identify the largest set of environmental conditions and actuations such that:
Expected response exceeds threshold
Probability of terrible response is very low

18. LSAT Formulation Learning to Satisfy (LSAT)
Given points find the set G that solves:
subject to
where Ci(G,P) are constraints.

19. Two types of constraints
Point-wise constraints: C(G,P) = C(x,G,P) is a function of the input variable x and the constraint takes the form
Example:
Set-average constraints: C(G,P) > 0 is only satisfied on-average over the entire set.

20. Solution Derive equivalent empirical constraints
Consider solution to empirical constraints
subject to
If empirical constraints are close to ideal constraints then solution is satisfactory.
Algorithm: extension of support vector machine
Lagrangian formulation allocating different penalties to violations of individual constraints.

21. Comments Actuator networks present a host of problems
Assessment of whether causal relationships exist and evaluation of their strength
Design of actuation strategies that yield a satisfactory (optimal?) environmental response
These problems are difficult in a centralized setting – the extension to distributed algorithms poses an even greater challenge.