Motion Prediction for Online Gaming Rynson W.H. Lau May 14, 2011

1. Network Latency According to our earlier experiments, the one- way network latency values for different network connections are shown as follows: According to our earlier experiments, the one- way network latency values for different network connections are shown as follows: Network Connection Average Latency LAN (2 hosts within the university network) 0.64ms Internet (local connection, 2 hosts within HK) 10ms Internet (overseas connection, between US and HK) 0.16s Internet (overseas connection, between UK and HK) 0.2s May need to consider other overheads, such as I/O and CPU.

Problems of Network Latency in Online Gaming Motion Synchronization Problems Motion Synchronization Problems  Motion discrepancy of shared objects among clients.  In a collaborative game, it will be difficult to synchronize interactions among remote players. Object Distribution Problem Object Distribution Problem  In a streaming environment, geometry of visible objects may not be readily available locally at the time when they are needed.

Solution to the Network Latency Problems We accept that prediction is a prediction. We can never have a 100% accuracy. We accept that prediction is a prediction. We can never have a 100% accuracy. However, we do want a reliable motion prediction method: However, we do want a reliable motion prediction method:  Given a specific time in the future, the prediction method should return the most likely position of the target object.

2. Related Work Roughly two types of motion prediction methods: Short-term prediction Short-term prediction  short prediction length (typically < 1s). Long-term prediction Long-term prediction  Long prediction length (typically >> 1s).

Short-term Prediction General predictors: General predictors:  Dead reckoning + polynormial predictor (Singhal & Zyda, 1999)  Modified EWMA scheme (Chim et al., 1998) Specialized predictors: Specialized predictors:  Kalman filter for head motion prediction (Azuma & Bishop, 1995)  Gauss-Markov process model for head prediction (Liang et al., 1991) and for human motion prediction (Capin et al., 1997)  Sequential Monte Carlo for motion tracking (Isard & Balke, 1998) In general, specialized prediction methods are more accurate but are only suitable for predicting motion of specific objects.

Long-term Prediction Mobility modelling: Mobility modelling: For mobile handovers -  Random process (Thomas et al., 1988; Xie & Goodman, 1993; Broch et al., 1998)  Discrete state conditional probability (Bar-Noy et al., 1995; Choi & Shin, 2002) For bandwidth optimization -  Zone-based sampling statistics (Tabbane, 1995; Bhattacharya & Das, 2002)  Zone-based path retrieval (Liu & Maguire, 1996; Liu et al., 1998; Akyildiz & Wang, 2004)

Path planning: Path planning:  Collision avoidance using prior knowledge (Tado et al., 1995) or statistics from samples (Tana, 1992)  Moving obstacle avoidance by considering obstacle motion trajectories (Kruse et al., 1997) To find an optimal path to move from one place to another while satisfying some predefined constraints.

Our Streaming-Based DVE We have developed a DVE, which is based on geometry streaming (Chim et al., 1998; Chim et al., 2003). We have developed a DVE, which is based on geometry streaming (Chim et al., 1998; Chim et al., 2003). <demo video> <demo video>  Objects formatted for progressive transmission.  Send initial packet as a user joins the game.  Send additional geometry packets as the user moves around in the game. This DVE was later extended into a game engine prototype (Li et al., 2004). This DVE was later extended into a game engine prototype (Li et al., 2004).

Problems: Too late if we request objects from the server only when the objects become visible. Too late if we request objects from the server only when the objects become visible. Need to ask the server for objects before they become visible. Need to ask the server for objects before they become visible.  Need a reliable motion prediction method.  Predict future visible objects based on the predicted future position from the predictor.  Request server for the predicted visible objects.

3. Short-term Motion Prediction We needed a reliable motion prediction method for 3D navigation – to address the network latency problem. We needed a reliable motion prediction method for 3D navigation – to address the network latency problem. As most desktop DVE systems still use the 2D mouse for navigation input, we studied the mouse behavior during such navigation. As most desktop DVE systems still use the 2D mouse for navigation input, we studied the mouse behavior during such navigation. We noted that a typical navigation step (from mouse button down to release) contains a number of pulses. We noted that a typical navigation step (from mouse button down to release) contains a number of pulses.

Mouse-based Hybrid Motion Prediction When plotting the mouse acceleration against velocity, we noted that each pulse can be approximated by an ellipse: When plotting the mouse acceleration against velocity, we noted that each pulse can be approximated by an ellipse: velocity Hence, we model it as: Hence, we model it as: v = K 1 (cos(K 2 t)-1) v = K 1 (cos(K 2 t)-1) where K 1 and K 2 are constants determined by the shape of the pulse. where K 1 and K 2 are constants determined by the shape of the pulse.

We proposed an elliptic model to model the mouse motion behavior (Chan et al., 2005). We proposed an elliptic model to model the mouse motion behavior (Chan et al., 2005). Two predictors are needed, one for the motion in x axis and the other for motion in y axis. Two predictors are needed, one for the motion in x axis and the other for motion in y axis. The predicted mouse velocity is then mapped to the 3D game scene to obtain the player’s future position. The predicted mouse velocity is then mapped to the 3D game scene to obtain the player’s future position. <Video Demo> <Video Demo>

Performance of the Predictor

Hand Motion Prediction We observed similar motion behavior in finger motion. We observed similar motion behavior in finger motion. We extended the elliptic model for hand motion prediction, by considering constraints within and between fingers (Chan et al., 2008). We extended the elliptic model for hand motion prediction, by considering constraints within and between fingers (Chan et al., 2008). Preformance comparison: Preformance comparison: <Video Demo> <Video Demo> Rock-Paper-Scissor game: sop -, ehm - Rock-Paper-Scissor game: sop -, ehm - <Video Demo> <Video Demo> <Video Demo> <Video Demo>

4. Long-term Motion Prediction We need a reliable motion prediction method for object prefetching and resource allocation (such as multi-server load-balancing). We need a reliable motion prediction method for object prefetching and resource allocation (such as multi-server load-balancing). We are developing a statistical-based method. We are developing a statistical-based method. The environment is divided into square zones. The environment is divided into square zones. We collect statistics as users go in and out of each zone. We collect statistics as users go in and out of each zone.

Given the boundary position that a user enters the zone, we compute the conditional probability that the user will leave the zone at a particular boundary position. Given the boundary position that a user enters the zone, we compute the conditional probability that the user will leave the zone at a particular boundary position.

wheren is the number of mixture, f i (y; C i, m i ) is the i th Gaussian distribution function, with covariance C i and mean m i, 0 < a i < 1, and We model such a distribution function f(y|x) as a Gaussian mixture: We model such a distribution function f(y|x) as a Gaussian mixture:

Performance of the Predictor [Choi02] computes the probability of the user moving to different neighboring zones based on the user’s previous and current zones.

An interesting feature of this method is that we may consider a zone as a basic block. An interesting feature of this method is that we may consider a zone as a basic block. We may aggregate this probability values from zone to zone to construct a path tree. We may aggregate this probability values from zone to zone to construct a path tree.

Current Work on Long-term Prediction How should we measure prediction accuracy with multiple predicted paths, each with a different probability? How should we measure prediction accuracy with multiple predicted paths, each with a different probability? Does prediction at lower spatial resolution help reduce computational cost, given that the prediction accuracy is high enough? Does prediction at lower spatial resolution help reduce computational cost, given that the prediction accuracy is high enough?

5. Combined Motion Prediction Short-term prediction Short-term prediction  Precise predicted location  Very short prediction length  Prefetching performance can be significantly affected by other factors, such as amount of geometry to be streamed, available network bandwidth, CPU processing delay, network latency fluctuation, etc. When applied to our streaming-based DVE, the two types of prediction methods have different behaviors:

Long-term prediction Long-term prediction  Imprecise predicted location  Generally lower prediction accuracy  Much longer prediction length  May start prefetching much earlier in time, but most of the objects prefetched may not be needed at all, due to the imprecise predicted location and the lower prediction accuracy.

Long-term predictor helps prefetch objects in low resolution, e.g., base meshes. Long-term predictor helps prefetch objects in low resolution, e.g., base meshes. Short-term predictor helps prefetch refinement information, e.g., progressive records, to bring the visible objects to their optimal resolution. Short-term predictor helps prefetch refinement information, e.g., progressive records, to bring the visible objects to their optimal resolution. One approach that we are considering is to combine the two types of predictors to optimize the prefetching performance:

6. Problems and Future Works Can we improve the prediction accuracy by incorporating user interests? Can we improve the prediction accuracy by incorporating user interests? Can we improve the prediction accuracy of the long-term predictor with results from the short- term predictor (or vise versa)? Can we improve the prediction accuracy of the long-term predictor with results from the short- term predictor (or vise versa)? Investigation on prediction errors and bounding of prediction errors. (Lau & Lee, 2010) Investigation on prediction errors and bounding of prediction errors. (Lau & Lee, 2010)

Q & A