1. The Rich and the Poor: A Markov Decision Process Approach to Optimizing Taxi Driver Revenue Efficiency
Huigui Rong, Xun Zhou, Chang Yang, Zubair Shafiq, Alex Liu - Presented by Team 3

2. Improving Taxi Efficiency is important because:
Improves Drivers’ income
Reduces gas emissions
Reduces fuel costs

3. Drawbacks of current strategies:
No consideration of passenger destination for future potential passengers
Some papers have considered only the next ‘single’ trip without thinking of the next 2-3 trips that might follow
Some papers have not considered the temporal variation of passenger distribution

4. What this paper intends to do
Increase Driver Revenue per Time Unit
How does it do so?
Markov Decision Process
Find the best move for a vacant taxi at a given time-slot to maximize the total revenue in that time-slot

5. Each state of MDP corresponds to:
Current Grid
Time
Driving Direction
For each state, a vacant taxi may choose to travel to a subset of its neighboring grids and cruise through that grid

6. The Problem of the Paper:
To perform data analysis to quantify the ‘success’ of the driver
To identify the most important factors that improves drivers’ businesses
To formulate the problem as an optimal decision problem

7. Question 1: How do you define ‘driver success’?
Driver Business Time = Operation Time + Seek Time
Revenue Efficiency = Money Earned / Driver Business Time
Using these, the authors identified the top 10% and bottom 10% drivers

8. Results of the formulas applied:

9. Results:
80% of drivers had revenue efficiency between

10. Question 2: What makes Top Drivers Successful?
How much money can a driver make by driving for 1 minute, ie find out the driving efficiency
How quickly can a driver find the next passenger

11. Question 3: What is the Optimal Strategy?
Given the historical passenger trips and seeking trips for each time slot, the current status of an vacant taxi, the authors aim to find the next movement for the driver.
The objective is to maximize the total expected revenue for this taxi in the rest time of the current time slot.

12. Markov Decision Process!
A stochastic decision process with a set of states (S) and a set of possible actions (A) that transfer the states from one to another
The catch?
The next state of the process must only depend on the current state and the action, but not any previous state or action
Each action will transfer the process from the current state to a new state at a probability (P) with a corresponding reward (R).

13. System States
Each state of the vacant taxi is decided by three parameters:
Current Location (Grid Cell Number)
Current Time
Direction from which taxi arrived at the current location

14. Actions
Each vacant taxi has 9 possible actions as shown above which is used as (1-9) by the authors to indicate directions

15. Actions
For each grid cell i, the authors examine all the seeking trips that origin from i and end at each of i’s eight neighbors. For each of i’s neighbors j, they calculate how many percent of these seeking trips are between i and j. If the percentage of such trips between i and j is lower than a minimum threshold then they could conclude that there is no road connecting these grids. In this case, they do not allow the taxi to move from i to j. The threshold is 5%.

16. State Transition and Objective Function
Current State -----> s = (i,t,d)
An action ‘a’ taken to move the taxi from ‘i’ to ‘j’ in t_seek_(j) minutes will have 2 possibilities:
Taxi successfully finds passenger in grid j.
Taxi does not find a passenger in grid j.

17. Scenario 1 (Finds Passenger)
Passenger asks to go to grid k at a probability p(j,k)
After reaching k, find t(j,k) which is the total time to travel from j to k
The fare received in r(j,k)
Finally, new state s’ = (k, t + tseek(j) + tdrive(j, k), 0)

18. Scenario 2 (Did not find Passenger)
Taxi must leave j to a new grid to find passenger
Thus, new state will be s0 = (j, t + tseek(j), 4)

19. Summary
To summarize, a vacant taxi in any state s0 = (i, t, d), s0 ∈ S may take one of the possible actions a ∈ Allowed(s0) to cruise to a nearby grid j. With the probability 1 − Pf ind(j) the taxi will transition to state s1 = (j, t + tseek(j), 10 − a) with no reward. With the probability of Pf ind(j) × Pdest(j, k), (k = 1, 2, ..., |L|), the taxi will end up in the next state s2 = (k, t + tseek(j) + tdrive(j, k), 0) and receive a reward of r(j, k). The state transition diagram of the proposed MDP model is shown in Figure 5. Each circle represents a state with the three parameters listed beside it.

21. Learning Parameters for MDP
Learning the passenger pickup probability - Pfind
Learning passenger destination probability - Pdest
Estimating the reward function - r(i, j).
Estimating the driving time tdrive.
Seeking time for each grid tseek.

22. Solving MDP - to find the optimal policy
For each time slot → build a MDP with parameters learned from data.
For each state in the MDP model, the result is → the best action to take for that state to maximize total expected rewards in the remaining time of the time slot.
No fixed number of steps.
Instead, there are a few terminal states → dynamic programming
Once the system reaches a state with t=60 the system terminates.

23. Solving MDP - to find the optimal policy

24. Evaluation
Revenue Efficiency - How much more revenue can be generated if a driver uses the proposed strategy?
Is the proposed method better than methods in related work?

25. Revenue Improvement
Simulation done and compared with top 10% and bottom 10% drivers
For top 10% drivers
The average revenue efficiency is improved from yuan/min to yuan/min, yielding a 7.6% improvement.
For bottom 10% drivers
The average revenue efficiency is improved from 0.75 yuan/min to yuan/min, yielding a 9.8% improvement.

27. Results
Figure show the average revenue efficiency of the top drivers in reality compared to the simulated revenue efficiency using the bottom 10% drivers’ driving efficiency for all the day- time slots.
The improvement is between 3.6% and 12%.

28. Results
Figure shows the comparison of revenue efficiency between the bottom drivers and the simulated results using the bottom 10% drivers’ driving efficiency for the day-time slots.
The corresponding improvement are between 4.2% and 15.5%.

29. Results
Figure shows the simulated revenue efficiency (red line) is higher than the top 10% drivers’ revenue efficiency for almost all the time slots.

30. Comparison with baseline method
Figure shows the average revenue in the simulated results. This method (red curve) outperformed the baseline method (blue curve) in average revenue efficiency (Yuan per minute) in all the time slots.
The maximum improvement is about 8.4%.

31. Conclusion
Investigated how to tackle the best taxi seeking strategy from historical data → to improve taxi drivers’ business efficiency over an extended period of time.
Proposed to model the passenger-seeking process as a Markov Decision Process → to get the best move for a vacant taxi at any state.
Evaluation results showcased→
Improved revenue efficiency of inexperienced drivers by 15%
Outperformed baseline by 8.4%

32. Future Work
To improve MDP model to incorporate the total revenue and driving time covering the taxi shift time, and recommend the best strategy to improve the overall revenue efficiency.
Include other attributes such weather condition and traffic conditions to improve the results of our method.