1. Speeding up multi-task learning Phong T Pham

2. Multi-task learning  Combine data from various data sources  Potentially exploit the inter-relation between different data  M datasets D = {D 1,….,D M } for M tasks  Each dataset D m ={(x m,t,y m,t ),t=1..T m } is i.i.d

3. Maximum Entropy Discrimination  Similar to Bayes, assume some prior p(Θ) and solve for p(Θ|D)  Instead of using Bayes rule, finds p(Θ|D) that minimize KL(p(Θ|D) || p(Θ))  Subject to classification constraints  Has close form solution

4. Log-linear MED  Assume log-linear model  Prior p(Θ) factorizes, and all terms are white Gaussians  Leads to support vector machines

5. Kernel selection

6. Feature selection  Special case of kernel selection  Kernels:  k d (x,y) = x (d) y (d)

7. Speeding up  Convex optimization problem  Can be solved using standard convex optimization algorithms  Impractically slow: M x T variables  Need speeding up

8. Method  Optimize its lower bound instead  This upper bound for f(x) is quadratic  Can use fast quadratic optimization methods

9. Optimization procedure 1.Initialize λ 2.Set λ~ = λ 3.Optimize the quadratic equation 4.Re-compute coefficients and return to step 2

10. Sequential Minimal Optimization  The quadratic optimization is similar to standard SVM  Implement a variant of SMO

11. Experiments  Feature selection  Landmine dataset  29 binary tasks  9 features  450-700 examples per task  Parameters  Training examples: 20i, i=1..5  Trade-off constant: 10 (i/2), i=1..5  Alpha: 5 (i-1), i=1..5  Performance average over 5 runs with random choice of training examples

12. Result – Running time (1)

13. Result – Running time (2)

14. Result - Performance

15. Future work  Further improve running time  Evaluate on more datasets

16. Thank you