1. PHE YEONG KIANG A152076

2. Introduction For this course LMCK1531 KEPIMPINAN & KREATIVITI, I will talk about what I've try to do in the Robot Soccer's Club.

3. What I doing in the Robot Soccer's Club? 1. Calculate the angle of robot to shoot the ball. 2. Study some machine learning knowledge.

4. Why I doing the task? 1. Because I want improve robot accuracy shooting the ball and reduce the process when robots shoot the ball. 2. Because I want improve robot learning technique.

5. How I doing the task? A.Improve robot accuracy shooting the ball. State1: When camera get State2: The computer will the position ball and robot. calculate the angle for the ball to find the location for the robot.

6. State3: The robot will change State4: The computer will the position to 0 degree.calculate the angle for robot to change the position to shoot the ball.

7. State5: Then the ball will be shot into the goal.

8. How I doing the task? B. Reduce the process when robots shoot the ball. State1: When camera get State2: The computer will the position ball and robot. calculate the angle for the ball to find the location for the robot.

9. State3: The computer will State4: Then the ball will be shot calculate the angle for robot into the goal. to direct change the position and shoot the ball, without change the position to 0 degree. I already successfully completed the part A, but part B still exploring.

10. Improve robot learning technique I using the Q- Learning to try improve robot learning technique. In fact, I have no any knowledge for Q-Learning, then I do a lot of investigation and to interrogate my seniors, finally I have some knowledge about the theory of Q- Learning. But in the meantime I was very upset, I asked the senior, senior said he did not know, because he never learned, so, I can only themselves to explore. Now I show what I learned in this Q- Learning.

11. This is the sample how the object get the state from start point to end point.

12. Algorithm State1: For each state-action pair (s, a), initialize the table entry Q i (s, a) to zero State2: Observe the current state s State3: Do forever: State3.1: Select an action a and execute it State3.2: Receive immediate reward r State3.3: Observe the new state s[new] State3.4: Update the table entry for Q[new](s, a)as follows: Q[old](s,a)=r + γ *max a * Q[new](s,a) s[old]=s[new]

13. Q(s2, a23) = r + 0.5 * max(Q(s3,a32),Q(s3,a36)) = 0 +0.5 *0=0, 0 +0.5 *0=0

14. S6 is FINAL STATE so Q(s3, a36) = r = 100

15. After the end point, the program will loop again until all the state get the value. Q(s2, a23) = r + 0.5 * max(Q(s3,a32), Q(s3,a36)) = 0 + 0.5 * 0 = 0, 0 + 0.5 * 100 = 50

16. The is the output after all state get the value. And the next part is using neural network to find the good way from start point to end point. But this part I also exploring.

17. I believe these tasks will improve the robot system. And I feel happy because can try to do this task, and I will continue to explore these tasks. Conclusion