1. An AV Control Method Using Natural Language Understanding Author ： M. Matsuda, T. Nonaka, and T. Hase Date ： 2012-01-09 Speaker ： Sian-Lin Hong IEEE Transactions on Consumer Electronics, vol 52, no. 3, pp. 990-997, 2006. 1

2. Outline 1. Introduction 2. Applying The Proposal To Consumer Use 3. Verification Experiments And Results 4. Conclusion 2

3. 1. Introduction （ 1/8 ） Audio-visual (AV) systems are becoming multifunctional and larger in scale Audio-visual (AV) systems are becoming multifunctional and larger in scale Therefore, users must understand the structure of the system, plan the sequence of using the AV devices, and operate and control the devices following the appropriate methods Therefore, users must understand the structure of the system, plan the sequence of using the AV devices, and operate and control the devices following the appropriate methods 3

4. 1. Introduction （ 2/8 ） Remote controller (RC) for each AV device has a unique arrangement of switches and original functions Remote controller (RC) for each AV device has a unique arrangement of switches and original functions This forces users to operate various RCs for the AV devices which make up the AV system at the same time This forces users to operate various RCs for the AV devices which make up the AV system at the same time 4

5. 1. Introduction （ 3/8 ） Generally, when users operate several AV devices, they need to manage a series of procedures as follows Generally, when users operate several AV devices, they need to manage a series of procedures as follows i. Visualizing the result ii. Determining the status of each device iii. Planning various operations iv. Setting up certain functions for each device 5

6. 1. Introduction （ 4/8 ） When users control this AV system by the current method, they manage all the procedures from (i) to (iv) When users control this AV system by the current method, they manage all the procedures from (i) to (iv) The status of the AV system is changed by each input, so the user needs to follow the procedures from (ii) to (iv) The status of the AV system is changed by each input, so the user needs to follow the procedures from (ii) to (iv) In addition, as more AV devices are added to the system, the combination of their functions becomes more complicated In addition, as more AV devices are added to the system, the combination of their functions becomes more complicated Therefore, these procedures are even more difficult for users who are unused to operating AV devices Therefore, these procedures are even more difficult for users who are unused to operating AV devices 6

7. 1. Introduction （ 5/8 ） 7

8. 1. Introduction （ 6/8 ） 8

9. 1. Introduction （ 7/8 ） This paper proposes a method of generating the user interface of an AV controller that reduces the sequences of operation by using natural language understanding and reasoning technology, and controls the AV system automatically This paper proposes a method of generating the user interface of an AV controller that reduces the sequences of operation by using natural language understanding and reasoning technology, and controls the AV system automatically 9

10. 1. Introduction （ 8/8 ） When the AV system has a lot of functions, the AV controller doesn’t need to display a lot of icons When the AV system has a lot of functions, the AV controller doesn’t need to display a lot of icons The user doesn’t need to select an icon from many icons displayed on the screen or touch panel, because the user can communicate with the AV controller The user doesn’t need to select an icon from many icons displayed on the screen or touch panel, because the user can communicate with the AV controller 10

11. 2. Applying The Proposal To Consumer Use （ 1/7 ） The proposed system consists of the four categories The proposed system consists of the four categories parsing parsing semantic analysis semantic analysis reasoning reasoning controlling controlling 11

12. 2. Applying The Proposal To Consumer Use （ 2/7 ） Parsing Parsing The syntax of the sentence input by the user is analyzed The syntax of the sentence input by the user is analyzed When words in the input sentence are not in its lexicon, the AV controller generates questions and tries to get more useful information When words in the input sentence are not in its lexicon, the AV controller generates questions and tries to get more useful information 12

13. 2. Applying The Proposal To Consumer Use （ 3/7 ） Semantic analysis Semantic analysis The contents of the sentence are analyzed based on the information obtained from parsing, and then the information necessary for the reasoning operation to control the AV device is determined The contents of the sentence are analyzed based on the information obtained from parsing, and then the information necessary for the reasoning operation to control the AV device is determined 13

14. 2. Applying The Proposal To Consumer Use （ 4/7 ） Reasoning Reasoning Based on the information obtained from parsing and semantic analysis, how to control which AV devices is finally decided Based on the information obtained from parsing and semantic analysis, how to control which AV devices is finally decided 14

15. 2. Applying The Proposal To Consumer Use （ 5/7 ） Controlling Controlling The AV controller controls the various AV equipment connected to each other by a network using the results of reasoning The AV controller controls the various AV equipment connected to each other by a network using the results of reasoning 15

16. 2. Applying The Proposal To Consumer Use （ 6/7 ） It is difficult to apply the natural language processing method used for general computers, because embedded microcomputers for consumer use have limited processing performance and memory capacity It is difficult to apply the natural language processing method used for general computers, because embedded microcomputers for consumer use have limited processing performance and memory capacity Therefore, a method of natural language processing by generative grammar is used to solve these limitations Therefore, a method of natural language processing by generative grammar is used to solve these limitations 16

17. 2. Applying The Proposal To Consumer Use （ 7/7 ） 17

18. 3. Verification Experiments And Results （ 1/2 ） 18

19. 3. Verification Experiments And Results （ 2/2 ） The resulting dialog between the system and the user displayed on the LCD panel The resulting dialog between the system and the user displayed on the LCD panel The system started the playback of the tape in VCR2 correctly after displaying “OK.” U means the user’s inputs and S means responses of the system The system started the playback of the tape in VCR2 correctly after displaying “OK.” U means the user’s inputs and S means responses of the system 19

20. 4. Conclusion This paper has proposed an AV control method based on natural language understanding This paper has proposed an AV control method based on natural language understanding Experiments to verify the effectiveness of the proposal were conducted by using an AV system with a prototype controller with an embedded microcomputer Experiments to verify the effectiveness of the proposal were conducted by using an AV system with a prototype controller with an embedded microcomputer This experimental system restricted the input method to typed sentences using natural language because it does not support a voice interface This experimental system restricted the input method to typed sentences using natural language because it does not support a voice interface 20