1. Speaking patterns -MAS.662J, Fall 2004
Diane Hirsh & Xian Du
Dec

2. Outline Introduction Data at hand Objectives Applied Methods Results
Conclusion and Comments
References

3. Introduction
Group debate always leads to only two kinds of final decision: right or wrong
Debating Member always influences each other by different speaking patterns
The speaker’s pattern and other speakers’ influences lead to the final decision of the debate
Identifying those patterns and influences can be helpful to the prediction of the debate result and member’s option

4. Data at hand Raw data is Speaker ID and stamp time
Labeled data indicates the initial position and final outcome of the speakers
Fig. 1 talking sequences of the four members in group: study_07_task1

5. Objectives
Find the distinct feature to discriminate the right-decision from wrong-decision group
Predict the winner of the project
Tell the individual’s position

6. Applied Methods Preprocessing
“Turn” [1,2] : “for each unit of time we estimate how much time each of the participants speaks, the participants who has the highest fraction of speaking time is considered to hold the “turn” for that time unit. For a given interaction, we can easily estimate how a pair participating in the conversation transitions between turns.”

7. Applied Methods Recognition techniques
Parzen Window & Linear Discriminant Function with one-leave-out validation
Hidden Markov Models (HMM)

8. Parzen Window & Linear Discriminant Function with one-leave-out validation
Goal:
- to discriminate the right-decision group from wrong-decision group
Assumption:
- Wrong decision group has a “wrong” density function (Parzen window)
- There is a hyperplane H to divide the “turns” space into half spaces: right or wrong
Group 07 and 12 are two wrong groups in the training groups

9. Results for Parzen Window & Linear Discriminant Function application
- 07 group: 6 in 10 right groups are identified but Minimum error rate~0.5
- 12 group: fail (5 in 10 and ~0.5)
Linear Discriminant
07 group: 8 in 10 right groups are identified with Minimum error rates: 0.057~0.47, AVG=0.269

10. Hidden Markov Models (HMM)
Single HMM
Identify Group option (wrong/right)
Parallel HMM
Identify members’ state option (probability of the final decision)
Influence model
- Improve the result of parallel HMM by considering the influence between members in the group

11. Implementation of HMM Assumptions:
- each member in one group has influence on others by turns “amount” and more turns contribute to higher influence.
- each member retains its initial state or changes to be opposite. The transition is strictly one direction.
Initializations:
- randomize the initialization of transition matrix while keeping the HMM strictly left-to-right.
- two states for each group: right or wrong; two observation symbols: 0 or 1.
- The initial states of each member in the group are set according to the initial position in labeled data (e.g.1/4)

12. Implementation of HMM 1 4 2 3 Influence model S2 S1 a11 a22=1 a12
Fig. 2 The dynamic structure of influence model with four members’ HMMs (arrow in the influence model indicates the influence weight on c by c’) [3, 5]

13. Result for HMM
Single HMM for right-wrong groups’ separation: cannot find wrong groups
e.g..
training data: [1,2,3,4,5,6,8,9,10,11] and [7,12]
testing data: [1,2,3,5,6,7,8,9,10,12] and [4,11]
Confusion Matrix for the Test Data (test 9)
Recognized as right Recognized as wrong
Class
Class

14. Result for HMM Parallel HMM (group 8 and group 11 used)
Recognition accuracy= 54.4%
Members’ state transition matrix:
- group 08: 1/4 meet labeled data
- group 11: 3/4 meet labeled data
Influence HMM model
Recognition accuracy= 61.0%
- group 08: 2/4 meet labeled data

15. Conclusion and Comments
The data set is not friendly for HMM because different training group has different members and each group has only one speech sequence;
Influence model improves the HMM recognition accuracy but its random initial state probability limits its application in this project (it needs more training) and its result up to now failed to find the winner;
Linear discriminant function recognizes some right-wrong group well but not all (more data needed for the testing);
The length of talking time varies a lot among different groups which limits the recognition;
More features may be helpful for this project.

16. References
Tanzeem Khalid Choudhury, “Sensing and modeling human networks”, PhD thesis, MIT, Cambridge, MA, 2004
Chalee Asavathiratham, “the influence model: a tractable representation for the dynamics of networked Markov chains”, PhD thesis, MIT, Cambridge, MA, 2001
A Pentland, Learning communications-understanding information flow in human networks, BT technology Journal, vol. 22, No4, October 2004
Shi Zhong and Joydeep Ghosh, A new formulation of coupled hidden markov models, A new formulation of coupled hidden markov models, Tech. Report, June, 2001
YongHong Tian, etc. Incremental learning for interaction dynamics with the influence model, IEEE, www-2.cs.cmu.edu/~dunja/LinkKDD2003/papers/Tian.pdf
Lawrence Rabiner and Biing-Hwang Juang, Fundamentals of speech recognition, Prentice Hall, 1993