Cognitive Modelling in Computer Game Pidgins

Cognitive Modelling in Computer Game Pidgins (2002-2005)

ARC Linkage Grant

Investigators: M. Kavakli, T.Bossomaier, M. Cooper

This project models the way human users exploit languages in virtual environments and develops a pidgin language for computer games. It has implications not only for computer assisted collaborative tasks in virtual worlds and other educational or interactive entertainment environments like computer games, but also for speech markup.

An APAI  (Rudra Tarashankar) started working on the project. Rudra's PhD project is to develop "pidgin" languages to be used by game characters to communicate with a human player. The research project has been transferred from Charles Sturt University to Macquarie University at the end of 2005.  Within 2 years that the APAI working on the project, we have developed a model for The Cognitive Game Pidgin system. The Cognitive Game Pidgin system has a pidgin language with limited grammar and vocabulary [1].

The human player and the game characters communicate with each other using the GPL. The cognitive model and the ontology govern the behaviour of the game characters. The cognitive model has an XML coding all possible reaction to an action or activity. A mathematical model using graph theory is used to develop the GPL and test the cognitive model. The entire theme of the game play is centred on the game environment.

We have developed the grammar for the Game Pidgin language along with sample vocabulary and its representation in XML. We also worked on Emotion classification by Support Vector Machines (SVM). We have used the software PRAAT to annotate the speech and SVM for Matlab to classify the emotion states of neutral and anger from the speech signal as these are the most common emotion elicited during Game-play.

The result of our research is encouraging and further research with more samples is required to prove our hypothesis. We added an emotion tag to the coding scheme adapted from Suwa & Tversky for capturing the context of complex behaviour in game-play. We developed an XML for storing cognitive actions and their responses. 6 papers were produced.