Unsupervised modeling of Twitter conversations

{{Summary
 * title=Unsupervised modeling of Twitter conversations
 * authors=A. Ritter, C. Cherry, B. Dolan,
 * url=http://www.cs.washington.edu/homes/aritter/twitter_chat.pdf
 * tags=Twitter, LDA, dialogue acts, speech act theory,
 * summary=This paper models dialog acts in Twitter conversations and presents a corpus of 1.3 million conversations. They provide a status diagram showing the likelihood of transitions between dialogue acts.



Methodology
Unsupervised LDA modelling of Twitter conversations, evaluated by held-out test conversations. Uses a conversation+topic model (segmenting post words into those that involve the topic of conversation, the dialogue act, or something else). Trained on 10,000 randomly sampled conversations (conversation length 3-6) from the corpus.

Corpus
1.3 million conversations with each conversation containing between 2 and 243 posts. In summer 2009, they selected a random sample of Twitter users by gathering 20 randomly selected posts per minute, then queried to get all their posts. Followed any replies to collect conversations. Removed non-English conversations and non-reply posts.