Argument Mining

In 1987, computer scientist Robin Cohen outlined a theory of argument structure which laid the groundwork for modern argument mining tasks. Taking argument to a process in which a speaker intentionally tries to convince a hearer, her approach focused on understanding the structure arguments can take.

This structure is generally tree-like: the speakers primary claim is the root, and supporting arguments appear as branches. Secondary arguments may further expand the tree, as the speaker makes claims to reinforce a supporting argument. That is, a simple argument can take the form A and B, therefore C, or could take the form A therefore B therefore C.

In this way a complex argument can be modeled a tree with all the various supporting and secondary arguments point back up to the core argument root.

The problem that Cohen noted, which has continued to be a challenge in more recent argument mining techniques, is that core premises often go unsaid.

Take, for example, the simple argument structure of “P therefore Q.” In many contexts, a speaker will state P and Q, but leave out the primary claim: P therefore Q. As human interpreters, filling this gap is often a trivial task. Consider the simple argument:

Joey is dangerous.
Joey is a shark.

It is left the reader to infer that Joey is dangerous because he is a shark…and that all sharks are dangerous. (This, of course, could be debated…)

While there are no doubt instances where this lack of clarity causes confusion for a human reader, in general, this is a challenge which is easy for people with their broad array of contextual knowledge – and terribly difficult for machines.

Joel Katzav and Chris Reed formalize this missing argument (enthymeme) challenge. Defining an argument as “a representation of a fact as conveying some other fact,” a complete argument then has three elements: a conveying fact, the appropriate relation of conveyance, and the conveyed fact.

In parsing content, then, an algorithm could work to define a sentence or otherwise defined element as either a “non-argument” or as one of the argument types above. This makes the computer’s job a little easier: it only has to recognizes pieces of an argument and can flag which arguments are incomplete.

Furthermore, syntactic clues often give both humans and machines some insight into the structure of an implied argument: because X, therefore Y. Annotated debate texts can then help machines learn the relevant syntactic clues, allowing them to better parse arguments.

This is still somewhat unsatisfying, though, as annotating texts is difficult, expensive…and may still be inaccurate. In one study of online-debate, Rob Abbott et al employed 5-7 annotators per post and still found not-insignificant disagreement on some measures. Most notably, it seems, people are not much better at recognizing sarcasm than people.

Furthermore, arguments are not always…formal.

In legal texts or a public debate, it might be reasonable to assume that a given speaker makes the best possible argument as clearly as possible for a general human audience. This assumption can not be extended to many online forums or other domains, such as student essays. Colloquially, syntactic clues may be missing…or may even be miss used.

Latest work in argument mining has focused on over coming these challenges.

A 2015 paper by Ivan Habernal and Iryna Gurevich, for example, aimed to build an argument mining system that could work across domains, on unlabeled data. An earlier paper by Christian Stab and Iryna Gurevich focused on trying to parse (often poorly-formated) student essays.

By projecting argument elements into a vector space – or argument space – researchers can use unsupervised techniques to cluster arguments and identify argument centroids, which represent “prototypical arguments” not actually observed in the text.

There’s still more work to do, but these recent approaches have been reasonably successful and show a lot of promise.


Leave a Reply

Your email address will not be published. Required fields are marked *