Author Archives: sshugars

Gender and Language

Both gender and language are social constructs, and sociological research indicates a link between the two.

In Lakoff’s classic 1973 paper, Language and woman’s place, she argues that “the marginality and powerlessness of women is reflected in both the ways women are expected to speak, and the ways in which women are spoken of.” This socialization process achieves its end in two ways: teaching women the ‘proper’ way to speak while simultaneously marginalizing the voices of women who refuse to follow the linguistic norms dictated by society. As Lakoff writes:

So a girl is damned if she does, damned if she doesn’t. If she refuses to talk like a lady, she is ridiculed and subjected to criticism as unfeminine; if she does learn, she is ridiculed as unable to think clearly, unable to take part in a serious discussion: in some sense, as less than fully human. These two choices which a woman has – to be less than a woman or less than person – are highly painful.

Lakoff finds numerous lexical and syntactic differences between the speech of men and women. Women tend to use softer, more ‘polite’ language and are more like to hedge or otherwise express uncertainty with in their comments. While she acknowledges that – as of the early 70s – these distinctions have begun to blur, Lakoff also notes that the blurring comes almost entirely in the direction of “women speaking more like men.” Eg, language is still gendered, but has acceptable language grown in breadth for women, while ‘male’ language remains narrow and continues to be taken as the norm.

A more recent study by Sarawgi et al looks more closely at algorithmic approaches to identifying gender. They present a comparative study using both blog posts and scientific papers, examining techniques which learn syntactic structure (using a context-free grammar), lexis-syntatic patterns (using n-grams), and morphological patterns using character-level n-grams.

Sarawgi et al further argue that previous studies made the gender-identification task easier by neglecting to account for possible topic bias, and they therefore carefully curate a dataset of topic-balanced corpora. Additionally, their model allows for any gamma number of genders, but the authors reasonably restrict this initial analysis to the simpler binary classification task, selecting only authors who fit a woman/man gender dichotomy.

Lakoff’s work suggests that there will be lexical and syntactic differences by gender, but surprisingly, Sarawgi et al find that the character-level n-gram model outperformed the other approaches.

This, along with the fact that the finding holds in both formal and informal writing, seems to suggest that gender-socialized language may be more subtle and profound than previously thought. It is not just about word choice or sentence structure, it is more deeply about the very sounds and rhythm of speech.

The character n-gram approach used by Sarawgi is taken from an earlier paper by Peng et al which uses character n-grams for the more specific task of author attribution. They test their model on English, Greek, and Chinese corpora, achieving impressive accuracy on each. For the English corpus, they are able to correctly identify the author of text 98% of the time, using a 6-gram character model.

Peng et al make an interesting case for the value of character n-grams over word n-grams, writing:

The benefits of the character-level model in the context of author attribution are that it avoids the need for explicit word segmentation in the case of Asian languages, it captures important morphological properties of an author’s writing, it can still discover useful inter-word and inter-phrase features, and it greatly reduces the sparse data problems associated with large vocabulary models.

While I initially found it surprising that a character level n-gram approach would perform best at the task of gender classification, the Peng et al paper seems to shed computation light on this question – though the area is still under theorized. If character n-grams are able to so accurately identify the single author of a document, and that author has a gender, it seems reasonable that this approach would be able to infer the gender of an author.

Still, the effectiveness of character n-grams in identifying an author’s gender indicates an interesting depth to the gendered patterns of language.  Even as acceptable language for women converges to the acceptable language of men, the subtleties of style and usage remain almost subconsciously gendered – even in formal writing.

facebooktwittergoogle_plusredditlinkedintumblrmail

TERFs

Last week, an altercation related to a “What is Gender?” event occurred in Speaker’s Corner – “a traditional site for public speeches and debates” in London.

The event was organized by a group self-identified “gender-critical feminists” – essentially, women who don’t believe that all women deserve equal rights.

As you might imagine, in the face of such an event a group of protestors showed up to demonstrate in favor of the opposite: all women deserve to be treated with dignity and respect.

From there, details begin to get fuzzy, but it appears that a woman from the first group – the “gender-critical feminists” – began harassing and attacking a woman from the second group – those supporting equality. The attacker was eventually pulled off the victim, getting clocked in the face in the process.

Afterwards, pictures of the attacker’s bruised face began to circulate online, along with a questionable story. The woman – who can be seen in a video to be shaking another woman like a rag doll until a third woman intervenes – claimed that she was the real victim; the other women attacked her.

Except, she didn’t say women.

“Gender-critical feminist” is a palatable label adopted by women more colloquially known as TERFs – Trans-Exclusionary Radical Feminists. They are fervently passionate self-identified feminists whose feminism does not have space for all women.

In short, the attacker, having incited violence with seeming intention, proceeded to misgender her victims and successfully paint herself in popular media as just a normal old woman who was wrongly attacked while attempting to mind her own business.

This narrative is exceedingly dangerous.

Taken by itself, the event is unfortunate. Indeed, any time a person is attacked in the street is cause for concern.

But the narrative that emerged from this incident plays dangerously into broader misconceptions and stereotypes. It reinforces the idea that some women are inherently dangerous and that other women would be wise to distance themselves; it tacitly assumes that only some women are ‘truly’ women in some mystically vague sense of the word, while other women are not; and it erases and attempts to overlay the experience of women for whom these first two statements ring so obviously false.

It is gaslighting on a social scale.

Consider the account described in a statement by Action For Trans Health London, one of the organizations leading the demonstration against the TERFs:

Throughout the action, individuals there to support the ‘What is Gender?’ event non-consensually filmed and photographed the activists opposing the event. Often photos and videos taken by transphobes are posted online with the intention of inciting violence and harassment against trans activists. Due to this clear and documented history of transphobes violently ‘outing’ individuals of trans experience, visibility can be a high risk to trans individual’s personal safety.

During the action, a transphobe approached activists whilst filming with their camera. An individual then attempted to block their face from the lens of camera, leading to a scuffle between both individuals. This altercation was quickly and efficiently broken up by activists from both sides.

Action for Trans Health London later shared personal accounts from women who were assaulted by TERFs during the events of that evening.

Activists had good reason to be concerned for their safety.

Yet the stories emerging from that night don’t talk about the women who were assaulted. They don’t talk about the valid fear these women experienced when someone got up in their face with a camera. They didn’t talk about the pattern of violence and harassment these women face while just trying to lead their normal lives.

In fact, the stories do worse than ignore the incident all together. They blare the headline that a woman was hit during the altercation while reserving the full sense of ‘woman’ for the perpetrator; implicitly directing compassion to the person who did the attacking.

If you’re not familiar with the term, gaslighting is “a form of manipulation that seeks to sow seeds of doubt in a targeted individual or members of a group, hoping to make targets question their own memory, perception, and sanity.”

If you have never experienced gaslighting, be glad. If you have experienced gaslighting, you know that it is one of the worst possible sensations. You lose the ability to trust yourself, to trust your own instincts and senses. You lose the ability to know what is real due to the unwavering insistence of those around you that your reality is false.

And make no mistake, the dominant narrative emerging from the incident at Speaker’s Corner is a sophisticated form of gaslighting.

It is gaslighting when an attacker is allowed to mischaracterize their victims, it is gaslighting when the injuries suffered by an attacker are treated as more concerning than the injuries they inflicted, and it is gaslighting to pretend that people who have been systematically and zealously victimized are somehow the real perpetrators deserving of our scorn.

The sad truth is that there is an epidemic of violence against trans people. In the United States alone, at least 20 transgender people have been violently killed so far in 2017. Seven were murdered within the first six weeks of the year. Almost all were transgender women of color.

We cannot pretend that this violence isn’t occurring, and we cannot stay silent in the face of false narratives which wrongfully defame and mischaracterize an entire population of women.

I don’t know how to say it more plainly than that. To deny the rights of all women, to deny the existence of all women, and to deny the richly varied experiences of all women is simply unconscionable. You cannot do those things and call yourself a feminist.

I am not much of anyone, and it is always daunting to wonder what one small person can do in the face of terrible, complex, and systemic problems. I endeavor to do more, but literally the least I can do is to say this:

To all my transgender sisters: I see you. I believe you. And I will never, ever, stop fighting for you. I will not be silent.

facebooktwittergoogle_plusredditlinkedintumblrmail

Robot Humor

Text processing algorithms are notoriously bad at processing humor. The subtle, contradictory humor of irony and sarcasm can be particularly hard to automatically detect.

If, for example, I wrote, “Sharknado 2 is my favorite movie,” an algorithm would most likely take that statement at face value. It would find the word “favorite” to be highly correlated with positive sentiment. Along with some simple parsing, it might then reasonably infer that I was making a positive statement about an entity of type “movie” named “Sharknado 2.”

Yet, if I were indeed to write “Sharknado 2 is my favorite movie,” you, a human reader, might think I meant the opposite. Perhaps I mean “Sharknado 2 is a terrible movie,” or, more generously, “Sharknado 2 is my favorite movie only insofar as it is so terrible that it’s enjoyably bad.”

This broader meaning is not indicated anywhere in the text, yet a human might infer it from the mere fact that…why would Sharknado 2 be my favorite movie?

There was nothing deeply humorous in that toy example, but perhaps you can see the root of the problem.

Definitionally, irony means expressing meaning “using language that normally signifies the opposite,” making it a linguistic maneuver which is fundamentally difficult to operationalize. A priori, how can you tell when I’m being serious and when I’m being ironic?

Humans are reasonably good at this task – though, suffering from resting snark voice myself, I do often feel the need to clarify when I’m not being ironic.

Algorithms, on the other hand, perform poorly on this task. They just can’t tell the difference.

This is an active area of natural language processing research, and progress is being made. Yet it seems a shame for computers to be missing out on so much humor.

I feel strongly that, should the robot uprising come, I’d like our new overlords to appreciate humor.

Something would be lost in a world without sarcasm.

facebooktwittergoogle_plusredditlinkedintumblrmail

The Nature of Failure

I had the pleasure of attending a talk today by Dashun Wang, Associate Professor at Northwestern’s Kellogg School of Management. While one of our lab groups is currently studying the ‘science of success,’ Wang – a former member of that lab, is studying the nature of failure.

Failure, Wang argued, is much more ubiquitous than success. Indeed, it is a “topic of the people.”

It is certainly a topic those of us in academia can relate to. While people in all fields experience failure, it can perhaps more properly be considered as a way of life in academia. The chances of an average doctoral student navigating the long and winding road to success in academia are smaller than anyone wants to think about. There aren’t enough jobs, there’s not enough funding, and the work is really, really hard. More than that, it’s ineffable: how do you know when you’re ‘generating knowledge’? What does that look like on an average day?

Mostly it looks like failure.

It looks like not knowing things, not understanding things, and not getting funding for the things about which you care most. It looks like debugging for hours and it looks like banging your head against the wall.

It looks like a lot of rejections and a lot of revise & resubmits.

Those successful in academia – subject, as they are to the fallacy of survival bias – often advise becoming comfortable with the feeling of failure. With every paper, with every grant, simply assume failure. It is even becoming common for faculty to share their personal CV of Failures as a way to normalize the ubiquity of failure in academia.

But, Wang argues, failure is the key to success.

I suppose that’s a good thing, since, as he also points out, “in life you don’t fail once, you fail repeatedly.”

Failure is a thinning process, no doubt – many people who experience significant failure never come back from it. But a series of failures is no guarantee of future failure, either.

People who stick with it, who use failures as an opportunity to improve, and who learn – not just from their most immediate failure but from their history of failure – can, with time, luck, and probably more failures, eventually succeed.

facebooktwittergoogle_plusredditlinkedintumblrmail

So Long and Thanks for All the Fish (Or, A Tribute to Cassini)

At 7:55 EST this morning, the Cassini spacecraft sent its final message to Earth before plunging into Saturn’s atmosphere. Reaching speeds over 77,200 miles (144,200 kilometers) per hour, Cassini experienced temperatures hotter than the surface of the sun, causing the spacecraft to char and break apart, its elemental components ultimately diluting in the atmosphere of the gas giant. As NASA poetically put it, Cassini is now a part of the planet it studied.

It sent data back to Earth right up until the end.

It may seem strange that the spacecraft was slated for destruction while previous missions, such as Voyagers 1 and 2 continue, with both probes still heading deeper into space after 40 years of exploration. Yet no such fate was appropriate for Cassini.

Among the most remarkable findings of the Cassini mission came from Saturn’s moons: icy Enceladus was found to have a global ocean and geyser-like jets spewing water vapor. Saturn’s largest moon, Titan, was discovered to have seas of liquid methane and an ocean of water and ammonia deep below the surface. Both moons provide promising avenues of research for that most thrilling and elusive topic: life itself.

Allowing Cassini to live out the rest of its life naturally, transmitting data well past when it had depleted its fuel reserves, would have put both those moons at risk. Cassini had to devote its final fuel to its own safe disposal.

It seems a strange thing to find the destruction of a spacecraft so moving. Cassini was machine: it felt nothing, desired nothing. It undertook an impressively successful mission and now, nearly 20 years after its launch from Cape Canaveral, it was time for that mission to come to an end.

Yet don’t we all wish to live and die so nobly? To make profound contributions through our efforts and to gracefully exit in a poetic blaze at the appropriate time?

It is beautiful to think of Cassini – a spacecraft I have loved and followed for over a decade – reduced to dust and becoming one with the planet to which it devoted much of its existence; and doing so in service to the remarkable moons with which it was intimately and uniquely acquainted.

If we are to believe Camus, all our fates are absurd; the workman toils everyday at the same tasks. Yet, in itself, this fact need not be tragic.

Truly, there is no meaning in the destruction of a spacecraft which has served well its purpose. Yet it is in these moments – when we find beauty and profoundness in the absurd; when we ascribe nobility to practical acts which mean nothing – these are the moments of consciousness. When we experience wonder generated from the mere act of living. The struggle itself is enough to fill a man’s heart.

So thank you, Cassini, for your decades of service. Thank you for the rich array of data you have shared with us, and thank you to the many, many people who made this mission possible. Because of you, I – and the rest of humanity – have seen and learned things we would have never experienced otherwise. There can be no greater gift than that.

 

facebooktwittergoogle_plusredditlinkedintumblrmail

Normalizing the Non-Standard

I recently read Eisenstein’s excellent, What to do about bad language on the internet, which explores the challenge of using Natural Language Processing on “bad” – e.g., non-standard – text.

I take Eisenstein’s use of the normative word “bad” here somewhat ironically. He argues that researchers dislike non-standard text because it complicates NLP analysis, but it is only “bad” in this narrow sense. Furthermore, while the effort required to analyze such text may be frustrating, efforts to normalize these texts are potentially worse.

It has been well documented that NLP approaches trained on formal texts, such as the Wall Street Journal, perform poorly when applied to less formal texts, such as Twitter data. Intuitively this makes sense: most people don’t write like the Wall Street Journal on Twitter.

Importantly, Eisenstein quickly does away with common explanations for the prevalence of poor language on Twitter. Citing Drouin and Davis (2009), he notes that there are no significant differences in the literacy rates of users who do or do not use non-standard language. Further studies also dispel notions of users being too lazy to type correctly, Twitter’s character limit forcing unnatural contractions, and phones auto-correcting going out of control.

In short, most users employ non-standard language because they want to. Their grammar and word choice intentionally convey meaning.

In normalizing this text, then, in moving it towards the unified standards on which NLP classifiers are trained, researchers explicitly discard important linguistic information. Importantly, this approach has implications for not only for research, but for language itself. As Eisenstein argues:

By developing software that works best for standard linguistic forms, we throw the weight of language technology behind those forms, and against variants that are preferred by disempowered groups. …It strips individuals of any agency in using language as a resource to create and shape their identity.

This concern is reminiscent of James C. Scott’s Seeing Like a State, which raises deep concerns about the power of a centralized, administrative state. In order to function effectively and efficiently, an administrative state needs to be able to standardize certain things – weights and measures, property norms, names, and language all have implications for taxation and distribution of resources. As Scott argues, this tendency towards standardization isn’t inherently bad, but it is deeply dangerous – especially when combined with things like a weak civil society and a powerful authoritarian state.

Scott argues that state imposition of a impose a single, official language is “one of the most powerful state simplifications,” which lays the groundwork for additional normalization. The state process of normalizing language, Scott writes, “should probably be viewed, as Eugen Weber suggests in the case of France, as one of domestic colo­nization in which various foreign provinces (such as Brittany and Occitanie) are linguistically subdued and culturally incorporated. …The implicit logic of the move was to define a hierarchy of cultures, relegating local languages and their regional cultures to, at best, a quaint provincialism.”

This is a bold claim, yet not entirely unfounded.

While there is further work to be done in this area, there is good reason to think that the “normalization” of language disproportionally effects people who are outside the norm along other social dimensions. These marginalized communities – marginalized, incidentally, because they fall outside whatever is defined as the norm – develop their own linguistic styles. Those linguistic styles are then in turn disparaged and even erased for following outside the norm.

Perhaps one of the most well documented examples of this is Su Lin Bloggett and Brendan O’Connor’s study on Racial Disparity in Natural Language Processing. As Eisenstein points out, it is trivially impossible for Twitter to represent a coherent linguist domain – users around the globe user Twitter in numerous languages.

The implicit pre-processing step, then, before even normalizing “bad” text to be in line with dominant norms, is to restrict analysis to English-language text. Bloggett and O’Connor find that  tweets from African-American users are over-represented among the Tweets that thrown out for being non-English.

Dealing with non-standard text is not easy. Dealing with a living language that can morph in a matter of days or even hours (#covfefe) is not easy. There’s no getting around the fact that researchers will have to make difficult calls in how to process this information and how to appropriately manage dimensionality reduction.

But the worst thing we can do is to pretend that it is not a matter of concern; to begin our work by thoughtlessly filtering and normalizing without giving significant thought to what we’re discarding and what that discarded data represents.

facebooktwittergoogle_plusredditlinkedintumblrmail

Graduate Workers Need a Union

Last night I attended a great panel hosted by the Graduate Employees of Northeastern University (GENU), a union of research and teaching assistants. The union is currently working towards holding its first election and becoming certified with the National Labor Relations Board (NLRB), an independent federal agency which protects employees’ rights to organize and oversees related laws.

Those of you immersed in academic life may have noticed a recent increase in organizing efforts among graduate workers at many institutions – this is due to a 2016 ruling by the NLRB that “student assistants working at private colleges and universities are statutory employees covered by the National Labor Relations Act.”

In other words, graduate employees have the right to organize.

Those not immersed in academic life, or less familiar with graduate education, might find this somewhat surprising. As someone said to me when I told them about this panel, “wait, you’re an employee? Aren’t you a student?”

Well, yes. I am an employee and a student. These two identities and lives are complexly intertwined and can be difficult to distinguish – when am I a worker and when am I a learner?

Often I am both simultaneously.

But I think about the perspective of the student program staff at the college where I worked for several years before starting my PhD. Collectively, we made a lot of student employment decisions – hiring student workers to help around the office and selecting paid student fellows to work at local organizations. Those students – primarily undergraduates – were workers, too, but every decision we made was centered around the question: how will this improve the student’s education?

That is, their student identity was always centered. Work expectations always deferred to course expectations. We looked to hire students who were prepared to learn a lot from their experiences, and we created structured mentorship and other activities to ensure student learning was properly supported and enhanced. The work was good work which needed to be done, but the primary purpose of these opportunities was always to create space for students to learn.

Graduate student work is…a bit more complicated. I have been fortunate in my own graduate experience, but I couldn’t even begin to enumerate the horror stories I’ve heard from other graduate employees whose work is most definitely work.

Even assuming good faculty members and good departments, the entire structure of American higher education is designed to exploit graduate students as cheap labor. Their labor may serve to enhance the undergraduate experience, but is rarely designed to enhance their own.

This problem is exacerbated by the fact that graduate student workers have virtually no power while the faculty, department, and administration they serve have a great deal of power over them. For graduate workers it is often not a possibility to simply “get another job” – a difficult undertaking for any vocation. International students are particularly vulnerable, as their Visa status could be taken away in a heartbeat.

As several of the panelists mentioned last night, many graduate students simply try to “keep their head down” in the face of this power imbalance. Stay quiet, don’t complain, and do your best to keep focused on the research you’re passionate about.

This is a reasonable copping response, but the reality is that silence never fixes a problem, and sometimes trouble will find you no matter how hard you try to avoid it.

Nearly all of the panelists had a story of someone who was unfairly targeted for termination, who was entirely taken by surprise when a department in which they “had no problems” suddenly had a serious problem with them.

Without a union these become the isolated stories of isolated individuals. They are personal problems to be worked out and ignored at the local level. In the absence of clear rules and expectations, they will happen again, and again, and again – in good departments and bad – with very little recourse for the individuals involved and with no resulting structural change to prevent it from happening again.

Unions build collective power. They build the ability of a people to come together, to share their ideas and concerns, and to work together with a common voice in order to achieve mutually-agreed upon outcomes.

As one of the panelists from a faculty union described, forming a union was a clarifying experience. It brought the community together and generated a clear, shared understanding of common problems and collective solutions. It created venues for enabling structural and policy changes that had been deeply needed for years.

Perhaps most fundamentally, it is important to understand that a union is not some abstract outside, thing. It is a living thing. It is the workers. It is a framework which allows us to work together, learn together, and build together. It is formed from our voices in order to address our concerns and to protect our interests.

We are the union.

And graduate student workers need a union.

The live-stream of the event, which focused specifically on STEM workers, can be seen here. Particularly for those at Northeastern, please check the GENU-UAW website, Facebook page, and Twitter.

facebooktwittergoogle_plusredditlinkedintumblrmail

Social and Algorithmic Bias

A commonly lamented problem in machine learning is that algorithms are biased. This bias can come from different sources and be expressed in different ways, sometimes benignly and sometimes dramatically.

I don’t disagree that there is bias in these algorithms, but I’m inclined to argue that in some senses, this is a feature rather than a bug. That is: all methodical choices are biased, all data are biased, and all models are wrong, strictly speaking. The problem of bias in research is not new, and the current wave of despair is simply a reframing of this problem with automated approaches as the culprit.

To be clear, there are serious cases in which algorithmic biases have led to deeply problematic outcomes. For example, when a proprietary, black box algorithm regularly suggests stricter sentencing for black defendants and those suggestions are taken to be unbiased, informed wisdom – that is not something to be taken lightly.

But what I appreciate about the bias of algorithmic methods is the visibility of their bias; that is – it gives us a starting point for questioning, and hopefully addressing, the inherent social biases. Biases that we might otherwise be blind to, given our own personal embedding in the social context.

After all, strictly speaking, an algorithm isn’t biased; its human users are. Humans choose what information becomes recorded data and they choose which data to feed into an algorithm. Fundamentally, humans – both specific researchers and through the broader social context – chose what counts as information.

As urban planner Bent Flyvbjerg writes: Power is knowledge. Those with power not only hold the potential for censorship, but they play a critical role in determining what counts as knowledge. In his ethnographic work in rural appalachia, John Gaventa similarly argues that a society’s power dynamics become so deeply entrenched that the people embedded in that society no longer recognize these power dynamics at all. They take for granted a shared version of fact and reality which is far from the unbiased Truth we might hope for – rather it is a reality shaped by the role of power itself.

In some ways, algorithmic methods may exacerbate this problem – as algorithmic bias is applied to documents resulting from social bias – but a skepticism of automated approaches opens the door to deeper conversations about biases of all forms.

Ted Underwood argues that computational algorithms need to be fundamentally understood as tools of philosophical discourse, as “a way of reasoning.” These algorithms, even something as seemingly benign as rank-ordered search results – deeply shape what information is available and how it is perceived.

I’m inclined to agree with Underwood’s sentiment, but to expand his argument broadly to a diverse set of research methods. Good scientists question their own biases and they question the biases in their methods – whether those methods are computational or not. All methods have bias. All data are biased.

Automated methods, with their black-box aesthetic and hopefully well-documented Git pages,  may make it easier to do bad science, but for good scientists, they convincingly raise the specter of bias, implicit and explicit, in methods and data.

And those are concerns all researchers should be thinking about.

 

facebooktwittergoogle_plusredditlinkedintumblrmail

Bag of Words

A common technique in natural language processing involves treating a text as a bag of words. That is, rather than restrict analysis to preserving the order in which words appear, these automated approaches begin by simply examining words and word frequencies. In this sense, the document is reduced from a well-ordered, structured object to a metaphorical bag of words from which order has been discarded.

Numerous studies have found the bag of words approach to be sufficient for most tasks, yet this finding is somewhat surprising – even shocking, as Grimmer and Stewart note – given the reduction of information represented by this act.

Other pre-processing steps for dimensionality reduction seem intuitively less dramatic. Removing stop words like “the” and “a” seems a reasonable way of focusing on core content words without getting bogged down in the details of grammar. Lemmatization, which assigns words to a base family also makes sense – assuming it’s done correctly. Most of the time, it doesn’t matter much whether I say “community” or “communities.”

But reducing a text – which presumably has been well-written and carefully follows the rules of it’s language’s grammar seems surprisingly profane. Do you lose so little when taking Shakespeare or Homer as a bag of words? Even the suggestion implies a disservice to the poetry of language. Word order is important.

Why, then, is a bag of words approach sufficient for so many tasks?

One possible explanation is that computers and humans process information differently. For a human reading or hearing a sentence, word order helps them predict what is to come next. It helps them process and make sense of what they are hearing as they are hearing it. To make sense of this complex input, human brains need this structure.

Computers may have other shortcomings, but they don’t feel the anxious need to understand input and context as it is received. Perhaps bag of words works because – while word order is crucial for the human brain – it provides unnecessary detail for the processing style of a machine.

I suspect there is truth in that explanation, but I find it unsatisfactory. It implies that poetry and beauty are relevant to the human mind alone – that these are artifacts of processing rather than inherent features of a text.

I prefer to take a different approach: the fact that bag of words models work actually emphasizes the flexibility and beauty of language. It highlights the deep meaning embedded in the words themselves and illustrates just how much we communicate when we communicate.

Linguistic philosophers often marvel that we can manage to communicate at all – the words we exchange may not mean the same thing to me as they do to you. In fact, they almost certainly do not.

In this sense, language is an architectural wonder; a true feat of achievement. We convey so much with subtle meanings of word choice, order, and grammatical flourishes. And somehow through the cacophony of this great symphony – which we all experience uniquely – we manage to schedule meetings, build relationships, and think critically together.

Much is lost in translating the linguistic signal between me and you. We miss each other’s context and reinterpret the subtle flavors of each word. We can hear a whole lecture without truly understanding, even if we try.

And that, I think, is why the bag of words approach works. Linguistic signals are rich, they are fiercely high-dimensional and full of more information than any person can process.

Do we lose something when we reduce dimensionality? When we discard word order and treat a text as a bag of words?

Of course.

But that isn’t an indication of the gaudiness of language; rather it is a tribute to it’s profound persistence.

facebooktwittergoogle_plusredditlinkedintumblrmail

Opinion Change

While there are differing views on whether or not a person’s opinions are likely to change, there’s a general sense of “opinion change” as some clear and discrete thing: one moment I think X, and the next moment I think Y…or perhaps, more conservatively, not X.

Coming to opinion change from a deliberation background, I’m not at all convinced that this is the right framework to be thinking in.

Perhaps in a debate the goal is to move your opponent from one discrete position to another, or to convincingly argue that your discrete position is better than another. But in deliberation – which very well may include aspects of debate – the very notion of “opinion change” seems misplaced.

I think of deliberation more as process of collaborative storytelling: you don’t know the ending a priori. You create the ending, collectively and uniquely. A different group would tell a different story.

As the story unfolds, you may shift your voice and alter your contributions, but the X -> Y model of “opinion change” doesn’t seem to fit at all. 

The challenge, perhaps, is that standard conceptions of opinion change take it as a zero-sum game. One person wins and another person loses. Or no one changes their mind and the whole conversation was a waste.

But deliberation isn’t like that. It is creative and generative. It is a collective endeavor through which ideas are born, not a competitive setting with winners and losers. In deliberation, all participants leave changed from the experience. They come to think about things in new ways and have the opportunity to look at an issue from a new perspective.

They may or may not leave with the same policy position they had going in, but either way, something subtle has changed. A change that may effect their future interactions and future judgements.

Standard conceptions of “opinion change” as a toggle switch are just too narrow to capture the rich, transformative interplay of deliberation.

facebooktwittergoogle_plusredditlinkedintumblrmail