Category Archives: Miscellaneous Musings

Our Secrets

“None of is us perfect, and each one of us has their own secrets, no doubt. None of is is flawless…but we are sane fanatics of reality living in a treadmill of good compromises.” That is what Comrade Pánczél tells István Balla Bán to get him to spy on his best friend; to get him to give the government incriminating evidence on his friend in exchange for keeping his own dark secret private. None of us is perfect.

This scene comes from a play I saw last night: Our Secrets, by Hungarian actor, playwright, and director Béla Pintér. It’s about government surveillance and control in Communist Hungary, a topic which seemed particularly timely as our own country – which has been no stranger to mass surveillance efforts – prepares to transfer power to a strongly nationalist leader.

There are shows through the weekend at the Emerson/Paramount Center in Boston’s Theater district. I highly recommend you get tickets and go. Spoilers below.

The story focuses on a group of Hungarian folk-music performers. As the play synopsis describes, “Communist Hungary’s dictatorship labeled the cultural acts and their corresponding community events throughout the country as either ‘banned,’ ‘tolerated,’ or ‘supported.’ The folk music scene was labeled ‘supported’ by the authoritarian government, therefore becoming a supposedly safe space for anti-Communist organizers to operate clandestinely, with little government oversight or interference to disrupt communications.”

The staging of the show fully incorporates the role of music in the era, with a giant reel-to-reel playing in the background and the musicians/cast members playing on the sides of the stage.

The story explores the individual tragedies of its characters and “exposes the hypocrisy and violence of the Communist regime, which infiltrated every corner of society to stamp out any whiff of dissent and by any means necessary.”

István Balla Bán and his friend Imre Tatár are both great folk performers. And while Tatár’s girlfriend is zealously pro-Communist, he secretly works as the editor for the underground, ant-Communist magazine, The Iron Curtain. Balla Bán is a pedophile and when the government finds out they offer him a deal: inform on your friend or go to jail. None of us is perfect.

The whole show is fantastic, but perhaps the most startling moment – though undertoned in it’s drama – is when the government turns Balla Bán. They bring him in and Comrade Pánczél asks him to spy. Balla Bán refuses. Comrade Pánczél excuses himself for a moment.

Then out of nowhere another folk-dancer friend comes in. It’s disorienting at first – what is that person doing here? The friend reveals that he’s been working with the government the whole time; that he placed bugs in people’s apartments and therefore recorded Balla Bán confiding in his therapist. The government knows everything because they already have informers.

It reminded me of that moment in 1984 when heroes Winston Smith and Julia seem like they’re going to escape control of the Thought Police, only to discover that the shop keeper who was helping them was actually a Thought Police agent. The whole world gets turned upside down.

And this, perhaps, is the most insidious thing about this kind of government surveillance; about a regime’s domineering demand for control. It’s not just that the possibility of dissent carries grave punishment. It’s that anyone may be turned against you; even your closest friends.

In part, it is this ability to isolate which gives a regime it’s power: if you can’t trust your neighbors; if you have no one in whom to confide, if at any moment your very thoughts could be used against you – organized resistance becomes impossible.

Yet I can’t help but think of the saying: they tried to bury us; they didn’t know we were seeds.


The White Moderate

In honor of Martin Luther King Day, today I wanted to share one of my favorite passages from Dr. King. It’s from a Letter from a Birmingham Jail, as Dr. King reflects upon the motivation for his work. He calls out the ‘white moderate’ – that person who “constantly says: ‘I agree with you in the goal you seek, but I cannot agree with your methods of direct action.'”

The white moderate is the greatest stumbling block of justice.

All of us in social justice work are all too familiar with the wide range of views and opinions on what actions are right and what actions are effective. These disagreements are good and healthy and productive. But those of us with positions of relatively more power – us white activists in particular – need to be mindful not to become just another white moderate; to never “paternalistically believe he can set the timetable for another man’s freedom.”

The full passage is below:

I must make two honest confessions to you, my Christian and Jewish brothers. First, I must confess that over the past few years I have been gravely disappointed with the white moderate. I have almost reached the regrettable conclusion that the Negro’s great stumbling block in his stride toward freedom is not the White Citizen’s Counciler or the Ku Klux Klanner, but the white moderate, who is more devoted to “order” than to justice; who prefers a negative peace which is the absence of tension to a positive peace which is the presence of justice; who constantly says: “I agree with you in the goal you seek, but I cannot agree with your methods of direct action”; who paternalistically believes he can set the timetable for another man’s freedom; who lives by a mythical concept of time and who constantly advises the Negro to wait for a “more convenient season.” Shallow understanding from people of good will is more frustrating than absolute misunderstanding from people of ill will. Lukewarm acceptance is much more bewildering than outright rejection.

I had hoped that the white moderate would understand that law and order exist for the purpose of establishing justice and that when they fail in this purpose they become the dangerously structured dams that block the flow of social progress. I had hoped that the white moderate would understand that the present tension in the South is a necessary phase of the transition from an obnoxious negative peace, in which the Negro passively accepted his unjust plight, to a substantive and positive peace, in which all men will respect the dignity and worth of human personality. Actually, we who engage in nonviolent direct action are not the creators of tension. We merely bring to the surface the hidden tension that is already alive. We bring it out in the open, where it can be seen and dealt with. Like a boil that can never be cured so long as it is covered up but must be opened with all its ugliness to the natural medicines of air and light, injustice must be exposed, with all the tension its exposure creates, to the light of human conscience and the air of national opinion before it can be cured.


The Knowledge Economy and (Ab)use of Symbols

I’m taking a Network Economics class this semester, and we’ve reasonably begun by reading The Use Knowledge in Society – in which Hayek addresses the economic problem of information scarcity.

The economic problem faced by society, Hayek argues, is that “the knowledge of the circumstances of which we must make use never exists in concentrated or integrated form, but solely as dispersed bits of incomplete and frequently contradictory knowledge which all the separate individuals possess.” That is, the problem is “how to secure the best use of resources known to any members of society, for ends whose relative importance only these individuals know.”

Hayek, of course, sees this problem as one which is best solved by the free market – by decentralization of economic decisions. On its face, his argument makes a lot of sense: “If we can agree that the economic problem of society is mainly one of rapid adaptation to changes in the particular circumstances of time and place, it would seem to follow that the ultimate decisions must be left to the people who are familiar with these circumstances, who know directly of the relevant changes and of the resources immediately available to meet them. We can’t expect that this problem will be solved by first communicated all this knowledge to a central board which, after integrating all knowledge, issues its orders. We must solve it by some process of decentralization.”

There is a lot of Hayek’s argument that I agree with. In the civic space, we often talk about the danger of expertise – technical knowledge is valuable and important, but reducing a community problem to a technocratic solution overlooks the expertise of the people themselves. No expert, no matter how well educated, can parachute into a community they know nothing about and successfully solve it’s problems without engaging community solutions.

But I don’t follow Hayek’s jump – just because a purely technocratic solution is clearly bad it does not necessarily follow that a purely populist solution is therefore good.

Hayek praises the pricing system of the open market as a mechanistic marvel – as an emergent behavior which continually tends towards the equilibrium of an instantaneous time and context. In other words, pricing becomes a tool for coordination, a “mechanism for communicating information.” It operates as “a kind of symbol” ensuring that “only the most essential information is passed on and only to those concerned.”

This is a inspiring description of market pricing, but it obscures the problems with such an approach – namely, it is unclear just how much people know and how much of that information is accurate.

Hayek’s invocation of ‘symbols’ immediately makes me think of Lippmann’s work – symbols can be powerful tools for coordination, but they are also props for propaganda and manipulation.

John Dewey describes the positive impact of symbols, writing, “Events cannot be passed from one to another, but meanings may be shared by means of signs. Wants and impulses are then attached to common meanings. They are thereby transformed into desires and purposes, which, since they implicate a common or mutually understood meaning, present new ties, converting a joint activity into a community of interest and endeavor. Thus there is generated what, metaphorically, may be termed a general will and social consciousness: desire and choice on the part of individuals in behalf of activities that, by means of symbols, are communicable and shared by all concerned.”

The problem, as Lippmann points out, is that elites are too easily able to manipulate those signs and symbols – to manufacture a shared experience and expectation which comes, not truly from the knowledge possessed by individuals, but which are myths designed solely to fulfill elite’s goals.


The Use of Faces to Represent Points in k-Dimensional Space Graphically

This is my new favorite thing.

Herman Chernoff’s 1972 paper, “The Use of Faces to Represent Points in k-Dimensional Space Graphically.” The name is pretty self-explanatory: it’s an attempt to represent high dimensional data…through the use, as Chernoff explains, of “a cartoon of a face whose features, such as length of nose and curvature of mouth, correspond to components of the point.”

Here’s an example:


I just find this hilarious.

But, as crazy as this approach may seem – there’s something really interesting about it. Most standard efforts to represent high dimensional data revolve around projecting that data into lower dimensional (eg, 2 dimensional) space. This allows the data to be shown on standard plots, but risks loosing something valuable in the data compression.

Showing k-dimsional data as cartoon faces is probably not the best solution, but I appreciate the motivation behind it – the questioning, ‘how can we present high dimensional data high dimensionally?’


Text As Data Conference

At the end of this week, Northeastern will host the seventh annual research conference on “New Directions in Analyzing Text as Data.”

I’m very excited for this conference which brings together scholars from many different universities and disciplines to discuss developments in text as data research.  This year’s conference is cohosted by David Smith and my advisor Nick Beauchamp, and I’ve been busily working on getting everything in order for it.

Here is the description from the conference website:

The main purpose of this conference is to bring together researchers from the social sciences, computer science and linguistics to investigate new approaches to utilizing text in social science research. Text has always been a valuable resource for research, and recent developments in automatic language-processing methodologies from the fields of information retrieval, natural language processing, and machine learning are creating unprecedented opportunities for searching, categorizing, and extracting social science information from text.

Previous conferences took place at Harvard University, Northwestern University, the London School of Economics, and New York University. Selection of participants and papers for the conferences is the responsibility of a team led by Nick Beauchamp (Northeastern) and David Smith (Northeastern), along with Ken Benoit (LSE), Yejin Choi (University of Washington), and Arthur Spirling (NYU).


Design Aesthetic and Chart Junk

In my visualization class today, we had a guest lecture by Michelle Borkin, another Northeastern professor who works in the field of information and scientific visualization. She gave us a great overview of the foundational design aesthetics of Edward Tufte.

Whether you know him by name or not, you may be familiar with some of his principles. He writes extensively about “graphical integrity,” highlighting the importance of clearly labeling of data and cautioning against distorted or misleading axes. But, perhaps more fundamentally, the Tufte-ian mantra seems to be summed in one word: simplify.

Tufte advocates for removing as much extraneous ink as possible. Non-data ink should be minimized as much of possible; clearing away the clutter and letting the data speak for themselves.

Generally, his arguments make sense – there’s no need to create a 3D bar-chart just because Microsoft Office says that you can. But in this day of infographics and data journalism, Tufte’s style can seem rather…dull.

This has led to a great debate over chart junk: a topic so real it has its own wikipedia page. “Chart junk” refers to any element of a visualization which doesn’t explicitly need to be there – elements which may make the visualization more interesting, but which don’t directly convey the data. The term was actually coined by Tufte, who, as you may have guessed, was adamantly anti-chart junk.

Recent research, though, has shown that “chart junk” isn’t necessarily inherently bad. Infographics and other visualizations designed for broad public consumption may not have the precision of a scientific visualizations, but they are more memorable and impactful.

Is chart junk okay? The answer, I guess, depends entirely on the audience, the task, and the context.


Democratic Distributions

Gaussian, Poisson, and other bell-shaped distributions are some times called “democratic.” This colloquial term is intended to indicate an important feature: an average value is a typical value.

Compare this to heavy-tailed distributions which follow generally the so-called 80/20 rule: 80% of your business comes from 20% of your clients, 80% of the wealth is controlled by 20% of the population. Indeed, this principle was originally illustrated by Italian economist Vilfredo Pareto when he demonstrated that 80% of the land in Italy was owned by 20% of the population.

In these distributions, an average value is not typical: the average household income doesn’t mean much when a small group of people are vastly more wealthy than the rest. This skew can be shown mathematically: in a bell curve, the variance – which measures the spread of a distribution – is well defined, while it diverges for a heavy-tailed distribution.

Yet while heavy-tailed distributions are clearly not democratic, I’m still struck by the use of the term for normal distributions. I’m not sure I’d call those distributions democratic either.

I’m particularly intrigued by the use of the word “democratic” to nod to the idea of things being the same. Indeed, such bell-shaped distributions are known primarily for being statistically homogeneous.

That’s starting to border on some Harrison Bergeron imagery, with a Handicapper General tasked with making sure that no outliers are too intelligent or too pretty.

That’s not democratic at all. Not really.

This, of course, leads me to the question: what would a “democratic” distribution really look like?

I don’t have a good answer for that, but this does raise an broader point about democracy: most real-world systems are heavy-tailed. Properties like hight and weight follow normal distributions, but power, money, and fame are heavy-tailed.

So the real question isn’t what a democratic distribution looks like; it is how do we design a democratic system in a complex system that is inherently undemocratic?


Gestalt Principles

In Parts I and II of Gestalt Principles, Bang Wong describes core elements of Gestalt psychology, a 1920s German theory of “how people organize visual information.” The German term Gestalt means shape or form. As Wong summarizes in Part II, “our visual system attempts to structure what we see into patterns to make sense of information.” In other words, we naturally and reflexively process visual input by attempting to group objects into “unified wholes.”

In Part I, Wong explores the principles of similarity, proximity, connection and enclosure. “The fundamental concept behind these principles is grouping;” he argues. “We tend to perceive objects that look alike, are placed close together, connected by lines or enclosed in a common space as belonging together.” Color schemes, visual clustering, and lines on a graph are all tools which can differentiate datasets.

In Part II, he examines the principles of visual completion and continuity:  “Because we have a strong tendency to see shapes as continuous to the greatest degree possible, we fill in voids with visual cues found elsewhere on the page.” This principle has an important implication: “every element on a page affects how we perceive every other element.”

Wong presents all these principles as helpful design tools which can leverage human mental processing in order to present data clearly.

What’s missing from these short essays, however, is any discussion of possible misuse of these design principles. Presumably, an altruistic designer would solely use these tools to “let the data speak for itself;” using Gestalt principles to highlight and clarify the ground truth which is already there.

But this seems to gloss over an important detail: all design choices are choices. Even putting aside the occasional malicious designer, who deliberately presents a warped visualization in order to leave viewers with an erroneous impression; it seems entirely possible that a lazy designer could accidentally imply something unintended, or that a researcher could be mislead by the Gestalt of their own visualization.

Furthermore, while these principles may be the simplest way to communicate data, there is no discussion of whether they are the right way to communicate data.

Last semester, Lauren Klein of Georgia Tech gave a talk at Northeastern in which she highlighted the visualization work of Elizabeth Peabody. Remembered primarily as an educator, Peabody created of elaborate mural charts of history, intended to provide historic “outlines to the eye.” Her work was intentionally complex and difficult to engage with; people had to interact with it to understand it. In the mid-1800s, this approach pushed the question who is authorized to produce knowledge? And subversively answered: everyone.

So Gestalt principles may make it easier to process information, but it should also be acknowledged that this may diminish the agency of the viewer – whose brain reflexively interprets visual stimuli in a given way, even if it’s not accurate and even if they know it’s not accurate.

At the beginning of the two articles, Wong quotes founding Gestalt scholar Kurt Koffka, in saying “The whole is ‘other’ than the sum of its parts.” While this is sometimes translated as “greater than the sum of its parts,” Wong is clear that this was not Koffka’s meaning: “the emergent entity is ‘other’ (not greater or lesser) than the sum of the parts.”

This quote highlights the need to think more robustly of the experience of the viewer. The design that is created, the visualization that expresses some aspect of the data, is a new thing, other than what existed before. Peabody’s visualizations were exhaustingly interactive, but they did invite the viewer to become an active participant in the act of creating this other.


Knowledge and Wonder

In his autobiography, Life on the Mississippi, Samuel Clemens – better known as Mark Twain – describes his changing relationship with the great river.

He grew up along the Mississippi, working as a typesetter and dreaming of some day becoming a steamboat pilot. In fact, his chosen pen name, “Mark Twain” is a steamboat cry, indicating a safe depth of 2 fathoms. In his early 20s, Twain was taken on as an apprentice pilot and he spent the next two years learning everything there was to know about the Mississippi.

He describes a magnificent sunset which left him bewitched in when steam boating was new to him, and he describes the awe he felt at the secret knowledge he was learning to glean from the river’s captivating surface.

The face of the water, in time, became a wonderful book ‐ a book that was a dead language to the uneducated passenger, but which told its mind to me without reserve, delivering its most cherished secrets as clearly as if it uttered them with a voice. And it was not a book to be read once and thrown aside, for it had a new story to tell every day. Throughout the long twelve hundred miles there was never a page that was void of interest, never one that you could leave unread without loss, never one that you would want to skip, thinking you could find higher enjoyment in some other thing. There never was so wonderful a book written by man; never one whose interest was so absorbing, so unflagging, so sparklingly renewed with every reperusal. The passenger who could not read it was charmed with a peculiar sort of faint dimple on its surface (on the rare occasions when he did not overlook it altogether); but to the pilot that was an italicized passage; indeed, it was more than that, it was a legend of the largest capitals, with a string of shouting exclamation points at the end of it, for it meant that a wreck or a rock was buried there that could tear the life out of the strongest vessel that ever floated. It is the faintest and simplest expression the water ever makes, and the most hideous to a pilotʹs eye. In truth, the passenger who could not read this book saw nothing but all manner of pretty pictures in it, painted by the sun and shaded by the clouds, whereas to the trained eye these were not pictures at all, but the grimmest and most dread‐earnest of reading matter.

Twain knew something the “uneducated passenger” didn’t know. He could see more and feel more as his knowledge of the river deepened. But, eventually, something changed:

Now when I had mastered the language of this water and has come to know every trifling feature that bordered the great river as familiarly as I knew the letters of the alphabet, I had made a valuable acquisition. But I had lost something, too. I had lost something which could never be restored to me while I lived. All the grace, the beauty, the poetry, had gone out of the majestic river!

…No, the romance and beauty were all gone from the river. All the value any feature of it had for me now was the amount of usefulness it could furnish toward compassing the safe piloting of a steamboat. Since those days, I have pitied doctors from my heart. What does the lovely flush in a beautyʹs cheek mean to a doctor but a ʺbreakʺ that ripples above some deadly disease? Are not all her visible charms sown think with what are to him the signs and symbols of hidden decay? Does he ever see her beauty at all, or doesnʹt he simply view her professionally, and comment upon her unwholesome condition all to himself? And doesnʹt he sometimes wonder whether he has gained most or lost most by learning his trade?

Gaining full knowledge of the river removed the mystery, removed the wonder. The river was no long a thing a beauty – it was an object to be analyzed factually.

Interestingly, Henry Thoreau expressed something similar as he worried about his work as a surveyor and found himself complicit in defining the wilderness of land as private property:

I have lately been surveying the Walden woods so extensively and minutely that I now see it mapped in my mind’s eye – as, indeed, on paper – as so many men’s wood-lots, and am aware when I walk there that I am at any given moment passing from such a one’s wood-lot to another’s. I fear this particular dry knowledge may affect my imagination and fancy, that it will not be easy to see so much wildness and native vigor there as formerly. No thicket will seem so unexplored now that I know that a stake and stones may be found in it.

As Kent Ryden describes in Landscape With Figures, “In the end, Thoreau viewed his profession of surveyor with a profound and deep-seated ambivalence, in that it simultaneously sustained and destroyed the visual, spiritual, emotional, and imaginative relationships with landscape and nature that he valued so highly.”

Knowledge has practical purpose and value, both Twain and Thoreau seem to find, but it also destroys something greater; knowledge is incompatible with beauty and wonder.

I don’t believe I could disagree with that sentiment more strongly.

In his autobiography, A Mathematician’s Apology, the brilliant G. H. Hardy wrote: “It may be very hard to define mathematical beauty, but that is just as true of beauty of any kind — we may not know quite what we mean by a beautiful poem, but that does not prevent us from recognizing one when we read it.”

Physicist and Nobel laureate Frank Wilczek has written extensively on the beauty of natural laws, which he argues is a sentiment with deep historical roots in physics:

The nineteenth-century physicist Heinrich Hertz once described his feeling that James Clerk Maxwell’s equations, which depict the fundamentals of electricity and magnetism, “have an independent existence and an intelligence of their own, that they are wiser…even than their discoverers, that we get more out of them than was originally put into them.” Not long after, Albert Einstein called Niels Bohr’s atomic model “the highest form of musicality in the sphere of thought.” More recently, the late Nobel laureate Richard Feynman, describing his discovery of new laws of physics, declared, “You can recognize truth by its beauty and simplicity.” Similar sentiments are all but universal among modern physicists.

Both Twain and Thoreau describe the loss of beauty through a process of learning, but more importantly, through a process of objectification. Through their respective work they come to see nature as a thing to be conquered, an object which can be possessed. They come to view the river or the woods through completely utilitarian means. They domesticate the natural world.

Real knowledge isn’t about that. It is about understanding the world, about reading the wonderful book as Mark Twain so eloquently describes; but ultimately it’s about constantly unlocking deeper levels of mystery, finding new layers of awe.

Knowledge builds beauty; the book never ends.