Fighting over the Nature of the Indus Valley Symbols

The most competent research is usually done by the scientist with the best right hook and feint.

If you are talking about research related to the Indus Valley symbols, that is.

A Brief Introduction

As most of us know, the Indus Valley civilization existed for more than half a millennium, about 4500 years ago. They were a rather sophisticated urban peoples and left behind quite a few clues about how they lived. But not enough to settle the most puzzling questions about the ancient history of the Indian subcontinent. They suddenly disappeared, almost overnight in historic terms, after nearly 700 years of city-dwelling, over an extensive region extending from modern Af-Pak to Madhya Pradesh.

Of all the toys they left us- pottery, bricks, bath complexes, drainage systems, statuettes etc.- the most intriguing is the thousands of tablets with several symbols and illustrations.  Ever since they were excavated in late 19th century, archeologists and linguists have attempted to explain what these symbols mean.

The two most famous successes in deciphering ancient scripts have been the deciphering of the ancient Egyptian script and the Linear-B from Mycanean Greece. The key to deciphering the Egyptian script was the discovery of the Rosetta stone, a set of inscriptions which consisted of Egyptian language script written alongside their Greek translations. The Linear-B was deciphered thanks to the fact that the symbols represented a pronounciation and syntax almost identical to ancient Greek. Unfortunately for those studying Indian history, the Indus equivalent of a Rosetta stone is yet to be found. It is unlikely that a Rosetta stone will be found on this side of Af-Pak, as extensive inscriptions of Indian languages don’t occur till Asoka, a full one and a half millennia after the ancient Indus peoples disappeared. However, archeological evidence tells us that the Indus Valley civilization conducted trade with Mesopotamia and other ancient civilizations of the near-East. Perhaps, one day we will come across a tablet with the Indus symbols written alongside the languages of ancient Mesopotamia. A Linear-B type decipherment, however, can be attempted only after a few problems described in the following paragraphs are resolved.

An example of the Indus symbols.

Cutting to the Chase

The most important problem yet to be resolved concerns the fundamental nature of these symbols. Do they represent a language, say, like Devanagari, or do they represent a set of symbols, like say, a set of smileys. For over a hundred years this question had been debated the way most scientific issues are- through rigorous research and conjecturing. Recently, in 2004, a paper was published by Steve Farmer, an independent language researcher, Richard Sproat, a Professor of linguistics and EE from UIUC and Michael Witzel, an Indologist from Harvard. I first came across the paper last year about the same time, following an interesting discussion with Vishwesha who had referred me to an article that had appeared in the journal Science, on 6 June 2008.

What struck me when I first saw the paper (in Electronic Journal of Vedic Studies) by Farmer, et al., was the title, which appeared rather arrogant to me- ‘The Collapse of the Indus Script Hypothesis: The Myth of a Literate Harappan Civilization‘. From the title, I thought that the paper was perhaps a survey of all the published work thus far, and that these overwhelmingly pointed towards the fact that the Indus symbols were non-linguistic and did not representing any underlying language. As I read on I realized that the authors were talking about their own work, and proclaiming the ‘collapse’ of an opposing point of view based on their own rigorous and extensive studies. Now, I am not competent in archeology/linguistics/history to comment about the academic merit of their paper. But what disturbed me was the ad hominem attacks that were included in the paper. The paper employed no subtlety in downplaying opposing points of view, and in discrediting researchers who didn’t toe the line of the paper. The paper repeatedly linked the personal/political opinions of the authors from the opposing camp to their research. In addition to presenting arguments supporting their own work, the authors alleged that researchers who thought that the Indus valley script represents a language were motivated by their political affiliations. They repeatedly used words such as “Hindu nationalists” and “Dravidian nationalists”. I agree, the work done by some of the researchers working in that field is trash, but one should counter bad or incomplete research through logical, coherent arguments, which are not adulterated by accusations of the persons intentions. I believe that when arguing about academic work published in peer reviewed journals, it is unprofessional to question the intentions of the author, no matter how bad their research is. Such personal attacks are not permissible in the academic literature. It surprised me how this paper was published in a peer reviewed journal. Perhaps because Michael Witzel is its editor :O ?  I wonder if it does any good for the academic merit of one’s arguments to make personal accusations against other researchers in peer-reviewed journal papers!

That said, there are some on the other side of the argument who make similar allegations of ‘imperialist’ research- like N. S. Rajaram, N. Jha etc. But few people take them seriously. They are not as scholarly as Farmer, et al, and their research is far from rigorous. Add to that, the fact that their work is mostly presented in self-published books. However, with a sweeping generalization, Farmer, et al., intended their barbs to all researchers who propose that the Indus symbols represent a language. My suspicion to this effect was confirmed following recent academic backs-and-forths between the Indus-script and the Indus-non-script camps.

Conditional Entropy and the Indus Script

The reputed journal Science, on 23 April 2009, published a paper by Rajesh P.N. Rao, an Associate Professor of Computer Science at UWash, Nisha Yadav and Mayank Vahia from TIFR, Hrishikesh Joglekar, R. Adhikari from IMSc, Chennai, and the reputed linguist Iravatham Mahadevan. The paper was titled, with a humility characteristic of most academic literature, as ‘Entropic Evidence for Linguistic Structure in the Indus Script’. Any EE/CS graduate student will have enough exposure to Information Theory to understand the work presented in Rao, et al. Then again, I am not sure if I can competently judge the direct link they draw between conditional entropy and the nature of the Indus symbols. I am convinced (with some skepticism, though) by their argument that the Indus symbols represent a language. However, I believe it would have been good if Rao, et al. had included an analysis of the conditional entropy of non-linguistic symbols too, rather than just abstract symbols. It would have helped convince the archeologists/linguists to some extent, of the discriminatory ability of conditional entropy, with respect to the linguistic/non-linguistic nature of a set of symbols. The basic premise of Rao, et al. is that symbols representing languages, like say Devanagari or English, form a Markov process with a regularity and sequential order that is somewhere in between that of repetitive non-linguistic symbols (what they refer to as Type 2 non-linguistic symbols) and random non-linguistic symbols (Type 1). The paper goes on to show, as in the figure below, that the conditional entropy of the Indus symbols follows a trend similar to scripts that represent a language (like Tamil or English) rather than non-linguistic. The sequential order is captured by the modelling of symbols as a 1st order Markov process with various transition probabilities. Markov processes are basically characterized by the fact that the future state is dependent only on the present state, irrespective of the past states. Symbols of a language are often modeled as Markov processes. Languages as Markov processes(like the opening roll of Star Wars 😐 ) is the opening sequence of Claude E. Shannon’s seminal paper ‘A Mathematical Theory of Communication‘, in the sections ‘The Series of Approximations to English’ and ‘Graphical Representation of a Markoff Process’. As I understand, the conditional entropy used in the paper by Rao, et al. is the event ‘Symbol j follows Symbol i’ or ‘Symbol j occurs given Symbol i has occured’. The data used was the 417 Indus symbol corpus of Iravatham Mahadevan.

Plot of conditional entropy of symbols of various languages. Image borrowed from the paper by Rao, et al. Science, 23 April 2009

A footnote: Entropy as you may know, is universally and simplistically speaking, a measure of disorder/uncertainty/randomness. Consider two discrete random variables X and Y. The conditional entropy is a measure of the uncertainty/randomness in choosing a value for the random variable Y, given that the event ‘X = x’ has occurred. If Y is completely determined by X, then there is no uncertainty in ‘Y given X = x’, and the conditional entropy will be zero. On the other hand, if X contains no information about Y, then ‘Y given X = x’ is  ‘Y’, and the conditional entropy is equal to the simple entropy of ‘Y’.

This is a common problem in cryptanalysis, where you try to determine if a sequence of symbols is a language.

So all’s well – an interesting paper with potentially groundbreaking work published in a reputed journal like Science, etc. Now Farmer, et al. have come up with their refutation of Rao, et al. The refutation is again characteristic of ad hominem attacks against other researchers, and the outright dismissal of  opposing viewpoints! A technicality: Farmer, Sproat and Witzel fail to note the novelty of the sequential modeling. I quote a few lines from the acerbic and rather grating refutation by Farmer, Sproat and Witzel below. Your opinions, please:

If the paper (by Rao, et al.) had been properly peer reviewed it would not have been published

Why are Farmer, Sproat and Witzel (FSW) so bellicose in defending their work? If their work has the academic merit, there is absolutely no reason for them to cry this way. I am surprised and disturbed. Is this is the kind of mudslinging that characterizes research on Indian history? Now, I am not alleging ‘imperialist’ suppression of our history,  but seriously, why do FSW write like angry little kids who have just been told that they may be wrong?

It is a matter of concern that attempts at conducting objective research about Indian history are scuttled this way- by reputed scientists like Farmer, Sproat and Witzel writing in the language of politicians.

As an EE guy, to me, Rao, et al.’s conclusions are rather convincing. Perhaps an archeologist/linguist would think otherwise. There are many who do, for criticisms of Rao, et al. please refer: Michael Libermann of UPenn, Steve Farmer’s condescension.

Here is Iravatham Mahadevan’s article in the Hindu (which was actually published as I was writing this blog).

Perhaps there are more blogs which support Rao, et al.’s work. I will put up links as and when I find them.