Last year there was a big rumble-grumble about Sotheby’s Paris’ sale of a portion of the Barbier-Mueller collection of Pre-Columbian antiquities.

Initial controversy was focused on a very strange claim from exactly one Peruvian that any object exported from Peru since independence was cultural property of the nation and thus stolen property in need of repatriation. I wrote about that in a blog post entitled The Sotheby’s sale of the Barbier-Mueller Collection: waxing and waning on Peruvian law. Next we started hearing the 1970 UNESCO convention date floating around, and, in particular, controversial assertions from Sotheby’s that this was an ‘old collection’. I discussed those issues in a blog post entitled The collection’s issues: Sotheby’s Paris Barbier-Mueller sale 2. Finally, focus shifted to calls for repatriation of the artefacts for sale. Quite a few countries piled on, but the sale went through none the less. When it was over, the sale was dismissed as a flop.

I couldn’t get the 1970s/Old Collection/Good Provenance issue out of my head, though. I kept coming back to it. Even wise commentators claimed that the poor sales in the auction pointed to buyers rejecting objects with poor provenances and favouring objects that either imported at an earlier (pre-1970) date or were not subject to repatriation claims. The sort of claims I snarked meanly about in a post called No, NYT, listing a couple of lots doesn’t prove that antiquities buyers care about provenance!. A rep for the UNESCO convention made this assertion to me verbally…specifically about the Barbier-Mueller auction. I was puzzled. There were no numbers. No evidence for a connection. Why was everyone drawing this conclusion?

So I told anyone who would listen that this just isn’t how auctions work and isn’t how statistics work. Speaking of, no one had any statistics, they were just saying these things based off their impressions and a light reading. Because they wanted it to be true. But auctions and auction catalogues are so murky. There are so many variables. The results of auctions, I contend, do not truly show what antiquities buyers favour…or if they do we can’t see it through the noise. I know that is pretty rich coming from someone who did a masters on antiquities auctions, but if you read it (don’t) you will see I mostly conclude that catalogue analysis doesn’t get us very far. That it does NOT represent the market in general or, perhaps, even the market for antiquities at auction.

My colleague, a statistician at the University of Tasmania who makes the best (and probably only) smoked wallaby spring rolls on earth, and I put the Barbier-Mueller auction through the wringer a bit. We asked the standard questions that the commentators asked, did the standard stats to answer them, presented the results, and then outlined why this was not right, good, viable, or useful. That’s right. It is a grand lesson in inferential statistics and how we art and antiquities market and trafficking folks need to not make unsubstantiated claims.

Researchers in this field see numbers as some sort of ultimate end in themselves. We want to find a number for everything and policy makers push us to try to do so. But this is dangerous. I am not a stats expert. Neither are you (well, maybe you are. Are you reading this, brilliant co-author?). If our work is to have any meaning, any credence, we have to stick to what is there, not what we want to be there. To take a bit from the paper (a co-author bit):

Inferential statistics rely on the assumption that available data are a representative sample of some larger population. Samples are analysed in order to make generalisations about the population from which the sample is drawn. Every rigorous analysis requires careful consideration of the constraints on the population to which inferences may plausibly be applied. Identification of the appropriate scope of inference must be considered with the utmost care, to maximise the utility of results and to avoid unwarranted generalisations.

Yes.

Am I trashing auction catalogues across the board? Not yet, no, but I think we need to use them quite differently and combine them with other sources.

This is the most advanced item on my research agenda. The paper is just about ready to submit to, I believe, the Journal of Field Archaeology who were nice enough to ask for something. I am sorry I was so slow with the turn around, JFA folks, but it is a better paper for it. We are also presenting this work at the Society of American Archaeology Meeting in Austin.