Schneier on Security

Name • November 10, 2018 5:27 AM

There is a lot more conflict of interest in ‘the competing values inherent in data stewardship’ than just the security perspective. Admittedly this is off-topic here but some of those other issues don’t lend themselves very well to a honest public debate, so I’ll write them here where nobody minds the writer hiding behind anonymity…

As the article says, releasing your research data is an increasingly common requirement for getting grants and publishing paper. But pushing it too far can backfire, similarly to other supposedly well-meaning initiatives like the Vancouver Recommendation (a one size fits all monstrosity from the medical and pharma research where the conflicts of interest are immensely greater then almost any other field). Here are two real examples from my field, somewhere in the natural/physical sciences:

1) As a scientist your success depends not only on the impact of your publications. It also depends substantially on how impressive your publications list appears to the random bureaucrats scoring your ‘academic productivity’. The two things are much less related than outsiders may think. I worked 3 years for then publishing one single monumental paper whose publicly-released dataset 10 years later is still being used by virtually all papers in my field. Data are in a sane format and well documented so nobody needs to bring me in as a coauthor just to make any sense of my gigabytes of data. And since it was complete and exaustive, I didn’t need to publish countless followups, further developments and other ways of padding my publications list while wasting my competitors/readers time with a forever moving ‘half-baked state of the art’. That was a very high impact paper, lots of citations. But still one paper in three years and just a handful of co-authorships in the following 10 years when people genuinely needed my contribution. In most countries, I’d have seriously risked my academic job for under-performance. A somewhat complementary paper and dataset came out around the same time as mine, so the two datasets are often used together. That other dataset, also publicly released, is I’d say a masterpiece in data obfuscation. Your chances are almost nil of using it properly without inviting its original author to join your paper. And that is not for any intrinsic complexity: similar datasets are routinely used even for undergrad lab assignments with no problems. Of course this trick earned the guy a very impressive list of coauthorships in top journals for very little work – well at least after the work it must have gone into obfuscating his ‘public’ data in the first place…

2) nowadays the bulk of my own research funding comes with an obligation to release the entirety of my data as soon after collection in the field as practical. This puts me in a difficult spot: producing said data takes up more than half of my and my group’s time (actual fieldwork, data quality control, designing, building, testing and maintaining the field instrumentation, logistics planning and contracting, and so on). If we release it all immediately, there is a near 100% chance that one or another of the high flying colleagues who use all of their time analyzing ‘public’ data will outcompete us and publish first on our own data. On the next grant I’m still evaluated on my publications output so the negative incentive to collecting data under these conditions is immense. So what do we do? First, we pretend that quality control takes a lot more than it really does. And secondly – well I learned from the more senior and acclaimed scientist mentioned above so when we are careful to express some of the key variables in the most esoteric and idiosyncratic way, here and there even leaving them in engineering units like millivolts or pulses per second which are worthless without the sensor calibration factors… If anybody wants to use them, they need to involve some of us in their papers. Or wait for our own papers, which come with sanely formatted data (so they get cited) and in countless ‘masters project’-sized bites (so we get to publish a lot of them). Honest? No. Efficient in our and our readers’ time? No. In line with the Vancouver criteria and stuff? No, and to hell with them anyways. But it allows for the survival of the relatively few who do actually go out and collect unique data in the field at great expense of time. It only really pains me when every new young colleague comes and candidly offers to improve our output formats to make them easier for anybody to use… Then I have to keep a straight face while hand-waving about consistency with legacy formats, following some arcane forgotten standard and other bs. At some point they decide I’m an old fool, give up, and go back co-authoring and and first-authoring on salaries I couldn’t give them otherwise.

Apologies for the off topic, and happy data releasing!