David M Rothschild on Posted on

I was getting my PhD in the Business and Public Policy (now call Applied Economics) program at the University of Pennsylvania, when my adviser, Justin Wolfers, got me really into prediction markets (markets that buy and sell contracts on upcoming events, thus created a revealed probability of the outcome). And, as I started learning about how they work, fiddling with new market designs and new interpretations of the data, I started PredictWise to serve two key purposes: (1) publishing my work in real-time keeps my academic work honest. It is actually a nice precursor for the newly popular pre-registering of experiments. If I publish the results of my work in real-time, no one could accuse me of tweaking the work after the results are known, to fit the end result. Also, it ensured that I had all of the data I needed, as people emailed me (now tweeted at me) if they saw a problem! (2) Since my work was producing interesting stuff, like election predictions, having a website was a good way for people to read it.

From prediction markets, my work moved towards studying other ways that individuals interacted with a controlled environment and their actions provided data that could help create market intelligence. I have written academically on: survey design, search, online browsing, social media, administrative data, and other fundamental or behavioral data-sets. And, I have examined how they help explain politics, along with finance, economics, marketing, and market for news.

But, while PredictWise remained a vehicle for disseminating research, it mainly focused on election predictions through the 2016 election, because that was where there was obvious demand. Sure, some people will read pieces on more general mapping of public sentiment, finance, marketing, or other topics, but election prediction have verifiable outcomes and are of general interest. After the 2016 election I immediately felt two serious regrets over my focus on election predictions: (1) my methodology and data could have been put to so much better use (2) concern that any popularizing of probabilities may have actually negatively influenced the election.

I cannot turn back time, so I went to work right away on what I could do, which was to transform my research and public writing into something more useful moving forward. Building off of methods originally developed with Andrew Gelman, Doug Rivers, and Sharad Goel (highlighted in this study), I have been spending much more of time, along with Tobias Konitzer, mapping public sentiment and understanding the market for news. The new PredictWise 2.0 (coming soon!) will center around both the blog and continuous mapping of support for key public policy and underlying value-frame (like populism and authoritarianism). Election predictions will be relegated in the site.

Further, with Duncan Watts, Markus Mobius, and others I have been exploring new projects on understanding the market for news. We are doing something unique: building an amazing data-set of news production and consumption so that we can explore the full market. Too many assumptions are made on partial views the market, and we hopeful that our work will transform how people study and consider the market for news. Fake news is interesting to explore, but it is just a sliver of the news people consume and news is just a sliver of the information most people consume.

My second concern is highlighted in Former FBI James Comey stating that he likely took polling into account when going after Democratic candidate Hillary Clinton publicly for her IT security (which was an absurdly meaningless issue), while keeping a lid on concerns about Republican candidate Donald Trump being compromised to Russians (which is a really important issue).

While PredictWise and David Rothschild was just bit player behind: Nate Silver at FiveThirtyEight, and many other established journalists who focused on predictions in 2016, it nonetheless is very disturbing to me. This guilt pushes me every day to make progress my work, and PredictWise, to be more useful for society. Yet, while I have a lot of respect for the work data journalists do, I do not see any evidence journalists will treat 2018 any differently.

Nate Silver tweeted out his take:

I find it startling that Nate Silver, the person most responsible for popularizing election predictions, can attack someone so directly, and repeatedly, for using election predictions to make decisions. While ideally only campaign investors and practitioners would use election predictions, certainty not law enforcement, how does this not cause Nate to reconsider the impact of his work? It is easy to blame poor decision maker James Comey for misusing his data, but whenever you publish anything, you need to worry about people will misinterpret and misuse.

This concern about how my work will be used is something I take with me whenever I hit publish.