Archive for November, 2012

Predicting everything

November 12, 2012

So I just got back from an awesome workshop put on by the DAGGRE project this weekend, where I was invited to give a presentation.  I learned a lot, and had a chance to discuss some really interesting ideas with some really smart people. It all came together for me, over the weekend and on my trip home, into something that could possibly launch me into a new career, or at least be a really interesting project.

So the first thing I learned was that I really do seem to be good at forecasting and predicting things a lot of different things.  I sorta already knew that, as I had attended DAGGRE’s March workshop, did really well on their workshop prediction market, and went home committed to get myself up to #1 on the DAGGRE leaderboard.  I did so over the course of about 6 months, and as a result of my success got invited to give a presentation at this week’s workshop on my winning strategies.  But beyond that, I learned that I seem to have a broader knack for making well-calibrated and accurate predictions and estimates, over a really wide range of subjects.

It was also reinforced just how hard it has been to transition a lot of new forecasting techniques (prediction markets especially) into widespread use, often for a quite fundamental reason: we really don’t *want* better forecasts of many things in life, especially information about our own (organization’s) prospects.  IARPA is sponsoring DAGGRE and several other projects to develop these techniques for use in the intelligence community, but I’m much more interested in their broader application.  There does seem to be demand for forecasts about other people (industry trends, economic indicators, and all the stuff financial markets do), but most of those markets are already served by incumbents of one sort or another.  So in order to apply new techniques, you either have to convince those companies to adopt them, or you have to outcompete them and overcome all their incumbency advantages.  Either of those are way more difficult than just making good forecasts.

But a third thing I’ve noticed is that, except for a few narrow fields like election forecasting, there aren’t a lot of people releasing public forecasts of the likelihood of various events.  There is a lot of raw data out there, from Twitter feeds, to prediction markets like Intrade, to online sportsbooks, to options and derivatives on financial markets, but it is often left as an exercise to the reader to translate that data into anything representing a forecast, such as a probability of a certain event.  So maybe there’s an unfilled niche in making better public forecasts about the kinds of things people have already proven their interest in by the money they have on the line…

So here’s my idea:

Firstly, we should put together a general-purpose open-source prediction model that can take a large number of prediction-like inputs, such as options chains, and convert them into statements like “the market is predicting an X% chance of Y happening by Z date.”  Perhaps we run a website that continuously recalculates that probability based on market prices.  But perhaps in order to see the prediction, you first have to give us your estimate (maybe in Delphi form, as a best guess with a minimum and maximum probability and a % confidence).  We can then take all the estimates provided by the website’s users, and calculate (and present) them as something like “but our users are predicting an A% chance, with a 90% confidence interval of B%-C%”.  Obviously, that could be a source of insight into areas where the estimates of our users differ from those of the financial markets.  A naive Delphi forecast by itself isn’t probably going to outperform the market, but it may point to areas where additional forecasting effort using some of the techniques below might lead to some profitable options trading opportunities.

Once we have a large number of predictions being imported from financial markets (and maybe being estimated by interested users), we should also start linking them up using combinatorial methods.  Through a process such as Bayesian inference, such methods can be used to express the impact a particular event should have on any particular prediction.  For example, a worse-than-expected jobs report will have obvious impacts on S&P500 options and futures, but may have more wide-ranging direct and indirect impacts that could be usefully represented in the model, and used to flag market prices that may not have fully adjusted yet.  This is undoubtedly a simplistic version of the kinds of analysis that hedge funds are already doing, but it may be useful to present a publicly accessible model of how things are likely to change with changing conditions, and allow people to provide input on the inter-relationships, and also use the model themselves to do their own scenario testing, etc.

If such a model gets enough traction to be developed into something that can robustly represent (and collect estimates for) a large number of interrelated predictions about the world, it may also be useful to start incorporating other predictions, models, relationships, etc. in and linking them to the economic data.  For example, The Economist recently published an article about the coming transition to driverless cars, and all the changes that will bring to industries from hotels to insurance to the country pub.  It would be quite interesting to actually represent such impacts in a prediction model, in a way that they (and their many interactions) can be evaluated.  I’m pretty sure there are investment insights lurking there, in addition to all the impacts that people like transportation planners should be considering.

For another example, last year I predicted that unsubsidized solar photovoltaics would hit grid parity sometime between 2015 and 2020, that annual solar PV production could grow exponentially until it matches the annual growth in world energy consumption (maybe sometime next decade), and that after that point, the cost of solar PV and grid electricity will be inextricably linked, and will continue to fall over time.  I’ve done a poor job of blogging my thoughts on the potential impacts of that, but for one thing it would put us on a path to solving the climate change problem via a dramatic decrease in carbon emissions (and by eventually making energy cheap enough to make it cost-effective to actually pull carbon back out of the air).  That would of course also mean desalination would be so cheap that coastal areas would no longer have to worry about water shortages.  There are probably at least thousands of other ways that cheap energy would affect hundreds of different industries, and there’s no way I could think of even a fraction of them.  But if we had a model where different people could input their ideas about how such scenarios would affect the parts of the economy they know about, we might be able to come a lot closer to actually understanding the potential impacts of such a change.

Now, just because technological improvements will eventually eliminate the need for most carbon emission doesn’t mean we aren’t at risk of significant disruption between now and then.  But there are already a lot of really smart people working on estimating exactly how likely such scenarios are.  If we could also incorporate predictions from their models, and more importantly get them to link them via conditional estimates to the rest of the prediction model, that would create a tremendously useful tool for actually coming up with decent probabilistic estimates of the uncertainties, risks, costs, and benefits of various potential approaches to dealing with climate change.

Anyway, that probably captures the vision.  Obviously there’s a long road between here and there, but I can see several ways to start incrementally producing useful insight.  My next step will probably be to start doing some digging to see if there’s anything in the literature about how to convert options chains into probability estimates, or if I need to learn how to use Black-Scholes to calculate them myself.  If you know of anything related to that or any of the other ideas above, please point me to any information available online (or put me in touch with anyone I should talk to). I’m not really interested in reinventing any wheels here, but I think there’s an opportunity to put some existing parts together in a novel way to make something that’s actually useful (and who knows, maybe even be able to make some money with it).