I’m a regular reader of Kent’s Imperative.  I don’t know how many of you read the blog, but I found this perspective  on Prediction Markets interesting. The notion of Predictions Markets has been getting a lot of attention lately.  And I wonder how many of today’s startups might be considering Prediction Markets as a way to determine which way the wind is going to blow.

About two years ago, I agreed to participate in a trial of a Prediction Market in the area of storage.  After a month of watching people trade on opinions, I began to wonder how they would validate the results.  Turns out they validated the results from the first published report of a top market research company. 

Here are the top 4 problems with the methodology, in my opinion:

  1. Every good market research company uses a taxonomy.
  2. Every good market research company’s taxonomy is different from every other good market research company’s taxonomy.
  3. Some bad market research companies use no taxonomy.
  4. Every market research company must estimate certain things that are unknowable.

The business of sizing markets is a process of information gathering, information cleaning, information transformation, and estimation in the absence of good information.  I did know one analyst that delivered “near-perfect” market estimates.  He had convinced every supplier to provide precise results on unit shipments.  He would collect them all, keep everyone’s information confidential, and publicly report back the total market.  That was it for the public reporting.  For the suppliers that participated, he also reported back to each of them individually their own ordinal position and their own market share.  For example, “The total market was 200,000 units, you were #2, and your market share was 10.2%.”  The only problems were, of course, that:

  1. Some suppliers don’t know what they’ve shipped, and thus can’t report accurately
  2. From time to time a supplier fails or refuses to report

Anyone who asks, “How hard can it be to count?” clearly doesn’t remember the chads from the 2000 election in the U.S.