Andreas Weigend
Stanford University
Data Mining and Electronic Business
Stat 252 and MS&E 238
Spring 2008

Class 4


The MP3 recordings for this class can be found here.
Video lectures for the class can be found here.

Must read before class: http://weigend.com/files/teaching/stanford/readings/PredictionMarkets%2520CowgillWolfersZitzewitz2008.pdf
Additional material on prediction markets are in http://weigend.com/files/teaching/stanford/readings/PredictionMarkets

Introduction (by Andreas)

Customer Behavior

Examples: despite knowing how it really is, we cannot help perceiving it differently
  • Optical illusions (brain persists on seeing image one way even if you know logically your perception is not true).
    • Example: The following two tables have the same length (longest side). However, even when told this, people still perceive the left table as being longer. This shows how people have deep systematic biases that they can't help having.
twotables.gif
  • Organ Donations: countries with a default opt-in organ donation system vs. a default opt-out system have far higher participation
  • Hip Replacement: To have surgery or not have surgery? The tipping point is if someone has to take 2 pills a day instead of 1 pill a day – then they are more likely to choose having surgery instead of having to take pills.
  • Online dating site: when people receive more matches they perceive themselves as happier, however they don't have time to read all the emails / matches so they are actually less successful
  • Draeger's Markets Jam Test – Draeger's supermarkets conducted a taste test.
  • Number of Reasons : One group in an experiment were told to write down 3 reasons they loved their partner. The next group was told to write down 10. Which group was happier? The one that had to write down less were determined to love their partner more
  • Economist Subscriptions Experiments:
    • Initial Setup: Given 3 choices for a subscription, 1) online subscription only for $55, 2) print version only for $125 3) online plus print for $125, people ignored 2 because it was not a good deal, and 84% chose 1 and 16% chose 3.
    • Revised Setup: Remove choice 2.
    • Result: 68% chose 1, and 32% chose 3. The majority choice moved from 1 to 3.
    • Takeaway: by putting more weights to choice 2 the print version subscription,more attentions are drawn to choice 3 to which choice 2 is closely related.
  • SSN and anchoring
    • Wine example: In an experiment to see what people were willing to pay for wine, there was a positive correlation between the price they were willing to pay and the last 2 digits of their SSN. Here is a paper (free to download) regarding SSN and anchoring willingness to pay.
  • Here is a book written by Dan Ariely that contains many other examples about how people tend to behave irrationally. In this book he argues that although people tend to behave irrationally when making many decisions, they do so in a predictive way.

The Lesson? 1) It's difficult to understand what people believe about the world 2) People are good at comparisons 3) People don't know their preferences

Why do people trade so much? Why is the data volume as high as it is? Answer: Different preference functions. If you give the same set of objective information to different people, they will have different conclusions because they have different preference functions. When you have many interactions between people, you get a price discovery.

Wisdom of the Crowds by James Surowiecki
In order for prediction markets to be successful, you need the following ingredients:
  1. crowd must be diverse – if everyone has the same preferences, you won't learn anything
  2. people need to be decentralized (no social influence)
  3. crowd needs some way to summarize their opinions
  4. people need to make their decisions individually


The following sections are discussed in Bo Cowgill's presentation about Google's implementation of Prediction Market.

Why Prediction Markets (In General)?

From Bo's slides:
  • To predict something
    • Purpose of setting up prediction market is to leverage “Wisdom of the Crowd” to measure the likelihood of certain events in the future. Good probability estimation provided by some specialized prediction markets would be extremely useful. e.g. Probability of having flu epidemic in an area for the coming year, such prediction improves decision makers’ ability to set the appropriate policy. In addition to social issues, prediction markets add values in business setting by predicting critical market events.
    • Prediction markets serve more than just business, politics and curiosity. Iowa Health Prediction Market will provide early signal for healthcare professionals to prepare for potential outbreaks of flu.
  • To provide an objective measure of something “everyone knows”
    • Within an organization or a community, difference in members’ knowledge creates different probabilistic opinions of certain event’s outcomes. Their trading of securities (i.e. outcomes) in a market reflects their probability belief. When new information is available to each individual member, market prices will quickly capture his/her improved knowledge. While trading signal indicates inflow of new knowledge or information, a frequently moving or highly liquid market can show participants’ shift in probability belief or optimism of certain outcomes instantly. The correlation between shift in participants’ probability and changes in external factors, which can be stock fluctuation, progress of product development, customers’ reaction to product introduction and even newly-gained personal experience, or rumors, can be captured.
  • To communicate priorities
    • Using prediction market, organization and community can draw participants’ attention to matters that are important, or simply awareness of some new initiatives. Priority of these issues can be communicated to other members by setting up the markets.
  • To create transparency
    • In a large organization, things happen on the side may not be known to people on the other side. Creation of markets for these issues give traders (members of the organization) incentive to investigate what is going on in order to make the best bet. Through this discovery process, they will get a better understanding of work other people’s work within the organization.
  • To create a Hawthorne effect
    • When a market exists for a project you are working on, two things are likely to happen. 1.) You feel that your work is under surveillance, which gives you pressure to deliver good results. 2.) You also as a trader are encouraged to work harder since you have the power to influence the outcome. Either of them will drive higher productivity.
  • To improve morale
    • Prediction market is a powerful channel to engage members of an organization in the decision making process. It offers a accessible platform for members to show their own belief or confidence in certain matters, thus allowing them to raise their concerns. For decision market specifically, members’ knowledge often becomes an important input for setting future directions. Ultimately, feeling empowered will improve members’ morale and motivate them to achieve the common goal. (Corporate goal in business sense)
  • Good video introductions on prediction market

Setting up the (Google) market

From Bo's slides
  • Iowa Electronic Markets style market
    • Avoids framing bias from short selling aversion
    • Requires reasonable amount of comfort with bids and asks
    • No automated market maker => no initialization bias, but requires many participants for liquidity
    • Complete set of outcomes – Each component of a market represent one possible outcome. Altogether the components should cover all possible outcomes. E.g. A.) You get an “A+” in this course, B.) You receive a grade worse than “C+” this course, C.) You receive a grade between A and C, inclusively, in this course.
    • Mechanism – Use currency (i.e. Goobles) to buy component/security of a market, if the associated outcome turns out to be true, 1 unit of Goobles is rewarded and zero otherwise. Alternatively, one can exchange complete set of a market’s components (i.e. a bundle) with 1 Gooble and sell unwanted components (expecting falling probabilities for those outcomes).
    • Arbitrage opportunities are available – In any market, when sum of bid prices for complete set of components/securities is greater than 1, one can first exchange 1 Gooble for a bundle, then sell all components. Arbitrage can also be achieved when ask prices sum up to less than 1 Gooble. Trader can buy all component and sell it as a bundle for 1 Gooble. However, arbitrage opportunities are rare and usually disappear very fast.
  • Markets open to all
    • 1.) Close to people who know too much, reason: when they know too much on certain topic and know what is going to happen with high accuracy, thus unfair to other traders.
    • 2.) Close to those who know too little, reason: when traders don’t know anything about the market (e.x. an engineer in Mountain View betting on a product under development in India) they will add noise to the market and affect the probability measure.
    • Counter argument: Trading is an information revealing process. When price of an outcome has been driven up to 0.90 Goobles, we will assume that traders have ability to form good judgment on what results in this pricing. In addition, the information disparity also helps inducing liquidity essential for an active market.
    • Certain topics are off-limits – market that lack objective outcomes. E.g. Google is awesome? A.) Yes, B.) No
  • Everyone starts each quarter with same $ endowment
    • With this endowment, participants can buy shares in any market, which are subject to up’s and down’s.
    • Final wealth will be compared at the end of the quarter
  • Linear monetary incentives
    • Predictive Value is important! There is need to encourage people to do well.
    • Problem with non-linear market: If money prizes are only rewarded to top 10, people will be indifferent between being 11th and the last. This potentially causes people to long securities with very low probabilities hoping to get high return from extremely risky investments, thus reducing the predictive power of the markets.
    • The Google solution – people receive raffle tickets with amount commensurate with their trading performance. More raffle tickets, higher chance to win the 8000 dollars (not Goobles) check.
    • This paper examines the impact of different incentive systems in prediction markets, and finds that both monetary and non monetary incentives may be effective in the performance of the market.
  • Participation incentives (T-shirts)
    • With the raffle ticket policy, some traders who do well in the markets might end up with no prize
    • To incentivize participation and induce liquidity, 1000 dollars is awarded to the most active trader, along with T-shirts for other active traders.
  • Special Market One - Fun markets
    • Main purpose is to lure people into trading system by setting up markets on interesting topics. When they start to trade, liquidity results and will eventual spill over to other market.
    • Analogy – New casinos opened seldom cannibalize existing businesses.

  • Special Market Two - Decision markets
    • Decision market is set up to inform a particular decision
    • For example, does Gmail user search more than regular Google user? If majority of participants respond “Yes”, action will be taken to acquire more Gmail users

Rich datasets inside the firm

Can extract social networks from the following:

  • Who performed a code review for whom? 2002-2007.
  • Historical seating chart, PLUS Long/Lat of all desks.
    • Physical proximity (very close proximity) increases social interaction and influences biases and the way people bid
    • Has the biggest impact in the market
  • Who works for a common boss(es) or common department?
  • Who works on common cross-departmental projects?
  • Who is on common email lists?
  • Who named who in a company-wide survey of social and professional networks? (this is the only dataset that is static with no start/stop date).
  • Common HR records -- background and demographic info such as start date, college and grad school, languages spoken, demographics, etc.

Discussion

  • Advice: give traders lots of money
    • At one point in the development of the Google PM, the creators inflated the currency dramatically. As Bo explained, more money can be split up across more securities, making the market more interesting.
  • Liquidity: users have a bad experience if nobody else is hanging out there;
    • One more recent design feature of the Google PM is including the fun markets, on subjects ranging from pop culture, sports, current news, entertainment, etc. The point is to get more people into the system, get them familiar with trading on something they enjoy. You then get liquidity into the market.
    • Alternatively, economist Paul C Tetlock, in this paper , observes that securities markets that are too liquid overprice low probability events and underprice high probability events
  • Should prices in the internal market be public? Legal implications if firm makes predictions public, particularly predictions regarding financial performance.

How really does it work?

  • What motivates people best? Money, reputation or social rewards? Discussion here.
  • Placing a bet: The market poses a question, e.g. "How many new gmail users will there be this month?" Go to prediction market homepage, select market, select price and quantity. View the high bid and low ask but not the entire book of bids.
    • This paper provides supporting evidence that prediction market prices tend to provide useful estimates of people's beliefs about the probability of an event.
  • No Marketmaker in Google's PM. For more on this aspect of market design, refer to this basic primer here
  • Markets come with links attached: to give more information
  • Annotate your order: Though not a current feature of Google's PM, a possibility would be to add a discussion / message board/ etc. explaining your bid, allowing one to influence others and drive up prices; integrate into strategy.
  • Should prediction markets be anonymous? There are trade-offs. Some people may have incentive to conceal their identify (employee going against boss) but in general Bo believes the Google culture would be more open to sharing.

Role of experience

  • How experience can improve your trading performance
    • Work experience: the longer you work at Google, the better you trade
    • Trading experience: the more you trade, the better you get
      • trading in the market is a process that helped correct the bias, and thus improves performance over time.
  • See more on page 6 of The promise of prediction markets: A roundtable, Bo's answer to the question " Do crowds learn over time?"

Typical trader in the Google markets

  • Quantitatively oriented
  • Centrally located
  • Hired before 2006
  • Traders were much less likely to leave Google
  • Male
  • Slightly younger but slightly more senior

Manipulation of the markets

  • Can attempts at manipulation be distinguished from “noise” trading by other traders?
  • Academic studies:
    • “Sheep bring out the wolves.” – Hanson (2005)
    • Historic failures to manipulate – Strumpf (2005)
  • In the paper Information Aggregation and Manipulation in an Experimental Market, the author argues that "attempts at market manipulation in fact end up increasing the accuracy of the market because they provide that much more profit incentive to bet against the manipulator." However, in this post, one counter reasons that "accuracy isn’t everything".

People at the bottom doing better than people at the top

  • Even taking into account their limited trading experience
  • In all types of markets(decision market, fun market, etc), overal this is true
  • why: people at the bottom know detailed information of the frontline, they interact directly with products, customers, etc
  • robustness of this finding:
    • people from high up tend to trade more
    • this finding is after normalizing performance at the level of trade

How Betting Changes People's Perceptions

  • If people are able to bet on something and see that they are wrong, they might change their mind (or at least shut up)
  • question from Harvard Business School:
    • The case: In one company, engineers believed the company was about to go out of business next month, and kept saying that month after month. So in this case a predicition market about whether the company is to go out of business next month may converge to the answer 'yes', and people may start looking for jobs.
    • The upside: This market will allow the engineers to bet on that while the whole company is watching, and see they were wrong, and they would eventually stop saying that.

Market Activity Patters

  • People used to trade a lot on day 1 of the quarter, and then lock in their trades until the end of the quarter. Now they get weekly installments every week (1/12) of their allotment
  • There are often surges in trading activity around the close of the quarter, when a lot of things are happening. Additionally, the closer it is to the actual date of an event (such as the launch of a new product), the more accurate the results are.

Conclusions from the Google paper:

Google’s markets reveal some biases
  • Optimism: Especially on good days for GOOG stock.
  • Under anticipation of extreme events
  • Favorites overpriced
Some traders contribute disproportionately to these biases
  • New hires
  • Inexperienced traders
  • Non-coders
  • Non-centrally located traders
We can use correlations in trading to learn about information flows
  • Geography matters
  • Common languages matter
  • Most other factors do not

Bots

  • A wiki introduction to bots at here
  • Use a screenscraper to write bots
  • Note: track for arbitrage, which is not very common
  • But bots do generally well
    • Better measurement of the knowledge people have, e.g., whether they are taking advantages of little opportunities
    • Different bots have different strategies
      • A most sophisticated one: UI to the market (put up a lot data, store new data from snapshots, etc.), implement a series of trades or something that would monitor the market
    • Time scales of bots very different from humans
  • Fair VS. unfair
    • Open source to bots
    • Have to be programmed by human
    • Cannot tell who has a bot

How to Evaluate

  • 0-10%: what percentage actually happened
  • 10-20%: What percentage actually happened, ect.
  • Good thing: the highest price stock happens 65% of the time, and the highest price is always 65%
  • Bad thing: the highest price stock is 95% of the time, but only happens 65%; or the highest price stock is 35% of the time, but happens 65%
  • Borrwing against future income is possible

Polls vs Prediction Markets

  • Here is a paper arguing that markets out perform a naive reading of the polls.
  • Here is another paper written by a Columbia Political Science professor suggesting that prediction markets don't outperform polls if you properly discount and adjust the polls.
    • There are several biases in election polls that political scientists are aware of. For example, in the beginning of a presidential race, the people take out their frustrations with the incumbent and assume that the challenger will be able to answer all of their problems. Therefore, poll data in the beginning of the race will be favorable for the challenger. However, as the race goes on, the people realize the challenger has his own flaws, and his poll numbers go down over time. If these biases are taken into account (by taking poll data and applying certain formulas to discount for the biases), better estimates can be obtained. When this algorithm is applied to the poll data, the results are roughly the same as the results of the prediction markets.
    • There are several criticisms, one of which is that the paper doesn't use many elections as its data source. Currently, larger studies that involve other elections (statewide and congressional district races), are being done.
  • Barry Ritholtz has written a blog post about the predictive ability of prediction markets. He states, "They [Prediction markets] are too small, have too little money at stake, and are therefore readily susceptible to undue “influence.” Despite this, they enjoy disproportionate credibility and media attention to their prognostications."

How People Use Prediction Market Results


  • Nowadays, management teams in many companies notice the results of prediction market while making decisions. Most of them are extremely interested to see the implication behind these results. Some of them would put the results from prediction market in the meeting agenda and bring them on the table during the meeting.
Are decisions being made based on the prediction market results?

  • Generally speaking, a decision is normally made by consulting a great number of sources of information and results from detailed analysis. Obviously, a prediction market is one of many data sources, but there is ample evidence that the prices are being viewed and discussed on internal blogs, wikis, meetings agendas and meeting notes.Nevertheless, it’s hard to quantify or describe explicitly the relationship between prediction market results and the quality of decisions.

  • In Prof. Tom Davenport's article “Prediction Markets: Is Anybody Really Predicting?”, he praised Google’s culture and its fit with prediction markets, but even they may have a problem with participation. Bo Cowgill, McKinseyQ-711424.PNGGoogle’s prediction markets evangelist, and a couple of academic economists have written a paper describing two-and-a-half years of prediction market activity at Google. It’s well done and worth a read. But one important fact is the authors say that 6,425 employees had a prediction market account, but only 1,463 placed at least one trade. In the McKinsey roundtable summary Cowgill notes that new Google recruits have ensured a continuing stream of users, but he isn’t sure about participation levels in a lower-growth environment. "It wouldn’t be surprising if participation levels over time were low for prediction markets. Companies can’t really pay you in anything but trinkets, or it becomes online gambling. In this upstanding country that’s illegal. Further, as Mark Turrell pointed out to me, any prediction market works by continual trading based on new information. The participants in prediction markets would have to be pretty highly motivated to keep going back to the site and placing new bets. In any case, as the McKinsey roundtable participants suggest, if you want to run one of these things, you’ve got to devote a lot of attention to marketing them to employees.”

Insider Trading in Prediction Markets

  • Generally, insiders avoid the market in order to avoid legal issues
insidetrading.JPG
Definition (US Securities and Exchange Commission)


  • Insider trading" is a term that most investors have heard and usually associate with illegal conduct. But the term actually includes both legal and illegal conduct. The legal version is when corporate insiders—officers, directors, and employees—buy and sell stock in their own companies. When corporate insiders trade in their own securities, they must report their trades to the SEC.
  • Illegal insider trading refers generally to buying or selling a security, in breach of a fiduciary duty or other relationship of trust and confidence, while in possession of material, nonpublic information about the security. Insider trading violations may also include "tipping" such information, securities trading by the person "tipped," and securities trading by those who misappropriate such information.
  • Corporate officers, directors, and employees who traded the corporation's securities after learning of significant, confidential corporate developments;
  • Friends, business associates, family members, and other "tippees" of such officers, directors, and employees, who traded the securities after receiving such information;
  • Employees of law, banking, brokerage and printing firms who were given such information to provide services to the corporation whose securities they traded;
  • Government employees who learned of such information because of their employment by the government; and

  • Other persons who misappropriated, and took advantage of, confidential information from their employers.

Some famous insider trading cases:


A paper that discusses how to change insider trading rules to improve prediction markets.

Spread

  • The bid-ask spread is the price difference between the most money a buyer is willing to pay for an asset (the bid price) and the least money for which a seller is willing to sell the asset (the ask price). The bid price is the price available for an immediate sale of the asset and the ask price is the price available for an immediate purchase of the asset.
  • The average spread for Google's PM is about 8 points. Certain traders and bots make money by being a market maker, slowly closing the gap.
  • In Google's PM, traders can put multiple bids and multiple asks on the same security.

Prediction Markets and Data Mining

  • A prediction market is one way of collecting rich data, much richer than polls or historical data.
  • Prediction markets have the quality of being about future markets, and understanding how to set up a prediction market helps us learn more about people's beliefs.
  • From the perspective of creating transparency in an organization, prediction markets allow us to get to the nuances that we otherwise wouldn't have with just click data.

Firms Related to Prediction Markets


Initial Contributors:
Ming Chen: chenm18@stanford.edu
Daniel Cheng: ychengd@stanford.edu
Bo Cowgill bolc@stanford.edu
Jingxia Lin: jingxia@stanford.edu
Ashlee Miller: ashleem@stanford.edu
Harry Wang: huai@stanford.edu
Yi-Fu Wu: yifuwu@stanford.edu