In this beautifully written and delivered talk, computer game development company CEO Kevin Slavin tells an engrossing story about how algorithms affect our physical reality and minds. Surprisingly, we do not understand those algorithms well. We have written things that have big effects on us, yet we cannot read what we have written. Some of the highlights of the talk:
 |
| Flash Crash timeline (from New York Times, 10/01/2010). |
- 70% of the stock market relies on algorithmic trading, or programs competing mostly with each other. This trading probably includes your pension fund and your mortgage.
- Algorithmic trading is poorly understood - witness the 2010 Flash Crash, when 9% of the market value just disappeared (now that's efficient market for you).
- Some of this trading (including the so called high frequency trading) requires high data transfer speed, which in turn requires close proximity to the fiber optic cables. That's why many trading companies buy space close to one such distribution point near Carrier Hotel in New York. And that's why a new high speed cable is being build between Chicago and New York - all to decrease the time it takes to place a trade by a few microseconds. Interestingly, future data centers may be built in the ocean (probably, on platforms) to improve connection speed.
- Algorithms drive prices in online stores and auctions. Sometimes such programs executing without human supervision run amok: one book was priced on Amazon at $1.7 million, and in a few hours its price increased to $23.6 million. Needless to say, no one bought anything - these were just algorithms competing with each other on price.
- More than 60% of the film rentals on Netflix are done using the "recommended movies" feature. This algorithm is truly shaping our culture. Another company called Epagogix is providing "accurate predictive analysis of the Box Office value of individual film scripts, and [...] identifying and quantifying how and where to improve their commercial value" (as their webs site claims). Shaping the script before the movie is made? I can only imagine the impact of support vector machines on the movies by Andrei Tarkovsky or Bela Tarr - the algorithm would probably recommend reducing the length of cuts, include some more upbeat music, and spruce it up with a good love story...
His parting message is: whether we like it or not, algorithms, along with nature and humans, are the forces that shape the world.
So why do we keep collecting these huge amounts of data which are becoming illegible and we are losing meaning?
ReplyDeleteThey keep the data and run the algorithms because these algorithms produce on average something useful most of the time - for the people who are running them. The data probably has to be huge due to the nature of the algorithms.
ReplyDeleteKevin Slavin simply points to the many cases where algorithms produce something very unintended, either for the people running them, or, worse, for the rest of the world.