The Open Source Economics That Caused Heartbleed and How to Prevent it From Happening Again

The Heartbleed bug has seemingly shone a light on the dark side of our open source architecture. The benefits of open source have long been known: it's free and you have a large community of users that will continuously improve the software. It's a naturally occurring collaborative arrangement that seems to beat the other corporate options out there. 

The downsides have previously been less publicized, but have always been there. Many large corporations have moved to open source, but there is a reason why some stick to commercial solutions. I work for a company that creates commercial data analytics software, and while I am a huge proponent of open source alternatives, I see why clients turn to us rather than open source alternatives like R, Python, or Gretl. If you're using an open source option, answers are almost always a google or openstack search away. If you're using a commercial option like my company's, you'll be able to get an expert on the phone who can show you what to do (that expert would probably be me.) At its core, the main service my company provides when compared with open source options is our company assumes responsibility for the software we produce.

This dynamic is why it took two years to notice Heartbleed. There was always an active development community surrounding it that in many ways is more dynamic than any commercial community can be, but no one ultimately can be held responsible if things go wrong. Cyber security is something that users won't notice unless it fails. Open source dynamics do many things right, but this is not one of them.

In my industry there is also the beginnings of a solution to this problem. Companies such as Revolution Analytics and Continuum Analytics have emerged as the commercial face for open source R and Python respectively. The underlying architecture is free, but companies like these are able to add consulting services or custom addins to open source software.

The dream of open source was that users will be actively maintaining the environment. While this has come to fruition in terms of upgrading user centric functionality, there are some holes, and ultimately no responsibility. This evolution in open source economics allows us to have it both ways. We can get large open source communities, but also have pay options available for those who need it. The providers of pay options can begin taking responsibility for software, and care about it in the same way commercial providers do. Large open source userbases provide the externality of a well maintained infrastructure that these consulting companies can take advantage of. Consulting companies, worried that their paying clients wouldn't trust the software if it had security and other non-user centric bugs that would never be noticed by volunteer communities will work to fix them, providing an externality to the free user community.

Much has already been written about how we need to pay people to solve security issues. Grants might be feasible in the short term, but the industry arrangement I describe above came about fairly naturally in data analytics. I don't see why something like this can't be encouraged elsewhere.

Read more ...

Uber Wants to Replace Your Local Department of Transportation

If you were unsure about what Uber's long term goals are, their April Fool's joke makes things crystal clear. Along with surge pricing, Uber is testing long held consumer assumptions about transportation.

It's not acknowledged as such, but there are truly limited choices when it comes to transportation. You have your selection among available cars, but car ownership itself is a bundled package. You're paying a hefty premium to be able to use your car whenever you want. On the other side of the spectrum, you have the option of a completely fixed public transit system. For a very low price, you're able to get from fixed points on a preset schedule. Compared to your other options you're getting a bargain, but you are constrained by the lines and timetables on your transit map. Cabs have always been in the mix as well, but are closer to cars on this spectrum: you're paying a premium to choose your pickup and dropoff location on your schedule.

I previously thought it would take a widescale adoption of driverless cars to bring about this sort of change. Uber is trying to break long standing consumer expectations and make people truly think about what they want when it comes to transportation. Looking at private cars and public transit as two sides of a spectrum, Uber is trying to give consumers a choice of any point between them. You can pay through the nose for a cab to pick you up right outside the bar now. Paying $2.50 is great, but you may not want to wait 45 minutes for a subway. I'm sure there are lots of people out there who would be willing to pay something in the middle to walk a block and wait 8 minutes for the next shared cab to come. Especially when you know exactly where it is on your phone.

This system is not even novel. In New York, several major outer borough streets leading to express subway stations have informal rush hour cab share systems. Over time, cabbies have begun picking up people at bus stops, and usually charge around $2 per person. Someone waiting for the bus in the morning can choose to pay $2 more than they normally would to get a faster ride now rather than a slower trip 8 minutes from now to the subway station.

When Uber builds up its fleet of cars and institutes this sort of pricing en masse, this April Fool's joke will look hilarious in hindsight.

Read more ...

Surge Pricing, Markets, and Expectation in the Taxi Industry

Recently Uber's practice of "surge pricing" during storms or other periods of increased demand. On one level, this is a fairly standard story of a more efficient market beginning to form in a highly regulated and currently distorted market.

But, I have to admit, seeing receipts like these just lead to a visceral reaction that you just can't control if you're a New Yorker. Cab rides just SHOULDN'T cost that much!

This uncovers a few more layers of what's going on. There aren't just entrenched interests that want to keep the medallion system in place, but deeply entrenched expectations among consumers.

First, consumers have an aversion to what's perceived as unfair price gauging. Gothamist asks the question in this way:

But when does surge pricing become price gouging?
Basically, there is no discernible difference.
"The best way to look at it is during Hurricane Sandy," says one financial expert. "You had these long lines because there was a limited amount of gas. But if you had surge pricing, gas could have been raised to $10 a gallon and there would have been less people on the line."

Raising gas prices during hurricane Sandy has to be the most egregious example of gauging. But, it's still true that if gas prices could have been raised to $10 a gallon there would have been no lines. Those who absolutely needed gas would have been able to get it without an issue, and those who could have done without would be able to wait for prices to come down and find other ways to get around.

This begs the question of what is a fair way to ration things during situations like this. Is it ethically more desirable to give gas only to those willing to put in the time to wait in line? 

Sandy was a disaster however, surge pricing happens more regularly. If Uber continues its growth and eventual domination of car services, it won't take long for things to find a new equilibrium.

Lets say you're a potential UberX driver, and you see how this surge pricing happens. As a driver who has control over their schedule, you begin taking note of when these surge prices occur, and will begin to act accordingly. Surge pricing will train drivers to allocate themselves when their needed, and eventually the surges will be minimal.

There are still ethical questions that need to be answered. This still won't solve the problem of ensuring universal access of cabs for example, as the introduction of metered green outer borough cabs have. Even in a perfect Uber world, cabs will still be incentivized to cluster in downtowns.

On any side of a transaction though it's always healthy to try to get rid of a sense of entitlement. Medallion owners have no legitimate reason to keep their system in place, and consumers don't have an intrinsic right to a cheap cab ride regardless of the weather. An Uber like system will allow these unsustainable expectations to come to light, and end up in an overall better system.

Read more ...

Premise and The Hurdles Big Data Will Face With Economics

Premise is a fascinating new company that aims to disrupt how macroeconomic data is collected. Individual companies are using every conceivable aspect of consumer data to predict their own sales. Why can't this same insight be used to measure the economy and revolutionize economic indicators?

There is a reason Big Data started out in the collection and analysis of the data within individual companies. A huge component of the value of information is access. Within a company, this variable is taken out of the equation. All the data is proprietary to the company and everyone within the company is working towards a common goal.

The value of information about the economy depends on the information everyone else has, and this value expresses itself in a few unique ways.

The first and relatively simple way this expresses itself is the value of exclusive information. It's clear to anyone working on Wall Street how valuable having an information edge is. This drives researchers to search for more and more esoteric datasets to try to tease out a relationship no one has ever thought of. As a relationship becomes more known, its value quickly drops to zero as stock prices or any other market being watched already has this information baked into it.

There is a related converse dynamic to this value of uniqueness that comes into play once data becomes widely known. People use information to make decisions. Once information is "out there," its value increases the more people use it. This is because multiple actors are all making decisions that affect each other.

Think of a specific industry as an example. If you're an automotive company and have information that no other automotive company has access to, you can use that as an information edge. This edge pretty much disappears the minute one other company learns of it.

At this point the second dynamic takes precedence. If 10% of the industry is using a certain piece of information, there's not too much of an incentive to use that information. On the other hand, if 95% of an industry is using a certain piece of information, the 5% who aren't are at a distinct disadvantage, and if you're a new entrant or analyzing the industry from the outside, you're going to want this information too.

Economic data responds to these dynamics in some weird ways. Headline indicators are venerated as describing the pulse of the economy, and maintain their momentum through their own heavy use. GDP is a deeply flawed measure of the economy. There are some alternatives out there, and there is more and more data every day that will allow us to create better measures of the economy, but GDP is still the agreed upon standard and continues to be because of inertia.

A company like Premise can take advantage of these dynamics in two ways. The first is to simply create indicators that compete with the standard indices we now have available. If Premise creates a few "rockstar" economic indicators, it will have a stable business as long as there exists an economy to be measured.

This isn't truly unleashing the power of big data into the economy. In my opinion this would be an upsetting outcome: using all of that power to simply create a new static status quo in how we measure the economy.

I see true success coming in a second path: the creation of a platform where anyone can create their own indicators. Instead of using a headline CPI number that is created using certain statistical methods from the Bureau of Labor Statistics, these methods would be available to anyone using the platform.

With this platform in place, the dynamics that now allow certain data to become stodgy and out of touch will start leading towards innovation. All the platform will have to do is link to already existing economic data.

How will this process play out? A headline indicator like CPI has a gravitational pull that makes it a benchmark of the economy. However, if a user is able to make a few tweaks to that CPI data using its atomized components, they will gain the benefits of BOTH dynamics. Their measurement is close enough to CPI to benefit from its wide use, but different enough to create an informational edge. Instead of crystallizing into a series of discrete indicators that never change despite the changing economy, the dynamics will now be incentivizing the discovery of new and unique indicators.

Our understanding of the economy as a whole will be revolutionized, and we'll all be making smarter decisions. People will be encouraged to have a deeper understanding of the economy, instead of relying on measures of the economy that are used simply because everyone uses them.

Read more ...

Apple and the End of Technological Progress

Hyperbolic title, but as I'm just starting to read Average is Over by Tyler Cowen, I couldn't help but think about what sort of limits there are to some of the technologies Cowen predicts in his new book.

All of the smart technologies we can hopefully expect depend on interconnectivity, and any sort of technology that depends on this sort of interconnectivity faces the same monopolistic temptations as any other ambitious company. Every company in a competitive market, at least on some level, has the goal of destroying its competition.

If you have a traditional monopoly, you have traditional consequences. A utility company with a natural monopoly doesn't face the same price pressures that a competitive market does, and won't innovate as fast as if it were in a competitive market. for these traditional monopolies, we also have traditional solutions: governments come in to regulate prices. In terms of innovation, I'd argue that natural monopolies are also industries in which innovation has a lower value. Think about how much improvement one can reasonably expect in a city's water infrastructure (a contrary example to his might be the fact that we don't have true high speed rail in the United States yet.)

Things are different in the tech sector. The temptation to create "walled gardens" and closed app ecosystems could be compared to the natural utility monopolies. Prices have completely different meanings in the tech sector and discussing that would probably warrant its own post, but there is a very clear link between the creation of these walled gardens and innovation.

Consider Apple. Apple makes several products that they sell directly to consumer, and maintain an ecosystem surrounding these products. They don't charge directly for the ecosystem, but the ecosystem adds value to Apple's products, and therefore Apple is incentivized to make this ecosystem as flawless as possible, in order to make their products more valuable. This is a positive externality: assuming the price of the products remain the same, consumers get the benefits of the ecosystem for free. Of course, Apple recaptures some of this value by selling its products at a premium.

Part of this value is a "stickiness." Consumers begin to depend on the ecosystem. The longer a consumer remains in the ecosystem, and the more consumers are in the ecosystem, the harder it is for that consumer to leave the ecosystem. As an ecosystem becomes more successful, the company faces fewer incentives to improve it. This is a classic negative feedback loop, and acts as a drag on innovation. The ecosystem sows the seeds of its own destruction.

Google addresses this problem as a company. It realizes it is in its own best interest to continue to innovate, and while a walled ecosystem will lead to monopoly power, it cannot afford to stultify its own evolution. This is why Android is open, why Google lets you (relatively) easily get your data out of their services. It's far from perfect, but at least it's a recognition that this is an issue, rather than the blatant ignorance of the issue by a company like Apple.

There are unfortunately more Apples out there than Googles. At this point in our economic development, the benefits that Google is foreseeing are too long term for a vast majority of companies. Even Google is only to a degree paying lip service to this concept rather than fully embracing the concept (cough Google+.) As far as I'm aware, there does not exist a way to properly align incentives within our economy to avoid this drag on innovation.

Read more ...

Why You Will Never Own a Driverless Car

This Atlantic piece talks about the ramifications of a new innovation in electric cars by BMW. Rather than purely technological, BMW is simply packaging zipcar-like access to a conventional SUV for several weeks a year. It correctly gets to the heart of why people are leery of electric cars:

This consumer peccadillo has driven alternative transportation advocates bonkers for years. So, why do people buy 7-passenger SUVs with four-wheel drive in California when they usually carry a single passenger in 70 degree weather on a highway? 

People buy cars for their peak (imagined) need. If you can imagine that one day you'll drive more than a handful of people to Lake Tahoe to go skiing, then (if you can afford it) you might choose a massive sport-utility vehicle.

This concept is short changed by only being applied to electric vehicles. Every time I have a conversation about driverless cars, I'm surprised by how many people apply today's car ownership paradigm to them. Driverless cars are certainly a huge technological advancement, but the biggest shift they will cause will be in how they are packaged as a service.

You may want to own a driverless car, but what if instead you had access to a service that guides a public driverless car to where you're standing within 5 minutes? What if you could have a cheaper service that would come within 15 minutes, or had to be scheduled in advance? Or another service that would pick someone else up on the way? Or another service that will drive you to a predetermined drop off point?

It's easy to see that driverless cars allow a seamless gradient to exist between public transit and private cars. With only two options in this spectrum available today, people drive only to have access to their car when they really need it. More people's needs will be met by these hybrid options, and more people will be swayed away from the hassles and costs of car ownership.

When access instead of ownership is the focus, and more options are available to people, more people will shift to being served at least partially by pure public transit as well. If you formerly drove from a suburban location to a downtown office for example, it would be cheaper and more efficient to take a driverless car to the fixed point of a commuter rail hub rather than taking a car all the way downtown.

This program started by BMW is only the start of these custom hybrid packages, and proof that we're ready for this paradigm shift.

Read more ...

Scientific Proof That Gentrification Will Hit Brownsville But Not Canarsie

Stephen Smith, writing in onion-esque form at the Observer, believes that gentrification in Brooklyn has finally reached its end game, and is calling Brownsville the last frontier (in Brooklyn at least.)

Well, if Brownsville truly is the end of the line, we should be able to compare interest in it to interest in Bushwick a few years ago. Those first steps: those pioneering hipster adventures out into wild urban frontiers, fueled by a burning question- "what's ACTUALLY out there?"- and a few PBRs, predate actual settlement. Lets check the Google trends:

This graph tells a very clear story! Bushwick has a very constant, if slightly increasing, rate of interest over the past 10 years. Which would make sense given what's happening on the ground. Until 2010 however, interest in Brownsville seems to remain flat. Isabella and Ferdinand holding court at Roberta's didn't authorize exploration until 2010, when it seems the fixie Nina, Pinta, and Santa Maria finally left for uncharted territory.

But Brownsville is over! It's obviously already been discovered. Lets compare this to the even more distant lands of East New York and Canarsie:

I'm not sure why there's such high variance screwing with the scale, but it seems like East New York and Canarsie are at least 3 years away from becoming as discovered as Brownsville is today. These are still magical lands, marked on maps as "there be dragons here."

The more interesting point in the article involves the South and East Asian immigrants filtering into these far flung neighborhood. The neighborhood I grew up in in the East Bronx, destined to forever be a boring backwater, is going through similar ethnic change. Ever since white flight in the 70s, city dynamics have been viewed as what's happening with white people vis-a-vis "undesirable" minorities. First the black people moved in and scared off the whites, and now the white people are displacing the blacks and hispanics. There are serious problems that are happening on both sides of this process, but focusing on just one process ignores everything else that's going on. Can you imagine hearing stories about what neighborhood the Asians are going to move to next? Flushing is so over, it's all about Bensonhurst now.

I think it's a good thing that we're not hearing things like this. Ethnic diversity is spreading through New York.  Recognize the negatives of gentrification and try to fight against them, but change your focus. Instead of focusing on how young white hipsters are expanding their sphere of influence in the city, focus on the more general trend that young people are more comfortable with living with people who are different than themselves, our formerly balkanized neighborhoods are starting to become less so, and this underlying trend has both positives and negative effects.

Read more ...