At some point on Tuesday there’s a decent chance you’ll read a story about how low cost bus service is reshaping intercity travel on the west coast of the United States. This is part of a concerted effort by new entrant FlixBus (already very successful in Europe) to drive demand for its services in this new market. The company commissioned a “study” and published a “report” outlining why travelers should spend more time on the bus. If whatever publication you’re reading does not cast MASSIVE suspicion on the data presented you should give up on that outlet and move on.
The 2018 West Coast Travel Index is an overview of the most convenient modes of transport, for travelers journeying across the West Coast of the United States, based on a number of selected factors, including Travel Cost, Travel Time, Convenience, Extra Services and Environmental Impact. The Index consists of 32 routes across the western coast of the United States, chosen based on popularity and importance.
The report is a dumpster fire of bad information and bad methodologies hiding under the guise of “data.” It is complete bullshit and should not be trusted.
The PR company pushing the report sent it out to many journalists, asking for a 9a Tuesday (no time zone given) embargo of the information. I never agreed to the embargo so I don’t feel bound by it. That said, I’m not going to publish the details in full. I can show just how awful the report is mostly by focusing on the methodology and skipping the underlying data itself.
Pick a route, any route
The report focuses on 32 routes up and down the west coast, stretching from Seattle to Tucson. For each city pair the company pulled information on what it would take to connect by plane, train, bus and rental car. Cost was a factor, of course. So was total travel time. And the company made sure to include the UberX ride from city center to the airport for those two score components. FlixBus also included a “convenience” score focused on:
- Total number of check-in touch points
- Assistance for disabled people
- Total number of payment options
- Hand luggage size/weight included
- Permitted amount of liquid in hand luggage (in ml)
The company considered “Extra Services” as well, including access to:
- Drinks and Food
- Oversize Luggage
Finally, the environmental impact was considered based on various greenhouse gas emissions per passenger.
So far, so good, right?
Alas, how quickly it all falls apart.
For the airline comparison only United, Delta and American are included. Southwest and Alaska, which each have a massive presence on the west coast, are excluded. American and Delta have far fewer options but are the basis for comparison. On a route like San Jose to Portland where only Alaska Airlines and Southwest Airlines operate nonstop excluding them presents a massive data bias. In this case it bumps the travel time up to 7:42 on a trip with a block time just under 2 hours. Excluding Southwest also affects the baggage option, though that might not matter as checked baggage allowance isn’t a criteria for any of the rankings.
Other missing bits
The bad data doesn’t stop there, unfortunately. Rental car costs are skewed by only searching one-way trips and neglecting to account for fuel or tolls. Frequency of service is irrelevant in the rankings; train service from Palm Springs is available only thrice weekly but it is considered just as viable as multiple daily flights. Tucson gets the same three trains per week and manages to beat planes or a rental car on most of its scores. Naturally the bus is even better.
So, skipping a couple of the obvious airline offerings is silly, but maybe that doesn’t affect the data too badly. After all, even with the horrible flight timing he San Jose-Portland route is still fastest by plane compared to the other three options. And screwing up the rental car costs might not be the end of the world. But the overall score calculation is a mess. It completely destroys any chance FlixBus had of making this report really matter.
Here’s what the company has to say about the final score calculation:
The results for each factor, for each individual route, across the four transport methods were scored from highest to lowest, with a maximum of 4 points for the best scoring mode of transport and a minimum score of zero, should the route not exist for that means of transport.
From there the totals are added up and averages computed. The additional math involved in computing those rankings is not particularly well explained, but there’s one key consideration: the factors are not weighted in the calculations.
It doesn’t matter if a bus ride is $0.10 or $110 cheaper than the next closest mode of transportation; the scoring is the same.
It doesn’t matter if the flight is 10 minutes or 32 hours faster than the other options; the scoring is the same.
The number of payment methods available matters just as much as the liquids permitted in hand luggage. And the environmental impact matters more than either of those because it is a separate, top level category all on its own.
Of course, it should come as no surprise that these decisions were made in the scoring. They ensure that the bus option wins. EVERY TIME.
So what that Seattle to Phoenix is 37+ hours on the bus? That ride is $20 cheaper than the flight and you can bring liquids in your carry-on bag so it must be the better choice. Or San Francisco to Seattle. You can spend 22 hours on a bus or make the trip in 6 hours by plane (which is high given the ~2:20ish block time). And there are more than 20 flights daily to make that trip. Surely the $75 fare difference there helps to cover some of that additional travel time. Ditto the $45 fare savings to spend 28 hours on a bus from Las Vegas to Portland rather than fly nonstop (or even connect).
Extinguishing the dumpster fire
There are good examples where bus service makes sense. Trips in the 4-6 hour range (e.g. Las Vegas to Southern California or Phoenix, though not with the 9ish hour ride time currently scheduled) can often be compelling by means other than flying. And growing options in the transportation world should generally be celebrated, especially when they lower prices and bring more convenience to the market. But trying to pass off some ridiculously biased marketing report as an authoritative “travel index” is all sorts of bad. Especially when the company cannot be honest about the fact that there are many, many, many times it is the wrong choice.
Header image: Dumpster fire via Flickr/CC BY-SA
Never miss another post: Sign up for email alerts and get only the content you want direct to your inbox.