If you’ve worked in marketing, advertising or any related field for more than about 72 hours you’ll have been part of at least one conversation where the phrase “return on investment” was used. Without belaboring the obvious, ROI is Amount Spent On Marketing MINUS Value of Action or Acquisition. If you spend $100 on an ad campaign and can track $1,000 in new or incremental spending to that campaign, your ROI is $900.

I’ve been thinking about ROI a lot in light of the news that broke a week or so ago after a set of numbers was leaked by someone of how Amazon tracks Prime Video subscriber value. Keep in mind these numbers did not come *from* Amazon and the company has refused to comment on their veracity. For the sake of this discussion I’m going to assume they’re at least directionally accurate.

Putting aside how Amazon uses Prime Video as a foot in the door for new members who it hopes will then use Prime to buy other goods, the leaked documents offer insights into what standards it uses to gauge the success of its original programming. Quite simply, a show’s value is dependent on the number of new Prime subscribers it attracts. Some of the math is a little fuzzy, but using the example offered in the Reuters article, “The Main in The High Castle,” here’s how it breaks down:

  1. Production and Marketing Cost: $72,000,000
  2. New Subscribers: 1,150,000
  3. Cost Per Subscriber: $63

Just using that last number – which is a perfect example of the kind of black-box calculation the leaked documents don’t offer insight into – and putting it against the $99 yearly Prime subscription fee, the ROI for “The Man in The High Castle” was $36. That’s the delta between the cost and the value per customer acquisition.

Amazon reportedly uses its proprietary formula solely on the first show a new subscriber watched after joining. That’s how it decides, at least in part, which shows to renew and which to let drop.

That’s a major flaw in the equation and one that is problematic given the growth of original programming on streaming services. Here are the questions that speak to the issues and problems I have:

  • Understanding that the line has to be drawn somewhere, how much should the longer-term value of a new subscriber be attributed to the show? If “Marvelous” brought in fewer new subscribers but they spent more in aggregate over the next six months on other goods offered by Amazon, shouldn’t that be a factor?
  • “First show watched” is useful for gauging how effective a marketing campaign is, but how much is the size of the campaign accounted for in the calculation? The first season of “High Castle,” based on what I’ve seen in the trade press, received a much larger marketing push than “Revolt,” so how does that math work? $10 spent to bring in $20 has the same ROI *ratio* as spending $1 to bring in $2, even if the dollar figures are the drastically different.
  • How much (if at all) are “other shows watched” given credit for retention if not acquisition? “High Castle” might have brought in scads of new subscribers, but has that audience stuck around in part because of other programming?

We might never know the anwsers to these and other questions, but they’re worth asking. Amazon, like Netflix, doesn’t offer viewership numbers willingly because it doesn’t have to. In part that’s because these shows aren’t ad-supported (though Amazon’s business model makes that statement iffy at best) and so don’t need to show anyone what it is they’re buying into.

If those questions aren’t being addressed internally within Amazon as part of its decision-making process, then those decisions are being made largely in a vacuum. At least they’re being made without considering a sufficiently broad range of data points. It leaves the fate of shows that have more niche appeal – those that are never going to draw more than small audience – in the balance as the company hopes a $500 million bet on a “Lord of the Rings” show pays off. That has the security executives like being based on an existing property and having a largely white, male audience.

Are those viewers going to do anything else, though? Will they re-up with Prime after they finish watching that one show? Or will they cancel as soon as they’re finished binging, offering no subsequent value to the company as a whole?

It’s not clear – at least not based on this one report – anyone’s actually asking those questions while they focus on the only the hook, not the rest of the line they have in the water.

Chris Thilk is a freelance writer and content strategist who lives in the Chicago suburbs.