Bid Caching and Commoditized Exchanges

Index was recently caught using bid caching. This caused a stir in the ad tech world and is among the sort of practices that, when disclosed, undermines the trust between buyers and sellers. In this Lift Letter, we discuss what bid caching is, what Index's response was, how this plays itself out in the display exchange ecosystem, and how this is the sort of thing that led to TripleLift creating HeaderDirect.


There are two important and different concepts when thinking about bid caching. Bid caching, naturally, is one. This is when a DSP bids into an exchange for impression 123 on site for user ABC at time T and the DSP loses this auction to some other DSP. In bid caching, the exchange will then use the losing DSP's bid for an impression that doesn't quite match all of the characteristics as the original impression. Generally it will be a different impression ID at a slightly later time, perhaps on a different site (e.g. instead of the homepage it could be on an article page). DSPs that are used to submitting a response for each individual ad impression are rightfully indignant that their bids would be applied to anything but the exact impression that they bid on.

Ad caching is the second concept in the context of bid caching. Ad caching is when an auction is executed and completed but not immediately delivered - instead it is stored in the browser (or app) and delivered later. This is a valid tactic to pre-load midroll video ads or mobile interstitials, for example, so there is no delay in their delivery. For a cached ad, only one DSP is the winner and no other bids from that impression are reused later. One of the more important concepts in delivering ads is session depth - this means how far into the user starting on the webpage or overall browsing journey is that particular user. Generally the earlier they are into their session the more likely they will be to click on an ad, so earlier in the session is better. With a cached ad, bid requests are still sent in an order, and this order can be used by the DSPs to reflect actual or potential session depth, so even if the ads are delivered out of order, there is only one "first in session" bid request and that ad - cached or otherwise - and all the bids for it, are only used once. In a cached bid concept, the bids for the "first in session" ad can be applied serially to ads much later in that user's session - meaning a bid for a first ad could be applied to a twentieth impression.

Index Exchange published a blog post where they disclosed the practice of bid caching (of course, only after being caught). They point to Section 7.2 of the OpenRTB 2.5 spec as justification for bid caching. Notably, this section is exclusively about ad caching, not bid caching. And thus it becomes clear that because their blog post on the behavior does not, in fact, address the behavior in question (bid caching), but rather a different and more acceptable behavior (ad caching) - that this was unacceptable. It is not unreasonable to expect to see DSPs divert open market spend away from Index as a result, albeit likely only for a short period of time.

Many other banner ad exchanges have taken this moment to cast disapproving glances at Index, including, hypocritically, those very same exchanges that previously were caught taking buy-side fees and doing shenanigans (the authors of prebid, for example, previously discussed implementing bid caching). This all should be expected. Header bidding has created a world where banner ad exchanges compete for the same impressions and have no particular advantage over their competition. It's hard to understand why any banner exchange would have unique demand (why, after all, would some buyer only want to buy through a specific channel as opposed to all the channels, and picking the best parts of each). As a result, banner exchanges have moved incrementally to higher clearing prices to win more auctions - moving ever closer to first price while dropping their margins. The newest trend is estimated market rate (EMR), which Rubicon Project has publicly discussed, where the auction is effectively first price minus some amount that the SSP calculates. The goal is maintain a high win rate but reduce the amount that the DSP needs to pay to win that particular impression based on imputed bid density - with the notion that Rubicon (or whoever is running the impression) has better knowledge about the particulars of the impression and the expected bid density than the DSP. Yet this also opens the door for a divergence in what the DSP thinks it has paid to what the publisher thinks it should receive (meaning variable margins or buy-side fees could once again be reintroduced). Indeed, perhaps coincidentally, Rubicon's margin has been increasing as EMR's uptake has been increasing.

Ultimately, for display exchanges, bid caching, buy-side fees, and whatever the next scandal is - they are all predictable. Exchanges and SSPs make money when a buyer / DSP wins an impression on that exchange. In header bidding, if every DSP sees the same auction from multiple sources, SSPs have every incentive to keep looking for auction dynamics to improve their win rate, especially if, when they're caught, the punishment is nominal if at all. Thus the incentive of the DSP becomes clear - circumvent the SSP wherever possible. If SSPs have incentives that are completely contrary to the DSP and have a history of acting against the DSP's interest, then that's the natural response. With header bidding, at least some of the larger DSPs may move to integrate directly into the wrapper, perhaps but for the need of having a publisher development and relationships team. At least some DSPs would like the header relationships but have staked their reputation on to being conflicted. The Trade Desk, Adobe and MediaMath, for example, don't have publisher relationships, don't have their own SSPs, and don't own media, etc - so they can present itself as completely neutral (contrasting with Google, AppNexus, Facebook, Criteo and others).

This is among the primary reasons for how Header Direct (TripleLift's low-cost, first-price display exchange) came to be. With first price auctions that are auditable in the header bidding wrapper, there is no room for pricing shenanigans. Further, if a DSP were to have a direct relationship with the publisher's wrapper, it would also be a first price exchange. At 5% margins, this is roughly the cost of having White Ops filter all impressions for fraud, pay for tech and bandwidth, and have a publisher team - with a tiny profit. If the world is moving to publishers and DSPs having a direct relationship for banner ads - a market that we don't particularly care about - we might as well facilitate that eventuality and make at least a couple dollars from it.