PBS2.0: bidders with dynamic hosts

[bretg]

This issue tracks one of the rules that will be enforced starting with Prebid Server 2.0: bidders must not have fully dynamic domain names. This rule is in place because it's a performance and security risk to allow Prebid Servers to open to connecting to any endpoint on the internet.

Fully Dynamic Domain Names

These are the bidders that strictly require an update for PBS 2.0... These 3 adapters will not be ported until the endpoint is at least only partially dynamic. We request this is done ASAP rather than waiting for September.

• adocean - @mmuras
• datablocks - @htang555
• tappx - @prebidtappx

Partially dynamic hostnames

As discussed in #2606, even partially dynamic host names are a potential performance and security risk for Prebid Server host companies.

We're not going to enforce this rule in September, but it may be next unless one of the entities here makes a compelling case. Quoted from that issue:

We do not believe dynamic sub-domains are required any scenario at all. In every case we can think of, having a dynamic component in the query string should suffice.

Let's take an explicit use case and guess that your requirement is to run an entirely separate cluster of servers for each customer, or perhaps clusters of customers. This seems like an extreme requirement, but might be useful for service-level reasons.

The obvious solution is to add a query string parameter, and use a load balancer to inspect the query string and send the request to the appropriate back-end cluster. Almost all modern load balancers allow inspection of query string parameters.

Here's the list of bidders that we want to engage in discussion with on this issue -- please either change your bid adapters, or post into this issue a compelling reason why dynamic subdomains are required.

• acuityads - @AcuityAdsIntegrations
• adhese - @sander-adhese
• adoppler - @vchimishuk
• algorix - @Bugxyb
• between - @digitalist , @egsk
• bidmachine - @meeeeeepo
• iionads - @imedvedko
• imds - @ecammit
• invibes - @invibes
• kayzen - @raghuteja
• limelightDigital - @imedvedko
• silvermob - @silvermob
• smarthub - @SmartHubSolutions (initiated Consider configurable subdomains #2606 !)
• smartyads - @SmartyAdsSSP
• taboola - @thetalkeren , @ahmadlob
• trafficgate - @vlaktion
• yeahmobi - @zhaojp
• zeroclickfraud - @htang555

Hostnames with regional elements

Also not a rule we're going to enforce in Sept, but found a potential performance issue for certain bidders.

While looking through all the domain names, we found 11 bidders whose endpoints are specific to a region, mostly the US. This isn't ideal for anyone. We're going to create a new page where the following bidders can, if desired, let PBS Host Companies know about other endpoints.

• aceex - @supportAceex
• apacdex - @thuyhq
• axonix - @cesarfd
• bizzclick - @BizzClick
• blue - @samuelfabel
• brightroll - ?
• logan - @WlsLogan
• rubicon - me
• smartrtb - @evanmsmrtb
• visiblemeasures - @VisibleMeasuresHB

[ecammit]

In both PBS and PBJS, the imds adapter uses a subdomain prefix of the customer originating the request. This is primarily as a shear-pin mechanism to be able to shed excessive traffic from a misconfigured customer (usually from improperly implemented QPS limits). DNS records for any misbehaving customer can be removed in order to halt a traffic flood while maintaining existing capacity and allowing other customers to be unaffected.

For PBS, the traffic is from a customer server, so it only originates from a limited set of IP addresses. Therefore a firewall can likely drop the traffic from a customer's IP addresses to manage a flood. While it's still possible that the traffic could overwhelm a firewall, it's much less likely.

-> So we're okay with having a single hostname for the imds adapter for PBS. However, if this decision is to be extended to PBJS as well, I think much more discussion is needed.

Why is this an issue for PBJS? PBJS traffic comes from end-users, so firewall rules cannot drop the traffic. Inspection of the URL or query parameter to determine the customer would be the only way to drop that customer's traffic (as mentioned). However, since it's end-user browser traffic, it requires HTTPS, and can only be filtered by whatever is terminating SSL.

Terminating SSL is CPU-heavy and sometimes the application servers handle termination because they're cheaper than a load balancer. Load balancers can also be a challenge to dynamically scale. ex. It can take more time to bring a new one online if it has new public IPs that have to propagate in DNS. Regardless, all that excess traffic saturates the capacity of whatever is terminating SSL, which can overwhelm it. Allowing dynamic DNS hostnames per customer can help prevent this by stopping traffic at the source. Otherwise you have to do an emergency scaling of your infrastructure to handle excess traffic that you shouldn't be receiving anyhow -- at least until you hit the limit of how far your infrastructure can scale -- and pray that the misconfigured customer is answering their phone.

[bretg]

Thanks for the explanation @ecammit . Any input from other adapter maintainers?

[mmuras]

Hi, @bretg, thanks for the information, we'll discuss the issue this week and I'll let you know about the results at the end of the week.

[rcheptanariu]

Hello @bretg
The invibes adapter uses subdomain prefix to direct traffic to our servers based on customer config for geolocation - hosting specific to countries or zones. We'd rather not send traffic originating in Africa to our ad servers in Europe, or a load balancer located there.
If you prefer, we could modify the adapter to have hardcoded a list of 15 subdomains that we will be using for the foreseeable future, instead of building the domains dynamically.
Is this acceptable?
Thanks

[bretg]

@rcheptanariu - our view is that geo-balancing should be solved in one of two ways:

1. Pay a Global Services Load Balancing vendor.
2. Publish a list of URL endpoints you have and ask PBS host companies to send traffic from their source datacenters to the most relevant of your datacenters.

It doesn't make that much sense, in our view, to tie a given publisher to a region because that web site's users could come from anywhere.

Are we missing something? Why wouldn't a GSLB vendor work for you?
That said, having a configurable list of allowable sub-domains might be a reasonable compromise.

[bretg]

@mmuras and @htang555 - gentle reminder. I also sent email to the maintainer contacts for Ad Ocean and Datablocks.

[bretg]

Closing this out. There will be documentation for bidders on best practices for dealing with regions.

[bretg]

For the record, a summary of the endpoind domain rules:

• They cannot be fully dynamic
• If the dynamic part is geographic, a regional 'host' parameter cannot be required. It may be optional. The vastly better solution to regional endpoints is that host companies deploy different static endpoints in their various datacenters. It is impractical to force publishers to make request parameters choose a relevant region specific to each bidder.
• Host companies are advised to watch for issues related to abuse of dynamic domains and be ready to disable that adapter if needed