Bug: Virtual Gateway Request Timeouts are misconfigured

[rishijatia]

Summary
When hostname rewrite is DISABLED on a Gateway Route resource, the corresponding route timeouts are not applied to the virtual gateway layer. This might cause high latency requests that would ordinarily succeed when timeouts are specified to fail when hostname rewrites are disabled.

Steps to Reproduce

• Set Route and Virtual Node Listener request timeouts
• High latency requests should succeed
• Configure rewrites and disable hostname rewrites on one of the gateway routes
• All high latency requests going through the Gateway Route will now fail

Route Request timeouts won't be applied to the gateway route anymore.

Are you currently working around this issue?

NA

Additional context

We are working on a fix for this and will keep this issue updated with our findings.

Attachments
If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)

[capraqua]

I have similar problem with 15s timeout.

Steps to Reproduce

• Setup virtual service A pointing to virtual node A (http with timeout 30s)
• Setup virtual service B pointing to virtual node B (http with timeout 30s)
• Service A calls service B always timeout at 15s, service B access log shows code 0 and service A access log shows code 504.

[rajal-amzn]

@capraqua Based on the steps mentioned, it looks like the timeouts are set only at the Routes. We need to set Routes at both Virtual Node and Routes. Please take a look at this troubleshooting guide

[visit1985]

@shwetasahuit Looks like this is fixed since yesterday. My envoys get dynamic routes from the management service for the self redirect to 15001, with timeouts properly configured for each GatewayRoute. Can you confirm that?

[ashanpw]

Awesome. We have fixed the issue and the change is deploying to all AWS regions. Depending on which region you're using, the change might already be there.

[shsahu]

@visit1985, yes we are rolling out the fix. Could you tell us the region where yours meshes are deployed. We can confirm if the fix is available there.

[visit1985]

@shwetasahuit eu-central-1

[ashanpw]

@visit1985, confirmed that the change has deployed to eu-central-1

[jslobotski]

We're in us-east-1 and experiencing a similar issue. We don't use virtual routers so we're setting the timeout in the virtual node to something higher than 15 seconds which appears to work for app to app communications but anything coming in through the virtual gateway still times out at 15 seconds regardless.

Has this fix been deployed to us-east-1? Just curious if this will fix our scenario as well.

[shsahu]

The timeouts at virtual gateways are calculated based on the timeout set at the routes of virtual routers. So in your scenarios, you need to define virtual router and routes with timeout to make it work.

[jslobotski]

The timeouts at virtual gateways are calculated based on the timeout set at the routes of virtual routers. So in your scenarios, you need to define virtual router and routes with timeout to make it work.

Thanks! Confirmed that worked for us, much appreciated.

[rajal-amzn]

Closing this as the fix is deployed in all regions.

[Richardmbs12]

Is there a way to do this if my virtual router is of type tcp?

TCP virtual routers cant set request timeouts, just idle connection timeout.

So I am experiencing the 15s timeout without a way to change this on my virtual gateway.