(@aws-cdk/integ-tests-alpha): Verifying snapshot doesn't read values from cdk.context.json

[sfriedlizuehlke]

Describe the bug

When running an integration test with integ-runner the verification of the snapshot fails if the integration test contains a VPC.fromLookup(...). The snapshot verification show a difference in the VpcId with the new value being the default VpcID dummy value of vpc-12345.

Expected Behavior

The integ-runner would read the VpcId from the cdk.context.json.

Current Behavior

#> integ-runner integ.bug.ts

Verifying integration test snapshots...

  CHANGED    integ.bug 5.097s
      Resources
[~] AWS::EC2::SecurityGroup sgxxxxxxx replace
 └─ [~] VpcId (requires replacement)
     ├─ [-] vpc-xxxxxxxxxxxxx
     └─ [+] vpc-12345



Snapshot Results: 

Tests:    1 failed, 1 total

As shown in the output above, the VpcId falls back to the default value of vpc-12345.

Reproduction Steps

Define the following integration test:

1. run with integ-runner --update-on-failed -> this will create the cdk.context.json with the lookup values.
2. run with integ-runner

import * as cdk from 'aws-cdk-lib';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import * as integ from '@aws-cdk/integ-tests-alpha';

const REGION = 'eu-central-2';
const ACCOUNT = 'SOME_ACCOUNT_ID';
const VPC_ID = 'some existing vpc id';

const app = new cdk.App({
  context: {
    '@aws-cdk/core:bootstrapQualifier': '<SOME_BOOTSTRAP_QUALIFIER_HERE>',
  },
});

const stack = new cdk.Stack(app, 'integBug', {
  env: {
    account: ACCOUNT,
    region: REGION,
  },
});

const vpc = ec2.Vpc.fromLookup(stack, 'vpc', {
  vpcId: VPC_ID,
});

new ec2.SecurityGroup(stack, 'sg', {
  vpc: vpc,
});

new integ.IntegTest(app, 'integTest', {
  testCases: [stack],
});

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.148.1 (build 283525d)

Framework Version

No response

Node.js Version

v20.15.0

OS

Linux

Language

TypeScript

Language Version

Typescript (5.5.4)

Other information

No response

[ashishdhingra]

@sfriedlizuehlke Good morning. Thanks for reporting the issue. I was able to reproduce it when my default bootstrapped environment had a different qualifier than the one set here:

const app = new cdk.App({
  context: {
    '@aws-cdk/core:bootstrapQualifier': '<SOME_BOOTSTRAP_QUALIFIER_HERE>',
  },
});

Could you please check:

• If the qualifier in your bootstrapped environment is same as the one you used in above code for that region (default qualifier is hnb659fds per Use the CDK CLI to customize bootstrapping).
• If the qualifier is different, then please
  IMPORTANT: Please test re-bootstrapping process in non-prod environment first. I don't think CDK allows to have different environments for different qualifiers (I tried it at my end and it tried to delete existing environment with default qualifier)
  • Re-bootstrap your environment with the --qualifier parameter (check CDK documentation Use the CDK CLI to customize bootstrapping for more details)
  • You would probably need to re-generate snapshots from scratch since it could have references to old qualifier and it my case, it was the default qualifier hnb659fds.

Thanks,
Ashish

[github-actions]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[sfriedlizuehlke]

Hey @ashishdhingra,

The bootstrap qualifier in the integration test is the same that was used for bootstrapping the environment. The deployment of the stacks works fine when I run the integration test with --update-on-failed. So I assume that it is not an issue with the bootstrapQualifier.

Note <SOME_BOOTSTRAP_QUALIFIER_HERE> needs to be replaced in the snippet above, it is just a placeholder.

[ashishdhingra]

Appears to be reprroducible:

> integ-runner


Verifying integration test snapshots...

  CHANGED    integ.vpc-lookup-test 3.822s
      Resources
[~] AWS::EC2::SecurityGroup sg29196201 replace
 └─ [~] VpcId (requires replacement)
     ├─ [-] vpc-<<redacted>>
     └─ [+] vpc-12345