Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(custom-resources): add logging property to AwsSdkCall and create Logging class #29648

Merged
merged 94 commits into from
Apr 14, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
94 commits
Select commit Hold shift + click to select a range
33c7ac8
logging
colifran Apr 4, 2024
3e0e2e1
v2 handler
colifran Apr 4, 2024
e87c3e6
docstring
colifran Apr 4, 2024
415b0c6
make logging abstract and implement render
colifran Apr 4, 2024
dcc0dc8
unit tests
colifran Apr 4, 2024
4e61b2d
unit test for off
colifran Apr 4, 2024
c8c0b64
log sdk version
colifran Apr 4, 2024
bcc1ab8
README
colifran Apr 4, 2024
97660cd
readme
colifran Apr 4, 2024
d20b6b4
logging integ tests
colifran Apr 4, 2024
ea8d4de
integ test
colifran Apr 4, 2024
840746a
selective logging should select things to log
colifran Apr 4, 2024
63914b9
align readme
colifran Apr 4, 2024
da86462
logging options
colifran Apr 4, 2024
43bdef9
snaps
colifran Apr 4, 2024
af67a3b
class doc string
colifran Apr 4, 2024
7404c00
docstring on logging property
colifran Apr 4, 2024
a534473
readme
colifran Apr 4, 2024
116d74c
readme
colifran Apr 4, 2024
d751faf
moved logging to AwsSdkCall, updated unit tests, updated integ test, …
colifran Apr 4, 2024
9c998c4
remove rogue snaps
colifran Apr 4, 2024
c8a13db
clarifying comment
colifran Apr 4, 2024
b48dc2e
fix unit tests
colifran Apr 4, 2024
02a68af
fix unit tests
colifran Apr 4, 2024
0214481
small unit test fix
colifran Apr 4, 2024
089c62b
all instead of on
colifran Apr 4, 2024
52e92d3
readme
colifran Apr 4, 2024
319fcc9
logging docstrin
colifran Apr 4, 2024
11caff0
comment
colifran Apr 4, 2024
c4d3790
scoped down flags and added with hidden data logging option
colifran Apr 4, 2024
6ed1e8b
updated integ test
colifran Apr 4, 2024
4bf84f1
readme
colifran Apr 4, 2024
27bf3c3
readme
colifran Apr 4, 2024
ea1bb11
fix unit tests
colifran Apr 4, 2024
34318e7
update snaps
colifran Apr 4, 2024
1f97930
dont need to export the logging props interface right now
colifran Apr 4, 2024
0bdc76e
docstring
colifran Apr 4, 2024
f77733a
stringify filtered response object
colifran Apr 4, 2024
6bd9564
Merge branch 'main' into colifran/response-data
colifran Apr 11, 2024
1ef4097
update physical resource id for aws custom resource ssm integ test
colifran Apr 12, 2024
80c2eae
export logging props interface
colifran Apr 12, 2024
2194742
cognito snaps
colifran Apr 12, 2024
a31701c
codedeploy snaps
colifran Apr 12, 2024
50240cc
open search snaps
colifran Apr 12, 2024
1505b2f
s3 snaps
colifran Apr 12, 2024
532bc14
elastic search snaps
colifran Apr 12, 2024
f9ba75f
aws custom resource snaos
colifran Apr 12, 2024
7c5ba5f
aws logs destinations snaps
colifran Apr 12, 2024
f7bbed7
route53 snaps
colifran Apr 12, 2024
30dca30
events targets logs snaps
colifran Apr 12, 2024
9dae21e
global accelerator snaps
colifran Apr 12, 2024
a4cafb5
global accelerator endpoints snaps
colifran Apr 12, 2024
92475fe
synthetics snaps
colifran Apr 12, 2024
221e55d
stepfunctions tasks emrcontainers snaps
colifran Apr 12, 2024
931bfee
elastic load balancing v2 actions snaps
colifran Apr 12, 2024
9464ca2
elastic load balancing v2 snaps
colifran Apr 12, 2024
cac77ed
elastic search domain unit tests
colifran Apr 12, 2024
6b0b1e7
opensearchservice domain unit tests
colifran Apr 12, 2024
b9c5c8b
cognito user pool client unit tests
colifran Apr 12, 2024
1df13ed
stepfunctions tasks emrcontainers start job run unit tests
colifran Apr 12, 2024
ac3cf58
route53 vpc endpoint service domain name unit tests
colifran Apr 12, 2024
f2f871e
codedeploy lambda custom deployment config unit tests
colifran Apr 12, 2024
3440250
open search service opensearch access policy unit tests
colifran Apr 12, 2024
f1b4578
elastic search access policy unit tests
colifran Apr 12, 2024
38c82f8
global accelerator security group unit tests
colifran Apr 12, 2024
5fbd77f
elastic search log group resource policy unit test
colifran Apr 12, 2024
bb439a7
open search service log group resource policy unit test
colifran Apr 12, 2024
e875058
events targets log group resource policy unit tests
colifran Apr 12, 2024
c366666
redshift alpha integ tests
colifran Apr 12, 2024
b729d15
msk alpha integ tests
colifran Apr 12, 2024
35cdf40
aws glue alpha integ tests
colifran Apr 12, 2024
5bad2c9
Merge branch 'main' into colifran/response-data
colifran Apr 12, 2024
358337d
iot actions alpha integ tests
colifran Apr 12, 2024
59cf170
Merge branch 'main' into colifran/response-data
colifran Apr 12, 2024
037bd61
global accelerator snaps
colifran Apr 12, 2024
4b14876
global accelerator endpoints integ tests
colifran Apr 12, 2024
d4fe078
rosetta
colifran Apr 12, 2024
f4f5bcf
response->responding
colifran Apr 12, 2024
cd5016d
comment
colifran Apr 12, 2024
6b3f7e9
update v3 handler
colifran Apr 12, 2024
770ef5d
fix v2 handler
colifran Apr 12, 2024
3ebb3cf
comment
colifran Apr 12, 2024
4630b9b
update aws custom resource snaps
colifran Apr 12, 2024
0a08663
update open searh snaps
colifran Apr 13, 2024
da38d95
events targets integ tests
colifran Apr 13, 2024
d8902ed
elastic search integ tests
colifran Apr 13, 2024
15ef34c
synthetics integ tests
colifran Apr 13, 2024
53640b3
s3 integ tests
colifran Apr 13, 2024
33d6a5b
route53 integ tests
colifran Apr 13, 2024
4d934d7
logs detinations integ tests
colifran Apr 13, 2024
a6384e7
global accel snaps
colifran Apr 13, 2024
90da2b2
elastic load balancing v2 snaps
colifran Apr 13, 2024
e3b2344
elastic load balancing v2 snaps
colifran Apr 13, 2024
1f08e31
Merge branch 'main' into colifran/response-data
colifran Apr 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,12 @@ function installLatestSdk(packageName: string): void {
}

interface AwsSdk {
[key: string]: any
[key: string]: any;
}

async function loadAwsSdk(
packageName: string,
logErrors: boolean,
installLatestAwsSdk?: 'true' | 'false',
) {
let awsSdk: AwsSdk;
Expand All @@ -49,7 +51,9 @@ async function loadAwsSdk(
// esbuild-disable unsupported-require-call -- not esbuildable but that's fine
awsSdk = require(`/tmp/node_modules/${packageName}`);
} catch (e) {
console.log(`Failed to install latest AWS SDK v3. Falling back to pre-installed version. Error: ${e}`);
if (logErrors) {
console.log(`Failed to install latest AWS SDK v3. Falling back to pre-installed version. Error: ${e}`);
}
// MUST use require as dynamic import() does not support importing from directories
// esbuild-disable unsupported-require-call -- not esbuildable but that's fine
return require(packageName); // Fallback to pre-installed version
Expand All @@ -71,6 +75,11 @@ async function loadAwsSdk(

/* eslint-disable @typescript-eslint/no-require-imports, import/no-extraneous-dependencies */
export async function handler(event: AWSLambda.CloudFormationCustomResourceEvent, context: AWSLambda.Context) {
let logHandlerEvent = event.ResourceProperties.LogHandlerEvent === 'true';
let logApiResponse = event.ResourceProperties.LogApiResponse === 'true';
let logResponseObject = event.ResourceProperties.LogResponseObject === 'true';
let logErrors = event.ResourceProperties.LogErrors === 'true';

try {
event.ResourceProperties.Create = decodeCall(event.ResourceProperties.Create);
event.ResourceProperties.Update = decodeCall(event.ResourceProperties.Update);
Expand All @@ -95,9 +104,11 @@ export async function handler(event: AWSLambda.CloudFormationCustomResourceEvent
if (call) {
const apiCall = new ApiCall(call.service, call.action);

let awsSdk: AwsSdk | Promise<AwsSdk> = loadAwsSdk(apiCall.v3PackageName, event.ResourceProperties.InstallLatestAwsSdk);
let awsSdk: AwsSdk | Promise<AwsSdk> = loadAwsSdk(apiCall.v3PackageName, logErrors, event.ResourceProperties.InstallLatestAwsSdk);

console.log(JSON.stringify({ ...event, ResponseURL: '...' }));
if (logHandlerEvent) {
console.log(JSON.stringify({ ...event, ResponseURL: '...' }));
}

let credentials;
if (call.assumedRoleArn) {
Expand Down Expand Up @@ -128,7 +139,9 @@ export async function handler(event: AWSLambda.CloudFormationCustomResourceEvent
flattenResponse: true,
});

console.log('API response', response);
if (logApiResponse) {
console.log('API response', response);
}

flatData.apiVersion = apiCall.client.config.apiVersion; // For test purposes: check if apiVersion was correctly passed.
flatData.region = await apiCall.client.config.region().catch(() => undefined); // For test purposes: check if region was correctly passed.
Expand Down Expand Up @@ -159,9 +172,9 @@ export async function handler(event: AWSLambda.CloudFormationCustomResourceEvent
}
}

await respond(event, 'SUCCESS', 'OK', physicalResourceId, data);
await respond(event, 'SUCCESS', 'OK', physicalResourceId, data, logResponseObject);
} catch (e: any) {
console.log(e);
await respond(event, 'FAILED', e.message || 'Internal Error', context.logStreamName, {});
await respond(event, 'FAILED', e.message || 'Internal Error', context.logStreamName, {}, logResponseObject);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ export function filterKeys(object: object, pred: (key: string) => boolean) {

type Event = AWSLambda.CloudFormationCustomResourceEvent

export function respond(event: Event, responseStatus: string, reason: string, physicalResourceId: string, data: any) {
export function respond(event: Event, responseStatus: string, reason: string, physicalResourceId: string, data: any, logResponseObject: boolean) {
const responseBody = JSON.stringify({
Status: responseStatus,
Reason: reason,
Expand All @@ -55,8 +55,10 @@ export function respond(event: Event, responseStatus: string, reason: string, ph
Data: data,
});

// eslint-disable-next-line no-console
console.log('Responding', responseBody);
if (logResponseObject) {
// eslint-disable-next-line no-console
console.log('Responding', responseBody);
}

// eslint-disable-next-line @typescript-eslint/no-require-imports
const parsedUrl = require('url').parse(event.ResponseURL);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import { Annotations } from '../../../core';
import { AwsCustomResourceSingletonFunction } from '../../../custom-resource-handlers/dist/custom-resources/aws-custom-resource-provider.generated';
import * as cxapi from '../../../cx-api';
import { awsSdkToIamAction } from '../helpers-internal/sdk-info';
import { Logging } from './logging';

// Shared definition with packages/@aws-cdk/custom-resource-handlers/lib/custom-resources/aws-custom-resource-handler/shared.ts
const PHYSICAL_RESOURCE_ID_REFERENCE = 'PHYSICAL:RESOURCEID:';
Expand Down Expand Up @@ -394,6 +395,13 @@ export interface AwsCustomResourceProps {
* @default - the Vpc default strategy if not specified
*/
readonly vpcSubnets?: ec2.SubnetSelection;

/**
* Enables logging
*
* @default Logging.on()
*/
readonly logging?: Logging;
}

/**
Expand Down Expand Up @@ -497,6 +505,10 @@ export class AwsCustomResource extends Construct implements iam.IGrantable {
update: props.onUpdate && this.encodeJson(props.onUpdate),
delete: props.onDelete && this.encodeJson(props.onDelete),
installLatestAwsSdk,
logHandlerEvent: props.logging?.logHandlerEvent ?? true,
logApiResponse: props.logging?.logApiResponse ?? true,
logRespondeObject: props.logging?.logResponseObject ?? true,
logErrors: props.logging?.logErrors ?? true,
},
});

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
/**
* Properties used to initialize Logging.
*/
export interface LoggingProps {
/**
* Whether or not to log the event object received by the lambda handler.
*
* @default true
*/
readonly logHandlerEvent?: boolean;

/**
* Whether or not to log the response returned from the API call.
*
* @default true
*/
readonly logApiResponse?: boolean;

/**
* Whether or not to log the response object that will be returned by the lambda.
*
* @default true
*/
readonly logResponseObject?: boolean;

/**
* Whether or not to log errors that were encountered during lambda execution.
*
* @default true
*/
readonly logErrors?: boolean;
}

/**
* A class that represents Logging that will take place during lambda execution.
*/
export class Logging {
/**
* Enables logging of all logged data in the lambda handler.
*
* This includes the event object, the API call response, all fields in the response object
* returned by the lambda, and any errors encountered.
*/
public static on() {
return new Logging();
}

/**
* Enables selective logging of logged data in the lambda handler.
*/
public static selective(props: LoggingProps) {
return new Logging({ ...props });
}

/**
* Turns off all logging in the lambda handler.
*/
public static off() {
return new Logging({
logHandlerEvent: false,
logApiResponse: false,
logResponseObject: false,
logErrors: false,
});
}

/**
* Whether or not to log the event object received by the lambda handler.
*/
public readonly logHandlerEvent: boolean;

/**
* Whether or not to log the API call response.
*/
public readonly logApiResponse: boolean;

/**
* Whether or not to log the response object that will be returned by the lambda.
*/
public readonly logResponseObject: boolean;

/**
* Whether or not to log errors that were encountered during lambda execution.
*/
public readonly logErrors: boolean;

private constructor(props: LoggingProps = {}) {
this.logHandlerEvent = props.logHandlerEvent ?? true;
this.logApiResponse = props.logApiResponse ?? true;
this.logResponseObject = props.logResponseObject ?? true;
this.logErrors = props.logErrors ?? true;
}
}
Loading