Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for enabling Point in time recovery (PITR) for DynamoDB tables #4063

Merged
merged 2 commits into from
Apr 26, 2018
Merged

Add support for enabling Point in time recovery (PITR) for DynamoDB tables #4063

merged 2 commits into from
Apr 26, 2018

Conversation

wjam
Copy link
Contributor

@wjam wjam commented Apr 5, 2018

Fixes #3927.

Note that it can take ~10 minutes for PITR to be enabled on newly created tables.

Also note that I've added waitForDynamoDbBackupUpdateToBeCompleted as the documentation implies that it's possible for PointInTimeRecoveryStatus to be ENABLING rather than just ENABLED or DISABLED (https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_PointInTimeRecoveryDescription.html).

@ghost ghost added the size/L Managed by automation to categorize the size of a PR. label Apr 5, 2018
@bflad bflad added enhancement Requests to existing resources that expand the functionality or scope. service/dynamodb Issues and PRs that pertain to the dynamodb service. labels Apr 5, 2018
@wjam
Copy link
Contributor Author

wjam commented Apr 13, 2018

@bflad Is there anything that needs to be fixed for this to be merged?

@drmaciej
Copy link

@bflad not rushing you or anything, but we're really looking forward to have that merged and released in the next version :)

@bflad bflad mentioned this pull request Apr 20, 2018
Closed
bflad
bflad reviewed Apr 24, 2018
View reviewed changes
Copy link
Contributor

@bflad bflad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for submitting this @wjam! I left some initial comments below. Can you please let us know if you have any questions or do not have time to implement the feedback? Cheers!

aws/resource_aws_dynamodb_table.go Outdated
@@ -238,6 +238,10 @@ func resourceAwsDynamoDbTable() *schema.Resource {
},
},
"tags": tagsSchema(),
"point_in_time_backup_enabled": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To future proof the resource, we should probably nest this as:

point_in_time_recovery {
  enabled = true/false
}

This can be done similar to ttl

aws/resource_aws_dynamodb_table.go Outdated
@@ -448,6 +452,12 @@ func resourceAwsDynamoDbTableUpdate(d *schema.ResourceData, meta interface{}) er
}
}

if d.HasChange("point_in_time_backup_enabled") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When running the acceptance testing in parallel, a large amount of the testing started failing with:

LimitExceededException: Subscriber limit exceeded: Only 10 tables can be created, updated, or deleted simultaneously

We handle this partially throughout the resource, but it seems this additional call takes a long time, even when setting it to be disabled. I think we will want to try to rework this so on resource creation, it only gets called if its enabled. d.IsNewResource() can help here. Its ugly but something like the below might work

if d.HasChange("point_in_time_recovery") {
  _, enabled := d.GetChange("point_in_time_recovery.0.enabled")
  if !d.IsNewResource() || enabled.(bool) {
    // call updateDynamoDbPITR()
  }
}

aws/resource_aws_dynamodb_table.go
return false, fmt.Errorf("Error reading backup status from dynamodb resource: %s", err)
}

pitr := output.ContinuousBackupsDescription.PointInTimeRecoveryDescription
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To prevent panics, we should perform nil check:

if output.ContinuousBackupsDescription == nil || output.ContinuousBackupsDescription.PointInTimeRecoveryDescription == nil {
  return false, errors.New("Error reading backup status from dynamodb resource: empty description")
}

aws/resource_aws_dynamodb_table.go
}

pitr := output.ContinuousBackupsDescription.PointInTimeRecoveryDescription
return *pitr.PointInTimeRecoveryStatus == dynamodb.PointInTimeRecoveryStatusEnabled, nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To prevent a panic in the unlikely scenario that status comes back nil, we should wrap it with aws.BoolValue(): return aws.BoolValue(pitr.PointInTimeRecoveryStatus) == dynamodb.PointInTimeRecoveryStatusEnabled, nil

aws/resource_aws_dynamodb_table.go Outdated
return 42, "", err
}

pitr := result.ContinuousBackupsDescription.PointInTimeRecoveryDescription
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here about nil checks

aws/resource_aws_dynamodb_table_test.go
return err
}

pitr := resp.ContinuousBackupsDescription.PointInTimeRecoveryDescription
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here about nil checks.

aws/resource_aws_dynamodb_table_test.go Outdated
@@ -937,6 +964,37 @@ func testAccCheckInitialAWSDynamoDbTableConf(n string) resource.TestCheckFunc {
}
}

func testAccCheckDynamoDbTableHasBackup(n string) resource.TestCheckFunc {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably name this more specifically: testAccCheckDynamoDbTablePointInTimeRecoveryEnabled

website/docs/r/dynamodb_table.html.markdown Outdated
@@ -90,6 +90,7 @@ attributes, etc.
* `stream_view_type` - (Optional) When an item in the table is modified, StreamViewType determines what information is written to the table's stream. Valid values are `KEYS_ONLY`, `NEW_IMAGE`, `OLD_IMAGE`, `NEW_AND_OLD_IMAGES`.
* `server_side_encryption` - (Optional) Encrypt at rest options.
* `tags` - (Optional) A map of tags to populate on the created table.
* `point_in_time_backup_enabled` - (Optional) Indicates whether point-in-time recovery is enabled (true) or disabled (false) - note that it can take a while to enable for new tables.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: "a while" means different things to different people. 😄 We should specify something more distinct like "a few minutes", "up to X minutes", or similar.

@bflad bflad added the waiting-response Maintainers are waiting on response from community or contributor. label Apr 24, 2018
@ghost ghost added the size/L Managed by automation to categorize the size of a PR. label Apr 26, 2018
@wjam
Copy link
Contributor Author

wjam commented Apr 26, 2018

@bflad Pushed changes to fix code review issues

@bflad bflad removed the waiting-response Maintainers are waiting on response from community or contributor. label Apr 26, 2018
bflad
bflad approved these changes Apr 26, 2018
View reviewed changes
Copy link
Contributor

@bflad bflad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work, @wjam! 🚀

21 tests passed (all tests)
=== RUN   TestAccAWSDynamoDbTable_importBasic
--- PASS: TestAccAWSDynamoDbTable_importBasic (59.29s)
=== RUN   TestAccAWSDynamoDbTable_importTimeToLive
--- PASS: TestAccAWSDynamoDbTable_importTimeToLive (60.50s)
=== RUN   TestAccAWSDynamoDbTable_basic
--- PASS: TestAccAWSDynamoDbTable_basic (120.65s)
=== RUN   TestAccAWSDynamoDbTableItem_withMultipleItems
--- PASS: TestAccAWSDynamoDbTableItem_withMultipleItems (121.65s)
=== RUN   TestAccAWSDynamoDbTableItem_rangeKey
--- PASS: TestAccAWSDynamoDbTableItem_rangeKey (122.20s)
=== RUN   TestAccAWSDynamoDbTable_streamSpecificationValidation
--- PASS: TestAccAWSDynamoDbTable_streamSpecificationValidation (3.30s)
=== RUN   TestAccAWSDynamoDbTableItem_basic
--- PASS: TestAccAWSDynamoDbTableItem_basic (128.04s)
=== RUN   TestAccAWSDynamoDbTableItem_update
--- PASS: TestAccAWSDynamoDbTableItem_update (139.17s)
=== RUN   TestAccAWSDynamoDbTableItem_updateWithRangeKey
--- PASS: TestAccAWSDynamoDbTableItem_updateWithRangeKey (140.40s)
=== RUN   TestAccAWSDynamoDbTable_importTags
--- PASS: TestAccAWSDynamoDbTable_importTags (167.65s)
=== RUN   TestAccAWSDynamoDbTable_attributeUpdateValidation
--- PASS: TestAccAWSDynamoDbTable_attributeUpdateValidation (4.15s)
=== RUN   TestAccAWSDynamoDbTable_streamSpecification
--- PASS: TestAccAWSDynamoDbTable_streamSpecification (137.67s)
=== RUN   TestAccAWSDynamoDbTable_tags
--- PASS: TestAccAWSDynamoDbTable_tags (146.14s)
=== RUN   TestAccAWSDynamoDbTable_ttl
--- PASS: TestAccAWSDynamoDbTable_ttl (157.41s)
=== RUN   TestAccAWSDynamoDbTable_gsiUpdateCapacity
--- PASS: TestAccAWSDynamoDbTable_gsiUpdateCapacity (176.81s)
=== RUN   TestAccAWSDynamoDbTable_encryption
--- PASS: TestAccAWSDynamoDbTable_encryption (231.46s)
=== RUN   TestAccAWSDynamoDbTable_gsiUpdateNonKeyAttributes
--- PASS: TestAccAWSDynamoDbTable_gsiUpdateNonKeyAttributes (297.82s)
=== RUN   TestAccAWSDynamoDbTable_enablePitr
--- PASS: TestAccAWSDynamoDbTable_enablePitr (372.63s)
=== RUN   TestAccAWSDynamoDbTable_extended
--- PASS: TestAccAWSDynamoDbTable_extended (435.02s)
=== RUN   TestAccAWSDynamoDbTable_attributeUpdate
--- PASS: TestAccAWSDynamoDbTable_attributeUpdate (592.64s)
=== RUN   TestAccAWSDynamoDbTable_gsiUpdateOtherAttributes
--- PASS: TestAccAWSDynamoDbTable_gsiUpdateOtherAttributes (615.26s)

@bflad bflad added this to the v1.17.0 milestone Apr 26, 2018
@bflad bflad merged commit 332005c into hashicorp:master Apr 26, 2018
bflad added a commit that referenced this pull request Apr 26, 2018
@bflad
Update CHANGELOG for #4063
6cb22eb
@bflad bflad mentioned this pull request Apr 30, 2018
Closed
@bflad
Copy link
Contributor

bflad commented May 2, 2018

This has been released in version 1.17.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

@MichaelPereira MichaelPereira mentioned this pull request May 16, 2018
Closed
@ewbankkit ewbankkit mentioned this pull request Feb 10, 2019
Closed
@ghost
Copy link

ghost commented Apr 6, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Apr 6, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@bflad bflad bflad approved these changes

Assignees
No one assigned
Labels
enhancement Requests to existing resources that expand the functionality or scope. service/dynamodb Issues and PRs that pertain to the dynamodb service. size/L Managed by automation to categorize the size of a PR.
Projects
None yet
Milestone
v1.17.0
Development

Successfully merging this pull request may close these issues.

Feature request: support DynamoDB Continuous Backups and Point-In-Time Recovery (PITR)
3 participants
@wjam @drmaciej @bflad