Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

provider/aws: Add support for AWS IoT #6961

Closed
wants to merge 52 commits into from

Conversation

jhedev
Copy link

@jhedev jhedev commented Jun 1, 2016

This is my very first PR to a go project so please be patient with me :)

This will add the following resources:

  • aws_iot_certificate
  • aws_iot_thing
  • aws_iot_policy
  • aws_iot_policy_attachment
  • aws_iot_topic_rule

I already added acceptance tests for all resources except aws_iot_policy_attachment (I have a test locally but it is still failing...).

Still todo:

  • Update function of aws_iot_policy_attachment
  • Update function of aws_iot_topic_rule
  • More tests for most resources
  • Clean up code
  • Documentation

Also see #6138

@jhedev
Copy link
Author

jhedev commented Jun 1, 2016

The aws_iot_certificate acceptance tests require a csr.pem file. This is still missing in the repo as I didn't know where to put it. Maybe someone can help me?

@jhedev
Copy link
Author

jhedev commented Jun 25, 2016

Added some basic documentation for all new resources.

@jhedev
Copy link
Author

jhedev commented Jun 27, 2016

Added the csr.pem file to the test-fixtures directory.

@jhedev
Copy link
Author

jhedev commented Jun 28, 2016

Rebased onto master. Unfortunately this messed up the commits a bit.

@jhedev jhedev mentioned this pull request Jun 28, 2016
@Flygsand
Copy link

Flygsand commented Aug 23, 2016

Thanks for putting work into this @jhedev. I've rebased our fork of this PR against current master, and made some other (minor) changes to get the tests running again. Feel free to pull that into yours.

As for aws_iot_policy_attachment, perhaps it'd be less surprising if it follows the interface of aws_iam_policy_attachment? That is: rather than attaching multiple policies to one principal, it attaches one policy to multiple principals. I'm not exactly sure why I had it the other way around, but I think it's more convenient also. You'd usually create one policy for a group of principals, so having the ability to perform this mapping in one go:

resource "aws_iot_policy_attachment" "pubsub_bulbs" {
  name = "pubsub-bulbs"
  policy = "${aws_iot_policy.pubsub.name}"
  principals = ["${aws_iot_certificate.bulb.*.arn}"]
}

seems easier to me.

@jwestboston
Copy link

Looking forward to this being supported within Terraform!

@jhedev
Copy link
Author

jhedev commented Oct 7, 2016

Thanks @protomouse! I pulled your changes in.

I remember wondering about your proposed interface of the aws_iot_policy_attachment, too. Not sure why I didn't ask, though.

I also agree that this should be kept similar to the aws_iam_policy_attachment. So I'll update the code.

@jhedev jhedev changed the title [WIP] provider/aws: Add support for AWS IoT provider/aws: Add support for AWS IoT Oct 7, 2016
@jhedev
Copy link
Author

jhedev commented Oct 7, 2016

Rebased onto master and updated the interface of aws_iot_policy_attachment as discussed above.

If there are no other remarks this can be merged now :)

@jhedev
Copy link
Author

jhedev commented Nov 2, 2016

Anything else I can do to speed up this PR getting merged?

@Flygsand
Copy link

Flygsand commented Nov 3, 2016

Pinging @catsby for review.

@jhedev
Copy link
Author

jhedev commented Nov 14, 2016

Any chance you have some time to review, @catsby ? :)

@Flygsand
Copy link

Flygsand commented Dec 5, 2016

I've resolved the merge conflicts in our fork, as well as brought the IoT dependency level with the other vendor'd AWS packages. Could you please @jhedev pull those changes into your branch, and then ping some more HashiCorp folk? (phinze, stack72, mitchellh)

@jhedev
Copy link
Author

jhedev commented Dec 5, 2016

@protomouse Done. Thanks for your help!

Any chance one of you can review, @catsby @phinze @stack72 @mitchellh ?

Copy link
Contributor

@stack72 stack72 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @jhedev

Thanks for the work here - I have gone through the first few resources and made some comments about what will need to be fixed up to make it conform to our standard way of writing a resource :)

Please have a read through and then apply the same changes across all of the resources

Thanks for the work here

Paul

d.SetId(*out.CertificateId)
d.Set("arn", *out.CertificateArn)

return nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The normal convention here is for the Create -> Read func

This means we can check that the Create has happened correctly. All of the d.Set("") then happens in the Read func

log.Printf("[DEBUG] Created certificate from csr")

d.SetId(*out.CertificateId)
d.Set("arn", *out.CertificateArn)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be removed as we set it in the Read func - we just need to make Create return the Read func

return err
}

d.SetId(*out.CertificateDescription.CertificateId)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to reset the ID here

}

d.SetId(*out.CertificateDescription.CertificateId)
d.Set("arn", *out.CertificateDescription.CertificateArn)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to deref, d.Set does this in a safe way

}

//TODO: Remove old cert??
//conn.DeleteCertificate(&iot.DeleteCertificateInput{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO we should clean the old cert up only if we have actually created the certificate in the first place

Either way, let's clean up these comments

return &schema.Resource{
Create: resourceAwsIotPolicyAttachmentCreate,
Read: resourceAwsIotPolicyAttachmentRead,
Update: resourceAwsIotPolicyAttachmentUpdate,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As this is an attachment, is there really an update? Wouldn't ForceNew (destroy attachment and then create a new attachment) be better?

conn := meta.(*AWSClient).iotconn

for _, p := range d.Get("principals").(*schema.Set).List() {
_, err := conn.AttachPrincipalPolicy(&iot.AttachPrincipalPolicyInput{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a potential eventual consistency issue here? Do we need to wait and check the attachment has happened before we move on to the next?

return nil
}

func resourceAwsIotPolicyAttachmentUpdate(d *schema.ResourceData, meta interface{}) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can make everything in the schema ForceNew and slip this


for _, p := range d.Get("principals").(*schema.Set).List() {
log.Printf("[INFO] %+v", p)
_, err := conn.DetachPrincipalPolicy(&iot.DetachPrincipalPolicyInput{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a potential time to wait for the detachment to happen?

}

func testAccCheckAWSIoTPolicyAttachmentDestroy_basic(s *terraform.State) error {
return nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why have we got a func that returns nil?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem here is that I'm not sure how to test if the attachment was destroyed and just copied this idea from the resource_aws_iam_policy_attachment_test.go which does the same:

func testAccCheckAWSPolicyAttachmentDestroy(s *terraform.State) error {
	return nil
}

@francknouama
Copy link

👍 Looking forward to have this feature available through terraform

@KoenR3
Copy link

KoenR3 commented Feb 21, 2017

It is one of the downsides we currently have with terraform that we cannot provision IoT. I hope this feature will be available soon!

@jhedev
Copy link
Author

jhedev commented Feb 22, 2017

@KoenR3 Same goes for me. Unfortunately I was pretty short on time the last couple months and couldn't make the requested changes. I hope to find some time in the next two weeks to make the required changes. So we can get this merged eventually :)

@Flygsand
Copy link

Cool @jhedev. I'm working on covering some of the more recent IoT functionality not covered by this PR, such as thing types and BYOC. I'll submit a separate PR for these. For now though, I'd like to suggest a small API change to aws_iot_certificate - namely, replacing the boolean attribute active with an enum status (ACTIVE, INACTIVE). This is because a certificate can also (irreversibly) be marked as REVOKED. Feel free to do as you wish w.r.t. actually implementing certificate revocation in this PR, but this API change will at least make adding this functionality in the future a bit smoother.

@alessandroben
Copy link

Hi guys, do you have any news about that? I'm looking to implement aws_iot on my terraform project and I'm stuck since there is no support at the moment :(
Let me know about the ETA of this feature, thanks you very much.

@ajlanghorn
Copy link
Contributor

IoT resources in Terraform would be awesome! Kudos for the great work on this so far.
Let me know if you need any help; happy to contribute, if I can!

@theherk
Copy link
Contributor

theherk commented Apr 12, 2017

@jhedev, you are doing a great thing for many of us. If there were anything I could do to help I would, but you're in the driver's seat. How far do you believe you are from this being ready for merge. I want very much not to write a CloudFormation template for our IoT bits.

@jhedev
Copy link
Author

jhedev commented Apr 24, 2017

I made some of the changes mentioned above and rebased on master again. However, there are still a few things that need to be change but I'm not sure how. So it would be great to get some input how these things are handled in terraform and if possible a pointer to some source code where this is done:

aws_iot_policy_attachment

  • Eventual consistency issue when attaching principals? see discussion
  • Wait some time when detaching? see discussion

aws_iot_thing

  • Same issue as above when attaching principals?
  • Same issue as above when detaching principals?

@stephencoe
Copy link
Contributor

@jhedev this is awesome! I don't have a great deal of experience with the code base but looking at other examples where this has been implemented it looks like it's wrapped in a resource.Retry.

Some examples I found of this were in lambda_function, api_gateway_account, aws_launch_configuration.

@jhedev
Copy link
Author

jhedev commented Apr 25, 2017

@stephencoe Thanks for your help. I'll have a look at the implementations.

@rob-smallshire
Copy link

Is this being worked on currently by anybody? No activity for a few months now...

@jhedev
Copy link
Author

jhedev commented Jun 23, 2017

Hey @rob-smallshire, I always planned to get this finished at some point. However, my free time is very rare at the moment and since I don't use AWS IoT at work anymore there is very little motivation to get this done.

Long story short: I do not work on it actively at the moment and don't think I will in the near future.

Feel free to take over and use any of my code if you like :)

@rob-smallshire
Copy link

@jhedev Thanks for a prompt response. It's good to have some clarity on the status of this feature. I'm still evaluating AWS IoT for my application, but if it works out for us I can dedicate some effort towards resurrecting your code. It looks like your were pretty close!

@jhedev
Copy link
Author

jhedev commented Aug 8, 2017

I guess we should close this PR now?

@rob-smallshire
Copy link

I guess so. Work continues over here:

hashicorp/terraform-provider-aws#143

and

hashicorp/terraform-provider-aws#986

@jhedev
Copy link
Author

jhedev commented Aug 8, 2017

Yeah, I've seen that. Thanks for taking over :)

@ghost
Copy link

ghost commented Apr 7, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@ghost ghost locked and limited conversation to collaborators Apr 7, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.