(@aws-cdk/aws-iot): New Rule does not call Action

[denxorz]

When I add a new IoT Rule+Action through CDK, the Action is not used until I make changes through the Console.

Reproduction Steps

1. Make a Lambda, for example:

exports.handler = async (event) => {  console.log("testlambda", event); };

2. Create an IoT rule + Action (Important: Create a new rule with a new name):

new CfnTopicRule(stack, "testAJKL1243242Topic",
    new CfnTopicRuleProps
    {
        RuleName = "testAJKL1243242Topic",
        TopicRulePayload = new CfnTopicRule.TopicRulePayloadProperty
        {
            Actions = new object[]
            {
                new CfnTopicRule.ActionProperty
                {
                    Lambda = new CfnTopicRule.LambdaActionProperty { FunctionArn = "arn:aws:lambda:eu-west-1:accountid:function:testlambda" }
                }
            },
            Sql = "SELECT * FROM \"test\"",
            RuleDisabled = false,
            AwsIotSqlVersion = "2016-03-23",
        }
    });

3. Open the Console and publish a message to the "test" topic
4. Check the CloudWatch output of the Lambda --> Lambda is not called

As a workaround you can make it work by adding a "Error action" through Console and remove it again.
5. Go to the Console, and add a CloudWatch log action to the "Error action"
6. Remove the CloudWatch log action
7. Repeat step 3 and 4, now you will see output in step 4.

What did you expect to happen?

I expect the Lambda to be called.

What actually happened?

Nothing?

Environment

• CDK CLI Version : 1.86.0 (build a10cfcf)
• Framework Version: (which framework?)
• Node.js Version: v14.15.3
• OS : Windows 10
• Language (Version): C# .NET 3.1, referenced CDK 1.86.0 Nuget packages

Other

Region: eu-west-1.
Using the CDK since version 1.73.0, and I think it was already an issue in that version, at least it is not new in 1.86.0.

---

This is 🐛 Bug Report

[denxorz]

Just ran into this again today. Do you guys need any more info?

[skinny85]

Hello @denxorz ,

do you feel like this is a CDK-specific problem? Or is that just a problem with how the IoT service works?

Thanks,
Adam

[denxorz]

Hello @skinny85 ,

Running the CDK example above results in a correct configuration in the Console, but it does not work. So definitely a bug.

If needed we could do a video call or something so I can show you the issue.

Best regards,
Dennis

[skinny85]

Right. I'm just wondering whether it's a CDK bug, or a CloudFormation/IoT bug 🙂.

Is the CloudFormation template the CDK generates correct with regards to the documentation for IoT?

[denxorz]

I am not sure. I did not use CloudFormation explicitly up till this point, so I did not learn to read those templates yet (that's the advantage of CDK).

Is there anything specific you are looking for in the template?

It might not be very explicit in my first post, but if you use a name of a rule that already existed, then it works through CDK. I am not sure where something like the name could be "cached", but it might help you determine if that is in CDK of CloudFormation.

[skinny85]

So, from looking at https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iot-topicrule.html , the example they use uses an existing Rule (its name is passed through a CloudFormation Parameter). When talking about creating Rules: https://docs.aws.amazon.com/iot/latest/developerguide/iot-create-rule.html , it only mentions creating them through the API, not through CloudFormation.

Perhaps this is a known limitation of IoT's CloudFormation support? Do you have AWS premium support by chance - perhaps you can open a ticket to clarify?

Basically, I think there's not much we can do here from the CDK side - everything seems to be working normally from the CDK perspective, it's the actual service's behavior (or its CloudFormation support) that's acting up. Unfortunately, there's not much we can do if that's the case 😕.

[github-actions]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[denxorz]

I did try to open a ticket at premium support, but it seems our businees contract expired 2 weeks ago. I am looking into this, but currently I am unable to at a ticket over there.

As a developer I understand that you cant just change your code base to fix this, if this is not part of your product. But as a user of the CDK I am also unable to use the CDK reliably (sometimes it works, some times it does not). And as reliability is one of the aspects of using infrastructure as code, this might be a thing that you could pick up with your collegues as well @skinny85 .

[olivierpichon]

Running into the very same issue, there is definitely a bug @skinny85. Will try to identify if it is more a CDK or cloudformation bug. In the meantime, here are some JS code:

    const fn = new Function(this, 'Lambda', {
      runtime: Runtime.NODEJS_12_X,
      handler: 'index.handler',
      code: ...,
      timeout: cdk.Duration.minutes(1),
      memorySize: 128
    })

  const rule = new CfnTopicRule(this, "TelemetryIngestion", {
      ruleName: 'TelemetryIngestion',
      topicRulePayload: {
        sql: "SELECT time, topic(2) as site_id, topic(3) AS device_type, topic(4) AS device_id, topic(5) AS endpoint, value FROM 'dt/+/+/+/#'",
        description: 'Ingestion rule',
        ruleDisabled: false,
        awsIotSqlVersion: '2016-03-23',
        actions: [
          {
            lambda: { functionArn: fn.functionArn }
          }
        ]
      }
    })

This code above creates all the resources: the lambda function and the rule but no data is transferred to the lambda function.

When going in the AWS console, editing the rule and just clicking on the "update" button without doing any modifications is enough to then have the rule working and the data starting flooding to the lambda function

Please consider reopening this issue @skinny85

[olivierpichon]

Can confirm it is a Cloudformation bug

[olivierpichon]

Found the reason for this issue, you need to setup a CfnPermission for the lambda function to be invoked. It can be defined this way:

    new CfnPermission(this, "InvokeLambdaToTimeScalePermission", {
      action: "lambda:InvokeFunction",
      functionName: fn.functionArn,
      principal: "iot.amazonaws.com",
      sourceArn: rule.attrArn
    })

Because it fails silently without this permission, and it is quite impossible for someone to tell what's missing, I would suggest that CDK automatically creates this resource (as its main goal and philosophy is to come with sensible default)