Snowflake.topic2table.map regex feature request

[shokhan-nulogy]

Hi Snowflake Team,

I request the feature of mapping Snowflake topics to tables through the use of regular expressions.

The snowflake.topic.regex config helps us automatically detect new tables in our source database and publish the topics, however we still face the issue of not being able to map these topics to specific tables in Snowflake. The topics2table.map works for the topics config but not the topics.regex config.

Thank you

[sfc-gh-gjachimko]

Thank you for inquiry. We will add that to our backlog.

[sfc-gh-gjachimko]

@shokhan-nulogy actually - I've checked with the team and this functionality is already supported (the documentation is lagging behind unfortunately).
According to this code you should be able to define both topic match via regexp as well as mapping to the tables. Please make sure your regexp definitions do not overlap.

I've checked if that is working with my test deployment and I can confirm it is (I've got 3 topics matching that regexp):

    "topics.regex": "TEST_TOPIC[0-9]",
    "snowflake.topic2table.map": "TEST_TOPIC[0-9]:target",