[cisco_meraki] Replace rsa2elk pipeline and add webhook input

[r00tu53r]

What does this PR do?

The PR

• Replaces the current rsa2elk implementation
• Adds support for Cisco Meraki syslog data reporting (for MX and MR devices only)
• Adds experimental support for webhook integration

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

• Add dashboards

How to test this PR locally

There are pipeline and system tests available for both the data streams.

Run pipeline tests -
elastic-package test pipeline -v

Run system tests -
elastic-package test system -v

Related issues

• Closes Cisco Meraki Integration #2691

Screenshots

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-04-26T08:01:34.312+0000

• Duration: 16 min 39 sec

Test stats 🧪

Test	Results
Failed	0
Passed	19
Skipped	0
Total	19

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[r00tu53r]

Thanks @andrewkroh for all the comments. I have made the changes.

[andrewkroh]

Can you switch these over to converts.

[andrewkroh]

Can you add a convert to validate the source/destination ip.

Do these logs also contain IPv6 addresses? I doubt this dissect works in that case. Unless you know the exact format or that it does not need to handle ipv6 addresses, then you might need a generic parser that handles most formats like https:/elastic/integrations/blob/main/packages/cisco_duo/data_stream/auth/elasticsearch/ingest_pipeline/default.yml#L50-L60.

[r00tu53r]

All the logs I have received and the docs only used IPv4. But I suppose nothing stops someone from using IPv6. I have added IP v4+v6 handling and converts across all pipelines where applicable. Thanks!

[efd6]

If there is no port associated with the IPv6, this pattern is not required (see #3117 (comment)).

[dainperkins]

Apologies if I'm jumping the gun in terms of planned improvements but I pulled all the expected json docs into a cluster and have some comments:

• theres no event.kind, event.outcome
• ids-alerts are not classified as event.kind alert, nor as category intrusion_detection, and should have an event.action & event.outcome
• dhcp no-acks aren't really an ECS accsss event, and definitely not a denial from an access control perspective - more an out of IPs perspective (which should probably also be an alert) could also use yo have event.outcome
• all user based authentication events (wpa, eap, 802.1x) should be category authentication with event.outcome populated to work with the SIEM app
• the http-access-error does not match up to the log - its actually indicative of a tls connection that the Meraki cannot see beyond the host portion of the URL
• rogue ssid & ssid-spoofing detected events should likely be an alert
• the client-vpn-connect event is mis-classified as site to site (event.action)
• client_vpn_connections should use "connected from" as source.ip, "local ip" as source.net.ip
• wifi association and disassociation events need to have the same type/category, etc. - based on current docs I would say event.category: network, session and event.type: access, (start | end), (allowed, denied) ,as appropriate. based on the ecs docs I wouldn't use connection for the actual association type events
• I am pretty sure a wpa-deauth is a connection end, not a deny
• vpn-connectivity-change should be event.type change
• ip flows with translated addresses should populate source.nat/ destination.nat ip and port fields and need start and end populated
• dhcp events should not be allowed/denied if missing information indicating a denial has occurred
• all flow type events should use the same sets of ecs categorization - type: (start | end), (allowed | denied) connection. category: network, and should use 2-3 event.action