Filters in MISP filebeat module are not working

[stevengoossensB]

Please post all questions and issues on https://discuss.elastic.co/c/beats
before opening a Github Issue. Your questions will reach a wider audience there,
and if we confirm that there is a bug, then you can open a new issue.

For security vulnerabilities please only send reports to [email protected].
See https://www.elastic.co/community/security for more information.

Please include configurations and logs if available.

For confirmed bugs, please report:

• Version: 7.14
• Operating System: Ubuntu
• Discuss Forum URL:/
• Steps to Reproduce: Setup Threat intel module, connect MISP, enable filters
  

The default config as described for filtering in the threat intel module is not working for MISP. Enabling the below results in an error message.

    # Optional filters that can be applied to the API for filtering out results. This should support the majority of fields in a MISP context.
    # For examples please reference the filebeat module documentation.
    var.filters:
      - threat_level: [4, 5]
      - to_ids: true

Could you update the syntax that is required for the filters to work?

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[syloktools]

Any updates on this?

[jamiehynds]

@efd6 can you confirm if your recent PR will resolve this, or is it a separate issue?

[marc-gr]

cc @P1llus

[efd6]

@jamiehynds I believe this is a separate issue.

[P1llus]

This is indeed a separate issue. We need to rewrite that part of the configuration, I have written a new way on how to deal with the filtering in the MISP package: https:/elastic/integrations/pull/1946/files#diff-d233009dee8a83779c004b766b5ec7ce2dba476a0a6c92c582353796f12ae353R17

I am however a bit concerned how the config change might affect existing installations. Do you know @marc-gr ?

[marc-gr]

If existing installations are using the filters option, is more likely that is not working as expected anyway if I understand the issue correctly? I think even if in theory this is a breaking change, would be beneficial to release it as a bugfix anyway to make the feature usable. @jamiehynds do you think this would be acceptable in this scenario?

[jamiehynds]

Agree @marc-gr - if filtering isn't working correctly today I'd classify this as a bug fix more so than a breaking change and happy to proceed with implementation.