Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add RPZ file output bot #1962

Merged
8 commits merged into from
Sep 17, 2021
Merged

Conversation

mariuskarotkis
Copy link
Contributor

Add new bot: RPZ file output

Example output file:

$TTL 3600
@ SOA rpz.yourdomain.eu. hostmaster.rpz.yourdomain.eu. 2105260601 60 60 432000 60
NS localhost.
;
; yourdomain.eu. CERT.XX Response Policy Zones (RPZ)
; Last updated: 2021-05-26 06:01:41 (UTC)
;
; Terms Of Use: https://rpz.yourdomain.eu
; For questions please contact rpz [at] yourdomain.eu
;
*.maliciousdomain.com CNAME rpz.yourdomain.eu.
*.secondmaliciousdomain.com CNAME rpz.yourdomain.eu.

@codecov-commenter
Copy link

codecov-commenter commented May 26, 2021

Codecov Report

Merging #1962 (eccded0) into develop (0750a3d) will increase coverage by 0.21%.
The diff coverage is 64.08%.

@@             Coverage Diff             @@
##           develop    #1962      +/-   ##
===========================================
+ Coverage    75.74%   75.96%   +0.21%     
===========================================
  Files          414      434      +20     
  Lines        22169    23283    +1114     
  Branches      2944     3111     +167     
===========================================
+ Hits         16793    17686     +893     
- Misses        4697     4879     +182     
- Partials       679      718      +39     
Impacted Files Coverage Δ
intelmq/bots/outputs/rpz_file/output.py 55.26% <55.26%> (ø)
intelmq/tests/bots/outputs/rpz_file/test_output.py 100.00% <100.00%> (ø)
intelmq/tests/bots/experts/wait/test_expert.py 50.00% <0.00%> (-50.00%) ⬇️
intelmq/bots/experts/wait/expert.py 48.57% <0.00%> (-31.43%) ⬇️
intelmq/bots/parsers/cert_eu/parser_csv.py 78.84% <0.00%> (-2.79%) ⬇️
intelmq/bots/parsers/misp/parser.py 85.45% <0.00%> (-2.79%) ⬇️
intelmq/bots/experts/tor_nodes/expert.py 46.42% <0.00%> (-2.33%) ⬇️
intelmq/bots/collectors/mail/_lib.py 67.27% <0.00%> (-1.96%) ⬇️
intelmq/bots/experts/asn_lookup/expert.py 35.83% <0.00%> (-1.89%) ⬇️
...telmq/bots/experts/recordedfuture_iprisk/expert.py 41.93% <0.00%> (-1.89%) ⬇️
... and 421 more

@ghost
Copy link

ghost commented May 26, 2021

Hi @mariuskarotkis thank you for contributing so much new stuff =) Looks all very good on the first glance.

Unfortunately I fear we won't have time for the reviews this week, so we'll need to keep you waiting a few days. But be assured that we don't ignore you :)

@ghost
Copy link

ghost commented Jun 26, 2021

You can be sure that we haven't forgot your PRs, and we're currently working hard. Unfortunately we need to hold a deadline for IntelMQ 3.0 and as we're short-staffed currently, we've had to postpone some activities, like reviewing the PRs. If I won't have time for a review before 3.0, we'll put your additions into 3.1 of course. Sorry for these circumstances :/

@ghost ghost self-assigned this Jul 12, 2021
@ghost ghost added component: bots feature Indicates new feature requests or new features labels Jul 12, 2021
@ghost ghost added this to the 3.1.0 milestone Jul 12, 2021
Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I am primarily missing is documentation for the bot which explains what it does, how it does it, how it can be configured and what can be achieved with it (linking to external sources is perfectly fine).

intelmq/bots/outputs/rpz_file/output.py Outdated Show resolved Hide resolved
intelmq/bots/outputs/rpz_file/output.py Outdated Show resolved Hide resolved
intelmq/bots/outputs/rpz_file/output.py Outdated Show resolved Hide resolved
intelmq/bots/outputs/rpz_file/output.py Outdated Show resolved Hide resolved
intelmq/bots/outputs/rpz_file/output.py Outdated Show resolved Hide resolved
intelmq/bots/outputs/rpz_file/output.py Outdated Show resolved Hide resolved
intelmq/bots/outputs/rpz_file/output.py Outdated Show resolved Hide resolved
intelmq/bots/outputs/rpz_file/output.py Outdated Show resolved Hide resolved
intelmq/bots/outputs/rpz_file/output.py Outdated Show resolved Hide resolved
@ghost ghost added the needs: feedback label Aug 20, 2021
@mariuskarotkis mariuskarotkis requested a review from a user August 26, 2021 09:10
@ghost ghost removed the needs: feedback label Sep 17, 2021
Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Apart from the minor change it looks good overall
I'll apply the change myself after merging, then we can finally conclude this as well. Sorry for the long waiting - with volatile priorities, it's not always easy to have enough time for the reviews. But I think that the current state is much better than what we had before.

class RpzFileOutputBot(OutputBot):
_file = None
format_filename: bool = False
__Bot_is_multithreadable = False
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
__Bot_is_multithreadable = False
_is_multithreadable = False

That was recently fixed/changed in 5c91a8c you could not have foreseen this

@ghost ghost merged commit dcf6f0e into certtools:develop Sep 17, 2021
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: bots feature Indicates new feature requests or new features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants