-
Notifications
You must be signed in to change notification settings - Fork 295
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bot debugging #975
Bot debugging #975
Conversation
You can always force push to your branch. |
(You are right, I gave up too early yesterday. :( :D ) I am working on the changes. Meanwhile, I've added another parameter, console, that would start an ipdb console in the bot. I find it very useful too. One doesn't have to write |
intelmq/bin/intelmqctl.py
Outdated
self.bot_stop(bot_id) | ||
else: | ||
paused = False | ||
self.logger.warning("Main instance of the bot is running in the background. You may want to launch: intelmqctl stop {}".format(bot_id)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I changed IntelMQProcessManager.bot_run
method so that it's not problem anymore to run multiple instance of bot.
If we run a bot that is already running, only warning "Main instance of the bot is running in the background. You may want to launch: intelmqctl stop {}"
is printed.
However, if there is a reason for the background instance is rather paused and relaunched after debugging is finished, I commented all the code so that I can restore it easily.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, now the background process will get stopped and relaunched afterwards.
Codecov Report
@@ Coverage Diff @@
## master #975 +/- ##
==========================================
- Coverage 77.98% 77.22% -0.77%
==========================================
Files 221 222 +1
Lines 9047 9197 +150
==========================================
+ Hits 7055 7102 +47
- Misses 1992 2095 +103
Continue to review full report at Codecov.
|
Everything's done. Please tell me if there is anything I can do more :) |
I'm currently traveling, I will review the code as soon as I have time again. You can feel certain that I am very eager to get this integrated! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How to quit the console without raising an Exception? Using quit()
, Ctrl+D and self.stop()
all lead to:
(Pdb) quit()
Traceback (most recent call last):
File "/usr/local/bin/intelmqctl", line 9, in <module>
load_entry_point('intelmq==1.0.0.dev7', 'console_scripts', 'intelmqctl')()
File "/home/sebastian/dev/intelmq/intelmq/bin/intelmqctl.py", line 902, in main
return x.run()
File "/home/sebastian/dev/intelmq/intelmq/bin/intelmqctl.py", line 548, in run
results = args.func(**args_dict)
File "/home/sebastian/dev/intelmq/intelmq/bin/intelmqctl.py", line 558, in bot_run
return self.bot_process_manager.bot_run(**kwargs)
File "/home/sebastian/dev/intelmq/intelmq/bin/intelmqctl.py", line 138, in bot_run
console_type, dryrun, message_action_kind, msg)
File "/home/sebastian/dev/intelmq/intelmq/lib/bot_debugger.py", line 51, in __init__
self._console(console_type)
File "/home/sebastian/dev/intelmq/intelmq/lib/bot_debugger.py", line 80, in _console
module.set_trace()
File "/usr/lib64/python3.4/bdb.py", line 52, in trace_dispatch
return self.dispatch_return(frame, arg)
File "/usr/lib64/python3.4/bdb.py", line 96, in dispatch_return
if self.quitting: raise BdbQuit
bdb.BdbQuit
And it seems the stop-method is not called in these cases, which is necessary for a proper shutdown of the bot.
docs/intelmqctl.md
Outdated
@@ -168,8 +276,8 @@ intelmqctl: Starting abusech-domain-parser... | |||
intelmqctl: abusech-domain-parser is running. | |||
intelmqctl: Starting abusech-feodo-domains-collector... | |||
intelmqctl: abusech-feodo-domains-collector is running. | |||
intelmqctl: Starting deduplicator-expert... | |||
intelmqctl: deduplicator-expert is running. | |||
intelmqctl: Starting file-output... |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why did you rename all these bot names? They do not make sense, as there is already a file-output in this example.
intelmq/bin/intelmqctl.py
Outdated
with open(filename, 'w') as fp: | ||
fp.write(str(os.getpid())) | ||
if pid and self.__status_process(pid): | ||
self.logger.warning("Main instance of the bot is running in the background. You may want to launch: intelmqctl stop {}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If there one bot is running twice in parallel, this will lead to duplicated messages as they are using the same (internal) source queue. Please restore the previous behavior.
Also see #710 (comment) and certat#92
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok then, now the background process will get stopped and relaunched afterwards.
# Never pops from source to internal queue, thx to disabling brpoplpush operation. | ||
# However, we have to wait manually till there is the message in the queue. | ||
pl = self.instance._Bot__source_pipeline | ||
pl.pipe.brpoplpush = lambda source_q, inter_q, i: pl.pipe.lindex(source_q, -1) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This does not read messages from the internal queue.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, corrected.
For parser bot, the messages are quite often too long. We can now just specify the filepath of a JSON than the JSON itself.
certtools#975 - debugger can accept files as messages certtools#993 - alienvault-otx conforming ParserBot
I've added a hint that "c" command will make you quit the console. However, if we break the process by Ctrl+D, the stopped bot won't get realunched. If that's a big problem, we may disable the possibility of to do |
This will be good to have. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We are almost there :)
intelmq/bin/intelmqctl.py
Outdated
fp.write(str(os.getpid())) | ||
if pid and self.__status_process(pid): | ||
self.logger.warning("Main instance of the bot is running in the background and will be stopped; " | ||
"when finished, we try to relaunch it again." |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing space between the two sentences
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where please? "and will be stopped; SPACE when finished"
I must be blind :D , I don't see where it's missing...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, I found it...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Either in this line at the end or at the beginning of the next one.
intelmq/bin/intelmqctl.py
Outdated
# log_bot_message('starting', bot_id) | ||
# filename = self.PIDFILE.format(bot_id) | ||
# with open(filename, 'w') as fp: | ||
# fp.write(str(os.getpid())) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Without writing the PID the bot can still be started in parallel leading to weird behavior and duplicate messages.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Got it! Thanks for reviewing me.
(This is a continuation of PR #973 , because I completely messed up a git history...)
BotDebugger is called via intelmqctl. It starts a live running bot instance,
leverages logging to DEBUG level and permits even a non-skilled programmer
who may find themselves puzzled with Python nuances and server deployment twists
to see what's happening in the bot and where's the error.
Depending on the subcommand received, the class either
Further help was added to argparse help of intelmqctl:
Possible commands:
There were commands I always wanted to have. I missed them when creating/quickly debugging the bots. If you find them useful, too, I'd be very glad to publish it in the main repository. I am open to any discussion concerning the new commands.
Wagner's response:
Dear @wagner-certat , please respond to my 3 little discussions:
Then I have to finish this: