Add multiline option in csv timeline command

[hitenkoku]

What Changed

• Add multiline option in csv timeline command

Evidence

・ help output

>./972.exe help csv-timeline
Hayabusa v2.3.1
Yamato Security (https:/Yamato-Security/hayabusa) @SecurityYamato)

Usage:
  hayabusa.exe csv-timeline <INPUT> [OPTIONS]

Input:
  -d, --directory <DIR>  Directory of multiple .evtx files
  -f, --file <FILE>      File path to one .evtx file
  -l, --live-analysis    Analyze the local C:\Windows\System32\winevt\Logs folder
  -J, --JSON-input       Scan JSON formatted logs instead of .evtx (.json or .jsonl)

Output:
  -G, --GeoIP <MAXMIND-DB-DIR>  Add GeoIP (ASN, city, country) info to IP addresses
  -H, --HTML-report <FILE>      Save Results Summary details to an HTML report (ex: results.html)
      --multiline               Output event field information in multiple rows
  -o, --output <FILE>           Save the timeline in CSV format (ex: results.csv)
  -p, --profile <PROFILE>       Specify output profile

Display Settings:
      --no-color            Disable color output
      --no-summary          Do not display Results Summary (slightly faster speed)
  -q, --quiet               Quiet mode: do not display the launch banner
  -v, --verbose             Output verbose information
  -T, --visualize-timeline  Output event frequency timeline (terminal needs to support unicode)

Filtering:
  -E, --EID-filter                Scan only common EIDs for faster speed (./rules/config/target_event_IDs.txt)
  -D, --enable-deprecated-rules   Enable rules with status of deprecated
  -n, --enable-noisy-rules        Enable rules set to noisy (./rules/config/noisy_rules.txt)
  -u, --enable-unsupported-rules  Enable rules with status of unsupported
  -e, --exact-level <LEVEL>       Scan for only specific levels (informational, low, medium, high, critical)
      --exclude-status <STATUS>   Ignore rules according to status (ex: experimental) (ex: stable,test)
  -m, --min-level <LEVEL>         Minimum level for rules (default: informational)
      --timeline-end <DATE>       End time of the event logs to load (ex: "2022-02-22 23:59:59 +09:00")
      --timeline-start <DATE>     Start time of the event logs to load (ex: "2020-02-22 00:00:00 +09:00")

General Options:
  -Q, --quiet-errors                     Quiet errors mode: do not save error logs
  -r, --rules <DIR/FILE>                 Specify a custom rule directory or file (default: ./rules)
  -c, --rules-config <DIR>               Specify custom rule config directory (default: ./rules/config)
      --target-file-ext <EVTX_FILE_EXT>  Specify additional file extensions (ex: evtx_data) (ex: evtx1,evtx2)
  -t, --threads <NUMBER>                 Number of threads (default: optimal number for performance)

Time Format:
      --European-time     Output timestamp in European time format (ex: 22-02-2022 22:00:00.123 +02:00)
      --ISO-8601          Output timestamp in ISO-8601 format (ex: 2022-02-22T10:10:10.1234567Z) (Always UTC)
      --RFC-2822          Output timestamp in RFC 2822 format (ex: Fri, 22 Feb 2022 22:00:00 -0600)
      --RFC-3339          Output timestamp in RFC 3339 format (ex: 2022-02-22 22:00:00.123456-06:00)
      --US-military-time  Output timestamp in US military time format (ex: 02-22-2022 22:00:00.123 -06:00)
      --US-time           Output timestamp in US time format (ex: 02-22-2022 10:00:00.123 PM -06:00)
  -U, --UTC               Output time in UTC format (default: local time)

・This PR

> ./972.exe csv-timeline --multiline -d ..\all-evtx\ -o 972.csv -p super-verbose -q --debug
...
Total event log files: 1858
Total file size: 6.1 GB

Loading detections rules. Please wait.

Excluded rules: 20
Noisy rules: 7 (Disabled)

Deprecated rules: 150 (4.34%) (Disabled)
Experimental rules: 1760 (50.93%)
Stable rules: 223 (6.45%)
Test rules: 1473 (42.62%)
Unsupported rules: 43 (1.24%) (Disabled)

Hayabusa rules: 149
Sigma rules: 3307
...
Results Summary:

First Timestamp: 2009-07-14 13:56:45.074 +09:00
Last Timestamp: 2022-09-18 23:37:13.088 +09:00

Events with hits / Total events: 1,594,356 / 4,817,181 (Data reduction: 3,222,825 events (66.90%))

Total | Unique detections: 1,627,665 | 152
Total | Unique critical detections: 0 (0.00%) | 0 (0.00%)
Total | Unique high detections: 12,043 (0.74%) | 20 (13.16%)
Total | Unique medium detections: 11,015 (0.68%) | 42 (27.63%)
Total | Unique low detections: 1,054,520 (64.79%) | 40 (26.32%)
Total | Unique informational detections: 550,087 (33.80%) | 50 (32.89%)

...
Saved file: 972.csv (2.1 GB)
Elapsed time: 00:04:37.194
Rule Parse Processing Time: 00:00:00.851
Analysis Processing Time: 00:04:09.165
Output Processing Time: 00:00:27.175

Memory usage stats:
heap stats:    peak      total      freed    current       unit      count
  reserved:    8.2 GiB    8.2 GiB   83.0 MiB    8.1 GiB
 committed:    7.5 GiB   66.1 GiB   58.7 GiB    7.3 GiB
     reset:      0          0          0          0                            ok
   touched:   64.2 KiB   16.5 MiB   55.6 GiB  -55.6 GiB                        ok
  segments:     14        264        255          9                            not all freed!
-abandoned:      0          0          0          0                            ok
   -cached:      0          0          0          0                            ok
     pages:      0          0      732.1 Ki  -732.1 Ki                         ok
-abandoned:      0          0          0          0                            ok
 -extended:      0
 -noretire:      0
     mmaps:      0
   commits:   54.3 Ki
   threads:     32         32          0         32                            not all freed!
  searches:     0.0 avg
numa nodes:       1
   elapsed:     277.207 s

[codecov]

Codecov Report

Patch coverage: 99.11% and project coverage change: +0.45 🎉

Comparison is base (996c919) 75.30% compared to head (fcc2299) 75.75%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #973      +/-   ##
==========================================
+ Coverage   75.30%   75.75%   +0.45%     
==========================================
  Files          24       24              
  Lines       16760    17088     +328     
==========================================
+ Hits        12621    12945     +324     
- Misses       4139     4143       +4     

Impacted Files	Coverage Δ
src/detections/configs.rs	55.97% <66.66%> (+<0.01%)	⬆️
src/afterfact.rs	46.91% <99.68%> (+8.92%)	⬆️
src/detections/detection.rs	69.41% <100.00%> (+0.06%)	⬆️
src/detections/rule/condition_parser.rs	96.96% <100.00%> (+<0.01%)	⬆️
src/detections/rule/count.rs	93.55% <100.00%> (+<0.01%)	⬆️
src/detections/rule/matchers.rs	96.43% <100.00%> (+<0.01%)	⬆️
src/detections/rule/mod.rs	94.59% <100.00%> (+<0.01%)	⬆️
src/detections/rule/selectionnodes.rs	90.67% <100.00%> (+0.01%)	⬆️
src/main.rs	27.25% <100.00%> (+0.05%)	⬆️
src/options/htmlreport.rs	100.00% <100.00%> (ø)
... and 2 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[YamatoSecurity]

Looks great! Thank you!
It will be a very useful option so I added a -M shorthand.

[hitenkoku]

Thank you for your review.

I will merge it .