Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: metrics, logon-summary and csv-timline show different total event records #1105

Closed
YamatoSecurity opened this issue Jun 22, 2023 · 2 comments · Fixed by #1106
Closed
Assignees
Labels
bug Something isn't working
Milestone

Comments

@YamatoSecurity
Copy link
Collaborator

When running the metrics, logon-summary and csv-timeline commands, the total event records count will be different.
Test:
./hayabusa-2.6.0-mac-intel metrics -d ../hayabusa-sample-evtx -o test.csv -C

./hayabusa-2.6.0-mac-intel logon-summary -d ../hayabusa-sample-evtx -o test.csv -C

./hayabusa-2.6.0-mac-intel csv-timeline -d ../hayabusa-sample-evtx -o test.csv -C

Results:
logon-summary: 94950
metrics: 47475
csv-timeline: 47,476

Also, there are no commas displayed with logon-summary and metrics so it is difficult to read the number when the event record count is in the millions. (ie. 234723049) So I would like to add commas like in csv-timeline (-> 234,723,049)

@hitenkoku Could you look at this whenever you have time? (I think you are the most knowledgable about this)

@YamatoSecurity YamatoSecurity added the bug Something isn't working label Jun 22, 2023
@YamatoSecurity YamatoSecurity added this to the v2.7.0 milestone Jun 22, 2023
@hitenkoku hitenkoku self-assigned this Jun 22, 2023
@hitenkoku
Copy link
Collaborator

hitenkoku commented Jun 23, 2023

`
@YamatoSecurity

The cause of the problem has been identified.
The logon-summary is double the number of records in metrics because it was double counted with the metrics tally.

The difference between metrics and csv-timeline is that the record count in metrics was counting the number of records after channel name and EID filtering.
The number in csv-timeline is the original record count.

Filtered record is following, filtered by Event>System>Channel is not found.

{"Event": Object {"SubscriptionBookmarkEvent": Object {"SubscriptionId": Null}, "System": Object {"Computer": String("dhcp01.offsec.lan"), "EventID": String("111"), "Provider_attributes": Object {"Name": String("Microsoft-Windows-EventForwarder")}, "TimeCreated_attributes": Object {"SystemTime": String("2021-04-27T15:03:16.983Z")}}}, "Event_attributes": Object {"xmlns": String("http://schemas.microsoft.com/win/2004/08/events/event")}}

@hitenkoku
Copy link
Collaborator

hitenkoku commented Jun 23, 2023

@YamatoSecurity There are two ways to fix it: 1. for 2. it affects the % of metrics and logon-summary display; and 2. for 3. it affects the % of logon-summary display. If you want to avoid affecting the % notation, need to separate the display variables from the aggregate variables. (In #1106, This adjusted)

  1. count the number of records in csv-timeline as the number of records after filtering for Channel and EID, instead of the total number of records
  2. the number of records in logon-summary and metrics should be the same as the overall number of records used in csv-timeline (this is currently done in the implementation at fixed metrics and logon-summary records and csv-timline show different total event records #1106) currently implemented at )

hitenkoku added a commit that referenced this issue Jun 23, 2023
hitenkoku added a commit that referenced this issue Jun 23, 2023
hitenkoku added a commit that referenced this issue Jun 23, 2023
#1105):

fix(main/metrics/timelines): separate variables for total and aggregated records in metrics and logon-summary #1105
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
2 participants