Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for timestamps with signed tz offsets #1389

Merged
merged 2 commits into from
Oct 1, 2020

Conversation

henridf
Copy link
Contributor

@henridf henridf commented Sep 29, 2020

Suricata outputs timestamp like 2015-04-13T11:32:45.143323-0200,
whereas Zeek does 2015-04-13T11:32:45.143323Z. This commit lets
us parse both.

zio/ndjsonio/ndjson_test.go Outdated Show resolved Hide resolved
@henridf henridf requested a review from a team September 29, 2020 15:30
Suricata outputs timestamp like 2015-04-13T11:32:45.143323+0200,
whereas Zeek does 2015-04-13T11:32:45.143323Z02:00. This commit lets
us parse both.
@henridf henridf merged commit 2f8d907 into master Oct 1, 2020
@henridf henridf deleted the json-suricata-timestamps branch October 1, 2020 20:51
brim-bot pushed a commit to brimdata/zui that referenced this pull request Oct 1, 2020
… by henridf

This is an auto-generated commit with a zq dependency update. The zq PR
brimdata/super#1389, authored by @henridf,
has been merged.

Add support for timestamps with signed tz offsets

Suricata outputs timestamp like `2015-04-13T11:32:45.143323-0200`,
whereas Zeek does `2015-04-13T11:32:45.143323Z02:00`. This commit lets
us parse both.
brim-bot pushed a commit to brimdata/zui that referenced this pull request Oct 5, 2020
This is an auto-generated commit with a zq dependency update. The zq PR
brimdata/super#1400, authored by @henridf,
has been merged.

Convert eve.json upon ingest

This PR adds a json type config that is suited for "alert" events output by the Brim suricata package. The config is used when transforming the Suricata-produced "eve.json" file. In addition, the suricata `timestamp` field is renamed to `ts`.

This PR is based on brimdata/super#1389 so if you come here first, please try to make the detour.

Closes brimdata/super#1213
@philrz
Copy link
Contributor

philrz commented Oct 6, 2020

Verified in zq commit af6d47e. Both formats are now accepted.

$ cat types-time.json
{
  "descriptors": {
    "timerecord": [
      {
        "name": "_path",
        "type": "string"
      },
      {
        "name": "ts",
        "type": "time"
      }
    ]
  },
  "rules": [
    {
      "descriptor": "timerecord",
      "name": "_path",
      "value": "timerecord"
    }
  ]
}

$ cat data.ndjson 
{"_path": "timerecord", "ts": "2015-04-13T11:32:45.143323Z"}
{"_path": "timerecord", "ts": "2015-04-13T11:32:45.143323-0200"}

$ zq -t -j types-time.json data.ndjson 
#0:record[_path:string,ts:time]
0:[timerecord;1428924765.143323;]
0:[timerecord;1428931965.143323;]

Thanks @henridf!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants