Vector doesnt ingest syslogs

[DuvelCorp]

A note for the community

No response

Problem

Not sure if this is a bug or a misconfiguration on my side as I am new to vector

I've got my pfsense router that sends syslogs (RFC 5424) UDP on a Ubuntu VM 192.168.1.3:514
Previously I had a syslog-ng on that machine listening to 514 and sending syslog to Promtail that send them to Loki. Everything works.

Now I want to use Vector only to ingest syslog itself, and send them to Loki, after enriching with geoip data (I thus stopped the syslog-ng and Promtail containers)

Note that everything listed above (except the pfsense) runs in docker on the target syslog machine 192.168.1.3, and on the same docker network.

My config is:

  sources:
    pfsense_syslog:
      type: syslog
      address: 0.0.0.0:514
      mode: udp

  enrichment_tables:
    geoip_table:
      path: /usr/share/GeoIP/GeoLite2-City.mmdb
      type: geoip
    geoip_table_asn:
      path: /usr/share/GeoIP/GeoLite2-ASN.mmdb
      type: geoip      

  transforms:
    pfsense_filterlog:
      type: "remap"
      inputs:
        - pfsense_syslog
      source: |-
        if .appname == "filterlog" {
          message_array = parse_csv!(string!(.message))
          # ipv4 and ipv6
          if message_array[8] == "4" || message_array[8] == "6" {
            .filteriface   = message_array[4]
            .filterflow    = message_array[7]
            .filteraction  = message_array[6]
            .filteripver   = message_array[8]
            # ipv4
            if message_array[8] == "4" {
              .filterproto   = message_array[16]
              .filterflags   = message_array[23]
              .filtersrcip   = message_array[18]
              .filtersrcport = message_array[20]
              .filterdstip   = message_array[19]
              .filterdstport = message_array[21]
            # ipv6
            } else if message_array[8] == "6" {
              .filterproto   = message_array[12]
              #.filterflags   = "IPv6 not fully supported"
              .filtersrcip   = message_array[15]
              .filtersrcport = message_array[17]
              .filterdstip   = message_array[16]
              .filterdstport = message_array[18]
            }
          }
        }


    pfsense_geotag:
      type: "remap"
      inputs:
        - "pfsense_filterlog"
      source: |-
        .src_geoip = get_enrichment_table_record!("geoip_table",
          {
            "ip": .filtersrcip
          }
        )
        .src_geoip_asn = get_enrichment_table_record!("geoip_table_asn",
          {
            "ip": .filtersrcip
          }
        )     

        .dst_geoip = get_enrichment_table_record!("geoip_table",
          {
            "ip": .filterdstip
          }
        )

        .dst_geoip_asn = get_enrichment_table_record!("geoip_table_asn",
          {
            "ip": .filterdstip
          }
        )        

  sinks:
    loki_pfsense:
      type: "loki"
      inputs:
        - "pfsense_geotag"
      encoding:
        codec: "json"
      endpoint: "http://192.168.1.3:3100"
      labels:
        service: pfsense

    debug_mode:
      type: console
      inputs:
        - pfsense_syslog
      encoding:
        codec: "json"
       

Container logs:

2024-09-22T16:58:16.367055Z  INFO vector: Vector has reloaded. path=[File("/etc/vector/vector.yaml", Some(Yaml))]

2024-09-22T17:00:39.564279Z  INFO vector::config::watcher: Configuration file changed.

2024-09-22T17:00:39.567012Z  INFO vector::topology::running: Reloading running topology with new configuration.

2024-09-22T17:00:39.567445Z  INFO vector::topology::running: Running healthchecks.

2024-09-22T17:00:39.567641Z  INFO vector::topology::running: New configuration loaded successfully.

2024-09-22T17:00:39.567702Z  INFO vector: Vector has reloaded. path=[File("/etc/vector/vector.yaml", Some(Yaml))]

2024-09-22T17:00:39.567809Z  INFO source{component_kind="source" component_id=pfsense_syslog component_type=syslog}: vector::sources::syslog: Listening. addr=0.0.0.0:514 type="udp"

And that is all. And it stays idle like this during hours, while I do have a bombing of syslogs every second.
It just does nothing, and more frustrating: it just says nothing.

I tried to change, step-by-step:

• protocol to tcp
• type to socket, udp and tcp
• Syslogs format to RFC 3164 in pfsense.

Nothing changes, it still does nothing.

Finally, I then tried this ultimate test : I re-enabled the syslog-ng container that listen to UDP 514, after having changed its config to make it write the syslogs on disk, AND to process them to port TCP 1514.
And I adapted the vector config accordingly, to :

  sources:
    pfsense_syslog:
      type: syslog
      address: 0.0.0.0:1514
      mode: tcp

After a restart of syslog-ng and vector containers, I can see the syslogs written on disk, but still nothing happens in Vector. No error, no hint, nothing ! Just the same idle logs

2024-09-22T19:42:08.190544Z  INFO vector::config::watcher: Configuration file changed.

2024-09-22T19:42:08.265461Z  INFO vector::topology::running: Reloading running topology with new configuration.

2024-09-22T19:42:08.284013Z  INFO vector::topology::running: Running healthchecks.

2024-09-22T19:42:08.284885Z  INFO vector::topology::running: New configuration loaded successfully.

2024-09-22T19:42:08.284966Z  INFO vector: Vector has reloaded. path=[File("/etc/vector/vector.yaml", Some(Yaml))]

2024-09-22T19:42:08.285799Z  INFO source{component_kind="source" component_id=pfsense_syslog component_type=syslog}: vector::sources::util::net::tcp: Listening. addr=0.0.0.0:1514

So I don't know what the heck is this and I hope you can help me to figure it out.

The syslogs do arrive on the server on the correct port, that is crystal clear, so why Vector doesn't see anything, and overall, without giving any error???

Configuration

No response

Version

timberio/vector:latest-alpine

Debug Output

No response

Example Data

No response

Additional Context

No response

References

No response

[jszwedko]

Hi @DuvelCorp !

Thanks for opening this. From what you've described so far, it sounds like the packets may never be making it to Vector (which is why Vector isn't logging anything). Have you verified that you can send packets directly from the host the sender is on to Vector? I'd suggest trying to use netcat to do so. You could also try using netcat to send packets from the host that Vector is running on to see if it works over localhost.