Shodan parser / mapping

[ghost]

shodan provides a hierarchical dictionary with a lot of fields. This is a list of fields we could possibly use for our internal format:

• hostnames: list of hostnames
• org: event_description.target
• port: source.port
• transport: protocol.transport
• http.location: source.urlpath (add urlpath harmonization type #1039)
• asn: source.asn (prefixed with AS)
• location.city, country_code, longitude, latitude: source.geolocation.cc
• timestamp: time.source
• ip_str: source.ip
• opts (.vulns): can be used for classification and event_description
• opts.screenshot: can be saved somehow "in" screenshot_url
• http: if available, protocol.application is http(s)
• _shodan.module: probably too
  I am not sure yet how we can how to derive the classification.

and everything else to extra

The collection can be done with either the http stream collector for the stream or the http collector for queries