Add runtime_mappings to search request

[nik9000]

This adds a way to specify the runtime_mappings on a search request
which are always "runtime" fields. It looks like:

curl -XDELETE -uelastic:password -HContent-Type:application/json localhost:9200/test
curl -XPOST -uelastic:password -HContent-Type:application/json 'localhost:9200/test/_bulk?pretty&refresh' -d'
{"index": {}}
{"animal": "cat", "sound": "meow"}
{"index": {}}
{"animal": "dog", "sound": "woof"}
{"index": {}}
{"animal": "snake", "sound": "hisssssssssssssssss"}
'

curl -XPOST -uelastic:password -HContent-Type:application/json localhost:9200/test/_search?pretty -d'
{
  "runtime_mappings": {
    "animal.upper": {
      "type": "keyword",
      "script": "for (String s : doc[\"animal.keyword\"]) {emit(s.toUpperCase())}"
    }
  },
  "query": {
    "match": {
      "animal.upper": "DOG"
    }
  }
}'

NOTE:
If we have to send a search request with runtime mappings to a node that
doesn't support runtime mappings at all then we'll fail the search
request entirely. The alternative would be to not send those runtime
mappings and let the node fail the search request with an "unknown field"
error. I believe this is would be hard to surprising because you defined
the field in the search request.

NOTE:
It isn't obvious but you can also use runtime_mappings to override fields
inside objects by naming the runtime fields with . in them. Like this:

curl -XDELETE -uelastic:password -HContent-Type:application/json localhost:9200/test
curl -uelastic:password -XPOST -HContent-Type:application/json localhost:9200/test/_bulk?refresh -d'
{"index":{}}
{"name": {"first": "Andrew", "last": "Wiggin"}}
{"index":{}}
{"name": {"first": "Julian", "last": "Delphiki", "suffix": "II"}}
'

curl -uelastic:password -XPOST -HContent-Type:application/json localhost:9200/test/_search?pretty -d'{
  "runtime_mappings": {
    "name.first": {
      "type": "keyword",
      "script": "if (\"Wiggin\".equals(doc[\"name.last.keyword\"].value)) {emit(\"Ender\");} else if (\"Delphiki\".equals(doc[\"name.last.keyword\"].value)) {emit(\"Bean\");}"
    }
  },
  "query": {
    "match": {
      "name.first": "Bean"
    }
  }
}'

Relates to #59332

[nik9000]

Draft because I expect @javanna and I to do a fair bit of iteration on this before we get it in.

This doesn't work with the fields option and, I think, it doesn't work with the fetch phase in general. But it seems to work for queries! So that is a start.

[javanna]

thanks @nik9000 I think that this is how it should look. It requires a bit more refactoring to make sure that when we lookup field types in the fetch phase, we use the method that knows about runtime mappings. I am on it, I think.

[javanna]

nice tricks! ;)

[javanna]

When I looked I think that this one gets a search source builder, so it seemed that runtime mappings can be specified, hence we should use them?

[nik9000]

@javanna and I talked about this offline - I believe we'll have to modify the method that looks like simpleMatchToFullName to pick up runtime fields as well.

[nik9000]

I've resolved the simpleMatchToFullName issues and pushed more on the tests. I'm down to 10 failures when running the core tests with search-time runtime fields.

[nik9000]

What do we expect to happen when folks do this:

curl -uelastic:password -XPOST -HContent-Type:application/json localhost:9200/test/_search?pretty -d'{
  "runtime_mappings": {
    "name": {
      "type": "object",
      "properties": {
        "first": {
          "type": "keyword",
          "script": "script"
        }
      }
    }
  },
  "query": {
    "match": {
      "name.first": "val"
    }
  }
}'

Right now you get a kind of silly error. I can make the error nicer. Or I could try and make it work?

[javanna]

@nik9000 users won't be able to create objects in the runtime mappings, but rather only runtime fields, eventually with fields in their names. If you throw an error now, that's good enough. With the introduction of the runtime section the parsers for runtime fields will be separate from the others hence this will be handled there.

[javanna]

@nik9000 maybe also mention overriding/shadowing in the description of the PR? For instance an example of shadowing a field within an object would help I think

[nik9000]

@nik9000 users won't be able to create objects in the runtime mappings, but rather only runtime fields, eventually with fields in their names. If you throw an error now, that's good enough. With the introduction of the runtime section the parsers for runtime fields will be separate from the others hence this will be handled there.

🤘 I'll make the error message reasonable and we can see what happens when we merge the runtime sections.

[javanna]

LGTM thanks @nik9000 !