Serverless API integration tests (elastic#192219)

Tests for serverless

- copies over and modifies all tests from stateful to work in
serverless. ~~deployment agnostic tests do not yet support enterprise
license for stateful, so are tests don't yet qualify as being deployment
agnostic~~. Given how difficult it is to see differences from the
stateful tests, I've added PR comments where I've changed something that
might be of interest.
- changes to `createObservabilityAIAssistantApiClient` to use supertest
without basic auth and accept headers for serverless and use roles
- removes creating persisted users when tests start and [use
roles](https:/elastic/kibana/blob/main/x-pack/test_serverless/README.md#roles-based-testing)
within tests. its not possible to create custom users with the
serverless test framework at the moment. See
elastic#192711

Skipped tests
- knowledge base tests elastic#192886
- any test suite that uses the LLM proxy has been skipped on MKI
elastic#192751
- all tests that depend on the config.modelId skipped in MKI
elastic#192757

TODO:

- [x] move over remaining tests
- [x]  test in MKI environment before merging
- [x] create issues for skipped tests
- [ ] this will not run on MKI (after merging) unless we ping the
appex-qa team to add it to the pipeline. this is due to creating a
separate config. ask appex-qa team to add our config.

Followup / related issues to be tracked in a newly created issue:

- [ ] elastic#192757
- [ ] elastic#192886
- [ ] elastic#192751
- [ ] elastic#192701
- [ ] elastic#192497
- [ ] elastic#192711
- [ ] elastic#192718
- [ ] serverless functional tests
- [ ] inquire with ml-ui-team to have the ability to delete system
indices which we do after uninstalling tiny elser with .ml indices

.buildkite/ftr_oblt_serverless_configs.yml

@@ -9,6 +9,7 @@ disabled:
  - x-pack/test_serverless/api_integration/test_suites/observability/config.feature_flags.ts
  - x-pack/test_serverless/api_integration/test_suites/observability/common_configs/config.group1.ts
  - x-pack/test_serverless/api_integration/test_suites/observability/fleet/config.ts
+ - x-pack/test_serverless/api_integration/test_suites/observability/ai_assistant/config.ts
  - x-pack/test_serverless/functional/test_suites/observability/config.ts
  - x-pack/test_serverless/functional/test_suites/observability/config.examples.ts
  - x-pack/test_serverless/functional/test_suites/observability/config.feature_flags.ts