[api] Add batch support to TextEmbeddingServingTranslator

[frankfliu]

• [api] Support "inputs" key in json request for text embedding translator
• [api] Add batch support to TextEmbeddingServingTranslator

Description

Brief description of what this PR is about

• If this change is a backward incompatible change, why must this change be made?
• Interesting edge cases to note here

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 92.07921% with 8 lines in your changes are missing coverage. Please review.

Project coverage is 72.29%. Comparing base (ae00167) to head (3fef316).
Report is 10 commits behind head on master.

Files	Patch %	Lines
...nlp/translator/TextEmbeddingServingTranslator.java	90.00%	4 Missing ⚠️
api/src/main/java/ai/djl/inference/Predictor.java	85.00%	3 Missing ⚠️
.../src/main/java/ai/djl/modality/nlp/TextPrompt.java	95.83%	0 Missing and 1 partial ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff            @@
##             master    #3084   +/-   ##
=========================================
  Coverage     72.28%   72.29%           
- Complexity     7302     7324   +22     
=========================================
  Files           722      723    +1     
  Lines         32593    32677   +84     
  Branches       3410     3427   +17     
=========================================
+ Hits          23561    23624   +63     
- Misses         7407     7432   +25     
+ Partials       1625     1621    -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[siddvenk]

does it make sense to report the average latency per item as the preprocess latency? do we also report the batch size metric somewhere?

I feel like we should report the total time for the full batch, but if this is consistent with how we're doing it elsewhere, or what users typically expect, i'm ok with it. same foes for the other methods we report a similar metric for

[frankfliu]

try to align with TEI for batch case. We do report DynamicBatchSize in metrics. We have total inference time as well