Infinite seeking for duration in MPEG-TS with fixed Content-Length

[stari4ek]

[REQUIRED] Issue description

MPEG-TS stream (as http progressive) served with fixed Content-Length (ignoring Range) (imitating live streaming?) never finishes preparing stage keeping seeking in TsDurationReader.readLastPcrValue():

input.getPosition()=14186      searchStartPosition=899887200  | timestampSearchBytes=112800 inputLength=900000000   bytesToSearch=112800
input.getPosition()=899887200  searchStartPosition=1799774400 | timestampSearchBytes=112800 inputLength=1799887200  bytesToSearch=112800
input.getPosition()=1799774400 searchStartPosition=2699661600 | timestampSearchBytes=112800 inputLength=2699774400  bytesToSearch=112800
input.getPosition()=2699661600 searchStartPosition=3599548800 | timestampSearchBytes=112800 inputLength=3599661600  bytesToSearch=112800
input.getPosition()=3599548800 searchStartPosition=4499436000 | timestampSearchBytes=112800 inputLength=4499548800  bytesToSearch=112800
input.getPosition()=4499436000 searchStartPosition=5399323200 | timestampSearchBytes=112800 inputLength=5399436000  bytesToSearch=112800
input.getPosition()=5399323200 searchStartPosition=6299210400 | timestampSearchBytes=112800 inputLength=6299323200  bytesToSearch=112800
input.getPosition()=6299210400 searchStartPosition=7199097600 | timestampSearchBytes=112800 inputLength=7199210400  bytesToSearch=112800
input.getPosition()=7199097600 searchStartPosition=8098984800 | timestampSearchBytes=112800 inputLength=8099097600  bytesToSearch=112800
input.getPosition()=8098984800 searchStartPosition=8998872000 | timestampSearchBytes=112800 inputLength=8998984800  bytesToSearch=112800
input.getPosition()=8998872000 searchStartPosition=9898759200 | timestampSearchBytes=112800 inputLength=9898872000  bytesToSearch=112800
input.getPosition()=9898759200 searchStartPosition=10798646400| timestampSearchBytes=112800 inputLength=10798759200 bytesToSearch=112800

This is clearly an issue on IPTV server side, but it would be nice if ExoPlayer stand firm against flawed media.

[REQUIRED] Reproduction steps

• demo player
• play MPEG-TS stream which is served with constant Content-Length: 900000000

[REQUIRED] Link to test content

sent to email

[REQUIRED] A full bug report captured from the device

not applicable

[REQUIRED] Version of ExoPlayer being used

• a3a5760 (current dev-v2 head)

[REQUIRED] Device(s) and version(s) of Android being used

not applicable

[stari4ek]

This is how http communication looks like:

--> GET <redacted>/?f=TraceUrban_SD
Icy-MetaData: 1
Accept-Encoding: identity
--> END GET

<-- 200 OK <redacted>/TraceUrban_SD.ts? (553ms)
Content-Type: video/MP2T
Cache-Control: no-cache
Content-Range: bytes */900000000
Accept-Ranges: bytes
Content-Length: 900000000
Date: Mon, 19 Oct 2020 13:32:12 GMT
Connection: keep-alive
<-- END HTTP


16:34:57.499 --> GET <redacted>/?f=TraceUrban_SD
16:34:57.502 Icy-MetaData: 1
16:34:57.505 Range: bytes=899887200-
16:34:57.510 Accept-Encoding: identity
16:34:57.515 --> END GET

16:34:58.371 <-- 200 OK <redacted>/TraceUrban_SD.ts? (852ms)
16:34:58.374 Content-Type: video/MP2T
16:34:58.376 Cache-Control: no-cache
16:34:58.379 Content-Range: bytes */900000000
16:34:58.381 Accept-Ranges: bytes
16:34:58.384 Content-Length: 900000000
16:34:58.387 Date: Mon, 19 Oct 2020 13:34:58 GMT
16:34:58.389 Connection: keep-

[stari4ek]

This issue can be identified by checking input size in TsDurationReader.readLastPcrValue. If size is growing with each attempt - we will never reach searchStartPosition

if okhttp is used instead of cronet used in demo, input length stays fixed.
it hangs in

input.peekFully(packetBuffer.getData(), /* offset= */ 0, bytesToSearch);
...
  OkHttpDataSource.skipInternal()

BTW, it looks like okhttp data source handles 200/206 status code properly, while cronet does not - input size grows over time.

[stari4ek]

After digging around I'm not sure how it can be solved/workarounded properly on ExoPlayer side.
I've added additional handling to bypass it:

• block huge seek-by-reading (OkHttpDataSource.skipInternal()).
• finish duration reading in TsDurationReader.readLastPcrValue() if input reading failed. Another attempt to load will be performed with duration reading disabled

[andrewlewis]

Could you describe where the bug is on the ExoPlayer side? Based on the block in your first comment it looks like the input length is increasing past the content length returned by the server, which looks wrong. TsExtractor won't try to determine the duration by reading the end of the file if the data source has an unset length (for progressively growing streams).

[stari4ek]

I was checking this issue in 2 cases:

• demo player, which is configured by default with cronet + caching data source
• exoplayer configured to use okhttp data source

In both cases ExoPlayer hangs in preparation stage.
Http log from my comment is valid for both cases. In fact media is MPEG-TS live stream, but server serves it as fixed-size (900MB) without ranged requests support.

First log with growing inputLength is from first case (demo player, cronet). After checking it with demo I switched to testing it with okhttp, which has next block:

// OkHttpDataSource.open()

// If we requested a range starting from a non-zero position and received a 200 rather than a
// 206, then the server does not support partial requests. We'll need to manually skip to the
// requested position.
bytesToSkip = responseCode == 200 && dataSpec.position != 0 ? dataSpec.position : 0;

And inputLength (in TsDurationReader.readLastPcrValue()) stays fixed (900000000), but exoplayer hangs identifying duration due another issue.

So, here is "issue" on ExoPlayer side:

• when cronet (wrapped to cache data source) is used - it does not handle case when "200 OK" is returned instead of "206 Partial Content" for range request. So, TsExtractor/TsDurationReader will seek infinitely.

And here is "improvement" for ExoPlayer. If issue above is fixed or okhttp data source is used:

• ExoPlayer hangs trying to identify MPEG-TS duration (TsExtractor/TsDurationReader) if server does not support range requests. It hangs cause it tries to skip to the end of stream by reading through it, which can be huge (900Mb here) and if there is no enough bandwidth from server or stream is live (=low bandwidth) it will take too long.

With provided media server declares it's content as 900Mb and it does not support ranged request (despite it's Accept-Ranges: bytes), so when ExoPlayer tries to play it - it hangs while identifying MPEG-TS duration.

[ojw28]

When cronet (wrapped to cache data source) is used - it does not handle case when "200 OK" is returned instead of "206 Partial Content" for range request

We'll fix this. Thanks for reporting!

ExoPlayer hangs trying to identify MPEG-TS duration (TsExtractor/TsDurationReader) if server does not support range requests. It hangs cause it tries to skip to the end of stream by reading through it, which can be huge (900Mb here) and if there is no enough bandwidth from server or stream is live (=low bandwidth) it will take too long.

I don't think it's really worth spending time on trying to handle this case. It's really an issue with the server, and it's not something we've seen reported before. Any client side solution would also need to pick an arbitrary threshold of some kind at which to "give up".

[ojw28]

Marking as a bug to resolve the CronetDataSource issue only.