Infinite loop loading first fragment with a hole

[mattzucker]

What version of Hls.js are you using?

1.4.7

What browser (including version) are you using?

Chrome 114.0.5735.135 (Official Build) (64-bit)

What OS (including version) are you using?

Windows 10

Test stream

https://raw.githubusercontent.com/mattzucker/hls-test-data/main/clean-bunny/bunny.m3u8

Configuration

{
  "debug": true,
  "enableWorker": true,
  "lowLatencyMode": true,
  "backBufferLength": 30
}

Additional player setup steps

I have to work with possibly corrupted video files. In this case, the first few bytes of a segment may be corrupt or missing, which results in a "hole". The sample stream that I provided has this happen on the very first file in the playlist, and you can see when the player loads that it starts the video buffer at 3.70833 instead of 0.

Checklist

• The issue observed is not already reported by searching on Github under https:/video-dev/hls.js/issues
• The issue occurs in the stable client (latest release) on https://hlsjs.video-dev.org/demo and not just on my page
• The issue occurs in the latest client (main branch) on https://hlsjs-dev.video-dev.org/demo and not just on my page
• The stream has correct Access-Control-Allow-Origin headers (CORS)
• There are no network errors such as 404s in the browser console when trying to play the stream

Steps to reproduce

1. Load the example stream.
2. The video player does not automatically start playing the video like it normally does, but click the play button and it will correctly skip over the buffer hole.
3. Seek to a part further in the video such that the first fragment is evicted from the buffer.
4. Seek to the buffer hole, meaning seek to a time in between 0 and 3.70833.
5. Entered an infinite loop of attempting to load the first fragment, but then calling this.backtrack(frag) and evicting it, then repeating.

Expected behaviour

I expect the first fragment to load and skip past the buffer hole, just like it does the first time.

What actually happened?

The fragment is never buffered and enters an infinite loop of downloading and evicting.

Console output

[log] > injecting Web Worker for "main"
transmuxer-interface.ts:227 [log] > [transmuxer-interface, main]: Starting new transmux session for sn: 0 p: -1 level: 0 id: 1
        discontinuity: true
        trackSwitch: true
        contiguous: false
        accurateTimeOffset: true
        timeOffset: 0
        initSegmentChange: true
base-stream-controller.ts:386 [log] > [stream-controller]: Loaded fragment 0 of level 0
blob:https://hlsjs.video-dev.org/8904339c-93e0-409a-b03d-5d85e9e7b577:542 [log] > Debug logs enabled for "main" in hls.js version 1.4.7
transmuxer-interface.ts:379 [warn] > MPEG2-TS detected but first sync word found @ offset 104
onWorkerMessage @ transmuxer-interface.ts:379
TransmuxerInterface.onwmsg @ transmuxer-interface.ts:88
transmuxer-interface.ts:379 [log] > [mp4-remuxer]: ISGenerated flag reset
transmuxer-interface.ts:379 [log] > [mp4-remuxer]: initPTS & initDTS reset
transmuxer-interface.ts:379 [log] > [mp4-remuxer]: reset next timestamp
transmuxer-interface.ts:379 [warn] > MPEG-TS PMT found at 205776 after unknown PID '256'. Backtracking to sync byte @104 to parse all TS packets.
onWorkerMessage @ transmuxer-interface.ts:379
TransmuxerInterface.onwmsg @ transmuxer-interface.ts:88
transmuxer-interface.ts:379 [warn] > [mp4-remuxer]: Dropped 89 out of 269 video samples due to a missing keyframe
onWorkerMessage @ transmuxer-interface.ts:379
TransmuxerInterface.onwmsg @ transmuxer-interface.ts:88
base-stream-controller.ts:1749 [log] > [stream-controller]: FRAG_LOADING->PARSING
stream-controller.ts:1264 [log] > [stream-controller]: Init video buffer, container:video/mp4, codecs[level/parsed]=[/avc1.42c01e]
base-stream-controller.ts:1749 [log] > [stream-controller]: PARSING->IDLE
base-stream-controller.ts:727 [log] > [stream-controller]: Loading fragment 0 cc: 0 of [0-59] level: 0, target: 0
base-stream-controller.ts:1749 [log] > [stream-controller]: IDLE->FRAG_LOADING
transmuxer-interface.ts:85 [log] > injecting Web Worker for "main"
transmuxer-interface.ts:227 [log] > [transmuxer-interface, main]: Starting new transmux session for sn: 0 p: -1 level: 0 id: 1
        discontinuity: true
        trackSwitch: true
        contiguous: false
        accurateTimeOffset: true
        timeOffset: 0
        initSegmentChange: true
base-stream-controller.ts:386 [log] > [stream-controller]: Loaded fragment 0 of level 0
blob:https://hlsjs.video-dev.org/e70fad47-2905-4872-a950-cbdee709fd31:542 [log] > Debug logs enabled for "main" in hls.js version 1.4.7
transmuxer-interface.ts:379 [warn] > MPEG2-TS detected but first sync word found @ offset 104
onWorkerMessage @ transmuxer-interface.ts:379
TransmuxerInterface.onwmsg @ transmuxer-interface.ts:88
transmuxer-interface.ts:379 [log] > [mp4-remuxer]: ISGenerated flag reset
transmuxer-interface.ts:379 [log] > [mp4-remuxer]: initPTS & initDTS reset
transmuxer-interface.ts:379 [log] > [mp4-remuxer]: reset next timestamp
transmuxer-interface.ts:379 [warn] > MPEG-TS PMT found at 205776 after unknown PID '256'. Backtracking to sync byte @104 to parse all TS packets.
onWorkerMessage @ transmuxer-interface.ts:379
TransmuxerInterface.onwmsg @ transmuxer-interface.ts:88
transmuxer-interface.ts:379 [warn] > [mp4-remuxer]: Dropped 89 out of 269 video samples due to a missing keyframe
onWorkerMessage @ transmuxer-interface.ts:379
TransmuxerInterface.onwmsg @ transmuxer-interface.ts:88
base-stream-controller.ts:1749 [log] > [stream-controller]: FRAG_LOADING->PARSING
stream-controller.ts:1264 [log] > [stream-controller]: Init video buffer, container:video/mp4, codecs[level/parsed]=[/avc1.42c01e]
base-stream-controller.ts:1749 [log] > [stream-controller]: PARSING->IDLE
base-stream-controller.ts:727 [log] > [stream-controller]: Loading fragment 0 cc: 0 of [0-59] level: 0, target: 0
base-stream-controller.ts:1749 [log] > [stream-controller]: IDLE->FRAG_LOADING
transmuxer-interface.ts:85 [log] > injecting Web Worker for "main"
transmuxer-interface.ts:227 [log] > [transmuxer-interface, main]: Starting new transmux session for sn: 0 p: -1 level: 0 id: 1
        discontinuity: true
        trackSwitch: true
        contiguous: false
        accurateTimeOffset: true
        timeOffset: 0
        initSegmentChange: true
base-stream-controller.ts:386 [log] > [stream-controller]: Loaded fragment 0 of level 0
9c266fb5-766d-49c9-b583-349c0f4bc094:542 [log] > Debug logs enabled for "main" in hls.js version 1.4.7
transmuxer-interface.ts:379 [warn] > MPEG2-TS detected but first sync word found @ offset 104
onWorkerMessage @ transmuxer-interface.ts:379
TransmuxerInterface.onwmsg @ transmuxer-interface.ts:88
transmuxer-interface.ts:379 [log] > [mp4-remuxer]: ISGenerated flag reset
transmuxer-interface.ts:379 [log] > [mp4-remuxer]: initPTS & initDTS reset
transmuxer-interface.ts:379 [log] > [mp4-remuxer]: reset next timestamp
transmuxer-interface.ts:379 [warn] > MPEG-TS PMT found at 205776 after unknown PID '256'. Backtracking to sync byte @104 to parse all TS packets.
onWorkerMessage @ transmuxer-interface.ts:379
TransmuxerInterface.onwmsg @ transmuxer-interface.ts:88
transmuxer-interface.ts:379 [warn] > [mp4-remuxer]: Dropped 89 out of 269 video samples due to a missing keyframe
onWorkerMessage @ transmuxer-interface.ts:379
TransmuxerInterface.onwmsg @ transmuxer-interface.ts:88
base-stream-controller.ts:1749 [log] > [stream-controller]: FRAG_LOADING->PARSING

Chrome media internals output

No response

[robwalch]

Hi @mattzucker,

Since your first segment has some media (it just does not begin with an IDR frame) it doesn't really call for a GAP tag (apologies for my earlier curt comment). Using the GAP prevents it from loading and handles seeking, but you miss out on the playable portion of the segment.

I was unable to reproduce loop loading with the stream prior to the GAP being added - maybe because I had the worker disabled. If you have another sample or updated steps let me know. Thanks!

[mattzucker]

Thank you for following up. I was just about to comment explaining why GAP wouldn't work, but I wanted to test to make sure I understood the behavior. It would skip the entire segment which is not desired, and I also do not know which segments have a problem so I wouldn't know where to add that tag anyway.

I am able to reproduce the bug consistently using the stream I linked in the ticket. I also tried with enableWorker: false using the demo site and I still see the bug. It does not happen the first time the player loads. You have to load, then seek far forward, then seek back to the very beginning, as close to 0 as you can.

[mattzucker]

Reproducing.Infinite.Load.mp4

[robwalch]

Hi @mattzucker,

Let me know if #5613 resolves this issue.

[mattzucker]

In the example stream, I reproduced the bug by breaking the first segment, but it is possible that a segment anywhere in the playlist could be missing the first IDR frame. I will work on updating my example to include this case.

I also noticed different behavior when trying to skip over the hole on your branch as compared to version 1.4.7. In 1.4.7, when a fragment with this problem is buffered and the playback gets stuck, I am able to nudge the currentTime forward (video.currentTime = video.currentTime + 0.1;). This is enough to trigger a bufferSeekOverHole error and the video skips to the next playable frame. In bugfix/holes-on-first-no-backtrack and master it does not seek over the hole unless I call video.play().

In my case the video may be paused, and I would rather not start playing the video just to get it to jump over the hole.

I should note that I really appreciate all your work on this and the fast responses! It is a great product and I realize that I am dealing with edge cases.

[robwalch]

it is possible that a segment anywhere in the playlist could be missing the first IDR frame.

In that case, an earlier fragment will be loaded and appended as long as it is independent. The hole is only there when the previous segment with the required IDR was not appended.

You can search the project history for 'backtracking' for more details on this issue and its history.

If your first segment had no IDR frames (not independent) and your second segment did not start with an IDR, then we might have an issue when seeking back to the start of segment two where backtracking ultimately fails because segment one is essentially a blank.

[mattzucker]

Hi @robwalch,

I was able to reproduce the behavior with a segment in the middle of the playlist. It gets into the same backtracking infinite loop, but this time frag.sn = 35 and details?.startSN = 0.
The m3u8 file is here: https://raw.githubusercontent.com/mattzucker/hls-test-data/main/clean-bunny/bunny_2.m3u8

Reproducing.Infinite.Load.2.mp4

[mattzucker]

Can you take another look at the second video I posted? The infinite loop of loading a segment is happening at frag.sn = 35, which is not resolved by the changes in #5613

[mattzucker]

Hi @robwalch. Not sure if you saw my last comment but I don't think this bug is completely fixed.

[robwalch]

Hi @mattzucker,

Please file a new issue for this comment #5609 (comment) with repro steps and logs specific to this test case.