-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Expose flow control information to transform #206
Comments
This resembles the proposal outlined in https:/alvestrand/hackathon-encoded-media |
RTCRtpScriptTransformer seems like an appropriate place for such API. If using events, For instance, a |
Side note here on keyFrameRequested : We found that you can get repeated keyFrameRequests which you need to ignore when when you know a keyframe is still inflight. Likewise you probably want to remove any frames in an output queue if they predate the keyFrame. So keyFrameRequested handling is by no means stateless. |
By default, it is up to the UA to not trigger too many key frames.
And skipping of frames could be done like:
Does this look ok? |
Yep, looks good. However you now have to require that transformer.generateKeyFrame() returns and resets isKeyFrameInflight before the first keyframe chunk is given to doTransform() otherwise you'll drop the keyframe. |
Right!
The intent is also for generateKeyFrame to be a no-op if a previous generateKeyFrame call did not yet resolve (see |
Sounds good. I'll prepare a PR that adds events and corresponding listeners / action functions, and we can use that as a basis for discussion. |
Discucssion of the late fanout use case at TPAC 2023 led to the realization that for this use case, and also for others such as "additional frame-level metadata", we need to expose congestion control signals from the transport via the packetizer to the transform, and allow them to be modified before (possibly) passing them on to the encoder (or, in the late fanout use case, to the incoming stream that it is doing fanout for).
Similarly, there may be signals that a decoder pipeline wishes to pass on to the depacketizer and to the transport, such as a keyframe request - these signals should also be exposed at the Javascript API.
A first suggestion would be to expose a series of events (one per feedback type identified) on the downstream interface of the transform, with a corresponding series of methods on the upstream interface of the transform that would take the same parameters as the content of the incoming events; this would make "default handling" easy by connecting the event directly to the handler if the transform did not desire to do any processing of them.
The text was updated successfully, but these errors were encountered: