Support CEA-608 embedded in H262 in MPEG-TS

[jvdave23]

Issue description

I have a few sample ts files that have CEA 608 closed captioning available in them. I know this because when I play it using VLC player for Android, it is able to successfully display the captions. However, VLC player has frame rate issues when rendering the captions which is why I'd like to use Exoplayer. I've hosted the files on a server, and then tried using the demo app to play the files. The demo app doesn't recognise the captions in the file.

To confirm that it wasn't a server issue, I used one of the samples provided in the exoplayer app :

https://tungsten.aaplimg.com/VOD/bipbop_adv_example_v2/master.m3u8

I downloaded the first 6 to 7 segments of the m3u8 file, concatenated them into a bigger ts file, uploaded it to same server as a ts file, and tried playing it using the Exoplayer demo app. The player was able to successfully render the captions from it using ExtractorMediaPeriod.

I found the point at which my sample and the concatenated sample differ in the code. At TsExtractor::run() , TsPayloadReader payloadReader is null which is why payloadReader.consume() is never called which would eventually lead to reading the captions. I don't understand what the payloadReader is so I was hoping to get some information from the dev team on that.

Reproduction steps

I added my sample ts files as entries in the media.exolist.json which I will send to [email protected]. Once you have the json file in your demo app, you will see a new subcategory that has all the sample files being used in the investigation of this issue.

Link to test content

I will send the test links to the email provided.

Version of ExoPlayer being used

r2.1.1

Device(s) and version(s) of Android being used

Tested on Samsung Galaxy Tab S2 - Android version 6.0.1
Tested on Nexus 5X - Android version 7.1.1
Tested on Client provided device with codecs to play video and audio - version 5.0.2

A full bug report captured from the device

Will provide if needed but don't think it is relevant.

[jvdave23]

Some more info about the sample files:

You may run into issues playing the video. It is MPEG2 video and MPEG1-L2 audio. There is one additional device we are using which is provided by our client. It has a custom Android OS on it, and some codecs that might not be available on all Android devices.

[AquilesCanta]

Please try the latest dev version. Since 2.1 we have added support for multiple CC channels in a Transport Stream. Note that Exoplayer automatically detects CC services only if they are signaled by the caption_service_descriptor. The provided TS files didn't include it. In this case, you need to specify yourself the channels you expect: I think VLC proactively offers all of them, even if they don't actually include anything (this is the case of channel 2 in CNN.ts).

You can define the channels through the closedCaptionFormats constructor argument of DefaultTrackSelector. To use this constructor you will also have to provide your own Extractor factory to Extractor Media Source. Update this issue if this was not helpful.

[jvdave23]

I've looked at the class DefaultTsPayloadReaderFactory in the dev-v2
branch and I can see the new parameter closedCaptionFormats. I've tried
to customize the closedCaptionFormats list but I cannot get ExoPlayer
to display the close caption.

One thing I noticed, closedCaptionFormats is only used in method
buildSeiReader() which is only used if the video is H264. The file
CNN.ts does not use H264, it is using the older MPEG2 video (H262).
Does ExoPlayer support extracting close caption from an MPEG2 video ?

[AquilesCanta]

I will use this issue to track CEA support in H262. A commit reference will appear here once it's done.

[needz]

Is there any chance to add DVB subtitle support somehow (through decoder or what not), since they are currently detected as CEA-608 and with supported=YES flag, but not visible on display?

Also, it's hard to detect, whether they will show up or not, since the same type of subtitles show fine if they are included in MKV as built-in subtitles.

I can share the stream sample TS if you need it. Thanks.

[AquilesCanta]

608 and DVB don't have much in common, and cannot be mistaken for one another by our extractor. For DVB support track #1781. Regarding 608 detection, it's because we expose 608 channel one by default. Considering it has DVB subtitles, the stream probably does not include 608 CC, as they are used in the ATSC standard.

[goffioul]

As I needed this feature, I've come up with this simple implementation, in case you're interested.
h262-cea608.zip

[ojw28]

Is it possible for you to send your change as a proper pull request to dev-v2, for consideration? We're not able to accept changes provided in other ways (because we need to maintain a proper history of where the change originated from, that the author signed the CLA, etc). Thanks.

[kiall]

@goffioul if you get a chance to submit a pull request, it would be much appreciated! I would do it myself, but the project wouldn't accept it as I didn't actually write the code :)

[ojw28]

This should be supported in dev-v2 as of 3c4b0aa

[okycelt]

@ojw28, when approximately do you expect this to be included in a release version?