Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Packaging to DASH does not work when output filename contains an ampersand (unterminated entity reference) #1107

Closed
andrej-peterka opened this issue Oct 20, 2022 · 0 comments · Fixed by #1395
Assignees
Labels
priority: P3 Useful but not urgent status: archived Archived and locked; will not be updated type: enhancement New feature or request

Comments

@andrej-peterka
Copy link

System info

Operating System: macOS Monterey 12.6
Shaka Packager Version: v2.6.1-634af65-release

Issue and steps to reproduce the problem

When packaging to DASH and the output filename contains an ampersand (&), an unterminated entity reference error is printed.
Packaging completes, but the resulting MPD manifest is invalid.

Packager Command:
Working:
packager-osx "input=SampleVideo.mp4,output=Sample_Video.mp4,output_format=mp4,stream_selector=video" -mpd_output manifest_ok.mpd

Not working:
packager-osx "input=SampleVideo.mp4,output=Sample&Video.mp4,output_format=mp4,stream_selector=video" -mpd_output manifest_nok.mpd

[1020/191510:INFO:demuxer.cc(89)] Demuxer::Run() on file 'SampleVideo.mp4'.
[1020/191510:INFO:demuxer.cc(155)] Initialize Demuxer for file 'SampleVideo.mp4'.
[1020/191510:INFO:single_segment_segmenter.cc(111)] Update media header (moov) and rewrite the file to 'Sample&Video.mp4'.
error : unterminated entity reference       Video.mp4
[1020/191511:INFO:mp4_muxer.cc(186)] MP4 file 'Sample&Video.mp4' finalized.
error : unterminated entity reference       Video.mp4
Packaging completed successfully.

Diff of the manifests:

diff manifest_ok.mpd manifest_nok.mpd
7c7
<         <BaseURL>Sample_Video.mp4</BaseURL>
---
>         <BaseURL/>

Extra steps to reproduce the problem?
(1) Package any video to DASH and set the output video filename to something containing an ampersand (&)
(2) Check the manifest

What is the expected result?
A manifest with filename filled out at BaseURL.

What happens instead?
The resulting manifest has an empty BaseURL

Additionally, if you run the following command:
packager-osx "input=SampleVideo.mp4,output=Sample&amp;Video.mp4,output_format=mp4,stream_selector=video" -mpd_output manifest_nok2.mpd

The BaseURL looks like....

<BaseURL>Sample&amp;Video.mp4</BaseURL>

But the resulting file has a filename Sample&amp;Video.mp4

@andrej-peterka andrej-peterka changed the title Packaging to DASH does not work when output file contains an ampersand (unterminated entity reference) Packaging to DASH does not work when output filename contains an ampersand (unterminated entity reference) Oct 20, 2022
@cosmin cosmin added type: enhancement New feature or request priority: P3 Useful but not urgent labels Apr 26, 2024
@cosmin cosmin added this to the Backlog milestone Apr 26, 2024
@cosmin cosmin self-assigned this May 7, 2024
cosmin added a commit that referenced this issue May 11, 2024
Currently `media_info.media_file_url()` is not escaped when placed into
MPD for things like BaseURL. This for example breaks when trying to us a
file name that contains special characters like &. Since these are
supposed to be URLs let's URL encode them.

Fixes #1107

---------

Co-authored-by: Joey Parrish <[email protected]>
@cosmin cosmin removed this from the Backlog milestone May 12, 2024
@github-actions github-actions bot added the status: archived Archived and locked; will not be updated label Jul 10, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 10, 2024
SteveR-PMP added a commit to SteveR-PMP/shaka-packager that referenced this issue Aug 28, 2024
…ovi 8.1 (shaka-project#35)

* fix: BaseURL missing when MPD base path is empty (shaka-project#1380)

The check for `!mpd_dir.empty()` is not needed because MakePathRelative
handles the case where the parent path is empty. As a result of this
check the base url, segment url, or segment template URLs were all
missing in cases where the mpd output was in the current working
directory.

Fixes shaka-project#1378

* chore(main): release 3.0.4 (shaka-project#1377)

:robot: I have created a release *beep* *boop*
---


## [3.0.4](shaka-project/shaka-packager@v3.0.3...v3.0.4) (2024-03-27)


### Bug Fixes

* BaseURL missing when MPD base path is empty ([shaka-project#1380](shaka-project#1380)) ([90c3c3f](shaka-project@90c3c3f)), closes [shaka-project#1378](shaka-project#1378)
* Fix NPM binary selection on ARM Macs ([shaka-project#1376](shaka-project#1376)) ([733af91](shaka-project@733af91)), closes [shaka-project#1375](shaka-project#1375)

---
This PR was generated with [Release Please](https:/googleapis/release-please). See [documentation](https:/googleapis/release-please#release-please).

* build: turn on integration tests in ctest by default (shaka-project#1381)

They can still be skipped by passing `-DSKIP_INTEGRATION_TESTS=ON` for
the build configuration. Fix integration tests so they run correctly when building out of tree.

Use FindPython3 in CMake to fix build and integration tests on Windows.

* feat: teletext formatting (shaka-project#1384)

This PR adds parsing of teletext styling, and rendering of the styling
in output TTML and WebVTT subtitle tracks.

Beyond unit tests, I've used the sample
https://drive.google.com/file/d/19ZYsoeUfH85gEilQkaAdLbPhC4CxhDEh/view?usp=sharing
which has rather advanced subtitling with two separate rows at the same
time, where one is left aligned and another is right aligned. This
necessitates two parallel cues to be rendered. It also has some colored
text.

Solve shaka-project#1335.

## parse teletext styling and formatting

Extend the teletext parser to parse the teletext styling and formatting.
This includes translating rows into regions, calculating alignment
from start and stop position of the text, and extracting text and
background colors.

The colors are limited to full lines.
Both lines and regions are propagated in the TextSample structures.
This is because the number of lines may differ from different sources.
For teletext, there are 24 rows, but they are essentially always
used with double height, so the number of output lines is 12
from 0 to 11.
There are also corresponding regions are denoted "ttx_R",
where R is an integer row number. A renderer can use either
the line number or the region ID to render the text.

## ttml generation for teletext to EBU-TT-D

Add support to render teletext input in EBU-TT-D (IMSC-1) format.
This includes appropriate regions ttx_0 to ttx_11 signalled
in the TextSamples, alignment and text and background colors.

The general TTML output has been changed to always include
metadata, layout, and styling nodes, even if they are empty.

EBU-TT-D is detected by the presence of "ttx_?" regions in the
samples. If detected, extra TTML elements will be added and
the EBU-TT-D linePadding used as well.

Appropriate styles for background and text colors are generated
depending on the color and backgroundColor attributes in the
text fragments.

## adapt WebVTT output to teletext TextSample.

Teletext input generates both a region with prefix ttx_
and a floating point line number (e.g. 9.5) in the
range 0 to 11.5 (due to input 0-23 as double lines).

The output is adopted to drop such regions
and convert the line number to an integer
since the standard only used floats for percent
values but not for plain line numbers.

* feat: add missing DASH roles from ISO/IEC 23009-1 section 5.8.5.5 (shaka-project#1390)



Fixes shaka-project#1149

---------

Co-authored-by: Joey Parrish <[email protected]>

* docs: Fix missing graphviz outputs in generated docs (shaka-project#1392)

Fixes shaka-project#1388

* feat: get start number from muxer and specify initial sequence number (shaka-project#879)

Set the start number in representation to the segment index that is sent by muxer.

With this enhancement, you can now specify the initial sequence number
to be used on the generated segments when calling the packager.
With the old implementation, it was always starting with "1".

---------

Co-authored-by: Cosmin Stejerean <[email protected]>

* refactor: merge Period::ProtectedAdaptationSetMap  into AdaptationSet (shaka-project#844)



---------

Co-authored-by: Cosmin Stejerean <[email protected]>

* chore(main): release 3.1.0 (shaka-project#1391)

:robot: I have created a release *beep* *boop*
---


##
[3.1.0](shaka-project/shaka-packager@v3.0.4...v3.1.0)
(2024-05-03)


### Features

* add missing DASH roles from ISO/IEC 23009-1 section 5.8.5.5
([shaka-project#1390](shaka-project#1390))
([fe885b3](shaka-project@fe885b3))
* get start number from muxer and specify initial sequence number
([shaka-project#879](shaka-project#879))
([bb104fe](shaka-project@bb104fe))
* teletext formatting
([shaka-project#1384](shaka-project#1384))
([4b5e80d](shaka-project@4b5e80d))

---
This PR was generated with [Release
Please](https:/googleapis/release-please). See
[documentation](https:/googleapis/release-please#release-please).

* fix: adaptation set IDs were referenced by lowest representation ID  (shaka-project#1394)

After change to add forced command line ordering adaptation set IDs in
places were referenced by their sort index (the minimum representation
index they contained).

Instead always refer to adaptation sets by their own ID, and use the
index only as an optional sort key.

Fixes shaka-project#1393

* docs: document --enable_entitlement_license option for Widevine (shaka-project#1399)

The option was never covered to the widevine docs when it was added,
requiring someone to read the source code or the --help to discover this
option.

Fixes shaka-project#983

* fix: escape media URLs in MPD (shaka-project#1395)

Currently `media_info.media_file_url()` is not escaped when placed into
MPD for things like BaseURL. This for example breaks when trying to us a
file name that contains special characters like &. Since these are
supposed to be URLs let's URL encode them.

Fixes shaka-project#1107

---------

Co-authored-by: Joey Parrish <[email protected]>

* fix: set yuv full range flag to 1 for VP9 with sRGB (shaka-project#1398)

If color_space is VPX_COLOR_SPACE_SRGB, the specs says that color_range
should be 1 i.e. yuv_full_range = true. 

However, yuv_full_range was initialized as false and wasn't set in the branch for color_space
is VPX_COLOR_SPACE_SRGB.

Fixes shaka-project#990

---------

Co-authored-by: Joey Parrish <[email protected]>

* feat: support Dolby Vision profile 8.x (HEVC) and 10.x (AV1) in HLS and DASH  (shaka-project#1396)

Support Dolby Vision profile 8.1, 8.2, 8.4, 10.1, 10.4 signaling in HLS
and DASH.

Adds new option `--use_dovi_supplemental_codecs` (off by default) to use
SUPPLEMENTAL-CODECS in HLS and `scte214:supplementalCodecs` and
`scte214:supplementalProfiles` for DASH.

To maintain compatibility with existing players the current behavior of
using two entries in the manifest remains the default. This will be
changed in a future version where `use_dovi_supplemental_codecs` will
become on by default.

Adds Dolby Vision compatible brands, 'db1p', 'db2g', 'db4g', 'db4h',
'dby1' based on https://mp4ra.org/#/brands

---------

Co-authored-by: Xingzhao Yun <[email protected]>

* chore(main): release 3.2.0 (shaka-project#1400)

:robot: I have created a release *beep* *boop*
---


##
[3.2.0](shaka-project/shaka-packager@v3.1.0...v3.2.0)
(2024-05-11)


### Features

* support Dolby Vision profile 8.x (HEVC) and 10.x (AV1) in HLS and DASH
([shaka-project#1396](shaka-project#1396))
([a99cfe0](shaka-project@a99cfe0))


### Bug Fixes

* adaptation set IDs were referenced by lowest representation ID
([shaka-project#1394](shaka-project#1394))
([94db9c9](shaka-project@94db9c9)),
closes
[shaka-project#1393](shaka-project#1393)
* escape media URLs in MPD
([shaka-project#1395](shaka-project#1395))
([98b44d0](shaka-project@98b44d0))
* set yuv full range flag to 1 for VP9 with sRGB
([shaka-project#1398](shaka-project#1398))
([f6f60e5](shaka-project@f6f60e5))

---
This PR was generated with [Release
Please](https:/googleapis/release-please). See
[documentation](https:/googleapis/release-please#release-please).

* lint

---------

Co-authored-by: Cosmin Stejerean <[email protected]>
Co-authored-by: Shaka Bot <[email protected]>
Co-authored-by: Torbjörn Einarson <[email protected]>
Co-authored-by: Joey Parrish <[email protected]>
Co-authored-by: sr90 <[email protected]>
Co-authored-by: Cosmin Stejerean <[email protected]>
Co-authored-by: Xingzhao Yun <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
priority: P3 Useful but not urgent status: archived Archived and locked; will not be updated type: enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants