Canvas text rendering and metrics

[AshleyScirra]

Description

A problem that has been painful for us for some time now is browsers don't agree on how to align text when drawing to a canvas. Further, TextMetrics gives different answers in different browsers for values like fontBoundingBoxAscent/Descent depending on the text baseline, and Firefox doesn't support those properties at all. In some situations that makes the API pretty much useless - it is either not supported or gives you different answers per browser.

Rationale

Positioning text precisely on canvases is important for things like buttons and text layout. For example a canvas game may have a text label drawn on top of a button graphic with a border. Carefully placing the text so it's aligned correctly in one browser will still show it misaligned in another browser. There doesn't seem to be any good workaround to this.

TextMetrics is important for things like canvas text layout. If you want to draw some text on a canvas vertically aligned inside a box, you may need to take in to account the fontBoundingBoxAscent/Descent. However those values return one value in Chrome, another in Safari, and aren't supported in Firefox - which basically makes the API useless for precisely aligning text. You have to pick one browser to display it right and the others will be wrong.

Specification

Not sure if this is part of the HTML or canvas specs.

There is a so far unresolved spec issue about this: whatwg/html#6731

Tests

A cursory search doesn't turn up any web-platform-tests for TextMetrics, but I guess here? https://wpt.fyi/results/html/canvas/element/drawing-text-to-the-canvas

[gsnedders]

A cursory search doesn't turn up any web-platform-tests for TextMetrics, but I guess here? https://wpt.fyi/results/html/canvas/element/drawing-text-to-the-canvas

Searching for measure( calls within canvas (IIRC, the only method which returns TextMetrics):

html/canvas/element/2d.conformance.requirements.missingargs.html
html/canvas/element/conformance-requirements/2d.conformance.requirements.missingargs.html
html/canvas/element/drawing-text-to-the-canvas/2d.text.drawing.style.fontKerning.html
html/canvas/element/drawing-text-to-the-canvas/2d.text.drawing.style.letterSpacing.change.font.html
html/canvas/element/drawing-text-to-the-canvas/2d.text.drawing.style.letterSpacing.measure.html
html/canvas/element/drawing-text-to-the-canvas/2d.text.drawing.style.wordSpacing.change.font.html
html/canvas/element/drawing-text-to-the-canvas/2d.text.drawing.style.wordSpacing.measure.html
html/canvas/element/drawing-text-to-the-canvas/2d.text.measure.actualBoundingBox.html
html/canvas/element/drawing-text-to-the-canvas/2d.text.measure.advances.html
html/canvas/element/drawing-text-to-the-canvas/2d.text.measure.baselines.html
html/canvas/element/drawing-text-to-the-canvas/2d.text.measure.boundingBox.direction.html
html/canvas/element/drawing-text-to-the-canvas/2d.text.measure.boundingBox.textAlign.html
html/canvas/element/drawing-text-to-the-canvas/2d.text.measure.emHeights.html
html/canvas/element/drawing-text-to-the-canvas/2d.text.measure.fontBoundingBox.ahem.html
html/canvas/element/drawing-text-to-the-canvas/2d.text.measure.fontBoundingBox.html
html/canvas/element/drawing-text-to-the-canvas/2d.text.measure.rtl.text.html
html/canvas/element/drawing-text-to-the-canvas/2d.text.measure.width.basic.html
html/canvas/element/drawing-text-to-the-canvas/2d.text.measure.width.empty.html
html/canvas/element/text-styles/2d.text.measure.width.space.html
html/canvas/offscreen/2d.conformance.requirements.missingargs.html
html/canvas/offscreen/2d.conformance.requirements.missingargs.worker.js
html/canvas/offscreen/conformance-requirements/2d.conformance.requirements.missingargs.html
html/canvas/offscreen/conformance-requirements/2d.conformance.requirements.missingargs.worker.js
html/canvas/offscreen/text/2d.text.drawing.style.fontKerning.html
html/canvas/offscreen/text/2d.text.drawing.style.fontKerning.worker.js
html/canvas/offscreen/text/2d.text.drawing.style.letterSpacing.change.font.html
html/canvas/offscreen/text/2d.text.drawing.style.letterSpacing.change.font.worker.js
html/canvas/offscreen/text/2d.text.drawing.style.letterSpacing.measure.html
html/canvas/offscreen/text/2d.text.drawing.style.letterSpacing.measure.worker.js
html/canvas/offscreen/text/2d.text.drawing.style.measure.direction.html
html/canvas/offscreen/text/2d.text.drawing.style.measure.direction.worker.js
html/canvas/offscreen/text/2d.text.drawing.style.measure.rtl.text.html
html/canvas/offscreen/text/2d.text.drawing.style.measure.rtl.text.worker.js
html/canvas/offscreen/text/2d.text.drawing.style.measure.textAlign.html
html/canvas/offscreen/text/2d.text.drawing.style.measure.textAlign.worker.js
html/canvas/offscreen/text/2d.text.drawing.style.wordSpacing.change.font.html
html/canvas/offscreen/text/2d.text.drawing.style.wordSpacing.change.font.worker.js
html/canvas/offscreen/text/2d.text.drawing.style.wordSpacing.measure.html
html/canvas/offscreen/text/2d.text.drawing.style.wordSpacing.measure.worker.js
html/canvas/offscreen/text/2d.text.measure.actualBoundingBox.html
html/canvas/offscreen/text/2d.text.measure.actualBoundingBox.worker.js
html/canvas/offscreen/text/2d.text.measure.advances.html
html/canvas/offscreen/text/2d.text.measure.advances.worker.js
html/canvas/offscreen/text/2d.text.measure.baselines.html
html/canvas/offscreen/text/2d.text.measure.baselines.worker.js
html/canvas/offscreen/text/2d.text.measure.emHeights.html
html/canvas/offscreen/text/2d.text.measure.emHeights.worker.js
html/canvas/offscreen/text/2d.text.measure.fontBoundingBox.html
html/canvas/offscreen/text/2d.text.measure.fontBoundingBox.worker.js
html/canvas/offscreen/text/2d.text.measure.width.basic.html
html/canvas/offscreen/text/2d.text.measure.width.basic.worker.js
html/canvas/offscreen/text/2d.text.measure.width.empty.html
html/canvas/offscreen/text/2d.text.measure.width.empty.worker.js
html/canvas/offscreen/text/2d.text.measure.width.space.html
html/canvas/offscreen/text/2d.text.measure.width.space.worker.js
html/canvas/tools/yaml-new/conformance_requirements.yaml
html/canvas/tools/yaml/element/drawing-text-to-the-canvas.yaml
html/canvas/tools/yaml/element/text-styles.yaml
html/canvas/tools/yaml/offscreen/text.yaml

[fserb]

There are two issues that Ashley has brought up:

1. making all TextMetrics available on all browsers, returning reasonably results consistent with what is described here.
2. making all browsers return the exact same value for all metrics/fonts/texts.

Number 2 is extremely complex (there are many corner cases, and Browsers may not even be 100% consistent even with themselves across different OSes). There has been a lot of discussion on how to address some of those issues in other forums, and progress is currently being made, but this is probably a bigger effort than Interop can take at this point.

That said, number 1 is a reasonable request, relatively small and that could already give huge benefits for users. It would also pave the way for more progress towards the second issue in the future.

My proposal is that we amend this proposal to be about it, with the intent of making it more practical for Browsers to accept it. To that effect we could agree, for this proposal to cover only the following:

Have a set of tests (similar to the ones that @gsnedders points out we already have) that cover the existence of all the metrics described on the spec, returning values within a reasonable generous margin of error and that are self-consistent (i.e., ascend + descent should make sense, values should respond to textAlign/textBaseline in a consistent way, etc...).

I think it's possible to build a set of tests that make those values still meaningful, even though they may not be identical everywhere. And that still has practical usefulness for web developers, that can rely on them for behavior, even though it wouldn't be pixel perfect in all cases.

If folks are okey with this on principle, I'd commit to have the tests in a good state by the end of year.

[AshleyScirra]

I accept that all platforms will have some subtle text metrics/rendering differences, such as if one platform optimises for screen legibility and another platform optimises for faithful rendering of the font design. That is not particularly concerning to me; the real problem is that existing TextMetrics measurements and rendering positions are significantly off in a way I don't think can be explained by system variation. On that basis I'm happy for this proposal to focus on point 1 from the previous comment, i.e. reasonably consistent as opposed to exactly consistent.

[gsnedders]

AIUI, a lot of the problem is the spec is fundamentally not clear enough about what the metrics should be, per the HTML issue Ashley linked in the original proposal.

Writing tests alone doesn't solve this; we also need to actually reach agreement on what the spec needs to say (see also all the pings of people more involved with the definition of inline layout in CSS in whatwg/html#5826).

[nt1m]

@AshleyScirra Do you have a small testcase of such a rendering difference so we can pinpoint what the exact issues/solution is?

[jfkthame]

Can we confirm that this proposal is only about the TextMetrics attributes that are currently in the spec? There are tests currently in WPT for an advances attribute that is not spec'd and has serious defects as an API; if we're going to proceed with this, I'd like it clarified that .advances is not part of it.

There's also a need for some spec clarification regarding how the various font properties behave and interact; see whatwg/html#8103.

[AshleyScirra]

Do you have a small testcase of such a rendering difference so we can pinpoint what the exact issues/solution is?

Here are the test cases I filed with browser vendors (Chrome report, Safari report), which ended up leading to the spec issue.

Drawing text with the "top" baseline renders inconsistently. Minimal repro: https://downloads.scirra.com/labs/bugs/text-top-baseline/index.html
Comparison screenshot: https://downloads.scirra.com/labs/bugs/text-top-baseline/comparison.png

There's another issue I filed for Chrome which is similar but affects TextMetrics. The minimal repro is here: https://downloads.scirra.com/labs/bugs/font-measurements.html
Look in the browser console. fontBoundingBoxAscent measures 0 in Safari, but ~21 in Chrome.

Can we confirm that this proposal is only about the TextMetrics attributes that are currently in the spec?

For my purposes I'm only really interested in fontBoundingBoxAscent/Descent. I haven't heard of advances before and I don't personally see any issue in excluding that from this proposal.

[jfkthame]

There's another issue I filed for Chrome which is similar but affects TextMetrics. The minimal repro is here: https://downloads.scirra.com/labs/bugs/font-measurements.html Look in the browser console. fontBoundingBoxAscent measures 0 in Safari, but ~21 in Chrome.

FWIW, in Firefox Nightly with the fontBoundingBox pref enabled, I get:

Text metrics for 'top' baseline: fontBoundingBoxAscent = 15.39106145251398, fontBoundingBoxDescent = 163.60893854748602

whereas Chrome gives me

Text metrics for 'top' baseline: fontBoundingBoxAscent = 20.875, fontBoundingBoxDescent = 158.125

So there's clearly some lack of consistency here.

[AshleyScirra]

An example of one of several "text is misaligned" reports we've got from customers complaining about this interop issue: Scirra/Construct-bugs#6255

[foolip]

In the MDN short survey on APIs & JavaScript, "Canvas text rendering" was selected by ~15% of survey takers, putting it in the middle third of options. (There is some uncertainty as with any survey data.)

[nairnandu]

Thank you for proposing Canvas text rendering and metrics for inclusion in Interop 2023.

We wanted to let you know that this proposal was not selected to be part of Interop this year. We had many strong proposals, and could not accept them all. The specifications for canvas metrics are not yet complete enough to allow interoperable implementations. To make progress on this area in the future it will first be necessary to ensure that the proposal is on a standards track and has cross-vendor support. Resubmitting a proposal for this feature as part of a future round of Interop would be welcome.

For an overview of our process, see the proposal selection summary. Thank you again for contributing to Interop 2023!

Posted on behalf of the Interop team.

[AshleyScirra]

Can I request that this proposal is resubmitted for Interop 2024? It's still actively causing us painful interop issues which customers keep complaining about. I could file a new issue but it seems valuable to preserve the existing discussion. (I don't appear to have permission to reopen this issue myself.)

[foolip]

@AshleyScirra since the template has changed a bit, please file a new issue. It's OK to point to this issue and focus on what's changed I think.

[AshleyScirra]

Submitted for 2024 at #427.