ES3 Vertex Formats

[lexaknyazev]

Since we don't want to make breaking changes until the next major release, we must make glTF 2.0 future-proof. I.e., make sure that some expected features could be added in non-destructive way (i.e., via glTF 2.1 minor update).

Modern APIs provide several vertex formats which are not yet available in glTF. Some of them have universal support, while others work only on particular platforms.

Here're all non-packed vertex formats, supported by intersection of OpenGL ES 3.0 / Vulkan / Metal / D3D11.

Legend

• Green - supported by OpenGL ES 2.0 (hence, glTF 1.0)
• Blue - supported by all modern platforms (should be added in future glTF update)
• Yellow - could require simple client-side processing (shouldn't be used much)

From the table above, it looks like we could add support for 16-bit floats and 32-bit integer attributes at almost no cost (just allow couple new enum values for accessor.componentType).

However, there's one possible issue here: OpenGL (ES) runtimes must differentiate true integer attributes (ES 3.0+) from float attributes filled with integer values, because there're two different API calls: VertexAttribPointer and VertexAttribIPointer.

From OpenGL perspective, it's a part of VAO state (VERTEX_ATTRIB_INTEGER), so it could be signaled via additional accessor property (e.g., accessor.integer) or modified accessor type (e.g., "IVEC3").

On the other hand, such distinction has nothing to do with actual vertex data and is tied to shader code (since real attribute type is specified there).

References

• OpenGL ES 3.0
• Vulkan Formats
• Metal MTLVertexFormat
• DXGI_FORMAT
• ANGLE Vertex Data Conversion

Edited:

Initial version of the table above contained incorrect info about packed formats. This is addressed in the comment below.

[lexaknyazev]

Packed vertex formats

Support for packed (10-10-10-2) vertex formats varies across platforms.

Normalized formats

OpenGL (ES)	Vulkan	Metal	D3D11
GL_UNSIGNED_INT_2_10_10_10_REV	VK_FORMAT_A2B10G10R10_UNORM_PACK32	uint1010102Normalized	DXGI_FORMAT_R10G10B10A2_UNORM
GL_INT_2_10_10_10_REV	VK_FORMAT_A2B10G10R10_SNORM_PACK32	int1010102Normalized	None

At least, GL_UNSIGNED_INT_2_10_10_10_REV (normalized) should be allowed for glTF.

Non-normalized formats for true integer (ivec4) attributes

OpenGL (ES)	Vulkan	Metal	D3D11
None	VK_FORMAT_A2B10G10R10_UINT_PACK32	None	DXGI_FORMAT_R10G10B10A2_UINT
None	VK_FORMAT_A2B10G10R10_SINT_PACK32	None	None

Not sure about Metal runtime conversion, however I don't think that such format (packed 10-bit integers) has broad usage.

Non-normalized formats for float (vec4) attributes

OpenGL (ES)	Vulkan	Metal	D3D11
GL_UNSIGNED_INT_2_10_10_10_REV	VK_FORMAT_A2B10G10R10_USCALED_PACK32	None	None
GL_INT_2_10_10_10_REV	VK_FORMAT_A2B10G10R10_SSCALED_PACK32	None	None

Again, I'm not sure about possible usage patterns here.

---

My assumptions on D3D types are based on ANGLE source code. @kainino0x, could you confirm?

[kainino0x]

I actually know very little about D3D, but a glance at the docs shows there are no SINT and SNORM variants.

[lexaknyazev]

ANGLE converts packed

• GL's SNORM, UINT (vec4), and SINT (vec4) to DXGI_FORMAT_R32G32B32A32_FLOAT, and
• GL's SINT (ivec4) to DXGI_FORMAT_R16G16B16A16_SINT.

So I was curious do these conversions have noticeable impact or not.

[kainino0x]

For large amounts of vertex data, that's a huge memory and bandwidth overhead (2x-4x). So I would expect it to be noticeable. ANGLE needs to have full compatibility so tradeoffs like this are acceptable. If glTF can somehow avoid them it probably makes sense.

[donmccurdy]

Is this resolved?

[lexaknyazev]

I think that the info here is quite useful for the possible future updates. Feel free to close or tag as you see fit.

[donmccurdy]

Ok, my personal preference is to close things when they no longer have next steps associated with them, but I'm open to handling it differently. Tagging this with "archive".

[mosra]

What would be the correct approach to expose the additional formats in glTF 2.0? Proposing an extension? I could see a lot benefit in having animation data encoded as half-floats or quaternions packed into 10-10-10-2 formats.

[lexaknyazev]

This could be a vendor or multi-vendor extension. The most important question is how to design fallback behavior. If there's no need to maintain compatibility with existing ecosystem, the extension could be marked as required for assets that use it.

As an alternative approach for compressing animation data, one might want to use a dedicated extension like #1407.

[mosra]

I am interested in packing the data in order to allow things like using animation data directly from memory-mapped files (so, without needing to apply the Draco decompression first) or reducing the amount of data uploaded to the GPU without needing to process the data on load time (there Draco doesn't apply either).

For fallback behavior I was thinking about a similar approach as in the Conformance section of #1407, having fallback buffers with full-sized data. But primary use case would be requiring the support -- it's to reduce the data amount after all, so dragging the uncompressed variant along doesn't make much sense.

[JoshKlint]

We use half floats for texture coordinates.

I thought 5125 = unsigned 32-bit int was official.

5124 is unofficial signed int.