[BUG] when merging meshes with buffers exceeding the maximum capacity

[KisliPlug]

Hello! I try convert big model from Navisworks (i think it's not important)
I can do it with part of this model, but when i try export whole model i got this error.

[ERR] System.OverflowException: Arithmetic operation resulted in an overflow.
at System.Linq.Enumerable.Sum(IEnumerable1 source) at SharpGLTF.Geometry.PackedBuffer.MergeBuffers() at SharpGLTF.Geometry.PackedPrimitiveBuilder1.MergeBuffers(IEnumerable1 primitives) at SharpGLTF.Schema2.Toolkit.CreateMeshes[TMaterial](ModelRoot root, Converter2 materialConverter, SceneBuilderSchema2Settings settings, IMeshBuilder1[] meshBuilders) at SharpGLTF.Scenes.Schema2SceneBuilder.AddGeometryResources(ModelRoot root, IEnumerable1 srcScenes, SceneBuilderSchema2Settings settings)
at SharpGLTF.Scenes.SceneBuilder.ToGltf2(IEnumerable`1 srcScenes, SceneBuilderSchema2Settings settings)
at SharpGLTF.Scenes.SceneBuilder.ToGltf2(SceneBuilderSchema2Settings settings)
at MeshToBim.Navis.Logic.MeshExporter.d__6.MoveNext()

[vpenades]

This is probably happening because one of the optimization steps (buffer merging) is unable to run because the resulting buffer will be too large.

Look into the export settings, I think there's a parameter that disables buffer merging.

[KisliPlug]

Hello! Thanks for clarification! I disabled buffer merge, but it this problem not gone. I think it happen in this place

https:/vpenades/SharpGLTF/blob/master/src/SharpGLTF.Toolkit/Schema2/MeshExtensions.cs on 87 line.

In this place buffer always merged, and settings not checked.
I'm not sure if it part of algorithm or can be skiped.

[vpenades]

I'll look into it

[vpenades]

I've uploaded some changes that fix the line you mentioned, could you try the code from the repository and tell me if it successfully loads the model?

[KisliPlug]

Hello! Thank you very match! I will check it today, and let you know about results!

[KisliPlug]

I thing now it's works in huge modes. thanks!

[KisliPlug]

Hello!
I have meet this error when try save big model as glb

System.InvalidOperationException: Can't merge a buffer larger than 2Gb
at SharpGLTF.Schema2.ModelRoot.MergeBuffers()
at SharpGLTF.Schema2.WriteContext._PreprocessSchema2(ModelRoot model, Boolean imagesAsBufferViews, Boolean mergeBuffers, Int32 buffersMaxSize)
at SharpGLTF.Schema2.WriteContext.WriteBinarySchema2(String baseName, ModelRoot model)

Also found that in this place
https:/vpenades/SharpGLTF/blob/master/src/SharpGLTF.Core/Schema2/Serialization.WriteContext.cs on 249 line
option for buffer merging always true.

Can problem happen in this place? Or for glb we always must merge buffers?

[vpenades]

Or for glb we always must merge buffers?

Yes, this is a requirement of the GLB format, because GLBs only support one buffer.

See:

• GLB packing has issues, stemming from its one buffer limit KhronosGroup/glTF#1318
• GLB file size is limited to 2^32 bytes because of uint32 length KhronosGroup/glTF#2114

And, although they state the maximum limit is 4gb, not 2gb, later it was established that most clients use a 32 bit signed integer to address the buffer, which reduces the buffer size by half, an example of this is:

• Buffer error when reaching 2GB (2.147.869.560) byteLength microsoft/glTF-SDK#116
• MergeBuffers causing "Destination array was not long enough." in _StaticBufferBuilder #101 (comment)

In practice, in c# the maximum size of a Byte Array is exactly 2 gb because it uses signed integers.

so if you're trying to save a very big model, your only choice left is to save it to glTF.

[KisliPlug]

Thanks for clarification! I's very sad 😢

[vpenades]

Indeed. My feeling is that the khronos board thought that GLB is to create relatively small, self contained packages, and whoever needs very large models should use glTF format, possibly compressed inside a ZIP file.