Crash while using Build --serve and modifying vault content live #798

Maldor · 2024-02-04T05:22:43Z

Describe the bug
Not really a clue... Seems to hang when attempting to emit files after they have been updated in some way.

To Reproduce
Reliable replication not possible or unknown

Start quartz using build command with --serve argument
Wait until webserver is spun up
Modify the page that the webserver most recently served
After the Detected change, rebuilding message, modify it again
(These steps are best achieved one immediately after the other, typing a long sentence can work)
It will parse successfully, then start emitting
it will then hang at this point until an error occurs

Expected behavior
Either
A. Emitting to complete and reprocessing to start occurring immediately
B. Emitting to self cancel and pause for a set period before reprocessing

Screenshots and Source
Entire log from start of command to error and crash:

Quartz v4.2.1

Cleaned output directory `public` in 3ms
Found 3 input files from `content` in 10ms
Parsed 3 Markdown files in 114ms
Filtered out 0 files in 46μs
Emitted 14 files to `public` in 62ms
Done processing 3 files in 191ms
Started a Quartz server listening at http://localhost:8080
hint: exit with ctrl+c
[200] /
[200] /index.css
[200] /prescript.js
[200] /postscript.js
[200] /static/contentIndex.json
[200] /Quests/Quest-Log
[404] /Adventurers
[200] /Build-Rules
Detected change, rebuilding...
Parsed 1 Markdown files in 44ms
Filtered out 0 files in 15μs
⠹ Emitting output files
<--- Last few GCs --->

[22440:0000021E689DDB20]   166362 ms: Mark-Compact 4046.6 (4135.3) -> 4035.6 (4139.8) MB, 939.96 / 0.00 ms  (average mu = 0.590, current mu = 0.019) allocation failure; scavenge might not succeed
[22440:0000021E689DDB20]   168154 ms: Mark-Compact 4051.5 (4139.8) -> 4040.0 (4144.3) MB, 1773.06 / 0.00 ms  (average mu = 0.314, current mu = 0.011) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 00007FF6B2F56E7B node::SetCppgcReference+16075
 2: 00007FF6B2ECD996 v8::base::CPU::num_virtual_address_bits+79190
 3: 00007FF6B2ECFBA5 v8::base::CPU::num_virtual_address_bits+87909
 4: 00007FF6B393D9E1 v8::Isolate::ReportExternalAllocationLimitReached+65
 5: 00007FF6B3927178 v8::Function::Experimental_IsNopFunction+1336
 6: 00007FF6B3788AA0 v8::Platform::SystemClockTimeMillis+659328
 7: 00007FF6B3785B28 v8::Platform::SystemClockTimeMillis+647176
 8: 00007FF6B379AE3A v8::Platform::SystemClockTimeMillis+733978
 9: 00007FF6B379B6B7 v8::Platform::SystemClockTimeMillis+736151
10: 00007FF6B37A42EE v8::Platform::SystemClockTimeMillis+772046
11: 00007FF6B37B4C36 v8::Platform::SystemClockTimeMillis+839958
12: 00007FF6B37B9808 v8::Platform::SystemClockTimeMillis+859368
13: 00007FF6B3530F9A v8::base::Thread::StartSynchronously+372186
14: 00007FF6B344E735 v8::CodeEvent::GetFunctionName+2549
15: 00007FF65399AAFA
fatal error: all goroutines are asleep - deadlock!

Desktop (please complete the following information):

Quartz Version: 4.1.2
node Version: v20.11.0
npm version: 10.2.4
OS: Windows 10 Pro build 19045.3930
Browser: Chrome

Additional context
Content and vault are symlinked through command mklink
Additional files and/or vault available on request.

The text was updated successfully, but these errors were encountered:

Maldor · 2024-02-04T05:25:14Z

Was able to replicate a second time, however no continuing edit was made to the content, ie didn't do step 4 in STRs
Error:

<--- Last few GCs --->

[2136:0000012E5B0BC3D0]   173148 ms: Mark-Compact 4047.9 (4137.1) -> 4036.7 (4141.3) MB, 915.05 / 0.00 ms  (average mu = 0.459, current mu = 0.020) allocation failure; scavenge might not succeed
[2136:0000012E5B0BC3D0]   174892 ms: Mark-Compact 4052.5 (4141.3) -> 4041.4 (4146.1) MB, 1724.38 / 0.00 ms  (average mu = 0.218, current mu = 0.011) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 00007FF6B2F56E7B node::SetCppgcReference+16075
 2: 00007FF6B2ECD996 v8::base::CPU::num_virtual_address_bits+79190
 3: 00007FF6B2ECFBA5 v8::base::CPU::num_virtual_address_bits+87909
 4: 00007FF6B393D9E1 v8::Isolate::ReportExternalAllocationLimitReached+65
 5: 00007FF6B3927178 v8::Function::Experimental_IsNopFunction+1336
 6: 00007FF6B3788AA0 v8::Platform::SystemClockTimeMillis+659328
 7: 00007FF6B3785B28 v8::Platform::SystemClockTimeMillis+647176
 8: 00007FF6B379AE3A v8::Platform::SystemClockTimeMillis+733978
 9: 00007FF6B379B6B7 v8::Platform::SystemClockTimeMillis+736151
10: 00007FF6B37A42EE v8::Platform::SystemClockTimeMillis+772046
11: 00007FF6B37B4C36 v8::Platform::SystemClockTimeMillis+839958
12: 00007FF6B37B9808 v8::Platform::SystemClockTimeMillis+859368
13: 00007FF6B361B639 v8::base::Thread::StartSynchronously+1332345
14: 00007FF6B3469A13 v8::CodeEvent::GetFunctionName+113875
15: 00007FF65399AAFA

jackyzha0 · 2024-02-04T06:04:45Z

@aarnphm can you take a look at this actually LOL i cannot reproduce this at all

jackyzha0 · 2024-02-04T06:11:25Z

Can you give us a link to a vault? Does this happen to you on the default vault?

Maldor · 2024-02-04T06:28:37Z

No, didn't seem to happen on default vault.
Link is https://vttnotes.maldor.tech/
Somehow, the last time I ran the build it was missing the index files (html, XML, css) and some others...

Maldor · 2024-02-04T06:38:31Z

And it just crashed without any action on my part...

<--- Last few GCs --->

[19432:000002058175AE60]   119631 ms: Mark-Compact 4048.9 (4136.7) -> 4037.8 (4141.2) MB, 917.77 / 0.00 ms  (average mu = 0.468, current mu = 0.020) allocation failure; scavenge might not succeed
[19432:000002058175AE60]   121327 ms: Mark-Compact 4053.7 (4141.2) -> 4042.5 (4146.0) MB, 1674.94 / 0.00 ms  (average mu = 0.226, current mu = 0.012) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 00007FF6B2F56E7B node::SetCppgcReference+16075
 2: 00007FF6B2ECD996 v8::base::CPU::num_virtual_address_bits+79190
 3: 00007FF6B2ECFBA5 v8::base::CPU::num_virtual_address_bits+87909
 4: 00007FF6B393D9E1 v8::Isolate::ReportExternalAllocationLimitReached+65
 5: 00007FF6B3927178 v8::Function::Experimental_IsNopFunction+1336
 6: 00007FF6B3788AA0 v8::Platform::SystemClockTimeMillis+659328
 7: 00007FF6B3785B28 v8::Platform::SystemClockTimeMillis+647176
 8: 00007FF6B379AE3A v8::Platform::SystemClockTimeMillis+733978
 9: 00007FF6B379B6B7 v8::Platform::SystemClockTimeMillis+736151
10: 00007FF6B37A42EE v8::Platform::SystemClockTimeMillis+772046
11: 00007FF6B37B4C36 v8::Platform::SystemClockTimeMillis+839958
12: 00007FF6B37B9808 v8::Platform::SystemClockTimeMillis+859368
13: 00007FF6B3530F9A v8::base::Thread::StartSynchronously+372186
14: 00007FF6B344E735 v8::CodeEvent::GetFunctionName+2549
15: 00007FF65399AAFA

If you want I can hop into a call in your discord to troubleshoot or get what you need, I have backups so I can mess with things

jackyzha0 · 2024-02-04T07:08:28Z

Can you link to the source? Want to see if I can reproduce

does the size of your content exceed 1.5gb (default stack size limit for node)
does this still happen without the symlink

Maldor · 2024-02-04T07:16:08Z

Source: Shillelagh.zip
Size is reported as 16 Kb (size on disk

I removed the entire setup (saving the config and layout files), redownloaded Quartz and then re-established the symlink. And it hasn't crashed yet.
I should point out that I created an semi-automatic batch script to automate some of the downloading and updating it and it wasn't used here.

Looking at some other unrelated things, its possible there might be an issue with the symlink, but only on my computer which will take some time to troubleshoot this. I'm not sure if its related or not yet.

jackyzha0 · 2024-02-04T07:18:54Z

Super weird, keep me posted. I haven't been able to reproduce this yet but let me know if you have something consistent

aarnphm · 2024-02-04T07:48:45Z

@jackyzha0 might be a apple silicon issue.

jackyzha0 · 2024-02-04T07:59:52Z

They are on a Windows machine

aarnphm · 2024-02-04T08:16:19Z

They are on a Windows machine

oh ok I forgot to ready the description 😄

aarnphm · 2024-02-04T10:38:31Z

I can reliably recreate this after editing source code and spamming refresh.

Will share some findings here later on.

Maldor · 2024-02-05T04:11:21Z

I've completed what troubleshooting I can to isolate this from my setups quirks and while I can't get everything isolated, I have been able to replicate this again still using a symlink, so I'll be removing the symlink in a second and trying again.
Log:

<--- Last few GCs --->

[26760:0000027B2DADDC30]   516426 ms: Mark-Compact 4046.4 (4136.1) -> 4035.2 (4140.3) MB, 1022.28 / 0.00 ms  (average mu = 0.458, current mu = 0.014) allocation failure; scavenge might not succeed
[26760:0000027B2DADDC30]   518298 ms: Mark-Compact 4051.0 (4140.3) -> 4040.0 (4145.3) MB, 1851.82 / 0.00 ms  (average mu = 0.222, current mu = 0.010) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 00007FF7AE8A6E7B node::SetCppgcReference+16075
 2: 00007FF7AE81D996 v8::base::CPU::num_virtual_address_bits+79190
 3: 00007FF7AE81FBA5 v8::base::CPU::num_virtual_address_bits+87909
 4: 00007FF7AF28D9E1 v8::Isolate::ReportExternalAllocationLimitReached+65
 5: 00007FF7AF277178 v8::Function::Experimental_IsNopFunction+1336
 6: 00007FF7AF0D8AA0 v8::Platform::SystemClockTimeMillis+659328
 7: 00007FF7AF0D5B28 v8::Platform::SystemClockTimeMillis+647176
 8: 00007FF7AF0EAE3A v8::Platform::SystemClockTimeMillis+733978
 9: 00007FF7AF0EB6B7 v8::Platform::SystemClockTimeMillis+736151
10: 00007FF7AF0F9FAF v8::Platform::SystemClockTimeMillis+795791
11: 00007FF7AEDBA565 v8::CodeEvent::GetFunctionName+116773
12: 00007FF7AF33EFAE v8::PropertyDescriptor::writable+677134
13: 00007FF7AF2B01B5 v8::PropertyDescriptor::writable+91925
14: 00007FF72F7BBD7A

jackyzha0 · 2024-02-05T04:16:05Z

Interesting, @aarnphm does your setup also use symlinks or separate?

Maldor · 2024-02-05T04:21:55Z

This is probably its own problem but when using build -serve it shows everything is linked correctly, however when uploaded to the website, all of the links just break down and throw 404s. Separate issue I assume?

aarnphm · 2024-02-05T04:25:17Z

Interesting, @aarnphm does your setup also use symlinks or separate?

separate.

aarnphm · 2024-02-05T04:25:31Z

This is probably its own problem but when using build -serve it shows everything is linked correctly, however when uploaded to the website, all of the links just break down and throw 404s. Separate issue I assume?

separate.

Maldor · 2024-02-05T04:44:04Z

Non-symlink still crashes:

<--- Last few GCs --->

[6624:000001E60755C4B0]   329520 ms: Mark-Compact 4051.7 (4138.9) -> 4040.5 (4143.7) MB, 860.18 / 0.00 ms  (average mu = 0.372, current mu = 0.024) allocation failure; scavenge might not succeed
[6624:000001E60755C4B0]   331212 ms: Mark-Compact 4056.3 (4143.7) -> 4045.3 (4148.4) MB, 1673.58 / 0.00 ms  (average mu = 0.167, current mu = 0.011) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 00007FF7AE8A6E7B node::SetCppgcReference+16075
 2: 00007FF7AE81D996 v8::base::CPU::num_virtual_address_bits+79190
 3: 00007FF7AE81FBA5 v8::base::CPU::num_virtual_address_bits+87909
 4: 00007FF7AF28D9E1 v8::Isolate::ReportExternalAllocationLimitReached+65
 5: 00007FF7AF277178 v8::Function::Experimental_IsNopFunction+1336
 6: 00007FF7AF0D8AA0 v8::Platform::SystemClockTimeMillis+659328
 7: 00007FF7AF0D5B28 v8::Platform::SystemClockTimeMillis+647176
 8: 00007FF7AF0EAE3A v8::Platform::SystemClockTimeMillis+733978
 9: 00007FF7AF0EB6B7 v8::Platform::SystemClockTimeMillis+736151
10: 00007FF7AF0F9FAF v8::Platform::SystemClockTimeMillis+795791
11: 00007FF7AEDBA565 v8::CodeEvent::GetFunctionName+116773
12: 00007FF7AF33EFAE v8::PropertyDescriptor::writable+677134
13: 00007FF7AF33F703 v8::PropertyDescriptor::writable+679011
14: 00007FF72F7BC6B7

Is there a flag I can set so that it gives more helpful information or is this all you need?

jackyzha0 · 2024-02-05T04:50:59Z

Can you run it in verbose mode (with the -v flag) and see if it crashes at any particular step?

Maldor · 2024-02-05T04:54:22Z

Will do. And also the quartz version in use is 4.2.1 not 4.1.2 as mentioned above

Maldor · 2024-02-05T05:07:55Z

Looks like it something that its not happy about with [emit:ComponentResources] public/postscript.js it kept stopping at that step and taking the longest, till it crashed. I've included everything from the last 3 rebuilds, but a total of 11 rebuilds were done before getting this.

Detected change, rebuilding...
Parsing input files using 1 threads
[process] content/Dungeon Master/DM Scratch paper....md -> Dungeon-Master/DM-Scratch-paper... (7s)
Parsed 1 Markdown files in 7s
Filtered out 0 files in 12μs
Emitting output files
[emit:ComponentResources] public/index.css
[emit:ComponentResources] public/prescript.js
[emit:ComponentResources] public/postscript.js
[emit:ContentPage] public/Adventurers.html
[emit:ContentPage] public/Build-Rules.html
[emit:ContentPage] public/Quests/Quest-Log.html
[emit:ContentPage] public/Sessions/Session-0.html
[emit:ContentPage] public/index.html
[emit:ContentPage] public/Dungeon-Master/DM-Scratch-paper....html
[emit:FolderPage] public/Quests/index.html
[emit:FolderPage] public/Sessions/index.html
[emit:FolderPage] public/Dungeon-Master/index.html
[emit:TagPage] public/tags/index.html
[emit:ContentIndex] public/sitemap.xml
[emit:ContentIndex] public/index.xml
[emit:ContentIndex] public/static/contentIndex.json
[emit:Static] public/static/icon.png
[emit:Static] public/static/og-image.png
[emit:404Page] public/404.html
Emitted 19 files to `public` in 1m
Done rebuilding in 2m
Detected change, rebuilding...
Parsing input files using 1 threads
Parsed 0 Markdown files in 1ms
Filtered out 0 files in 12μs
Emitting output files
[emit:ComponentResources] public/index.css
[emit:ComponentResources] public/prescript.js
[emit:ComponentResources] public/postscript.js

<--- Last few GCs --->

[14600:000001EEFB210080]   432085 ms: Mark-Compact 4042.8 (4135.3) -> 4031.4 (4139.8) MB, 1152.64 / 0.00 ms  (average mu = 0.390, current mu = 0.020) allocation failure; scavenge might not succeed
[14600:000001EEFB210080]   434101 ms: Mark-Compact 4047.4 (4139.8) -> 4036.0 (4144.3) MB, 1996.99 / 0.00 ms  (average mu = 0.185, current mu = 0.010) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 00007FF7AE8A6E7B node::SetCppgcReference+16075
 2: 00007FF7AE81D996 v8::base::CPU::num_virtual_address_bits+79190
 3: 00007FF7AE81FBA5 v8::base::CPU::num_virtual_address_bits+87909
 4: 00007FF7AF28D9E1 v8::Isolate::ReportExternalAllocationLimitReached+65
 5: 00007FF7AF277178 v8::Function::Experimental_IsNopFunction+1336
 6: 00007FF7AF0D8AA0 v8::Platform::SystemClockTimeMillis+659328
 7: 00007FF7AF0D5B28 v8::Platform::SystemClockTimeMillis+647176
 8: 00007FF7AF0EAE3A v8::Platform::SystemClockTimeMillis+733978
 9: 00007FF7AF0EB6B7 v8::Platform::SystemClockTimeMillis+736151
10: 00007FF7AF0F9FAF v8::Platform::SystemClockTimeMillis+795791
11: 00007FF7AEDBA565 v8::CodeEvent::GetFunctionName+116773
12: 00007FF7AF33EFAE v8::PropertyDescriptor::writable+677134
13: 00007FF7AF33F827 v8::PropertyDescriptor::writable+679303
14: 00007FF72F69FDF7
fatal error:

jackyzha0 · 2024-02-05T05:17:01Z

Seems like something esbuild specific in the script transpilation process, is there more to the error? Import cache busting in ESM is notoriously difficult

aarnphm · 2024-02-05T05:49:11Z

is there a -vvv for even more verbosity?

jackyzha0 · 2024-02-05T05:50:57Z

No lol, I already know the exact line it dies on quartz/plugins/emitters/componentResources.ts L64-L66 but this seems like esbuild. Nothing else there is computationally/memory expensive, it's just string concatenation that is <1MB

Maldor added the bug Something isn't working label Feb 4, 2024

MaelImhof mentioned this issue May 1, 2024

[BUG]: (Windows) EPERM: Operation not permitted, unlink: '<folder>\public\static\icon.png' #737

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Crash while using Build --serve and modifying vault content live #798

Crash while using Build --serve and modifying vault content live #798

Maldor commented Feb 4, 2024 •

edited

Loading

Maldor commented Feb 4, 2024

jackyzha0 commented Feb 4, 2024

jackyzha0 commented Feb 4, 2024

Maldor commented Feb 4, 2024

Maldor commented Feb 4, 2024

jackyzha0 commented Feb 4, 2024

Maldor commented Feb 4, 2024

jackyzha0 commented Feb 4, 2024

aarnphm commented Feb 4, 2024

jackyzha0 commented Feb 4, 2024

aarnphm commented Feb 4, 2024

aarnphm commented Feb 4, 2024

Maldor commented Feb 5, 2024

jackyzha0 commented Feb 5, 2024

Maldor commented Feb 5, 2024

aarnphm commented Feb 5, 2024

aarnphm commented Feb 5, 2024

Maldor commented Feb 5, 2024

jackyzha0 commented Feb 5, 2024

Maldor commented Feb 5, 2024

Maldor commented Feb 5, 2024

jackyzha0 commented Feb 5, 2024

aarnphm commented Feb 5, 2024

jackyzha0 commented Feb 5, 2024

Crash while using Build --serve and modifying vault content live #798

Crash while using Build --serve and modifying vault content live #798

Comments

Maldor commented Feb 4, 2024 • edited Loading

Maldor commented Feb 4, 2024

jackyzha0 commented Feb 4, 2024

jackyzha0 commented Feb 4, 2024

Maldor commented Feb 4, 2024

Maldor commented Feb 4, 2024

jackyzha0 commented Feb 4, 2024

Maldor commented Feb 4, 2024

jackyzha0 commented Feb 4, 2024

aarnphm commented Feb 4, 2024

jackyzha0 commented Feb 4, 2024

aarnphm commented Feb 4, 2024

aarnphm commented Feb 4, 2024

Maldor commented Feb 5, 2024

jackyzha0 commented Feb 5, 2024

Maldor commented Feb 5, 2024

aarnphm commented Feb 5, 2024

aarnphm commented Feb 5, 2024

Maldor commented Feb 5, 2024

jackyzha0 commented Feb 5, 2024

Maldor commented Feb 5, 2024

Maldor commented Feb 5, 2024

jackyzha0 commented Feb 5, 2024

aarnphm commented Feb 5, 2024

jackyzha0 commented Feb 5, 2024

Maldor commented Feb 4, 2024 •

edited

Loading