-
Notifications
You must be signed in to change notification settings - Fork 165
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Memory leakage #113
Comments
From the docs: "The max bytes accepted per request can be specified with maxFilesSize." Also, while OT, why on Earth would you want to use Node at all for uploading files of that size? |
Oh..Let me clear that node doesnt support such large file uploads?@csvan |
@csharathreddy pretty sure it does if you just push your limiters high enough, but it just doesn't make sense. Anyway, the answer you are looking for is in the docs. |
No if set 'maxFilesSize=3 * 1024 * 1024' it accepts only 3MB data and then raises error event saying that "maximum file length exceeded" |
I also happen to be looking for a multipart parser that just streams directly to disk. I've taken the example code with these options: {
maxFieldsSize: Infinity,
maxFields: Infinity,
autoFiles: true,
uploadDir: __dirname
} Uploading a file bigger then the RAM + Swap of the machine seems to result in the process getting killed by the kernel oom killer eventually. Am I doing this wrong?
Could you elborate on why you think node it a bad fit? I don't see anything inherently wrong in wanting to upload big files to a node-based file server. |
Tried @dougwilson' suggestion with piping to The ramp ups at the start is when it's uploading. After the file has been written completely, it seems that around 1 GB memory isn't freed. The short spike afterwards is when I kill the process. I also tried periodic |
Just dug deeper into this and I think it's an io.js 3.0.0 issue at least in my case. @csharathreddy what node/io.js version are you seeing this on? Related: mscdex/busboy#92 and nodejs/node#2308 |
Ahh..I have tried with different versions of nodejs starting from 0.8.0 to latest. |
Hmm that's a different issue than mine then. Maybe try my test case, it shouldn't leak in anything except io.js 3.0.0. |
@csharathreddy we cannot even begin to answer why your code has a memory leak, as the code posted above is not runnable by us; if you are unable to diagnose the memory leak, we need a complete test case where we can run the code and reproduce the issue. |
Testcase provided by @silverwind and mine is similar.I have posted just uploading part of code.Use @silverwind test case to test. |
Thanks, @csharathreddy , I just tried that, but I get the following error:
Can you provide a complete test case where we can run the code and reproduce the issue? Trying to substitute your code into @silverwind 's code results in numerous undefined variables in your code. |
Use @silverwind code and try uploading a large of size >30 GB @dougwilson.my code has variables which are used as per my usecase |
Ok, well, @silverwind already confirmed there is no memory leak with his code; it's an io.js 3.0 issue. Closing this issue then, as we confirmed there's no memory leak based on your instructions, thank you! If you want us to try on your code instead of silverwind's code, we'd need a way to run it. |
Here,when I tried to upload file of size >30GB, RAM uasge increases from 10% to 98%.After reaching 98% memory doesnt get freed up after file upload completes.Is there size limit for multiparty?
The text was updated successfully, but these errors were encountered: