s3 byte limit too low

[tooptoop4]

i saw

argo-events/sensors/artifacts/s3.go

Line 64 in a98978a

b, err := io.ReadAll(io.LimitReader(obj, 65536))

which means can only read data from s3 that is less than 65kb

its also common for logs to be larger than this

[whynowy]

hmmm.... we should increase it...

[VaibhavPage]

@whynowy what was the reason to add LimitReader?

[whynowy]

@whynowy what was the reason to add LimitReader?

We had a security audit that we used ioutil.ReadAll() in many places which would potentially cause DoS, then I added the limit to all the places, which might have overkilled. See #1946.

[whynowy]

@tooptoop4 - did you really experience any issue with this? This one is for reading the workflow spec definition from S3, we of course can increase the limit to 1 MB also, but it does not make sense to make it more than that.

[tooptoop4]

yes, my workflow is more than 65kb

[whynowy]

Has been raised to 1 MB in PR #2055