-
-
Notifications
You must be signed in to change notification settings - Fork 745
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
pageres swamping server with requests #416
Comments
I just tried on a local web-server and I cannot reproduce this. It correctly does a single request. Pageres doesn't have any retry logic, so either there's a bug in Puppeteer (headless Chrome) or the server is misconfigured. |
Thanks for the reply. I witnessed that on two servers. Let me know if you're interested in a URL to check it out -- I'll just have to send it to you privately.
…On Wed, Jun 23, 2021 at 05:03:56AM -0700, Sindre Sorhus wrote:
I just tried on a local web-server and I cannot reproduce this. It correctly does a single request. Pageres doesn't have any retry logic, so either there's a bug in Puppeteer or the server is misconfigured.
--
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub:
#416 (comment)
|
Try the latest version: https:/sindresorhus/pageres/releases/tag/v7.0.0 It may have resolved the issue. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Fresh pageres-cli installed today, it hammers the server with repeat requests before ultimately stopping and successfully producing the screenshot.
I noticed the issue on a production server when the generated screenshots were all errors with HTTP code 429 ("too many requests").
$ pageres http://www.mytestsite.com
Server logs, see timestamps (goes much faster on production server)
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:55 +0300] "GET / HTTP/1.1" 200 5736 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.045 "- : 0.044"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:55 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.042 "- : 0.040"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:56 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.046 "- : 0.048"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:56 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.047 "- : 0.048"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:56 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.148 "- : 0.148"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:56 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.049 "- : 0.048"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:56 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.046 "- : 0.048"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:56 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.044 "- : 0.044"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:57 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.045 "- : 0.044"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:57 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.061 "- : 0.060"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:57 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.055 "- : 0.056"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:57 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.044 "- : 0.044"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:57 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.064 "- : 0.064"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:57 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.052 "- : 0.052"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:58 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.052 "- : 0.052"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:58 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.054 "- : 0.052"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:58 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.052 "- : 0.052"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:58 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.045 "- : 0.044"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:58 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.043 "- : 0.044"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:58 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.042 "- : 0.040"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:58 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.051 "- : 0.048"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:58 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.048 "- : 0.048"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:59 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.039 "- : 0.040"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:59 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.056 "- : 0.056"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:59 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.055 "- : 0.052"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:59 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.044 "- : 0.044"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:59 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.042 "- : 0.044"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:59 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.042 "- : 0.044"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:09:00 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.063 "- : 0.064"
[...]
The text was updated successfully, but these errors were encountered: