Browser version mismatch between Windows development and AWS deployment

[wojtekKrol]

Which package is this bug report for? If unsure which one to select, leave blank

@crawlee/playwright (PlaywrightCrawler)

Issue description

Description

I'm developing a Crawlee-based application on Windows and deploying it on AWS (Amazon Linux 2). I'm encountering an issue where the application is unable to locate the correct browser directory after deployment.

Environment

• Development: Windows
• Deployment: AWS Elastic Beanstalk (Amazon Linux 2, Fedora-based)
• Crawlee version: 3.11.1
• Playwright version: 1.44.1
• Node.js version: >=18

Issue

When running the application on AWS, I receive the following error:

ERROR: Error processing batch: {
  "errorName": "BrowserLaunchError",
  "errorMessage": "Failed to launch browser. Please check the following:
  - Try installing the required dependencies by running `npx playwright install --with-deps` (https://playwright.dev/docs/browsers).

The original error is available in the `cause` property. Below is the error received when trying to launch a browser:
​",
  "stackTrace": "Failed to launch browser. Please check the following:
  - Try installing the required dependencies by running `npx playwright install --with-deps` (https://playwright.dev/docs/browsers).

The original error is available in the `cause` property. Below is the error received when trying to launch a browser:
​
browserType.launchPersistentContext: Executable doesn't exist at /home/webapp/.cache/ms-playwright/chromium-1117/chrome-linux/chrome

Details

• On Windows (development environment), Playwright installs Chromium version 1117.
• On Amazon Linux 2 (deployment environment), Playwright installs Chromium version 1140.
• The application is trying to locate the browser in the directory for version 1117, which doesn't exist on the deployment server.

Installation Process

I've set up the following to install Chromium and its dependencies:

1. In .ebextensions/chromium.config, I install necessary packages:

packages:
  yum:
    cups-libs: []
    dbus-glib: []
    libXrandr: []
    # ... (other packages)
    openssl-devel: []
    jq: []

commands:
  refresh-fonts:
    command: fc-cache -f -v
    
container_commands:
  enable_extra_packages:
    command: "sudo amazon-linux-extras install epel -y"
  install_chromium:
    command: "sudo yum install -y chromium" 

2. In .platform/hooks/predeploy/chromium.sh, I run the following script:

#!/bin/bash
# ... (log setup)

sudo -iu webapp bash << 'EOF'
# ... (logging functions)

log_message "Remove package-lock & node_modules"
rm -rf node_modules package-lock.json

log_message "npm installation"
npm install   

log_message "Uninstall all playwright browsers"
npx playwright uninstall --all

log_message "Install chromium to webapp/.cache"
npx playwright install chromium

# ... (check if chromium is installed)
EOF

Attempted Solution

I tried renaming the directory on the deployment server from chromium-1140 to chromium-1117, but this did not resolve the issue.

Questions

1. Why my application is trying to use chromium-1117 ? becasue I'm sending built app? But im building on jenkins pipeline so it's built on linux as well

Additional Information

• Full package.json can be provided if needed.

Code sample

No response

Package version

3.11.1

Node.js version

18

Operating system

Amazon Linux 2 (Fedora underneath)

Apify platform

• Tick me if you encountered this issue on the Apify platform

I have tested this on the next release

No response

Other context

No response