Scheduler fails to start with server when deployed on a cloud instance

[FrenchMajesty]

Package version

@adonis/core: 5.9.0
adonis5-scheduler: 2.1.1
Node: 20.3.1

Error Message & Stack Trace

I use the scheduler to run a CRON job on my Adonis application. Works perfectly fine locally. However, once deployed on a server, it fails with this error:

RuntimeException: E_INVALID_SCHEDULER_DIR: Not found task dir /workspace/build/app/Tasks
    at Function.notFoundTask (/workspace/node_modules/adonis5-scheduler/build/src/Exceptions/index.js:34:16)
    at Scheduler.run (/workspace/node_modules/adonis5-scheduler/build/src/Scheduler/index.js:103:57)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)

However, if I add a recursive console.log at the root of my server.ts, it outputs this:

DEFAULT 2023-12-28T06:01:52.743976Z 
All the files found for folder "build/app/Tasks" {
  [
    '/workspace/build/app/Tasks/SearchForTweet.js',
    '/workspace/build/app/Tasks/SearchForTweet.js.map'
  ]
}

Something is wrong where it's unable to find a folder that in facts exists and is present.

Relevant Information

Here is the recursive print.

This is being deployed on GCP's App Engine if that is any helpful. My app.yaml is simple it just has envVars and the target runtime.

runtime: nodejs20

env_variables:
  HOST: 'cool-host.com'

handlers:
  - url: /.*
    secure: always
    redirect_http_response_code: 301
    script: auto

[FrenchMajesty]

Upon gorking the module and adding debug tracking, it looks like the line that fails is the first one from Scheduler._fetchTask()

This is the real error message provided:

{"code":"ENOENT", "errno":-2, "hostname":"localhost", "level":60, "msg":""uncaughtException" detected", "name":"duome-api", "path":"/workspace/tmp/adonis5-scheduler/locks", "pid":11, "stack":"Error: ENOENT: no such file or directory, mkdir '/workspace/tmp/adonis5-scheduler/locks'

[FrenchMajesty]

Update:

I found out from App Engine docs that you cannot dynamically create files or folders except in the /tmp folder.

Problem is that on App Engine, puts all of your code in the folder /workspace. So this package was trying to create files in /workspace/tmp which would fail from a permission issue. The fix I used was simply to hardcode my target path to be do this:

// inside of Scheduler.ts
const oldSource = this.appRoot + '/tmp/adonis5-scheduler/locks' // <-- THIS WILL FAIL
const source = '/tmp/adonis5-scheduler/locks' // <-- DO THIS INSTEAD

And that should set you on your way.

Beyond the issue

However after much wrestling with trying to get this working, I do not recommend others to try to use this kind of package in production. Overall CRONs do not work well when they are ran alongside your server. You also have the problem of duplicate runs. If you cloud provider replicates your instance 6 times you are likely to have 6 CRONs running in parallel. Same issue if your traffic goes down and your instance scales down to 0 over night.

It is better to use your cloud provider's CRON scheduler. It will only run once at the given time and won't have the issues I've described above.

Happy coding.

[FrenchMajesty]

I think this package should provide an option to decide where to put the /locks folder because in some stateless cloud contexts, this might fail without a clear why.

The error message could also be more precise. Assuming all EONENT is the Task folder not findable is not helpful. We should still be passing down the raw error message down the chain just in case it's not what was initially assumed.