-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add delayed enqueue for workers that support it #15
Comments
Hi @gaffneyc The reason I added it was to deal with a race condition I was having between my services. I would have a 99.5% miss with a index job on it's first try, since the job would try to run before the record actually existed. Since indexing was not something that needed to be async, I added the functionality to have it as a scheduled job with a configurable delay. |
@vinniefranco that's interesting, I'm curious what could be causing the index job to be enqueued before the record exists in the database. Sunspot should be calling index during Was this with ActiveRecord and a single master database? Was the record being saved through an association? I think adding a delay is the quickest and simplest approach but I would like to understand that root cause if possible. |
Actually... it looks like the index is being done in the after_save callback and not |
Yeah, I figured as much since it had a id passed and yes, you are correct the record was being saved through an association. Single master |
Yup. Pretty much what I've experienced. The worker would run the index before the actual record saved. |
I'm using delay_for to resolve race condition issue: sunspot is using after_save callback, index job will start before db commit sometimes. |
Same problem here with Sidekiq. The job starts always before the object is saved. |
I'll throw my hat in - race conditions started happening after a added additional 'after_save' callbacks to my model. Near 100% lookup miss on first try. |
Hopefully if sunspot/sunspot#534 gets merged this problem will go away. |
A couple forks have added delay support. I don't fully understand the reasoning or what is causing the need for it. My best assumption is that the worker system is pulling the index job off the queue before the record can be saved to the database.
The text was updated successfully, but these errors were encountered: