Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Background Event batching based delay #450

Closed
wants to merge 5 commits into from

Conversation

markvdouw
Copy link
Contributor

@markvdouw markvdouw commented Nov 13, 2023

Background Event batching based delay

Instructions

  1. PR target branch should be against development
  2. PR title name should follow this format: https:/mParticle/mparticle-workflows/blob/main/.github/workflows/pr-title-check.yml
  3. PR branch prefix should follow this format: https:/mParticle/mparticle-workflows/blob/main/.github/workflows/pr-branch-check-name.yml

Summary

  • Adding JobScheduler utilities with current and legacy code to trigger batch upload
  • Implementing as the legacy impl, a delayed message with type UPLOAD based on dynamic delay
  • Adding feature flag ebb to "enableBackgroundBatching", read from config and default=false Adding Service for uploading
  • Adding class in proguard
  • Business logic change at the upload handler level to dismiss automatic upload interval and upload at session end when batching is enabled.

Testing Plan

  • Was this tested locally? If not, explain why.
    Please consider that the JobScheduler specifies that the job will be triggered at MOST one time within the interval but there is no guarantee when the job will be trigger, meaning the delay can be very inexact and this could be a challange while testing.

Images are attached:
Screen Shot 2023-12-06 at 13 33 52

Reference Issue (For mParticle employees only. Ignore if you are an outside contributor)

Sam Dozor and others added 2 commits November 3, 2023 14:05
… batch upload

Implementing as the legacy impl, a delayed message with type UPLOAD based on dynamic delay
Adding feature flag ebb to "enableBackgroundBatching", read from config and default=false
Adding Service for uploading
Adding class in proguard
Business logic changes on AppStateManager to route to different implementations based on feature flag.
Business logic change at the upload handler level to dismiss automatic upload interval and upload at session end when batching is enabled.
@markvdouw markvdouw force-pushed the feat/SQDSDKS-5924-event-batching branch 2 times, most recently from 1cc545c to e655a1f Compare November 16, 2023 12:19
@markvdouw markvdouw force-pushed the feat/SQDSDKS-5924-event-batching branch from e655a1f to 910b714 Compare November 27, 2023 16:44
@markvdouw markvdouw force-pushed the feat/SQDSDKS-5924-event-batching branch from 910b714 to 99ae712 Compare November 29, 2023 15:07
Copy link
Member

@samdozor samdozor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't tested yet with the job scheduler working - here's what I found if a customer just upgrades to the SDK.

As part of this PR - we need to update the README of the SDK to have the docs for how to setup the scheduler.

SchedulingBatchingType.PERIODIC,
true,
delay -> {
sendEmptyDelayed(UPLOAD_MESSAGES, uploadInterval);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's something funky here. I tested by setting the upload interval to 30 seconds and set the SDK to production mode. When using the legacy upload mode - i'm not seeing uploads every 30s - it seems to be much quicker.

context.scheduleJob(builder.build())
} catch (e: Exception) {
Logger.error("Error while trying to use JobSceduler for batch upload")
legacyAction.invoke(delayInSeconds.secondsToMillis())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

delayInSeconds is already in milliseconds when it's passed in....

}
context.scheduleJob(builder.build())
} catch (e: Exception) {
Logger.error("Error while trying to use JobSceduler for batch upload")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When using this I get spammed with error messages saying "Error while trying to use JobSceduler for batch upload" (including the typo..).

Can we check programmatically if the service is there rather than relying on an exception?

And - rather than a vague error, we should log more or a warning that includes the code that they need to add to their manifest, and let them know that as a result of this, we may not be able to upload in the background.

Deprecating function in Media API
Minor business logic changes in AppStatemanager
Removing featrue flag on ConfigManager for platform consistency
@markvdouw markvdouw force-pushed the feat/SQDSDKS-5924-event-batching branch from 99ae712 to d973342 Compare December 1, 2023 17:19
sdk is backgrounded and there are no messages stored pending and the upload table is empty.
@markvdouw markvdouw closed this Dec 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants