Repository navigation
UNOMI-1000: Run background tasks for every tenant - #911
Merged
Merged
Conversation
Profile purge, segment date recalculation, rule statistics refresh and router startup only saw the system tenant. Walk each tenant with executeAsTenant (restoring context afterward) instead of widening the system persistence filter.
Disable the scheduled profile purge in ITs so years-old snapshot data is not deleted once purge walks every tenant. Also harden tenant task execution and use tenant-scoped router route IDs.
A tenant that cannot be switched to during a route refresh no longer drops the refreshes already taken off the queue for the other tenants, nor skips the export pass of that tick. Also quiet the startup retry (stack trace once per tenant), stop the purge and segment tasks from logging "executed" after a tenant failed, log a missing export configuration as an error like the import one, and tighten two tests that could pass for the wrong reason. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…o end Router startup adds each recurrent route on its own once the Camel context runs, so a configuration whose route cannot be built is handed to the route refresh instead of stopping the routes of every tenant. Removing a route also removes a definition left by a failed start, and the refresh retries, then gives up on, a tenant it cannot switch to rather than dropping its queued updates. The date-relative recalculation now includes the segments and scoring plans inherited from the system tenant, which apply to every tenant's profiles. The rule statistics sync skips tenants with nothing to sync. Adds BackgroundTenantTasksIT (tenant rule statistics, system segment applied to tenant profiles, same import configuration id in two tenants), unit tests for the per-tenant helper, session and event purge and the router failure paths, and an upgrade note in the 3.0 to 4.0 migration guide. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Removing what a failed attempt left in the Camel context no longer keeps the configuration from being marked ROUTE_CREATION_FAILED when the refresh gives up. A tenant that cannot be switched to logs its stack trace once instead of once per queued configuration, and the related error messages name what actually failed. Covers the statistics sync skipping tenants with nothing to sync, makes the rule statistics IT wait until nothing is left to flush, keeps the routes IT cleanup from hiding the test's own failure, and notes in the migration guide that the migration tenant identifier must follow the tenant identifier rules. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A rule only applies to events of its own scope, so the event sent by ruleStatisticsOfATenantRuleArePersisted never fired the rule and the test failed before reaching the statistics refresh it is about. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Plain-language summary
Several background jobs in 4.0 only saw the internal "system" tenant. Purge, daily segment/scoring updates, rule statistics refresh and router startup after a restart skipped real tenants (including "default", where migrated data lives). Other jobs such as rule refresh already looped over every tenant. This PR applies the same pattern, and fixes the problems that running per tenant exposed.
What changed
Background tasks
executeAsSystemfor permissions, then run under each tenant (system included) through a sharedexecuteForEachTenanthelper inAbstractMultiTypeCachingService. One tenant's failure does not stop the others, and the context is restored afterward.syncRuleStatistics()no longer copies the system tenant's statistics into each tenant's in-memory map. Once the sync ran per tenant, that copy made a tenant save system statistics under its own tenant id and the system counts were lost. Reads already fall back to the system tenant. The sync skips tenants with nothing to sync.Router
ROUTE_CREATION_FAILED) and does not keep the routes of other configurations or tenants from starting.<tenantId>:<configId>(newRouteIdsinrouter-api). Configuration ids are only unique within a tenant, so two tenants using the same id previously shared one route id, and updating or deleting one tenant's configuration removed the other tenant's route.IRouterCamelContext.killExistingRoute,updateProfileImportReaderRouteandupdateProfileExportReaderRoutetake the tenant id as a new first parameter.RouterCamelContextis the only implementation and caller in this repository; a plugin calling these methods has to pass the tenant.NullPointerException.RouteIds.ofrefuses a tenant id containing a colon.Documentation
migrate-3.0-to-4.0.adoc: new section on background tasks and recurrent routes running for every tenant (first purge after upgrade, task failure reporting, route id format,IRouterCamelContextchange).Integration tests
New
BackgroundTenantTasksIT(registered inAllITsandCorePersistenceITs): statistics of a tenant rule are persisted for that tenant; a system date-relative segment is applied to a tenant profile by the recalculation; two tenants using the same import configuration id each get their own route, and deleting one keeps the other.The scheduled purge is switched off in the IT container. Now that it reaches every tenant, it deleted the years-old snapshot data of the migration ITs seconds after startup (the
Migrate16xToCurrentVersionITfailure on the first CI run of this PR). No IT relies on the scheduled purge; the purge ITs call the purge methods directly.The Camel route helpers in
BaseITtake the tenant id and the configuration id.Tests
BackgroundTenantTasksTest: purge (profiles, sessions, events), segment recalculation (including a system-owned segment) and rule statistics are checked on data in two tenants through the real scheduled executors; one tenant failing is reported and leaves the others processed; an interrupted run is not reported as done; system statistics stay in the system tenant over several ticks.AbstractMultiTypeCachingServiceTest:executeForEachTenantdirectly (every tenant plus system, a failing tenant, an interruption, both callback outcomes).RouterCamelContextTenantLoadTest: the real route setup builds the routes of two tenants and the system tenant sharing the same configuration ids; removing one tenant's route keeps the other's; a configuration whose route cannot be built does not block the others and ends up marked failed; startup failures are retried with back-off; a queued removal wins over the startup retry; refreshes of an unswitchable tenant are retried then given up on.RouteIdsTest,RouteCompletionProcessorTest: route id round trip (including colons in configuration ids) and completion through a tenant-qualified route id.Known limits
Test plan
mvn -pl services-common,services,extensions/router/router-api,extensions/router/router-core clean install(unit tests: services 820, services-common 51, router-core 99)🤖 Generated with Claude Code