-
Notifications
You must be signed in to change notification settings - Fork 119
Add retention policy knowledge to the privacy domain #177
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| codeunit 50563 "Contoso Activity Log Cleanup" | ||
| { | ||
| Access = Internal; | ||
|
|
||
| // The log table is never added to the allowed tables, so it cannot appear | ||
| // on the Retention Policies page. Cleanup is hard-coded here instead: | ||
| // the period is not configurable, the deletion is not written to the | ||
| // Retention Policy Log, and an administrator cannot switch it off. | ||
| trigger OnRun() | ||
| var | ||
| ContosoActivityLog: Record "Contoso Activity Log"; | ||
| begin | ||
| ContosoActivityLog.SetFilter( | ||
| SystemCreatedAt, '<%1', CreateDateTime(CalcDate('<-30D>', Today()), 0T)); | ||
| ContosoActivityLog.DeleteAll(); | ||
| end; | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| codeunit 50560 "Contoso Reten. Pol. Setup" | ||
| { | ||
| Access = Internal; | ||
|
|
||
| procedure AddAllowedTables() | ||
| var | ||
| ContosoActivityLog: Record "Contoso Activity Log"; | ||
| RetenPolAllowedTables: Codeunit "Reten. Pol. Allowed Tables"; | ||
| UpgradeTag: Codeunit "Upgrade Tag"; | ||
| begin | ||
| if UpgradeTag.HasUpgradeTag(AllowedTableTag()) then | ||
| exit; | ||
|
|
||
| if not RetenPolAllowedTables.IsAllowedTable(Database::"Contoso Activity Log") then | ||
| RetenPolAllowedTables.AddAllowedTable( | ||
| Database::"Contoso Activity Log", | ||
| ContosoActivityLog.FieldNo(SystemCreatedAt), | ||
| 28); // support cases need at least four weeks of log history | ||
|
|
||
| UpgradeTag.SetUpgradeTag(AllowedTableTag()); | ||
| end; | ||
|
|
||
| local procedure AllowedTableTag(): Code[250] | ||
| begin | ||
| exit('Contoso-ActivityLogAllowedTable-20260910'); | ||
| end; | ||
| } | ||
|
|
||
| codeunit 50561 "Contoso Reten. Pol. Install" | ||
| { | ||
| Subtype = Install; | ||
| Access = Internal; | ||
|
|
||
| trigger OnInstallAppPerCompany() | ||
| var | ||
| ContosoRetenPolSetup: Codeunit "Contoso Reten. Pol. Setup"; | ||
| begin | ||
| ContosoRetenPolSetup.AddAllowedTables(); | ||
| end; | ||
| } | ||
|
|
||
| codeunit 50562 "Contoso Reten. Pol. Upgrade" | ||
| { | ||
| Subtype = Upgrade; | ||
| Access = Internal; | ||
|
|
||
| // Install code does not run on upgrade, so tenants that already have the | ||
| // app get their registration here. | ||
| trigger OnUpgradePerCompany() | ||
| var | ||
| ContosoRetenPolSetup: Codeunit "Contoso Reten. Pol. Setup"; | ||
| begin | ||
| ContosoRetenPolSetup.AddAllowedTables(); | ||
| end; | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| --- | ||
| bc-version: [all] | ||
| domain: privacy | ||
| keywords: [retention-policy, allowed-tables, addallowedtable, reten-pol-allowed-tables, log-table-growth, mandatory-minimum-retention, install-upgrade-codeunit] | ||
| technologies: [al] | ||
| countries: [w1] | ||
| application-area: [all] | ||
| --- | ||
|
|
||
| # Extension-owned log tables must be registered as retention-policy allowed tables | ||
|
|
||
| ## Description | ||
|
|
||
| The retention policy engine only ever deletes from tables that appear in its allowed-tables list, and an extension may register only tables it owns — it cannot add a base application table or a table from another extension. Registration is a call to `Codeunit "Reten. Pol. Allowed Tables".AddAllowedTable`, passing the table ID and the field number of the Date or DateTime field that ages each record (`SystemCreatedAt` is the usual choice). Until that call has run in a company, the table cannot be selected on the **Retention Policies** page at all, so an activity log, integration log, or archive table the extension writes to grows with no supported way for an administrator to trim it. The registration is stored per company and is not part of the table's metadata — it exists only because install or upgrade code put it there. | ||
|
|
||
| ## Best Practice | ||
|
|
||
| Register every table the extension owns that accumulates rows over time: activity and audit logs, integration and API request logs, archived documents. Call a shared routine from both the install codeunit (`OnInstallAppPerCompany`) and an upgrade codeunit (`OnUpgradePerCompany`), because install code does not run when an existing installation moves to a new version — registration added only to install code never reaches tenants that already have the app. Guard the routine with `IsAllowedTable` and an upgrade tag so repeated runs are idempotent. Pass `MandatoryMinRetenDays` when the data must survive a minimum period for audit or support reasons; the platform then rejects any shorter period an administrator configures. When only a subset of rows should ever expire, build the filter with `AddTableFilterToJsonArray` and pass it to the `AddAllowedTable` overload that takes a `JsonArray` — a filter added as locked cannot be removed later by the administrator. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Install and upgrade coverage is necessary, but not sufficient. The Retention Policies pages raise |
||
|
|
||
| Registering a table only makes it eligible; the policy itself is a separate concern, covered by [`ship-a-default-retention-policy-setup.md`](ship-a-default-retention-policy-setup.md). | ||
|
|
||
| See sample: [`register-owned-log-tables-for-retention-policies.good.al`](register-owned-log-tables-for-retention-policies.good.al). | ||
|
|
||
| ## Anti Pattern | ||
|
|
||
| An extension-owned log table with no `AddAllowedTable` call anywhere in the app, cleaned instead by hand-rolled code — a job queue codeunit or scheduled task running `DeleteAll` against a hard-coded date window. The deletion happens outside the **Retention Policy Log**, the administrator has no page on which to lengthen, shorten, or disable it, and the table is invisible during a data-retention review. The same defect in slower form is registration performed only in the install codeunit: new tenants are covered, every existing tenant stays unregistered after the upgrade. | ||
|
|
||
| See sample: [`register-owned-log-tables-for-retention-policies.bad.al`](register-owned-log-tables-for-retention-policies.bad.al). | ||
|
|
||
| ## References | ||
|
|
||
| - [Clean up data with retention policies](https://learn.microsoft.com/en-us/dynamics365/business-central/admin-data-retention-policies), section *Include your extension in a retention policy*. | ||
| - [`RetenPolAllowedTables.Codeunit.al`](https://github.com/microsoft/BCApps/blob/main/src/System%20Application/App/Retention%20Policy/src/Retention%20Policy%20Allowed%20Tables/RetenPolAllowedTables.Codeunit.al) in microsoft/BCApps — the `AddAllowedTable` overloads, `IsAllowedTable`, and `AddTableFilterToJsonArray`. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| codeunit 50565 "Contoso Reten. Pol. Register" | ||
| { | ||
| Subtype = Install; | ||
| Access = Internal; | ||
|
|
||
| // The table becomes selectable on the Retention Policies page and nothing | ||
| // else happens: no Retention Policy Setup record is created, so no period | ||
| // is proposed and no data is ever deleted. The log table keeps growing | ||
| // until someone notices the tenant's storage consumption. | ||
| trigger OnInstallAppPerCompany() | ||
| var | ||
| ContosoActivityLog: Record "Contoso Activity Log"; | ||
| RetenPolAllowedTables: Codeunit "Reten. Pol. Allowed Tables"; | ||
| begin | ||
| RetenPolAllowedTables.AddAllowedTable( | ||
| Database::"Contoso Activity Log", ContosoActivityLog.FieldNo(SystemCreatedAt)); | ||
| end; | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| codeunit 50564 "Contoso Reten. Pol. Default" | ||
| { | ||
| Access = Internal; | ||
|
|
||
| var | ||
| SixMonthsTok: Label 'Six Months', MaxLength = 20; | ||
|
|
||
| procedure CreateDefaultPolicy() | ||
| var | ||
| RetentionPolicySetup: Record "Retention Policy Setup"; | ||
| UpgradeTag: Codeunit "Upgrade Tag"; | ||
| begin | ||
| // Created once per company: an administrator who deletes the policy | ||
| // does not get it back on the next upgrade. | ||
| if UpgradeTag.HasUpgradeTag(DefaultPolicyTag()) then | ||
| exit; | ||
|
|
||
| if not RetentionPolicySetup.Get(Database::"Contoso Activity Log") then begin | ||
| RetentionPolicySetup.Validate("Table Id", Database::"Contoso Activity Log"); | ||
| RetentionPolicySetup.Validate("Apply to all records", true); | ||
| RetentionPolicySetup.Validate("Retention Period", SixMonthRetentionPeriod()); | ||
| RetentionPolicySetup.Validate(Enabled, false); // the administrator opts in to deletion | ||
| RetentionPolicySetup.Insert(true); | ||
| end; | ||
|
|
||
| UpgradeTag.SetUpgradeTag(DefaultPolicyTag()); | ||
| end; | ||
|
|
||
| local procedure SixMonthRetentionPeriod(): Code[20] | ||
| var | ||
| RetentionPeriod: Record "Retention Period"; | ||
| begin | ||
| RetentionPeriod.SetRange("Retention Period", RetentionPeriod."Retention Period"::"6 Months"); | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please use the public |
||
| if RetentionPeriod.FindFirst() then | ||
| exit(RetentionPeriod.Code); | ||
|
|
||
| RetentionPeriod.Code := CopyStr(UpperCase(SixMonthsTok), 1, MaxStrLen(RetentionPeriod.Code)); | ||
| RetentionPeriod.Description := SixMonthsTok; | ||
| RetentionPeriod.Validate("Retention Period", RetentionPeriod."Retention Period"::"6 Months"); | ||
| RetentionPeriod.Insert(true); | ||
| exit(RetentionPeriod.Code); | ||
| end; | ||
|
|
||
| local procedure DefaultPolicyTag(): Code[250] | ||
| begin | ||
| exit('Contoso-ActivityLogDefaultPolicy-20260910'); | ||
| end; | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| --- | ||
| bc-version: [all] | ||
| domain: privacy | ||
| keywords: [retention-policy-setup, retention-period, default-policy, unbounded-table-growth, opt-in-deletion, upgrade-tag] | ||
| technologies: [al] | ||
| countries: [w1] | ||
| application-area: [all] | ||
| --- | ||
|
|
||
| # Registering a table is not a retention policy — ship a default setup | ||
|
|
||
| ## Description | ||
|
|
||
| `AddAllowedTable` only makes a table selectable on the **Retention Policies** page. Nothing is deleted until a `Retention Policy Setup` record exists for that table, names a `Retention Period`, and is enabled. An extension that registers its log tables and stops there ships unbounded growth as its default behaviour: the administrator has to discover the page, know which of the extension's tables are safe to trim, and pick a period the extension's own author never documented. Microsoft's `Codeunit 3907 "Retention Policy Installer"` shows the intended shape — it registers `Retention Policy Log Entry`, then creates a setup record with a six-month period on first install, inserted disabled, guarded by an upgrade tag so a policy the administrator later deleted is not recreated on the next upgrade. | ||
|
|
||
| ## Best Practice | ||
|
|
||
| In the same install and upgrade routine that registers the table (see [`register-owned-log-tables-for-retention-policies.md`](register-owned-log-tables-for-retention-policies.md)), create the `Retention Policy Setup` record: reuse an existing `Retention Period` whose `"Retention Period"` enum value matches the period you want and create one only when none exists, then `Validate` `"Table Id"`, `"Apply to all records"` and `"Retention Period"` before inserting. Gate the creation on an upgrade tag so it happens once per company rather than on every upgrade. Default to inserting with `Enabled` set to false: pre-creating the line puts a reviewed, sensible period in front of the administrator while leaving the decision to delete tenant data with them. Shipping the policy enabled is defensible for rows that are purely diagnostic and documented as transient — state that choice, and the default period, in the app's onboarding material either way. | ||
|
|
||
| See sample: [`ship-a-default-retention-policy-setup.good.al`](ship-a-default-retention-policy-setup.good.al). | ||
|
|
||
| ## Anti Pattern | ||
|
|
||
| Install code that calls `AddAllowedTable` and treats the table as covered by retention policies. The signal is an install or upgrade routine that touches `Codeunit "Reten. Pol. Allowed Tables"` but never `Record "Retention Policy Setup"`; the symptom is a support case where the extension's log table holds millions of rows on a tenant whose **Retention Policies** page has no line for it. The mirror-image defect is inserting the setup with `Enabled` set to true and no documentation, so the app begins deleting tenant data on a schedule nobody approved. | ||
|
|
||
| See sample: [`ship-a-default-retention-policy-setup.bad.al`](ship-a-default-retention-policy-setup.bad.al). | ||
|
|
||
| ## References | ||
|
|
||
| - [Clean up data with retention policies](https://learn.microsoft.com/en-us/dynamics365/business-central/admin-data-retention-policies) — retention periods, enabling a policy, and the job queue entry that applies it. | ||
| - [`RetentionPolicyInstaller.Codeunit.al`](https://github.com/microsoft/BCApps/blob/main/src/System%20Application/App/Retention%20Policy/src/Install/RetentionPolicyInstaller.Codeunit.al) in microsoft/BCApps — the platform's own register-then-create-disabled-setup pattern. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Retention policies and these System Application APIs were introduced with Business Central 2020 release wave 2 (v17), so
[all]makes both articles applicable to versions where the objects do not exist. Please scope both articles tobc-version: [17..], consistent with how this corpus versions other platform features.