Repository navigation
Conversation
plone-exporter accepts --path (repeatable) and --paths FILE, with content paths relative to the site root. Each path selects that content and everything inside it. Paths without content are skipped with a warning; the export aborts if none is left. The paths become options.query, used by the content exporter. The relations, translations, discussions, portlets and redirects exporters keep only data related to the exported content. Principals are exported in full. Refs #86
Each ZIP entry stores the file modification time, with a 2 second resolution, so two exports made a few seconds apart differ byte by byte even when their data is the same. Compare file names and contents instead.
|
@ericof thanks for creating this Pull Request and helping to improve Plone! TL;DR: Finish pushing changes, pass all other checks, then paste a comment: To ensure that these changes do not break other parts of Plone, the Plone test suite matrix needs to pass, but it takes 30-60 min. Other CI checks are usually much faster and the Plone Jenkins resources are limited, so when done pushing changes and all other checks pass either start all Jenkins PR jobs yourself, or simply add the comment above in this PR to start all the jobs automatically. Happy hacking! |
|
@jenkins-plone-org please run jobs |
|
Nice addition, but unfortunately for sites that have some folders with LOTS of content inside, that's not going to be that easy to handle with this approach. That's why I created the The three options are all valid (full export, path based, or custom logic), so we don't need to remove any of them, but ensure we keep all them working. 👍 |
|
@gforcada Could we merge this, then? |
3 participants
Summary
Adds a partial export to
plone-exporter: only the listed content, and everything inside it, is exported. The request is described in #86: exports of huge sites are too large to hand to developers or external partners, and may contain data that should not leave the site.Command line
Two new options, which can be combined:
--path CONTENT_PATH: a content path, relative to the site root. Can be repeated.--paths FILE: a file with one content path per line. Empty lines and lines starting with#are ignored./about/teamdoes not export/aboutor the site root./folder/pageto/page.Other exporters
When a partial export is requested, the other exporters keep only data related to the exported content:
Implementation
options.query.ContentExporteralready usedoptions.query; an explicitqueryargument toexport_datastill takes precedence.BaseExporter.exported_content()returns the UID and path of the exported content, orNonefor a full export. It runs the catalog query itself, so every exporter also works on its own.uids(orpaths) argument. Existing callers are unaffected.CLI_SPECoptions are nowargparse.add_argumentkeyword dicts, instead of help strings.Tests
tests/exporters/test_exporters_partial.py: content and each filtered exporter, on the multilingual fixture. Every filtering test fails when the filtering is disabled.tests/cli/test_cli_exporter_partial.py: the real CLI entry point with--pathand--paths.tests/utils/test_utils_cli.py: path file parsing and query building.This PR also fixes an intermittent failure in
test_export_import_roundtrip. The test compared two exported ZIP files byte by byte. Each ZIP entry stores the file modification time, with a 2 second resolution, so two exports made a few seconds apart could differ even though their data was the same. The test now compares file names and contents.Documentation
The command line options need to be documented in
docs/admin-guide/export-import.mdin plone/documentation. A follow-up issue will be opened there.Closes #86