Skip to content

Implement concurrent processing using multiprocessing for efficient CPU utilization #293

Description

@hakancelikdev

Unimport's work (AST analysis and libcst refactoring) is CPU-bound, so files should be processed in parallel processes rather than with asyncio: asyncio only overlaps I/O waits, threads are limited by the GIL, and the analyzers keep per-file state on class attributes, which makes threads unsafe.

Proposed: a -j/--jobs N option (and jobs config key) that analyzes files in a process pool, keeping output, prompts and file writes in the main process in input order.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

changelogChanges should be written to the changelog file.enhancementNew feature or request

Projects

Relationships

None yet

Development

No branches or pull requests

Issue actions