Skip to content
Mittal-AnalyticsPublic

About

These are small Django utilities we find helpful and are used across projects such as Screener.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

16 Commits

Folders and files

Repository files navigation

DjMAPL

These are small Django utilities we find helpful and are used across projects.

Utilities

call_concurrently

Runs an async function over many items, at most concurrency at a time, with an optional tqdm progress bar. Results are yielded in completion order. A failed item yields its exception, with a Failed item: <item> note, instead of raising. Check each result:

from djmapl.concurrency import call_concurrently

async for result in call_concurrently(
    concurrency=10,
    func=process_report,  # called as process_report(item, client=client)
    items=reports,
    show_progress=True,
    client=client,  # extra kwargs are passed to func
):
    if isinstance(result, Exception):
        # exc_info prints the traceback, including the `Failed item` note
        logger.error("Got error", exc_info=result)

progress_callback(pbar, result) runs after each item, even when the bar is hidden, e.g. to show a running total with pbar.set_postfix_str(...).

All items are scheduled up front, so split very large inputs into batches. If you break out of the loop early, the remaining items are cancelled when the generator is closed. Use contextlib.aclosing to close it right away:

from contextlib import aclosing

results = call_concurrently(...)
async with aclosing(results):
    async for result in results:
        if should_stop(result):
            break

set_logging_verbosity

Makes a management command's --verbosity control logging: 0 logs warnings and errors, 1 and up add info. For debug logs, raise your app's own loggers (e.g. when options["verbosity"] > 1), as debug on the root logger would show every library's debug logs.

from djmapl.commands import set_logging_verbosity

class Command(BaseCommand):
    @set_logging_verbosity
    def handle(self, *args, **options):
        logger.info("Shown with --verbosity=1 (the default)")

It sets the root logger's level and restores it afterwards. It adds a stderr handler only when the root logger has none, so a LOGGING setting's handlers are kept. A logger with its own level (or propagate = False) keeps that level.

Running Tests

uv run pytest

Publishing

uv version --bump minor
uv build

# save the credentials using `uv auth login upload.pypi.org`
uv publish --username __token__

About

These are small Django utilities we find helpful and are used across projects such as Screener.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages