redis: API startup ignores credentials and DB from REDIS_URL

[DivyamTalwar]

Problem

The API process and the ARQ worker build Redis connection settings differently. The worker parses the full REDIS_URL, but API startup and the local readiness check only use REDIS_HOST and REDIS_PORT.

This means authenticated, non-default database, username/password, or TLS Redis URLs can work for the worker while the API process connects with different settings or fails to connect.

Evidence

• core/workers/ingestion_worker.py uses RedisSettings.from_dsn(settings.REDIS_URL), which preserves the DSN host, port, database, username, password, and rediss:// TLS.
• core/app_factory.py builds the API Redis pool with only RedisSettings(host=settings.REDIS_HOST, port=settings.REDIS_PORT).
• start_server.py waits for Redis using only settings.REDIS_HOST and settings.REDIS_PORT.

Reproduction

Steps to reproduce:

1. Configure Redis with a DSN that includes credentials or a non-default database, for example:

   [redis]
   url = "redis://:secret@redis.example.com:6380/2"

2. Start the worker and API process.

3. Compare the Redis settings each path uses.

Actual behavior

The worker parses the full DSN and preserves redis.example.com, 6380, password secret, and database 2.

The API startup path ignores the DSN credentials and database because it constructs Redis settings from only host and port.

Expected behavior

API startup, worker startup, and readiness checks should derive Redis connection settings from the same source so REDIS_URL works consistently across the deployment.

The existing fallback behavior should still work when REDIS_URL is the default localhost URL and deployments override only REDIS_HOST or REDIS_PORT.

Impact

Managed Redis and security-hardened Redis deployments commonly require password, username, TLS, or non-default database settings. Dropping those settings can prevent Morphik from starting or can make the API and worker use different Redis databases.

Proposed fix

Centralize ARQ Redis settings construction in a shared helper, reuse it in the API lifespan and worker, and align the local readiness check with the same effective host/port target without logging credentials.

Validation plan

• Add unit coverage for DSN parsing with password/database/TLS.
• Add unit coverage for the existing default-URL plus REDIS_HOST/REDIS_PORT fallback.
• Add a test that API startup passes the shared Redis settings to arq.create_pool.
• Run targeted tests and lint for the changed files.

Contribution

I can open a focused PR for this.