Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 22 additions & 2 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ jobs:
- name: Run Lab 05 (persistent data directory)
run: python labs/05_embedded_persistent.py

self-contained:
external-self-contained:
# Path B: external self-contained server -> runs the native apphost, no system .NET.
# We do not set up .NET; the lab uses a bogus dot_net_path so a successful run proves the
# apphost path is genuinely dotnet-free even though the runner happens to ship a .NET.
Expand Down Expand Up @@ -93,5 +93,25 @@ jobs:
SERVER="$(find server -type d -name Server | head -1)"
RAVENDB_SELF_CONTAINED_SERVER="$SERVER" python labs/02_embedded_external_server.py

- name: Run Lab 03 (on-demand cached self-contained, no system .NET)
on-demand:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]

steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0

- name: Set up Python 3.13
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: "3.13"

- name: Install package
run: |
python -m pip install --upgrade pip
pip install -e .

- name: Run Lab 03 (on-demand self-contained, no system .NET)
run: python labs/03_on_demand_server.py
15 changes: 9 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# ravendb-embedded

`ravendb-embedded` runs a real RavenDB server from inside your Python program. You `pip install`
it, start the server in-process, and talk to it with the normal `ravendb` client. There is no
separate server to install, configure, or keep running: the server's lifetime follows your
process.
`ravendb-embedded` starts a real RavenDB server as a child process managed by your Python program.
You `pip install` it, start the server, and talk to it with the normal `ravendb` client. There is
no separate server to install, configure, or keep running: the server process's lifetime follows
your Python process.

Reach for it when you want:

Expand Down Expand Up @@ -95,14 +95,16 @@ Download the Server package for your platform from the [RavenDB downloads page](
the server files live in the archive's `Server/` folder. Runnable walkthrough:
[`labs/02-embedded-external-server.md`](labs/02-embedded-external-server.md).

Or skip the manual download and let the driver fetch and cache one for you on first use:
Or skip the manual download and let the package fetch and cache one for you on first use:

```python
options = ServerOptions()
options.with_auto_downloaded_server() # downloads + caches a self-contained server, no .NET needed
```

Walkthrough: [`labs/03-on-demand-server.md`](labs/03-on-demand-server.md).
Self-contained builds remove the system .NET requirement, but the normal RavenDB OS dependencies
still apply. In particular, minimal Linux images may need their distribution's ICU package.

### Don't manage a server at all (tests)

Expand Down Expand Up @@ -153,5 +155,6 @@ Call `open_studio_in_browser()` to open RavenDB Studio in your default browser.

## Labs

The `labs/` folder holds runnable, self-checking guides, one per usage case above. Start at
The `labs/` folder holds runnable, self-checking guides, one per usage case above. The scripts live
in this repository rather than in the installed wheel, so run them from a checkout. Start at
[`labs/README.md`](labs/README.md).
6 changes: 4 additions & 2 deletions labs/01-embedded-zero-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,5 +43,7 @@ with EmbeddedServer() as server:
## Takeaway

This is the most convenient path, but it buys a hard dependency on a system-wide .NET in a
specific major version. If you would rather not manage .NET, use Lab 02 (external
self-contained server) or Lab 03 (attach to a server you run yourself, e.g. via Docker).
specific major version. If you would rather not manage .NET, use Lab 02 (a self-contained server
you provide) or Lab 03 (one the package downloads and caches). To run RavenDB separately in Docker
and give each test an isolated database, use the testdriver's
[attach lab](https://github.com/ravendb/ravendb-python-testdriver/blob/v7.2/labs/01-attach-to-server.md).
6 changes: 4 additions & 2 deletions labs/02-embedded-external-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,5 +41,7 @@ A framework-dependent build (only a `framework` reference, no bundled runtime) s

## Takeaway

No .NET on the box, at the cost of fetching and caching the server build yourself. If you would
rather not manage a server at all, run one in a container and attach to it: Lab 03.
No .NET on the box, at the cost of providing and managing the server directory yourself. If you
want the package to download and cache the self-contained server, use Lab 03. To run RavenDB in a
container and attach test databases to it, use the testdriver's
[attach lab](https://github.com/ravendb/ravendb-python-testdriver/blob/v7.2/labs/01-attach-to-server.md).
24 changes: 15 additions & 9 deletions labs/03-on-demand-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,22 +30,28 @@ with EmbeddedServer() as server:
package's RavenDB line and caches under `~/.cache/ravendb-embedded`. Pass `cache_root` to point
it at a directory your CI restores between runs.

## Why pulling `latest` is fine (on purpose)
## Version and host requirements

Fetching the `latest` self-contained build for the version line is deliberate. A self-contained
build bundles its own .NET runtime, so it does not have to match anything on the host: whatever
`latest` returns is a server that just runs. There is no host .NET compatibility matrix to pin
against, which is exactly what makes "grab latest and run" safe here (a framework-dependent build
could not make that promise). The cache then freezes whatever you first pulled, so later runs stay
stable without any extra pinning.
The package fetches the latest self-contained build for the installed RavenDB version line. Its
bundled runtime removes the host .NET compatibility matrix, and the cache keeps later runs on the
same downloaded build.

Automatic downloads support Windows x64/x86, Linux x64/ARM64, and macOS x64/ARM64. Unsupported
targets fail with an explicit message instead of downloading a build for the wrong architecture.

Normal RavenDB operating-system dependencies still apply. Standard Linux distributions and hosted
CI images usually include them; minimal Linux images may need their distribution's ICU package.

## Cost to know about

The first use downloads a self-contained build (100 MB+) and the cache keeps it on disk. Every
run after that is offline and instant. If disk or first-run latency matters, prefer Lab 02 (you
provide the server) or Lab 01 (bundled server, needs .NET).

The cache is not refreshed automatically. To pick up a newer build from the same RavenDB line,
remove that line's cache directory or use a new `cache_root`.

## Takeaway

`with_auto_downloaded_server()` gives the no-.NET path of Lab 02 with zero manual steps: one call, then
ordinary client code.
`with_auto_downloaded_server()` gives the no-.NET path of Lab 02 without a manual server download:
one call, then ordinary client code.
1 change: 1 addition & 0 deletions labs/03_on_demand_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
def main() -> None:
with tempfile.TemporaryDirectory() as work:
options = ServerOptions()
options.dot_net_path = "__no_dotnet__"
options.with_auto_downloaded_server()
options.data_directory = str(Path(work, "data"))
options.logs_path = str(Path(work, "logs"))
Expand Down
4 changes: 4 additions & 0 deletions labs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@
Runnable, self-checking guides for getting a RavenDB server, from most convenient to most
portable. Each lab ships a script next to it, so you can run the exact code the guide shows.

The scripts are part of this repository and are not installed into `site-packages`. Clone or
download the repository first, then run the commands below from its root; `pip install` supplies
the released library and server binaries used by the scripts.

| Lab | Covers | Needs system .NET? |
|-----|--------|--------------------|
| [01](01-embedded-zero-config.md) | Embedded, zero-config (the default) | Yes (.NET 10 for 7.2.x) |
Expand Down
13 changes: 7 additions & 6 deletions ravendb_embedded/embedded_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -99,9 +99,7 @@ def _try_create_database(self, options: DatabaseOptions, store: DocumentStore) -
store.maintenance.server.send(CreateDatabaseOperation(options.database_record))
except Exception as e:
# Expected behavior when the database already exists
if (
"conflict" in e.args[0] or "already exists" in e.args[0]
): # todo: change exc type when python client will implement conflict handling
if "conflict" in e.args[0] or "already exists" in e.args[0]:
self._log_debug(f"{options.database_record.database_name} already exists.")
else:
raise e
Expand Down Expand Up @@ -171,7 +169,7 @@ def _run_server(self, options: ServerOptions) -> Tuple[str, subprocess.Popen]:
)

if url_ref["value"] is None:
error_string = self.read_output(process.stderr, startup_duration, options, None)
error_string = self.read_output(process.stderr, Stopwatch.create_started(), options, None)
self._shutdown_server_process(process)
raise RuntimeError(self.build_startup_exception_message(output_string, error_string, process))

Expand Down Expand Up @@ -199,7 +197,7 @@ def build_startup_exception_message(output_string: str, error_string: str, proce
sb.append(output_string)
sb.append(os.linesep)

sb.append("Check your ServerOptions, dotnet version, or run the command manually to see detailed error.")
sb.append("Check your ServerOptions and host dependencies, or run the command manually to see detailed error.")
return "".join(sb)

def online(
Expand All @@ -212,7 +210,7 @@ def online(
options: ServerOptions,
):
if line is None:
error_string = self.read_output(process.stderr, startup_duration, options, None)
error_string = self.read_output(process.stderr, Stopwatch.create_started(), options, None)
self._shutdown_server_process(process)
raise RuntimeError(self.build_startup_exception_message("".join(builder), error_string, process))

Expand Down Expand Up @@ -263,6 +261,9 @@ def output_reader():
if line is None:
break

if line == self.END_OF_STREAM_MARKER:
break

sb.append(line)
sb.append(os.linesep)

Expand Down
38 changes: 29 additions & 9 deletions ravendb_embedded/on_demand.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,34 @@ def _default_version_line() -> str:

def _platform_download() -> tuple:
machine = platform.machine().lower()
arch = "arm64" if machine in ("arm64", "aarch64") else "x64"
if machine in ("amd64", "x86_64"):
arch = "x64"
elif machine in ("arm64", "aarch64"):
arch = "arm64"
elif machine in ("x86", "i386", "i686"):
arch = "x86"
else:
raise RuntimeError(
f"Unsupported machine architecture for an automatic RavenDB download: {machine or 'unknown'}"
)

system = platform.system()
if system == "Windows":
if arch == "arm64":
raise RuntimeError(
"Automatic RavenDB downloads do not provide a Windows ARM64 build. "
"Use with_external_server() with a compatible server or attach to a separately running RavenDB."
)
return f"RavenDB for Windows {arch}", "zip"
if system == "Darwin":
return f"RavenDB for OSX {arch}", "tar.bz2"
return f"RavenDB for Linux {arch}", "tar.bz2"
if arch == "x86":
raise RuntimeError("Automatic RavenDB downloads do not provide a 32-bit macOS build.")
return f"RavenDB for MacOS {arch}", "tar.bz2"
if system == "Linux":
if arch == "x86":
raise RuntimeError("Automatic RavenDB downloads do not provide a 32-bit Linux build.")
return f"RavenDB for Linux {arch}", "tar.bz2"
raise RuntimeError(f"Unsupported operating system for an automatic RavenDB download: {system or 'unknown'}")


def _extract_safely(archive: Path, dest: Path, extension: str) -> None:
Expand Down Expand Up @@ -59,12 +80,11 @@ def _inside(name: str) -> bool:
def ensure_server(version: str = None, cache_root: str = None) -> str:
"""Return a local self-contained Server directory, downloading and caching on first use.

The download is a self-contained build (bundles its own .NET), so it runs with no system
.NET. Pulling `latest` for the version line is intentional: a self-contained build never has
to match anything on the host. A completed cache entry (keyed on version + platform) is reused
and never re-downloaded. Download and extraction happen in a private temp directory that is
moved into place only once complete, so an interrupted or concurrent first run never leaves a
half-populated cache.
The download is a self-contained build (bundles its own .NET), so it needs no system .NET.
Normal RavenDB operating-system dependencies still apply. A completed cache entry (keyed on
version + platform) is reused and never re-downloaded. Download and extraction happen in a
private temp directory that is moved into place only once complete, so an interrupted or
concurrent first run never leaves a half-populated cache.
"""
version = version or _default_version_line()
root = Path(cache_root) if cache_root else Path.home() / ".cache" / "ravendb-embedded"
Expand Down
2 changes: 0 additions & 2 deletions ravendb_embedded/options.py
Original file line number Diff line number Diff line change
Expand Up @@ -93,8 +93,6 @@ def secured(

return self

# todo: secured by cert exec and args

def with_external_server(self, server_location: str) -> None:
self.provider = ExternalServerProvider(server_location)
# A directory is already a runnable server: run it in place, so we neither copy it nor
Expand Down
37 changes: 33 additions & 4 deletions tests/test_on_demand.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,39 @@ def test_cache_hit_skips_download(self):

self.assertEqual(str(server_dir), resolved)

def test_platform_download_targets_a_known_os(self):
label, extension = _platform_download()
self.assertTrue(label.startswith("RavenDB for "))
self.assertIn(extension, ("zip", "tar.bz2"))
def test_platform_download_maps_supported_targets(self):
cases = [
("Windows", "AMD64", ("RavenDB for Windows x64", "zip")),
("Windows", "x86", ("RavenDB for Windows x86", "zip")),
("Linux", "x86_64", ("RavenDB for Linux x64", "tar.bz2")),
("Linux", "aarch64", ("RavenDB for Linux arm64", "tar.bz2")),
("Darwin", "x86_64", ("RavenDB for MacOS x64", "tar.bz2")),
("Darwin", "arm64", ("RavenDB for MacOS arm64", "tar.bz2")),
]

for system, machine, expected in cases:
with self.subTest(system=system, machine=machine):
with mock.patch("platform.system", return_value=system), mock.patch(
"platform.machine", return_value=machine
):
self.assertEqual(expected, _platform_download())

def test_platform_download_rejects_unavailable_targets(self):
cases = [
("Windows", "arm64"),
("Linux", "i686"),
("Darwin", "i386"),
("FreeBSD", "x86_64"),
("Linux", "mips"),
]

for system, machine in cases:
with self.subTest(system=system, machine=machine):
with mock.patch("platform.system", return_value=system), mock.patch(
"platform.machine", return_value=machine
):
with self.assertRaises(RuntimeError):
_platform_download()

def test_extract_rejects_path_traversal(self):
# A tampered archive must not be able to write outside the destination (zip/tar slip).
Expand Down
Loading