Skip to content

Latest commit

 

History

29 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

LunarSky ObjectStore banner

ObjectStore

Small S3-compatible object storage for LunarSky.

Source: GitHub · Gitea mirror

Current source docs: Operations · Java client · Rust client · Tests

Development software. Do not use it for production data. It is provided as is, without warranty, under the MIT License.

Stage: development License: MIT Runtime: Java 21 S3 API: partial support CodeFactor

The hosted v0.0.8 docs describe an older release. Use the source docs above for this checkout's behavior and limitations.

Capability checklist

ObjectStore creates the default bucket at startup. All buckets share one capacity limit.

  • ✅ Persistent single-node storage with checksum verification
  • ✅ Optional dm-crypt-backed data mounts with startup guards
  • ✅ Configurable size limits and CLI verification
  • ✅ Local cluster prototype with host-aware placement and optional node TLS
  • ✅ Opt-in repair, rebalance, guarded garbage collection, and metadata backup
  • ✅ Cold snapshot and restore tooling for the local Compose stacks
  • ✅ Manual two-machine durability and restore drill
  • ✅ Shared encrypted managed credentials across cluster gateways
  • ✅ Disposable metadata failover and recovery tests
  • ⬜ Production multi-server validation across independent machines

Legacy single-node objects remain readable but stay out of listings until overwritten because their keys were not stored.

S3 API support checklist

  • ✅ Header-based and presigned-query AWS Signature Version 4 authentication
  • ✅ Path-style and configurable virtual-hosted-style bucket addresses
  • ✅ PutObject, GetObject, HeadObject, and DeleteObject in both modes
  • ✅ Single-range GET and ListObjectsV2 in both modes
  • ✅ SHA-256 payload verification and x-amz-checksum-sha256 in both modes
  • ✅ Content-MD5 and CRC32, CRC32C, CRC64NVME, XXHash64, XXHash3, XXHash128, SHA-1, SHA-256, SHA-512, and MD5 checksum headers on PutObject and UploadPart
  • ✅ CreateMultipartUpload, UploadPart, ListParts, CompleteMultipartUpload, AbortMultipartUpload, and ListMultipartUploads in both modes
  • ✅ Presigned URLs and signed streaming Signature V4 uploads, including signed checksum trailers
  • ✅ CreateBucket, HeadBucket, DeleteBucket, and ListBuckets in both modes
  • ✅ CopyObject across owned buckets, with COPY and REPLACE content-type and user-metadata behavior
  • ✅ User metadata on object and multipart uploads; object tags on upload, copy, and the tagging subresource
  • ✅ CRC64NVME and XXHash checksums, checksum trailers, and persisted object checksum metadata
  • ✅ Bucket versioning with retained versions, delete markers, and version-specific reads, copies, deletes, and tags in both modes
  • ✅ Bucket and object ACL grants, bucket policies, Object Ownership controls, and Block Public Access
  • ✅ Scoped identity policies and externally provisioned, expiring session credentials
  • ⬜ Full IAM, STS issuance, cross-account ACL ownership, and conditional policies

This is an S3 subset. Unsupported operations and Amazon-specific headers are rejected. Bucket names use three to 63 lowercase letters, digits, or hyphens; the default bucket cannot be deleted. Bucket creation accepts an empty body for the configured region. Tags do not control access.

Versioning supports enabled and suspended states. Old versions and delete markers count toward capacity; deleting a specific version is permanent. Lifecycle expiration, MFA Delete, and ListObjectVersions delimiter grouping are unsupported. User metadata accepts printable ASCII only.

PutObject checksums are validated and retained; without one, ObjectStore stores CRC64NVME, including for completed multipart uploads. Request x-amz-checksum-mode: ENABLED on GET or HEAD to receive it. UploadPart checksums are validated but are not returned by ListParts or combined into a multipart checksum. Presigned URLs expire within seven days. Signed aws-chunked uploads support one checksum trailer; other streaming modes are unsupported. Copies obey the object size limit.

Single-node setup

Requires Docker Compose and OpenSSL. From a fresh checkout, one command creates local credentials, enables the loopback-only admin console, and starts the service:

bash scripts/install.sh

For a new machine, clone and install in one shell line:

git clone https://github.com/LunarSkyOSS/ObjectStore.git objectstore && cd objectstore && bash scripts/install.sh

The installer supports the stock single-node Compose stack and keeps existing .env, secrets, and Docker data volumes on reruns. It refuses encrypted storage, existing data bind mounts, and extra Compose overlays; update those deployments using their original Compose command. Credentials are in .env and secrets/admin-password; keep secrets/managed-master for recovery. Review admin access and key management before binding the console to a VPN interface.

For manual setup, copy .env.example to .env, then set S3_ACCESS_KEY and S3_SECRET_KEY to unique values. The access key must be at least 16 alphanumeric characters; the secret must be at least 32 characters.

cp .env.example .env
docker compose up -d --build
curl http://127.0.0.1:9000/health

Port 9000 binds to localhost. Data stays in the object-data Docker volume. docker compose down -v deletes that volume.

The optional local admin console manages buckets, ACLs, policies, scoped keys, bulk jobs, access checks, and live performance. It has its own password and is disabled in the base Compose stack.

The standalone defaults are 128 MiB per object and 2 GiB total. Set MAX_OBJECT_BYTES (up to 64 GiB) and MAX_TOTAL_BYTES in .env to change them. The total must be at least the object limit. Incomplete multipart uploads consume space until aborted.

Operations

The operations guide covers encrypted mounts, access keys and ACLs, the CLI, cluster setup and maintenance, migration, metadata routing, and public-client limits. Read it before exposing an endpoint or upgrading a cluster.

Java client

The JDK-only Java client works with ObjectStore and other S3-compatible endpoints. Build the current source with JDK 21 or newer:

bash client/scripts/build-maven-repo.sh

This creates the client JAR, Javadocs, and a local Maven repository under client/dist/, using the checkout's version. It does not publish anything.

Older published package (0.0.8)

Version 0.0.8 is available from https://repo.lunarsky.cloud/ without an account. The following examples install that older release, not the current source:

Gradle (Kotlin DSL):

repositories {
    maven { url = uri("https://repo.lunarsky.cloud/") }
}
dependencies {
    implementation("cloud.lunarsky:objectstore-client:0.0.8")
}

Maven (pom.xml):

<repositories>
  <repository>
    <id>lunarsky</id>
    <url>https://repo.lunarsky.cloud/</url>
  </repository>
</repositories>
<dependencies>
  <dependency>
    <groupId>cloud.lunarsky</groupId>
    <artifactId>objectstore-client</artifactId>
    <version>0.0.8</version>
  </dependency>
</dependencies>

See current client usage and limits. The AWT image manager is a small desktop example.

Rust client

The Rust adapter configures the official AWS SDK for path-style ObjectStore endpoints and streams file transfers. It is local source code, not a published crate.

Disclaimer

ObjectStore is independent software. It is not affiliated with, sponsored by, or endorsed by Amazon or Amazon Web Services (AWS). “S3-compatible” describes only the API subset listed above.

Keep independent backups; data loss is possible. The software is provided “as is” under the MIT License. To the extent permitted by applicable law, the authors and maintainers are not responsible for data loss or other damages arising from its use.

AI contributions

AI tools assisted with parts of this project. Maintainers review releases and remain responsible for what ships.

Releases

Packages

Contributors

Languages