Repository navigation
LocalAIย #2296
Description
Name of the Script
LocalAI
Script Type
CT (LXC Container)
Does this script support arm64?
arm64 not tested
๐ Script Details
LocalAI is an OpenAI-compatible API server for local inference โ text, images, transcription and TTS behind one endpoint, with a model gallery and pluggable backends. It overlaps with the existing llama.cpp script but is multi-modal rather than inference-only.
Activity
github-actions commentedon Sep 18, 2026
The LocalAI script is ready for testing:
No models are installed by default. Use the gallery in the web UI to pull one, or drop GGUF files into /opt/localai_data/models. Inference backends are fetched on first use and can be several GB each, which is what the 30 GB default disk is for.
The API is protected from the moment it starts. With user accounts off, a static API key is generated and saved to ~/localai.creds in the container; clients send it as 'Authorization: Bearer ' or an 'x-api-key' header, and it grants full admin access. With user accounts on, register in the web UI immediately after install - the first account created becomes the admin.
GPU passthrough is offered during creation and driver libraries inside the container are matched to the host driver version automatically. Without a GPU everything still works on the CPU, just slowly.
Models, backends and the auth database live in /opt/localai_data and configuration in /opt/localai.env, both outside the application directory, so updates never touch them. A mount point on /opt/localai_data is worth considering rather than growing the root disk.
User accounts need a database. Upstream ships the release binary without the CGO auth tag, so the built-in SQLite store is unavailable and PostgreSQL is installed alongside LocalAI when the option is enabled. Leaving it off keeps the container to a single service.
Note: This is not in the official repo yetโit's just a dev version! After merging into ProxmoxVE, it will need to be recreated.
Discord testing thread: https://discord.com/channels/1302816934508630047/1550399677265088562
wittend commentedon Sep 21, 2026
Perhaps two or three problems, perhaps they are just one:
Using: var_port='8083' bash -c "$(curl -fsSL https://raw.githubusercontent.com/community-scripts/ProxmoxVED/main/ct/localai.sh)"
at root console on Proxmox VE.
/ / ____ _________ / / | / /
/ / / __ / / __ `/ / /| | / /
/ // // / /__/ // / / ___ |/ /
//_/_/_,/// |/___/
โ๏ธ Using Default Settings on node foxbat
๐ก PVE Version 9.2.20 (Kernel: 7.0.14-15-pve)
๐ Container ID: 110
๐ฅ๏ธ Operating System: debian (13)
๐ฆ Container Type: Unprivileged
๐พ Disk Size: 30 GB
๐ง CPU Cores: 4
๐ ๏ธ RAM Size: 8192 MiB
๐ฎ GPU Passthrough: Enabled
๐ Creating a LocalAI LXC using the above default settings
โ๏ธ Storage local (Free: 62.7GB Used: 26.5GB) [Template]
โ๏ธ Storage zfs-vgroup (Free: 3.5TB Used: 158.8GB) [Container]
โ๏ธ Storage 'zfs-vgroup' (zfspool) validated
โ๏ธ Template storage 'local' validated
โ๏ธ Template search completed
โ๏ธ Template debian-13-standard_13.1-2_amd64.tar.zst [local]
โ๏ธ LXC Container 110 was successfully created.
๐ฎ Detected Intel GPU
โ๏ธ Automatically configuring INTEL GPU passthrough
โ๏ธ INTEL GPU passthrough configured (2 devices)
โ๏ธ Started LXC Container
โ๏ธ Network in LXC is reachable (ping)
โ๏ธ GPU passthrough configured (video:44, render:992)
โ๏ธ Customized LXC Container
โ๏ธ Set up Container OS
โ๏ธ Network Connected: 10.0.0.165
โ๏ธ IPv4 Internet Connected
โ๏ธ IPv6 Internet Not Connected
โ๏ธ Git DNS: github.com:(โ๏ธ ) raw.githubusercontent.com:(โ๏ธ ) api.github.com:(โ๏ธ )
โ๏ธ Updated Container OS
โ๏ธ Installed Dependencies
โ๏ธ Detected GPU: Intel Corporation AlderLake-S GT1 (rev 0c) (INTEL)
โ๏ธ Deployed: libigdgmm12 (26.35.39758.10)
โ๏ธ Deployed: intel-igc-core (2.41.5)
โ Fetching github release: intel-igc-opencl (2.41.5)curl: (22) The requested URL returned error: 504
๐ก Download failed or stalled (attempt 1/3), retrying...
curl: (22) The requested URL returned error: 504
๐ก Download failed or stalled (attempt 2/3), retrying...
curl: (22) The requested URL returned error: 504
โ๏ธ Download failed: https://github.com/intel/intel-graphics-compiler/releases/download/v2.41.5/intel-igc-opencl-2_2.41.5%2B22716_amd64.deb
โ๏ธ Failed to install 'intel-opencl-icd (26.35.39758.10-0)' โ both apt and dpkg reported errors
โ Package intel-igc-opencl-2 is not installed.
โ dpkg: error processing package intel-opencl-icd (--install):
โ dependency problems - leaving unconfigured
โ Errors were encountered while processing:
โ intel-opencl-icd
โ๏ธ Unmet dependencies: intel-igc-opencl-2 intel-igc-opencl-2
โ๏ธ Hint: Run 'apt-get install -f' inside the container to attempt automatic dependency resolution.
โ๏ธ Intel Gen 9+ GPU configured
โ๏ธ Setup Hardware Acceleration
โ๏ธ Deployed: localai (4.10.0)
โ๏ธ Configured LocalAI
โ๏ธ Created Service
โ๏ธ Customized Container
โ๏ธ Completed Successfully!
๐ LocalAI setup has been successfully initialized!
๐ก Access it using the following URL:
๐ http://10.0.0.165:8080
๐ก Install models from the gallery in the web UI or place GGUF files in /opt/localai_data/models
I'm not sure what to do about this.
My intent is to run local models on several seperate local hosts and hopefully some external ones as well (if LocalAI can do that).
Thanks for this work!
MickLesk commentedon Sep 21, 2026
wittend commentedon Sep 21, 2026
That seemed to do the trick.
TheOne299 commentedon Oct 4, 2026
Getting the following error when trying to chat with the loaded model:
Error: failed to load model with internal loader: could not load model: rpc error: code = Unavailable desc = error reading from server: EOF
github-actions commentedon Oct 9, 2026
A PR has been created for localai: community-scripts/ProxmoxVE#17801
MickLesk commentedon Oct 9, 2026
Merged with #17801 in ProxmoxVE
github-actions commentedon Oct 9, 2026
Files deleted with PR #2399