Reach vLLM over headscale with a forwarder, and fix millenia's Piper
Rebinding vllm-chat was the expensive option: Petal, Gogobee and Open WebUI all point at 127.0.0.1:8000, and Open WebUI keeps its endpoint in its own database rather than in env, so moving the bind address meant editing three consumers and reloading a 35B AWQ model. The socat unit adds a second listener on the headscale address instead -- local callers untouched, no downtime, and the only new exposure is on the VPN. Bound to 100.64.0.2 specifically, never 0.0.0.0: the far end is a public host. Verified: a grammar checkpoint from petal.parodia.dev returns real suggestions in ~3s over the VPN. Also documents two things found on millenia that were invisible from outside it: - Piper had been dead since the Jul 26 reboot, 26,800+ failed restarts, with read-aloud silently falling back to browser Web Speech. An OS upgrade moved /usr/bin/python3 from 3.13 to 3.14, and venv/bin/python3 is a symlink to the system interpreter, so lib/python3.13/site-packages went invisible -- sys.path had no site-packages at all. Recreating the venv lands Piper 1.6.0, which is what TTS_PATH exists for. - The canonical instance has no automated backup (newest snapshot a month old) and runs unsupervised with PPID 1. Both written down; the backup one is pending the encryption-at-rest decision. Backups on the VPS now ride parodia-backup (age-encrypted, offsite, S3), using VACUUM INTO rather than that script's iterdump helper -- iterdump does not reproduce an FTS5 virtual table, so a restore would have come back with cross-document search silently missing.
This commit is contained in:
@@ -0,0 +1,40 @@
|
||||
[Unit]
|
||||
Description=Expose millenia's vLLM chat server on the headscale interface only
|
||||
# Why a forwarder instead of just rebinding vLLM: vllm-chat.service is shared.
|
||||
# Petal, Gogobee and Open WebUI all talk to 127.0.0.1:8000, and Open WebUI keeps
|
||||
# its endpoint in its own database rather than in env, so moving vLLM's bind
|
||||
# address would mean editing three consumers and reloading a 35B AWQ model
|
||||
# (minutes of downtime for all of them). This adds a second door instead: local
|
||||
# callers keep loopback untouched, and only the headscale address gains a
|
||||
# listener. Nothing about vllm-chat changes.
|
||||
#
|
||||
# Deliberately NOT 0.0.0.0 — this reaches a public VPS over the VPN, and the
|
||||
# LAN has no business seeing an unauthenticated inference endpoint.
|
||||
After=network-online.target tailscaled.service vllm-chat.service
|
||||
Wants=network-online.target
|
||||
BindsTo=vllm-chat.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
# fork: one child per connection, so a single client can't block the others.
|
||||
# reuseaddr: survive a restart while sockets are still in TIME_WAIT.
|
||||
# The bind address is millenia's headscale IP; if tailscaled hasn't brought the
|
||||
# interface up yet the bind fails and Restart retries until it has.
|
||||
ExecStart=/usr/bin/socat -d TCP-LISTEN:8000,bind=100.64.0.2,fork,reuseaddr TCP:127.0.0.1:8000
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
# Long generations hold a connection open; don't let systemd reap a healthy one.
|
||||
TimeoutStopSec=10
|
||||
|
||||
# The process only shuttles bytes between two sockets — give it nothing else.
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
ProtectKernelTunables=true
|
||||
ProtectControlGroups=true
|
||||
RestrictAddressFamilies=AF_INET AF_INET6
|
||||
DynamicUser=true
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
Reference in New Issue
Block a user