veola/README.md

# Veola

Self-hosted Go web app that tracks items across e-commerce platforms and pushes deal alerts to a self-hosted [ntfy](https://ntfy.sh) instance. eBay marketplaces are polled through eBay's official [Browse API](https://developer.ebay.com/api-docs/buy/browse/overview.html); Amazon family, Yahoo Auctions JP, and Mercari JP go through the [Apify](https://apify.com) scraping API.

Track. Watch. Notice.

## Features

- Watch arbitrary items across multiple marketplaces with per-item search queries, target prices, and poll intervals
- eBay via the official eBay Browse API, with a per-day call quota tracked and enforced — polling halts before the limit and resets on eBay's Pacific-time clock
- Apify active-listing, sold-listing, and price-comparison actors for the non-eBay marketplaces
- Price-history chart and best-price badge once enough history accumulates
- Deal alerts pushed to ntfy when current price falls at or below target
- Single-binary deploy, SQLite storage, no CGO

See [`veola-spec.md`](veola-spec.md) for the full specification.

## Requirements

- Go 1.22+ (developed against 1.25)
- `make`, `curl`, and a POSIX shell (for the Makefile)
- A reachable [ntfy](https://ntfy.sh) instance (self-hosted or ntfy.sh)
- An [eBay developer](https://developer.ebay.com) keyset (App ID + Cert ID) — for eBay marketplaces
- An [Apify](https://apify.com) account + API key — for the non-eBay marketplaces

### Build-time tools

All non-Go tool versions are pinned in the Makefile at the top:

```make
TAILWIND_VERSION := v3.4.17
HTMX_VERSION     := 2.0.4
CHARTJS_VERSION  := 4.4.6
TEMPL_VERSION    := v0.3.1020
```

Install `templ` yourself at the pinned version; Tailwind is downloaded for you on first `make css`. htmx and Chart.js are already vendored in `static/vendor/` at the pinned versions above and committed to the repo.

```sh
# Install the pinned templ CLI.
make tools

# (or, equivalently)
go install github.com/a-h/templ/cmd/templ@v0.3.1020

# Confirm it's on PATH (typically $(go env GOPATH)/bin).
templ --version
```

You **do not need Node, npm, or Yarn**. Tailwind ships as a self-contained binary — the Makefile fetches the [standalone Tailwind CLI](https://github.com/tailwindlabs/tailwindcss/releases) into `./bin/tailwindcss` on first `make css`. The `bin/` directory is gitignored.

By default the Makefile downloads the **linux-x64** build. For other platforms, override `TAILWIND_URL` on the command line — pick the matching asset from the [Tailwind releases page](https://github.com/tailwindlabs/tailwindcss/releases/tag/v3.4.17):

```sh
# macOS Apple Silicon
make css TAILWIND_URL=https://github.com/tailwindlabs/tailwindcss/releases/download/v3.4.17/tailwindcss-macos-arm64

# macOS Intel
make css TAILWIND_URL=https://github.com/tailwindlabs/tailwindcss/releases/download/v3.4.17/tailwindcss-macos-x64

# linux-arm64
make css TAILWIND_URL=https://github.com/tailwindlabs/tailwindcss/releases/download/v3.4.17/tailwindcss-linux-arm64
```

Or drop a binary at `./bin/tailwindcss` yourself and the Makefile will use it.

## Build

First time:

```sh
go install github.com/a-h/templ/cmd/templ@latest   # if not already
make build                                          # fetches Tailwind, runs templ, compiles
```

Subsequent builds:

```sh
make build
```

This runs `templ generate`, compiles Tailwind utilities into `static/css/tailwind.css`, and produces `veola-bin`. Makefile targets:

| Target | What it does |
| --- | --- |
| `make generate` | Regenerate templ Go from `.templ` sources |
| `make css` | Compile `static/css/tailwind.css` from `static/css/input.css` (fetches the Tailwind standalone CLI into `bin/` on first run) |
| `make build` | `generate` + `css` + `go build -o veola-bin .` |
| `make run` | `build`, then run against `config.toml` |
| `make test` | `go test ./...` |
| `make clean` | Remove `veola-bin` |

The binary is named `veola-bin` rather than `veola` because the module is also `veola` — `go build` cannot write a binary with the same name as the module dir.

`static/css/tailwind.css` is committed, so a deploy box can `go build -o veola-bin .` without the Tailwind CLI or even `make`. Re-run `make css` whenever you change templates or `static/css/input.css`. The hand-written component layer in `static/css/app.css` is loaded separately and needs no rebuild.

## Configure

Copy the example and edit:

```sh
cp config.toml.example config.toml
```

Both `session_secret` and `encryption_key` must be at least 32 bytes and different from each other. Generate with:

```sh
openssl rand -hex 32
```

`encryption_key` encrypts secrets at rest in SQLite (Apify key, eBay credentials, ntfy settings). Rotating it invalidates stored secrets — re-enter them through `/settings` after rotation.

Other notable config:

- `[ebay]` — `client_id` (App ID), `client_secret` (Cert ID), `environment` (`production` or `sandbox`), and `daily_call_limit` (Browse API calls per day; default 5000). All of `client_id` / `client_secret` / the limit can also be set or overridden at runtime via `/settings`.
- `server.secure_cookies` — sets the `Secure` attribute on the session cookie. Defaults to `true`; keep it on for any HTTPS-reachable deployment, **including behind a TLS-terminating proxy**. Set `false` only for local plain-HTTP testing on a non-localhost address.

## Run

```sh
./veola-bin -config config.toml
```

CLI flags:

| Flag | Default | Notes |
| --- | --- | --- |
| `-config <path>` | `config.toml` | Path to the TOML config file |
| `-debug` | off | Verbose `log/slog` at `LevelDebug`. Logs raw external API payloads (eBay / ZenMarket / etc.) — useful when diagnosing parse failures. Not for production. |

First-run flow:

1. Visit `http://localhost:8080/`. With no users, you are redirected to `/setup`.
2. Create the admin account. The first user is always an admin.
3. Log in at `/login`.
4. Add items at `/items/new`. Optionally fill in your eBay/Apify credentials and ntfy URL via `/settings` if you didn't put them in `config.toml`. The Settings page also shows the running eBay API call count for the day.

Account registration is admin-only — there is no public signup. Once at least one user exists, `/setup` returns 404. New users are created from the Settings page by an admin.

The scheduler starts with the server and polls each active item on its configured interval. The bottom-of-hour global poll runs every `scheduler.global_poll_interval_minutes`.

## Layout

```
main.go              entry point: config, db open, scheduler, http server
internal/
  config/            TOML config loading and validation
  crypto/            AES-GCM encryption for secrets at rest
  db/                SQLite schema, migrations, store
  models/            domain types
  apify/             Apify API client
  ebay/              eBay Browse API client (OAuth2 + item search)
  ntfy/              ntfy push client
  auth/              session + CSRF
  scheduler/         poll loop, alert/dedup/badge logic
  handlers/          HTTP handlers
templates/           templ components
static/              compiled Tailwind + app.css, vendored htmx/Chart.js, JS
tailwind.config.js   Tailwind content globs
Makefile             build targets
```

## Test

```sh
go test ./...
```

Unit tests cover crypto round-trip, db round-trip and dedup, scheduler alert/badge logic, and eBay marketplace/filter mapping. No handler-level tests yet.

## Keeping dependencies current

Veola pulls from four sources, all version-pinned for reproducibility.

```sh
# Bump Go module deps to their newest compatible versions, then prints
# upstream-release URLs for the four pinned tools so you can spot bumps.
make update-deps

# After bumping any of TAILWIND_VERSION / HTMX_VERSION / CHARTJS_VERSION
# in the Makefile, refetch the vendored assets at the new pins:
make vendor

# After bumping TEMPL_VERSION:
make tools

# Then rebuild and run the test suite:
make build && make test
```

The pinned tool versions live at the top of the Makefile. The vendored JS at `static/vendor/htmx.min.js` and `static/vendor/chart.umd.min.js` is committed and updated only by `make vendor`. Tailwind v3.4.17 is intentionally pinned — v4 is a breaking release that drops the `tailwind.config.js` format Veola uses.

For automated tracking, point [Dependabot](https://docs.github.com/en/code-security/dependabot) or [Renovate](https://docs.renovatebot.com/) at the repo. Both can watch `go.mod` natively; Renovate's custom-regex managers can also track the `*_VERSION` lines in the Makefile.

## Operate

- The SQLite file lives at `server.db_path` (default `./veola.db`). Back this up — it holds your watched items, history, encrypted secrets, and user accounts.
- `config.toml` and `veola.db` (plus its `-wal`/`-shm`) hold secrets and live session tokens — keep them `chmod 600` and owned by the service user.
- The process responds to `SIGINT` / `SIGTERM` with a graceful HTTP shutdown (30s timeout) followed by scheduler stop.
- Logs go to stdout as structured `log/slog` records.

### Deploying publicly

Veola speaks plain HTTP and is meant to sit behind a TLS-terminating reverse proxy (e.g. Traefik, Caddy, nginx).

- Keep `server.secure_cookies = true` (the default).
- Terminate TLS at the proxy and set HSTS there — Veola does not emit HSTS itself.
- Veola sets `Content-Security-Policy`, `X-Frame-Options`, `X-Content-Type-Options`, and `Referrer-Policy` on every response, and trusts `X-Forwarded-For` for client IPs — configure the proxy to strip client-supplied `X-Forwarded-*` headers so they cannot be spoofed.
- `GET /healthz` returns 200 `ok` with no auth — wire it up to your proxy/uptime probe.

A hardened systemd unit template lives at [`deploy/veola.service`](deploy/veola.service). It assumes:

- Binary at `/usr/local/bin/veola-bin`
- Config at `/etc/veola/config.toml`
- A `veola` system user with `/var/lib/veola` as its working / data directory (the only writable path under `ReadWritePaths`)

Install sketch:

```sh
sudo useradd --system --home /var/lib/veola --shell /usr/sbin/nologin veola
sudo install -d -o veola -g veola -m 0750 /var/lib/veola /etc/veola
sudo install -m 0755 veola-bin /usr/local/bin/veola-bin
sudo install -m 0640 -o veola -g veola config.toml /etc/veola/config.toml
sudo install -m 0644 deploy/veola.service /etc/systemd/system/veola.service
sudo systemctl daemon-reload
sudo systemctl enable --now veola
```

## Aesthetic

Sega Genesis blue. Not dark mode, not light mode — blue mode. See the visual design section of `veola-spec.md` for the palette.