Operate
PostgreSQL and S3
Nipa ships two server binaries from the same module:
| Binary | Entrypoint | Metadata | Chunks |
|---|---|---|---|
nipad (free) | cmd/nipad | SQLite (DATABASE_DSN, default nipa.db) | local (CHUNK_STORAGE=local, CHUNK_STORAGE_DIR) |
nipad-ee (enterprise) | ee/cmd/nipad | PostgreSQL (DATABASE_DSN) | S3 only (CHUNK_STORAGE=s3, empty defaults to s3) |
The free binary links no ee/ package; internal/ must never import
ee/. Both binaries compose repositories into the shared
internal/serverapp registry and call serverapp.Run, which owns the
h2c/HTTP/gRPC/SPA lifecycle. CHUNK_STORAGE=s3 in the free binary fails with a
hint to use nipad-ee.
Layout
ee/
db/ postgres opener + migration runner
migrations/postgres/ single consolidated schema + embed.FS
nodelease/ per-instance snowflake node id leases
repository/postgres/ 13 repositories (the server repository surface)
queries/ sqlc inputs
sqlc/ generated code (excluded from Sonar)
storage/s3/ S3-compatible ChunkStore (+ DirectTransferStore)
cmd/nipad/ enterprise server binary
cmd/migrate/ postgres migrate CLI
e2e/ testcontainers smoke suite
Schema conventions (ee/db/migrations/postgres)
- Snowflake records keep their app-generated IDs:
BIGINT PRIMARY KEY, no identity. Surrogate row ids (tree_nodes,files,chunks,pbac_rules,project_paths_permissions,refresh_tokens) useBIGINT GENERATED BY DEFAULT AS IDENTITY. - Every integer column is
BIGINT(notINTEGER). This makes the generated Go types identical to the sqlite engine (int64/sql.NullInt64); nullableint4would otherwise becomesql.NullInt32. - Hashes and payloads are
BYTEA, timestampsTIMESTAMPTZ(defaultnow()), booleansBOOLEAN, and the partial unique indexes (branches,file_locks,webhook_deliveries) mirror the sqlite predicates. - The migration seeds the super admin, the default org/project/branch and the
owner membership, matching
db/migrations/sqlite. - There is one consolidated
000001_init_schema(no postgres deployments exist); future changes addmake migrate-createpairs.
Snowflake node ids (multi-pod)
App-generated ids embed an 8-bit node id (0–255) in internal/snow; two
instances sharing a node id would generate colliding ids. nipad-ee resolves
the node id as follows:
SNOWFLAKE_NODE_IDset (env,config.yamlor.env): that static id is used as-is.- unset: the instance claims a free or expired id in
snowflake_node_leases(ee/nodelease) and renews the lease everySNOWFLAKE_LEASE_TTL_SECONDS / 3(default TTL 60s) until shutdown, when the row is released. A crashed instance’s id becomes reclaimable once the TTL expires; if a lease cannot be renewed before it expires the process exits and re-claims on restart instead of risking collisions. Startup fails when all 256 ids are held by live instances. - The free
nipad(SQLite, local chunks) is single-instance by design and keeps usingSNOWFLAKE_NODE_ID(default 0).
All lease expiry math runs on the database clock (now()), so instance clock
skew does not matter.
Queries and sqlc
make sqlc reads ee/db/migrations/postgres + ee/repository/postgres/queries
and writes ee/repository/postgres/sqlc (package postgres). The generated
models are byte-for-byte equivalent to the sqlite ones, so repository ports are
mechanical. Dialect notes:
- Use
sqlc.arg('name')with quoted names: an unquoted keyword such assqlc.arg(limit)is a parse error on the postgres engine. - Cast nullable filters:
sqlc.narg('resolved')::boolean,sqlc.narg('last_id')::bigint. LIMIT/OFFSETparameters need an explicit::bigint(otherwise int4 is inferred); a cast directly aftersqlc.argis rejected, so writeLIMIT sqlc.arg('limit')::bigint— quoting the name first is required.INSERT OR IGNOREbecomesON CONFLICT DO NOTHING;excluded.*upserts andRETURNINGcarry over unchanged.FileLockGetusesbranch_id IS NOT DISTINCT FROM sqlc.narg('branch_id')to reproduce sqlite’sISNULL semantics.
Repository layer
ee/repository/postgres implements the same unexported usecase interfaces as
internal/repository/sqlite (structural typing; no interface changes needed).
common.go maps errors: sql.ErrNoRows → 404, postgres 23505 → 409,
anything else → 500, always preserving Cause so
errors.Is(err, sql.ErrNoRows) still matches. The only intentional behavior
difference is MergeRequestCreate: it allocates the per-project number while
holding SELECT ... FOR UPDATE on the project row (the sqlite single writer
made the old MAX(number)+1 race invisible).
Migrations
nipad-eeapplies pending migrations at startup (ee/db.MigrateUp).ee/cmd/migrateis the standalone CLI;make migrate-up-postgres DSN=.../make migrate-down-postgres DSN=...wrap it. The freemake migrate-up/migrate-downremain sqlite-only.
Testing
- Repository tests live in
ee/repository/postgresand are testify suites over testcontainers: onepostgres:17container per package (TestMain), a fresh schema per test (search_path +pgxmigrate), dropped on cleanup. They assert against real FKs, so fixtures seed real trees/commits. ee/dbhas an env-agnosticMigrateSuite;ee/e2eboots postgres + minio and runs the real client through login → clone → push (S3 presigned direct upload) → clone → tag. A second e2e test leases node ids for two in-process instances against the shared postgres and pushes through both.ee/nodeleasehas its own testcontainers suite (fresh schema per test) covering distinct ids, expired-lease steal, exhaustion, renew, release and the lost-lease callback.- Overrides for service-container CI:
NIPA_TEST_POSTGRES_DSN,NIPA_TEST_S3_ENDPOINT,NIPA_TEST_S3_IMAGE(defaultdocker.io/pgsty/minio:latest),NIPA_TEST_REDIS_ADDR(rate limit store, default aredis:7container). make test-eeruns the enterprise packages; plaingo test ./...runs everything when Docker is available.