Author SHA1 Message Date
iu e0b4db926f feat(server): log metadata and storage backends at startup 2026-10-11 10:34:30 +03:00
iu 8638d1bbfc chore(openspec): archive completed changes and sync main specs
- archive local-fs-storage-backend, sqlite-metadata-backend,
  storage-orphan-reconciliation, metadata-managed-jwt-secret, default-local-backends
- sync specs: storage-backends, metadata-backends, storage-reconciliation,
  app-secrets; update photos/share-links
- defaults reflected in specs: SQLite and local FS
2026-10-11 10:19:44 +03:00
iu cb272398ce feat(auth): store the token signing secret in the metadata layer
- app_settings table (PostgreSQL migration 000010, SQLite 000002)
- internal/settings repository (Get/Set/Ensure) wired into store.Repos
- secret generated with crypto/rand on first start and persisted; reused on restart
- drop required JWT_SECRET from env/config; app.New takes a TokenManager and
  storage.Open takes the resolved secret (URL_SECRET still overrides)
- docs updated
2026-10-11 10:18:31 +03:00
iu 176a8d8ced feat(metadata): SQLite backend, storage reconciliation and local defaults
- pluggable per-domain Repository interfaces; PostgreSQL and SQLite impls
- SQLite on modernc.org/sqlite (WAL, busy_timeout, foreign_keys, immediate tx)
- separate migrations/sqlite and db.MigrateSQLite; store.NewPostgres/NewSQLite
- cmd/reconcile compares object storage with the journal (report, then --delete)
- storage.Backend.List and photos.Repository.AllStorageKeys
- DB_BACKEND/SQLITE_PATH and STORAGE_BACKEND/FS_ROOT selectable; local defaults
  (sqlite ./galo.db, fs ./data)
- Makefile reconcile/reconcile-delete; docs and .env.example updated
2026-10-11 10:08:31 +03:00
iu ec8d907565 feat(storage): serve media through backend and add local FS backend
- introduce storage.Backend with S3 and local file-system implementations
- deliver all blobs via signed GET/HEAD /api/objects/<key> (Range, Content-Type)
- stream S3 objects through the backend; remove presigned URL path
- share shared HMAC Signer between backends
- remove S3_PUBLIC_URL; rename S3_PRESIGN_TTL to OBJECT_URL_TTL
- make storage label in health dynamic; nginx proxy_buffering off for objects
2026-10-11 09:15:03 +03:00
iu 7ae9bcf305 chore(openspec): archive media-object-layer and sync main specs
- move change media-object-layer to openspec/changes/archive/2026-10-10-media-object-layer
- add main spec specs/media-objects (opaque keys, object journal, shared
  objects, reference-based GC, lifecycle recovery, variant generation queue)
- update specs/photos deletion requirement to place photos in the trash
  without deleting storage objects
2026-10-10 23:32:21 +03:00
iu 96268ee16c feat(photos): separate object storage from metadata layer
Move physical file facts out of the photos table into an objects ledger:

- objects / photo_objects / variant_jobs tables (migration 000008)
- drop storage_key/thumb_key/preview_key/size/width/height/mime_type from
  photos (migration 000009); they are derived via joins to the original object
- album-free storage keys: objects/{id}/{role}.{ext}
- explicit GC via pending/ready/deleting states; objects deleted only when no
  photo references them
- background worker drives variant generation (durable variant_jobs) and GC
- copy/move are metadata-only; storage objects are shared between photos
- remove unused storage.DeletePrefix

Non-destructive to the external JSON API: size/dimensions/mime and URLs are
still returned, just sourced from the object layer.

OpenSpec change: media-object-layer
2026-10-10 23:31:41 +03:00
iu 9effc0fc82 Merge pull request 'feat: Lightroom-style workspace UI' (#2) from feature/lightroom-workspace-ui into develop
Reviewed-on: #2
2026-10-10 19:56:14 +00:00
126 changed files with 8651 additions and 281 deletions
+23 -5
View File
@@ -6,18 +6,31 @@ PG_PASSWORD=change_me
PG_DATABASE=shared_album
PG_SSLMODE=disable
# S3 (Garage on s3.localnet)
# Metadata backend: sqlite (default, embedded single instance) or postgres
DB_BACKEND=sqlite
# Path to the SQLite database file (used when DB_BACKEND=sqlite)
SQLITE_PATH=./galo.db
# S3 (Garage on s3.localnet) - internal storage, never exposed to clients
S3_ENDPOINT=http://s3.localnet:3900
S3_REGION=garage
S3_ACCESS_KEY=GKchange_me
S3_SECRET_KEY=change_me
S3_BUCKET=galo-data
S3_USE_PATH_STYLE=true
S3_PUBLIC_URL=http://s3.localnet:3900/galo-data
S3_PRESIGN_TTL=1h
# Auth
JWT_SECRET=change_me_long_random
# Storage backend: fs (default, local filesystem) or s3 (S3-compatible/Garage).
# Blobs are always delivered through the backend via /api/objects/<key>.
STORAGE_BACKEND=fs
# Root directory for object blobs (STORAGE_BACKEND=fs)
FS_ROOT=./data
# Signs object URLs; falls back to the metadata-layer secret when empty
URL_SECRET=
# Lifetime of signed object URLs
OBJECT_URL_TTL=1h
# Auth. The token signing secret is generated on first start and stored in the
# metadata layer (app_settings); it is not configured here.
JWT_ACCESS_TTL=15m
JWT_REFRESH_TTL=720h
@@ -28,3 +41,8 @@ APP_BASE_URL=http://localhost:8080
# SSE
SSE_HEARTBEAT_INTERVAL=25s
SSE_CLIENT_BUFFER=64
# Background worker (variant generation + object GC)
WORKER_INTERVAL=5s
WORKER_BATCH=10
WORKER_PENDING_TTL=10m
+5
View File
@@ -7,3 +7,8 @@ frontend/build/
frontend/.svelte-kit/
node_modules/
*.log
# Local data backends (defaults)
galo.db
galo.db-*
data/
+10 -1
View File
@@ -1,4 +1,4 @@
.PHONY: build frontend backend run migrate tidy test clean
.PHONY: build frontend backend run migrate reconcile reconcile-delete tidy test clean
frontend:
cd frontend && npm ci && npm run build
@@ -20,6 +20,15 @@ run: backend
migrate:
cd backend && go run ./cmd/migrate
# Reconcile object storage against the metadata journal (report only).
reconcile:
cd backend && go run ./cmd/reconcile
# Reconcile and delete orphaned blobs older than --min-age (default 1h).
# Override the grace period when needed: make reconcile-delete MIN_AGE=1m
reconcile-delete:
cd backend && go run ./cmd/reconcile --delete $(if $(MIN_AGE),--min-age $(MIN_AGE),)
tidy:
cd backend && go mod tidy
+31 -4
View File
@@ -30,8 +30,19 @@ cp .env.example .env
```
Ключевые переменные: `PG_*`, `S3_*` (обязательны `UsePathStyle=true` и регион
`garage`), `JWT_SECRET`, `JWT_ACCESS_TTL`, `JWT_REFRESH_TTL`, `HTTP_ADDR`,
`SSE_HEARTBEAT_INTERVAL`, `SSE_CLIENT_BUFFER`.
`garage`), `JWT_ACCESS_TTL`, `JWT_REFRESH_TTL`, `HTTP_ADDR`,
`SSE_HEARTBEAT_INTERVAL`, `SSE_CLIENT_BUFFER`. Секрет подписи токенов не задаётся
в окружении: он генерируется при первом старте и хранится в слое метаданных
(`app_settings`).
Слои данных выбираются параметрами (по умолчанию — локальный режим):
- `STORAGE_BACKEND=fs|s3` — где лежат блобы: `fs` (по умолчанию, `FS_ROOT=./data`)
или `s3` (S3/Garage); наружу всегда отдаётся бэкенд (`/api/objects/<key>`).
- `DB_BACKEND=sqlite|postgres` — где лежат метаданные: `sqlite` (по умолчанию,
`SQLITE_PATH=./galo.db`) или `postgres`. SQLite — встроенный режим без внешней БД,
**только для одного экземпляра приложения на файл** (без горизонтального
масштабирования).
## Сборка
@@ -94,6 +105,21 @@ TEST_DATABASE_URL='postgres://user:pass@pg.localnet:5432/galo?sslmode=disable' \
cd backend && go test ./internal/repo/ -v
```
## Реконсиляция хранилища
Сборка мусора воркера управляется журналом `objects` и не сканирует хранилище, поэтому
блобы без строки в журнале (после сброса БД в обход журнала и т.п.) накапливаются.
Ручная сверка:
```bash
go run ./cmd/reconcile # отчёт: сироты (есть в хранилище, нет в журнале) и пропавшие
go run ./cmd/reconcile --delete # удалить сироты старше --min-age (по умолчанию 1h)
```
- По умолчанию ничего не удаляется; удаление только по явному `--delete`.
- Область — префикс `objects/`; блобы старого формата `albums/` не затрагиваются.
- Пропавшие ключи (в журнале есть, в хранилище нет) только сообщаются.
## Деплой через systemd
1. Соберите бинарник: `make build`.
@@ -129,11 +155,12 @@ backend/
cmd/migrate/ # применение миграций
internal/config # чтение env
internal/db # pgx + golang-migrate
internal/storage # S3 (Garage)
internal/storage # S3 (Garage) / локальная ФС, отдача через бэкенд
internal/store # фабрика репозиториев (postgres / sqlite)
internal/auth # JWT, bcrypt, refresh
internal/albums # альбомы
internal/members # участники, роли, инвайты
internal/photos # загрузка, варианты, presigned URL
internal/photos # загрузка, варианты, отдача через бэкенд
internal/social # лайки, комментарии
internal/notifications # персональные уведомления
internal/sharelinks # ссылки для просмотра
+11 -4
View File
@@ -9,8 +9,9 @@ import (
"shared-album/internal/config"
"shared-album/internal/db"
"shared-album/internal/photos"
"shared-album/internal/settings"
"shared-album/internal/storage"
"shared-album/internal/store"
)
func main() {
@@ -35,14 +36,20 @@ func main() {
}
defer pool.Close()
store, err := storage.New(ctx, cfg.S3)
repos := store.NewPostgres(pool)
secret, err := repos.Settings.Ensure(ctx, settings.JWTSecretKey, settings.GenerateSecret)
if err != nil {
slog.Error("secret init failed", "error", err)
os.Exit(1)
}
blob, err := storage.Open(ctx, cfg, secret)
if err != nil {
slog.Error("storage init failed", "error", err)
os.Exit(1)
}
photoStore := photos.NewStore(pool)
updated, err := photoStore.BackfillEXIF(ctx, store.Get)
updated, err := repos.Photos.BackfillEXIF(ctx, blob.Get)
if err != nil {
slog.Error("backfill failed", "error", err)
os.Exit(1)
+169
View File
@@ -0,0 +1,169 @@
// Command reconcile compares the object storage with the metadata journal and
// reports orphaned blobs (present in storage, absent from the journal). With
// --delete it removes orphans older than --min-age. It never touches missing
// keys (in the journal but not in storage) and never runs automatically.
package main
import (
"bufio"
"context"
"flag"
"io/fs"
"log/slog"
"os"
"strings"
"time"
assets "shared-album"
"shared-album/internal/config"
"shared-album/internal/db"
"shared-album/internal/reconcile"
"shared-album/internal/settings"
"shared-album/internal/storage"
"shared-album/internal/store"
)
const objectPrefix = "objects/"
func main() {
loadDotEnv(".env")
loadDotEnv("../.env")
slog.SetDefault(slog.New(slog.NewTextHandler(os.Stdout, &slog.HandlerOptions{Level: slog.LevelInfo})))
deleteFlag := flag.Bool("delete", false, "delete orphaned blobs older than --min-age")
minAge := flag.Duration("min-age", time.Hour, "minimum blob age before deletion is allowed")
sample := flag.Int("sample", 20, "number of example keys to print")
flag.Parse()
cfg, err := config.Load()
if err != nil {
slog.Error("config error", "error", err)
os.Exit(1)
}
ctx := context.Background()
repos, closer, err := openRepos(ctx, cfg)
if err != nil {
slog.Error("metadata open failed", "error", err)
os.Exit(1)
}
defer closer()
secret, err := repos.Settings.Ensure(ctx, settings.JWTSecretKey, settings.GenerateSecret)
if err != nil {
slog.Error("secret init failed", "error", err)
os.Exit(1)
}
blob, err := storage.Open(ctx, cfg, secret)
if err != nil {
slog.Error("storage init failed", "error", err)
os.Exit(1)
}
objects, err := blob.List(ctx, objectPrefix)
if err != nil {
slog.Error("list storage", "error", err)
os.Exit(1)
}
journaled, err := repos.Photos.AllStorageKeys(ctx)
if err != nil {
slog.Error("list journal", "error", err)
os.Exit(1)
}
res := reconcile.Classify(objects, journaled)
slog.Info("reconciliation report",
"storage", cfg.Storage.Backend,
"storage_objects", len(objects),
"journal_keys", len(journaled),
"orphans", len(res.Orphans),
"missing", len(res.Missing))
printSamples("orphan", res.Orphans, *sample)
printSamples("missing", res.Missing, *sample)
if !*deleteFlag {
slog.Info("dry run: nothing deleted (pass --delete to remove orphans)")
return
}
targets := reconcile.Deletable(objects, res, objectPrefix, *minAge, time.Now())
if len(targets) == 0 {
slog.Info("no orphaned blobs old enough to delete", "min_age", minAge.String())
return
}
if err := blob.Delete(ctx, targets...); err != nil {
slog.Error("delete orphans", "error", err)
os.Exit(1)
}
slog.Info("deleted orphaned blobs", "count", len(targets), "min_age", minAge.String())
}
// openRepos builds the metadata repositories for the configured backend.
func openRepos(ctx context.Context, cfg *config.Config) (store.Repos, func(), error) {
if cfg.Database.Backend == "sqlite" {
migFS, err := fs.Sub(assets.Migrations, "migrations/sqlite")
if err != nil {
return store.Repos{}, nil, err
}
if err := db.MigrateSQLite(cfg.Database.SQLitePath, migFS); err != nil {
return store.Repos{}, nil, err
}
database, err := db.OpenSQLite(cfg.Database.SQLitePath)
if err != nil {
return store.Repos{}, nil, err
}
return store.NewSQLite(database), func() { _ = database.Close() }, nil
}
migFS, err := fs.Sub(assets.Migrations, "migrations")
if err != nil {
return store.Repos{}, nil, err
}
if err := db.Migrate(cfg.Postgres.DSN(), migFS); err != nil {
return store.Repos{}, nil, err
}
pool, err := db.Connect(ctx, cfg.Postgres.DSN())
if err != nil {
return store.Repos{}, nil, err
}
return store.NewPostgres(pool), pool.Close, nil
}
func printSamples(label string, keys []string, n int) {
if len(keys) == 0 {
return
}
if n > len(keys) {
n = len(keys)
}
for _, k := range keys[:n] {
slog.Info("sample", "kind", label, "key", k)
}
}
func loadDotEnv(path string) {
f, err := os.Open(path)
if err != nil {
return
}
defer f.Close()
scanner := bufio.NewScanner(f)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line == "" || strings.HasPrefix(line, "#") {
continue
}
key, value, ok := strings.Cut(line, "=")
if !ok {
continue
}
key = strings.TrimSpace(key)
value = strings.TrimSpace(value)
if _, exists := os.LookupEnv(key); !exists {
_ = os.Setenv(key, value)
}
}
}
+62 -7
View File
@@ -15,10 +15,13 @@ import (
assets "shared-album"
"shared-album/internal/app"
"shared-album/internal/auth"
"shared-album/internal/config"
"shared-album/internal/db"
"shared-album/internal/settings"
"shared-album/internal/sse"
"shared-album/internal/storage"
"shared-album/internal/store"
)
func main() {
@@ -34,9 +37,49 @@ func main() {
os.Exit(1)
}
switch cfg.Database.Backend {
case "sqlite":
slog.Info("metadata backend", "backend", "sqlite", "path", cfg.Database.SQLitePath)
default:
slog.Info("metadata backend", "backend", "postgres",
"host", cfg.Postgres.Host, "database", cfg.Postgres.Database)
}
switch cfg.Storage.Backend {
case "fs":
slog.Info("storage backend", "backend", "fs", "root", cfg.Storage.FSRoot)
default:
slog.Info("storage backend", "backend", "s3",
"endpoint", cfg.S3.Endpoint, "bucket", cfg.S3.Bucket)
}
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
defer stop()
var (
repos store.Repos
ping func(ctx context.Context) error
closer func()
)
switch cfg.Database.Backend {
case "sqlite":
migFS, err := fs.Sub(assets.Migrations, "migrations/sqlite")
if err != nil {
slog.Error("migrations fs", "error", err)
os.Exit(1)
}
if err := db.MigrateSQLite(cfg.Database.SQLitePath, migFS); err != nil {
slog.Error("migration failed", "error", err)
os.Exit(1)
}
database, err := db.OpenSQLite(cfg.Database.SQLitePath)
if err != nil {
slog.Error("database open failed", "error", err)
os.Exit(1)
}
repos = store.NewSQLite(database)
ping = database.PingContext
closer = func() { _ = database.Close() }
default:
migFS, err := fs.Sub(assets.Migrations, "migrations")
if err != nil {
slog.Error("migrations fs", "error", err)
@@ -46,28 +89,40 @@ func main() {
slog.Error("migration failed", "error", err)
os.Exit(1)
}
slog.Info("migrations applied")
pool, err := db.Connect(ctx, cfg.Postgres.DSN())
if err != nil {
slog.Error("database connect failed", "error", err)
os.Exit(1)
}
defer pool.Close()
repos = store.NewPostgres(pool)
ping = pool.Ping
closer = pool.Close
}
defer closer()
slog.Info("migrations applied", "db_backend", cfg.Database.Backend)
store, err := storage.New(ctx, cfg.S3)
secret, err := repos.Settings.Ensure(ctx, settings.JWTSecretKey, settings.GenerateSecret)
if err != nil {
slog.Error("secret init failed", "error", err)
os.Exit(1)
}
tm := auth.NewTokenManager(secret, cfg.Auth.AccessTTL, cfg.Auth.RefreshTTL)
blob, err := storage.Open(ctx, cfg, secret)
if err != nil {
slog.Error("storage init failed", "error", err)
os.Exit(1)
}
if err := store.EnsureBucket(ctx); err != nil {
if err := blob.EnsureBucket(ctx); err != nil {
slog.Warn("could not ensure bucket", "error", err)
} else {
slog.Info("storage bucket ready", "bucket", store.Bucket())
slog.Info("storage ready", "location", blob.Bucket())
}
hub := sse.NewHub(cfg.SSE.ClientBuffer)
server := app.New(cfg, pool, store, hub)
server := app.New(cfg, repos, ping, tm, blob, hub)
go server.StartWorker(ctx)
httpServer := &http.Server{
Addr: cfg.Server.Addr,
+2 -2
View File
@@ -7,7 +7,7 @@ import "embed"
//go:embed all:static
var Static embed.FS
// Migrations holds the SQL migration files.
// Migrations holds the SQL migration files for every supported backend.
//
//go:embed migrations/*.sql
//go:embed migrations/*.sql migrations/sqlite/*.sql
var Migrations embed.FS
+8 -1
View File
@@ -17,6 +17,7 @@ require (
github.com/google/uuid v1.6.0
github.com/jackc/pgx/v5 v5.11.0
golang.org/x/crypto v0.58.0
modernc.org/sqlite v1.60.1
)
require (
@@ -39,6 +40,7 @@ require (
github.com/cloudwego/base64x v0.1.6 // indirect
github.com/dsoprea/go-logging v0.0.0-20200710184922-b02d349568dd // indirect
github.com/dsoprea/go-utility/v2 v2.0.0-20221003172846-a3e1774ef349 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/ebitengine/purego v0.10.1 // indirect
github.com/gabriel-vasile/mimetype v1.4.12 // indirect
github.com/gin-contrib/sse v1.1.0 // indirect
@@ -56,12 +58,14 @@ require (
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-isatty v0.0.24 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
github.com/quic-go/qpack v0.6.0 // indirect
github.com/quic-go/quic-go v0.59.0 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/ugorji/go/codec v1.3.1 // indirect
go.mongodb.org/mongo-driver/v2 v2.5.0 // indirect
@@ -73,4 +77,7 @@ require (
golang.org/x/text v0.43.0 // indirect
google.golang.org/protobuf v1.36.11 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
modernc.org/libc v1.77.1 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.12.1 // indirect
)
+44 -3
View File
@@ -85,6 +85,8 @@ github.com/dsoprea/go-utility/v2 v2.0.0-20221003142440-7a1927d49d9d/go.mod h1:LV
github.com/dsoprea/go-utility/v2 v2.0.0-20221003160719-7bc88537c05e/go.mod h1:VZ7cB0pTjm1ADBWhJUOHESu4ZYy9JN+ZPqjfiW09EPU=
github.com/dsoprea/go-utility/v2 v2.0.0-20221003172846-a3e1774ef349 h1:DilThiXje0z+3UQ5YjYiSRRzVdtamFpvBQXKwMglWqw=
github.com/dsoprea/go-utility/v2 v2.0.0-20221003172846-a3e1774ef349/go.mod h1:4GC5sXji84i/p+irqghpPFZBF8tRN/Q7+700G0/DLe8=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/ebitengine/purego v0.10.1 h1:dewVBCBT2GaMu1SrNTYxQhgQBethzfhiwvZiLGP/qyY=
github.com/ebitengine/purego v0.10.1/go.mod h1:iIjxzd6CiRiOG0UyXP+V1+jWqUXVjPKLAI0mRfJZTmQ=
github.com/felixge/httpsnoop v1.1.0 h1:3YtUj32ZZkqZtt3sZZsClsymw/QDuVfpNhoA31zeORc=
@@ -129,8 +131,12 @@ github.com/golang/geo v0.0.0-20210211234256-740aa86cb551/go.mod h1:QZ0nwyI2jOfgR
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3 h1:LMLX+LgTNWpfvCBdFebv6EsYotImrt/Ppc5cXIriCSo=
github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3/go.mod h1:jl5iWTm0/hd5PjEYEOuwAJ57L/CibdZfrqZ5XA5GrCk=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/jackc/pgerrcode v0.0.0-20220416144525-469b46aa5efa h1:s+4MhCQ6YrzisK6hFJUX53drDT4UsSW3DEhKn0ifuHw=
github.com/jackc/pgerrcode v0.0.0-20220416144525-469b46aa5efa/go.mod h1:a/s9Lp5W7n/DD0VrVoyJ00FbP2ytTPDVOivvn2bMlds=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
@@ -155,8 +161,8 @@ github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=
github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0=
github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
github.com/moby/moby/api v1.54.2 h1:wiat9QAhnDQjA7wk1kh/TqHz2I1uUA7M7t9SAl/JNXg=
@@ -172,6 +178,8 @@ github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9G
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040=
@@ -187,6 +195,8 @@ github.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8=
github.com/quic-go/qpack v0.6.0/go.mod h1:lUpLKChi8njB4ty2bFLX2x4gzDqXwUpaO1DP9qMDZII=
github.com/quic-go/quic-go v0.59.0 h1:OLJkp1Mlm/aS7dpKgTc6cnpynnD2Xg7C1pwL6vy/SAw=
github.com/quic-go/quic-go v0.59.0/go.mod h1:upnsH4Ju1YkqpLXC305eW3yDZ4NfnNbmQRCMWS58IKU=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ=
github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
@@ -226,6 +236,8 @@ golang.org/x/crypto v0.58.0 h1:COkYLr4k7nDI4r6QtMML2AfzeIswWHAJthLS6K/M/54=
golang.org/x/crypto v0.58.0/go.mod h1:Xh+kl5A+M0gMiWDytH2Hhr5Z7WbNCpv/eB/EFgfSk+w=
golang.org/x/image v0.0.0-20191009234506-e7c1f5e7dbb8 h1:hVwzHzIUGRjiF7EcUjqNxk3NCfkPxbDKRdnNE1Rpg0U=
golang.org/x/image v0.0.0-20191009234506-e7c1f5e7dbb8/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/mod v0.41.0 h1:qJmnOUb4YB+FsEuM3HcWucdZASCPGhsX6uljO6pog0c=
golang.org/x/mod v0.41.0/go.mod h1:Ek9pY8RKWXwsWvd3rQiHYtMqkjSUV+s1Rj7j4H5Ur6o=
golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200320220750-118fecf932d8/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
@@ -241,7 +253,6 @@ golang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220728004956-3c1f35247d10/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220928140112-f11e5e49a4ec/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.49.0 h1:XbzkgYJHdqh/8m2Uu0W/dQv8nktxx4BFHp1M0gROTXA=
golang.org/x/sys v0.49.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
@@ -250,6 +261,8 @@ golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.43.0 h1:1QivrlhwAsnMOcqgOdl9my7s9OyZl8O5/UGUTJVXoWc=
golang.org/x/text v0.43.0/go.mod h1:hbSIYA/amXcRXTFSZqh8tWHEoqRvvbKPJ2uz/KU+a+s=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.51.0 h1:k4Xc/1Om9jwkBJBo4NVLMSARBoWtK10mx+W5BnXCeAI=
golang.org/x/tools v0.51.0/go.mod h1:9eEncMayCV6zRMGhR5eZEC2iBx98qWcF1HZ9Z7wJOoA=
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
@@ -262,3 +275,31 @@ gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
modernc.org/cc/v4 v4.29.7 h1:q+NXGJ0bK3b4TXFYQQVr9pYETGnmwFWkrUzJnMya/Tg=
modernc.org/cc/v4 v4.29.7/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI=
modernc.org/ccgo/v4 v4.36.1 h1:ZNIUZAryN0UgnJwtyxrdEzcFc3yD4Cu4AzjfPXsLsIE=
modernc.org/ccgo/v4 v4.36.1/go.mod h1:rrtGc2QkS239nYb/mQNuBMyjq3/y3ZXWbBjPoV3wqzA=
modernc.org/fileutil v1.4.0 h1:j6ZzNTftVS054gi281TyLjHPp6CPHr2KCxEXjEbD6SM=
modernc.org/fileutil v1.4.0/go.mod h1:EqdKFDxiByqxLk8ozOxObDSfcVOv/54xDs/DUHdvCUU=
modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
modernc.org/gc/v3 v3.1.5 h1:21ldfPfRYE31Tb7B3mwAK8gy1AxP4+dKjrOQPfqakoc=
modernc.org/gc/v3 v3.1.5/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
modernc.org/libc v1.77.1 h1:Ct8j47QtiZ1Enj2DtFXQtUqrPCAjdCmPjtCuvrYQ0Hs=
modernc.org/libc v1.77.1/go.mod h1:87/pZ4L6nD1zqW4nItuS12YO7hN1igAah34xjnQo/W0=
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
modernc.org/memory v1.12.1 h1:nFMiWrpStgZczNl6XI9GnIk/rWhYIyHGUaR04pGbp9g=
modernc.org/memory v1.12.1/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg=
modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.60.1 h1:/blz53O951KWFOso4QQvEs/Fq6cDBKLtMVrYNSeJVKw=
modernc.org/sqlite v1.60.1/go.mod h1:1dIoEagfDE72QytD5scH1lxARtaUgKgHC/NuApA27r0=
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
+18
View File
@@ -0,0 +1,18 @@
package albums
import (
"context"
"shared-album/internal/models"
)
// Repository is the persistence contract for the albums domain.
type Repository interface {
Create(ctx context.Context, a *models.Album) error
OwnerID(ctx context.Context, id string) (string, error)
GetByID(ctx context.Context, id string) (*models.Album, error)
ListForUser(ctx context.Context, userID string) ([]models.Album, error)
Update(ctx context.Context, a *models.Album) error
Delete(ctx context.Context, id string) error
TrashPhotosByAlbum(ctx context.Context, albumID string) error
}
+5 -5
View File
@@ -18,13 +18,13 @@ type MemberLookup interface {
}
type Service struct {
store *Store
store Repository
members MemberLookup
hub *sse.Hub
storage *storage.Client
storage storage.Backend
}
func NewService(store *Store, members MemberLookup, hub *sse.Hub, storage *storage.Client) *Service {
func NewService(store Repository, members MemberLookup, hub *sse.Hub, storage storage.Backend) *Service {
return &Service{store: store, members: members, hub: hub, storage: storage}
}
@@ -61,12 +61,12 @@ func (s *Service) List(ctx context.Context, userID string) ([]models.Album, erro
return out, nil
}
// attachCover fills CoverThumbURL from the presigned thumbnail key, when present.
// attachCover fills CoverThumbURL from the signed thumbnail URL, when present.
func (s *Service) attachCover(ctx context.Context, a *models.Album) {
if s.storage == nil || a == nil || a.CoverThumbKey == "" {
return
}
if url, err := s.storage.PresignGet(ctx, a.CoverThumbKey); err == nil {
if url, err := s.storage.URL(ctx, a.CoverThumbKey); err == nil {
a.CoverThumbURL = url
}
}
+142
View File
@@ -0,0 +1,142 @@
package albums
import (
"context"
"database/sql"
"errors"
"time"
"shared-album/internal/models"
"shared-album/internal/sqliteutil"
)
// SQLiteStore is the SQLite implementation of Repository.
type SQLiteStore struct {
db *sql.DB
}
func NewSQLiteStore(db *sql.DB) *SQLiteStore { return &SQLiteStore{db: db} }
func (s *SQLiteStore) Create(ctx context.Context, a *models.Album) error {
var createdAt, updatedAt string
err := s.db.QueryRowContext(ctx,
`INSERT INTO albums (id, owner_id, title, description, cover_photo_id)
VALUES ($1, $2, $3, $4, $5)
RETURNING created_at, updated_at`,
a.ID, a.OwnerID, a.Title, a.Description, a.CoverPhotoID,
).Scan(&createdAt, &updatedAt)
if err != nil {
return err
}
a.CreatedAt = sqliteutil.ParseTime(createdAt)
a.UpdatedAt = sqliteutil.ParseTime(updatedAt)
return nil
}
func (s *SQLiteStore) OwnerID(ctx context.Context, id string) (string, error) {
var ownerID string
err := s.db.QueryRowContext(ctx, `SELECT owner_id FROM albums WHERE id = $1`, id).Scan(&ownerID)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return "", ErrNotFound
}
return "", err
}
return ownerID, nil
}
const sqliteAlbumColumns = `a.id, a.owner_id, a.title, a.description, a.cover_photo_id,
a.created_at, a.updated_at, COALESCE(co.storage_key, ''),
(SELECT count(*) FROM photos WHERE album_id = a.id),
(SELECT count(*) FROM album_members WHERE album_id = a.id)`
const sqliteAlbumJoins = `
LEFT JOIN photo_objects cpo ON cpo.photo_id = a.cover_photo_id AND cpo.role = 'thumb'
LEFT JOIN objects co ON co.id = cpo.object_id`
func scanSQLiteAlbum(scan func(...any) error) (*models.Album, error) {
var a models.Album
var cover sql.NullString
var createdAt, updatedAt string
if err := scan(&a.ID, &a.OwnerID, &a.Title, &a.Description, &cover, &createdAt, &updatedAt,
&a.CoverThumbKey, &a.PhotosCount, &a.MembersCount); err != nil {
return nil, err
}
if cover.Valid {
a.CoverPhotoID = &cover.String
}
a.CreatedAt = sqliteutil.ParseTime(createdAt)
a.UpdatedAt = sqliteutil.ParseTime(updatedAt)
return &a, nil
}
func (s *SQLiteStore) GetByID(ctx context.Context, id string) (*models.Album, error) {
row := s.db.QueryRowContext(ctx,
`SELECT `+sqliteAlbumColumns+` FROM albums a`+sqliteAlbumJoins+` WHERE a.id = $1`, id)
a, err := scanSQLiteAlbum(row.Scan)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return nil, ErrNotFound
}
return nil, err
}
return a, nil
}
func (s *SQLiteStore) ListForUser(ctx context.Context, userID string) ([]models.Album, error) {
rows, err := s.db.QueryContext(ctx,
`SELECT `+sqliteAlbumColumns+` FROM albums a
LEFT JOIN album_members m ON m.album_id = a.id AND m.user_id = $1`+sqliteAlbumJoins+`
WHERE a.owner_id = $1 OR m.user_id IS NOT NULL
ORDER BY a.created_at DESC`, userID)
if err != nil {
return nil, err
}
defer rows.Close()
out := []models.Album{}
for rows.Next() {
a, err := scanSQLiteAlbum(rows.Scan)
if err != nil {
return nil, err
}
out = append(out, *a)
}
return out, rows.Err()
}
func (s *SQLiteStore) Update(ctx context.Context, a *models.Album) error {
var updatedAt string
err := s.db.QueryRowContext(ctx,
`UPDATE albums
SET title = $2, description = $3, cover_photo_id = $4, updated_at = $5
WHERE id = $1
RETURNING updated_at`,
a.ID, a.Title, a.Description, a.CoverPhotoID, sqliteutil.FormatTime(time.Now().UTC()),
).Scan(&updatedAt)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return ErrNotFound
}
return err
}
a.UpdatedAt = sqliteutil.ParseTime(updatedAt)
return nil
}
func (s *SQLiteStore) Delete(ctx context.Context, id string) error {
res, err := s.db.ExecContext(ctx, `DELETE FROM albums WHERE id = $1`, id)
if err != nil {
return err
}
if n, _ := res.RowsAffected(); n == 0 {
return ErrNotFound
}
return nil
}
func (s *SQLiteStore) TrashPhotosByAlbum(ctx context.Context, albumID string) error {
_, err := s.db.ExecContext(ctx,
`UPDATE photos SET deleted_at = $2 WHERE album_id = $1 AND deleted_at IS NULL`,
albumID, sqliteutil.FormatTime(time.Now().UTC()))
return err
}
+6 -4
View File
@@ -44,11 +44,12 @@ func (s *Store) GetByID(ctx context.Context, id string) (*models.Album, error) {
var a models.Album
err := s.pool.QueryRow(ctx,
`SELECT a.id, a.owner_id, a.title, a.description, a.cover_photo_id, a.created_at, a.updated_at,
COALESCE(p.thumb_key, ''),
COALESCE(co.storage_key, ''),
(SELECT count(*) FROM photos WHERE album_id = a.id),
(SELECT count(*) FROM album_members WHERE album_id = a.id)
FROM albums a
LEFT JOIN photos p ON p.id = a.cover_photo_id
LEFT JOIN photo_objects cpo ON cpo.photo_id = a.cover_photo_id AND cpo.role = 'thumb'
LEFT JOIN objects co ON co.id = cpo.object_id
WHERE a.id = $1`, id).
Scan(&a.ID, &a.OwnerID, &a.Title, &a.Description, &a.CoverPhotoID, &a.CreatedAt, &a.UpdatedAt, &a.CoverThumbKey, &a.PhotosCount, &a.MembersCount)
if err != nil {
@@ -63,12 +64,13 @@ func (s *Store) GetByID(ctx context.Context, id string) (*models.Album, error) {
func (s *Store) ListForUser(ctx context.Context, userID string) ([]models.Album, error) {
rows, err := s.pool.Query(ctx,
`SELECT a.id, a.owner_id, a.title, a.description, a.cover_photo_id, a.created_at, a.updated_at,
COALESCE(p.thumb_key, ''),
COALESCE(co.storage_key, ''),
(SELECT count(*) FROM photos WHERE album_id = a.id),
(SELECT count(*) FROM album_members WHERE album_id = a.id)
FROM albums a
LEFT JOIN album_members m ON m.album_id = a.id AND m.user_id = $1
LEFT JOIN photos p ON p.id = a.cover_photo_id
LEFT JOIN photo_objects cpo ON cpo.photo_id = a.cover_photo_id AND cpo.role = 'thumb'
LEFT JOIN objects co ON co.id = cpo.object_id
WHERE a.owner_id = $1 OR m.user_id IS NOT NULL
ORDER BY a.created_at DESC`, userID)
if err != nil {
+56 -29
View File
@@ -1,13 +1,14 @@
package app
import (
"context"
"errors"
"io/fs"
"net/http"
"strings"
"time"
"github.com/gin-gonic/gin"
"github.com/jackc/pgx/v5/pgxpool"
assets "shared-album"
"shared-album/internal/albums"
@@ -21,15 +22,16 @@ import (
"shared-album/internal/social"
"shared-album/internal/sse"
"shared-album/internal/storage"
"shared-album/internal/store"
)
type Server struct {
cfg *config.Config
pool *pgxpool.Pool
storage *storage.Client
repos store.Repos
ping func(ctx context.Context) error
storage storage.Backend
hub *sse.Hub
albumsStore *albums.Store
albums *albums.Service
auth *auth.Service
@@ -37,6 +39,7 @@ type Server struct {
membersSvc *members.Service
photos *photos.Service
worker *photos.Worker
social *social.Service
@@ -45,45 +48,47 @@ type Server struct {
share *sharelinks.Service
}
func New(cfg *config.Config, pool *pgxpool.Pool, st *storage.Client, hub *sse.Hub) *Server {
authStore := auth.NewStore(pool)
tm := auth.NewTokenManager(cfg.Auth.JWTSecret, cfg.Auth.AccessTTL, cfg.Auth.RefreshTTL)
authSvc := auth.NewService(authStore, tm)
// New wires the domain services over the selected metadata repositories. ping
// reports the health of the metadata backend so health stays backend-agnostic.
func New(cfg *config.Config, repos store.Repos, ping func(ctx context.Context) error, tm *auth.TokenManager, st storage.Backend, hub *sse.Hub) *Server {
authSvc := auth.NewService(repos.Auth, tm)
albumsStore := albums.NewStore(pool)
membersStore := members.NewStore(pool)
notifSvc := notifications.NewService(repos.Notifications, hub)
notifStore := notifications.NewStore(pool)
notifSvc := notifications.NewService(notifStore, hub)
albumsSvc := albums.NewService(repos.Albums, repos.Members, hub, st)
membersSvc := members.NewService(repos.Members, repos.Albums, hub, notifSvc)
albumsSvc := albums.NewService(albumsStore, membersStore, hub, st)
membersSvc := members.NewService(membersStore, albumsStore, hub, notifSvc)
photosSvc := photos.NewService(repos.Photos, albumsSvc, st, hub)
photosWorker := photos.NewWorker(photosSvc, cfg.Worker.Interval, cfg.Worker.Batch, cfg.Worker.PendingTTL)
photosStore := photos.NewStore(pool)
photosSvc := photos.NewService(photosStore, albumsSvc, st, hub)
socialSvc := social.NewService(repos.Social, albumsSvc, repos.Photos, hub, notifSvc)
socialStore := social.NewStore(pool)
socialSvc := social.NewService(socialStore, albumsSvc, photosStore, hub, notifSvc)
shareStore := sharelinks.NewStore(pool)
shareSvc := sharelinks.NewService(shareStore, albumsSvc)
shareSvc := sharelinks.NewService(repos.ShareLinks, albumsSvc)
return &Server{
cfg: cfg,
pool: pool,
repos: repos,
ping: ping,
storage: st,
hub: hub,
albumsStore: albumsStore,
albums: albumsSvc,
auth: authSvc,
membersSvc: membersSvc,
photos: photosSvc,
worker: photosWorker,
social: socialSvc,
notifications: notifSvc,
share: shareSvc,
}
}
// StartWorker runs the background reconciler until ctx is cancelled.
func (s *Server) StartWorker(ctx context.Context) {
if s.worker != nil {
s.worker.Run(ctx)
}
}
func (s *Server) Router() *gin.Engine {
r := gin.New()
r.Use(gin.Recovery())
@@ -98,6 +103,8 @@ func (s *Server) Router() *gin.Engine {
api := r.Group("/api")
api.GET("/health", s.health)
api.GET("/objects/*key", s.serveObject)
api.HEAD("/objects/*key", s.serveObject)
api.POST("/auth/register", authHandler.Register)
api.POST("/auth/login", authHandler.Login)
@@ -169,20 +176,40 @@ func (s *Server) Router() *gin.Engine {
func (s *Server) health(c *gin.Context) {
ctx := c.Request.Context()
dbStatus := "ok"
if err := s.pool.Ping(ctx); err != nil {
if err := s.ping(ctx); err != nil {
dbStatus = "error"
}
s3Status := "ok"
storageLabel := s.cfg.Storage.Backend
if storageLabel == "" {
storageLabel = "s3"
}
storeStatus := "ok"
if err := s.storage.Ping(ctx); err != nil {
s3Status = "error"
storeStatus = "error"
}
status := http.StatusOK
overall := "ok"
if dbStatus != "ok" || s3Status != "ok" {
if dbStatus != "ok" || storeStatus != "ok" {
status = http.StatusServiceUnavailable
overall = "degraded"
}
c.JSON(status, gin.H{"status": overall, "db": dbStatus, "s3": s3Status})
c.JSON(status, gin.H{"status": overall, "db": dbStatus, storageLabel: storeStatus})
}
func (s *Server) serveObject(c *gin.Context) {
key := strings.TrimPrefix(c.Param("key"), "/")
switch err := s.storage.ServeObject(c.Writer, c.Request, key); {
case err == nil:
return
case errors.Is(err, storage.ErrSignedURLInvalid):
c.Status(http.StatusForbidden)
case errors.Is(err, storage.ErrObjectNotFound):
httpx.RespondError(c, httpx.NotFound("object_not_found", "object not found"))
case errors.Is(err, storage.ErrRangeNotSatisfiable):
c.Status(http.StatusRequestedRangeNotSatisfiable)
default:
c.Status(http.StatusInternalServerError)
}
}
func (s *Server) albumSSE(c *gin.Context) {
@@ -209,7 +236,7 @@ func (s *Server) shareAlbum(c *gin.Context) {
httpx.RespondError(c, err)
return
}
album, err := s.albumsStore.GetByID(c.Request.Context(), link.AlbumID)
album, err := s.repos.Albums.GetByID(c.Request.Context(), link.AlbumID)
if err != nil {
httpx.RespondError(c, httpx.NotFound("album_not_found", "album not found"))
return
+61
View File
@@ -0,0 +1,61 @@
package app
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/gin-gonic/gin"
"shared-album/internal/config"
"shared-album/internal/storage"
)
func mustSigner(t *testing.T) *storage.Signer {
t.Helper()
signer, err := storage.NewSigner("http://app.test", "secret", time.Hour)
if err != nil {
t.Fatalf("signer: %v", err)
}
return signer
}
func TestHealthStorageLabel(t *testing.T) {
gin.SetMode(gin.TestMode)
fs, err := storage.NewFS(t.TempDir(), mustSigner(t))
if err != nil {
t.Fatalf("fs: %v", err)
}
ok := func(context.Context) error { return nil }
srv := &Server{cfg: &config.Config{}, ping: ok, storage: fs}
call := func() map[string]any {
rec := httptest.NewRecorder()
c, _ := gin.CreateTestContext(rec)
c.Request = httptest.NewRequest(http.MethodGet, "/api/health", nil)
srv.health(c)
var body map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil {
t.Fatalf("decode: %v", err)
}
return body
}
srv.cfg.Storage.Backend = "fs"
if body := call(); body["fs"] != "ok" {
t.Fatalf("expected fs label ok, got %v", body)
}
srv.cfg.Storage.Backend = "s3"
if body := call(); body["s3"] != "ok" {
t.Fatalf("expected s3 label ok, got %v", body)
}
srv.cfg.Storage.Backend = ""
if body := call(); body["s3"] != "ok" {
t.Fatalf("expected default s3 label, got %v", body)
}
}
+114
View File
@@ -0,0 +1,114 @@
package app
import (
"context"
"net/http"
"net/http/httptest"
"net/url"
"strconv"
"strings"
"testing"
"time"
"github.com/gin-gonic/gin"
"shared-album/internal/auth"
"shared-album/internal/config"
"shared-album/internal/sse"
"shared-album/internal/storage"
"shared-album/internal/store"
)
func newTestServerWithFS(t *testing.T) (*storage.FS, *gin.Engine) {
t.Helper()
gin.SetMode(gin.TestMode)
signer, err := storage.NewSigner("http://app.test", "secret", time.Hour)
if err != nil {
t.Fatalf("signer: %v", err)
}
fs, err := storage.NewFS(t.TempDir(), signer)
if err != nil {
t.Fatalf("new fs: %v", err)
}
noop := func(context.Context) error { return nil }
tm := auth.NewTokenManager("test-secret", 15*time.Minute, time.Hour)
srv := New(&config.Config{}, store.Repos{}, noop, tm, fs, sse.NewHub(1))
return fs, srv.Router()
}
func TestServeObjectRoute(t *testing.T) {
ctx := context.Background()
fs, r := newTestServerWithFS(t)
key := "objects/abc/original.jpg"
if err := fs.Put(ctx, key, strings.NewReader("0123456789"), 10, "image/jpeg"); err != nil {
t.Fatalf("put: %v", err)
}
raw, err := fs.URL(ctx, key)
if err != nil {
t.Fatalf("url: %v", err)
}
u, err := url.Parse(raw)
if err != nil {
t.Fatalf("parse: %v", err)
}
do := func(target string, rangeHeader string) *httptest.ResponseRecorder {
req := httptest.NewRequest(http.MethodGet, target, nil)
if rangeHeader != "" {
req.Header.Set("Range", rangeHeader)
}
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
return rec
}
t.Run("valid", func(t *testing.T) {
rec := do(u.Path+"?"+u.RawQuery, "")
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200", rec.Code)
}
if rec.Body.String() != "0123456789" {
t.Fatalf("body = %q", rec.Body.String())
}
})
t.Run("range", func(t *testing.T) {
rec := do(u.Path+"?"+u.RawQuery, "bytes=2-5")
if rec.Code != http.StatusPartialContent {
t.Fatalf("status = %d, want 206", rec.Code)
}
if rec.Body.String() != "2345" {
t.Fatalf("body = %q, want 2345", rec.Body.String())
}
})
t.Run("expired", func(t *testing.T) {
q := u.Query()
q.Set("e", strconv.FormatInt(time.Now().Add(-time.Minute).Unix(), 10))
rec := do(u.Path+"?"+q.Encode(), "")
if rec.Code != http.StatusForbidden {
t.Fatalf("status = %d, want 403", rec.Code)
}
})
t.Run("forged", func(t *testing.T) {
q := u.Query()
q.Set("s", "deadbeef")
rec := do(u.Path+"?"+q.Encode(), "")
if rec.Code != http.StatusForbidden {
t.Fatalf("status = %d, want 403", rec.Code)
}
})
t.Run("missing", func(t *testing.T) {
raw, err := fs.URL(ctx, "objects/def/original.jpg")
if err != nil {
t.Fatalf("url: %v", err)
}
mu, _ := url.Parse(raw)
rec := do(mu.Path+"?"+mu.RawQuery, "")
if rec.Code != http.StatusNotFound {
t.Fatalf("status = %d, want 404", rec.Code)
}
})
}
+213
View File
@@ -0,0 +1,213 @@
package app
import (
"bytes"
"context"
"encoding/json"
"image"
"image/color"
"image/png"
"net/http"
"net/http/httptest"
"net/url"
"os"
"strings"
"testing"
"time"
"github.com/gin-gonic/gin"
"github.com/jackc/pgx/v5/pgxpool"
"shared-album/internal/albums"
"shared-album/internal/auth"
"shared-album/internal/config"
"shared-album/internal/members"
"shared-album/internal/models"
"shared-album/internal/photos"
"shared-album/internal/sharelinks"
"shared-album/internal/sse"
"shared-album/internal/storage"
"shared-album/internal/store"
)
func integrationPool(t *testing.T) *pgxpool.Pool {
t.Helper()
dsn := os.Getenv("TEST_DATABASE_URL")
if dsn == "" {
t.Skip("TEST_DATABASE_URL not set")
}
pool, err := pgxpool.New(context.Background(), dsn)
if err != nil {
t.Skipf("cannot connect: %v", err)
}
if err := pool.Ping(context.Background()); err != nil {
t.Skipf("cannot ping: %v", err)
}
t.Cleanup(pool.Close)
return pool
}
func makeTestPNG(t *testing.T) []byte {
t.Helper()
img := image.NewRGBA(image.Rect(0, 0, 4, 4))
for x := 0; x < 4; x++ {
for y := 0; y < 4; y++ {
img.Set(x, y, color.RGBA{R: uint8(x * 40), G: uint8(y * 40), B: 128, A: 255})
}
}
var buf bytes.Buffer
if err := png.Encode(&buf, img); err != nil {
t.Fatalf("encode png: %v", err)
}
return buf.Bytes()
}
// runShareFlow uploads a photo and validates the read-only share flow returns a
// usable photo URL. verify inspects/serves the URL according to the backend.
func runShareFlow(t *testing.T, backend storage.Backend, verify func(t *testing.T, router *gin.Engine, raw string)) {
t.Helper()
ctx := context.Background()
pool := integrationPool(t)
gin.SetMode(gin.TestMode)
hub := sse.NewHub(1)
tm := auth.NewTokenManager("test-secret", 15*time.Minute, time.Hour)
srv := New(&config.Config{}, store.NewPostgres(pool), pool.Ping, tm, backend, hub)
router := srv.Router()
authStore := auth.NewStore(pool)
owner := &models.User{ID: auth.NewID(), Email: auth.NewID()[:8] + "@test.local", PasswordHash: "x", Name: "Owner"}
if err := authStore.CreateUser(ctx, owner); err != nil {
t.Fatalf("create user: %v", err)
}
albumsStore := albums.NewStore(pool)
albumsSvc := albums.NewService(albumsStore, members.NewStore(pool), hub, backend)
album, err := albumsSvc.Create(ctx, owner.ID, albums.CreateInput{Title: "Share Album"})
if err != nil {
t.Fatalf("create album: %v", err)
}
photosSvc := photos.NewService(photos.NewStore(pool), albumsSvc, backend, hub)
pngData := makeTestPNG(t)
if _, err := photosSvc.Upload(ctx, photos.UploadInput{
AlbumID: album.ID, UserID: owner.ID, Filename: "pic.png", Data: pngData,
}); err != nil {
t.Fatalf("upload: %v", err)
}
var keys []string
rows, err := pool.Query(ctx,
`SELECT o.storage_key FROM photo_objects po
JOIN objects o ON o.id = po.object_id
JOIN photos p ON p.id = po.photo_id
WHERE p.album_id = $1`, album.ID)
if err == nil {
for rows.Next() {
var k string
if rows.Scan(&k) == nil {
keys = append(keys, k)
}
}
rows.Close()
}
t.Cleanup(func() {
_, _ = pool.Exec(ctx, `DELETE FROM photos WHERE album_id = $1`, album.ID)
for _, k := range keys {
_, _ = pool.Exec(ctx, `DELETE FROM objects WHERE storage_key = $1`, k)
}
_ = backend.Delete(context.Background(), keys...)
_, _ = pool.Exec(ctx, `DELETE FROM albums WHERE id = $1`, album.ID)
_, _ = pool.Exec(ctx, `DELETE FROM users WHERE id = $1`, owner.ID)
})
shareSvc := sharelinks.NewService(sharelinks.NewStore(pool), albumsSvc)
link, err := shareSvc.Create(ctx, album.ID, owner.ID, time.Hour)
if err != nil {
t.Fatalf("share create: %v", err)
}
rec := httptest.NewRecorder()
router.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/api/share/"+link.Token+"/photos", nil))
if rec.Code != http.StatusOK {
t.Fatalf("share photos status = %d body=%s", rec.Code, rec.Body.String())
}
var body struct {
Photos []models.Photo `json:"photos"`
}
if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil {
t.Fatalf("decode: %v", err)
}
if len(body.Photos) != 1 {
t.Fatalf("photos len = %d, want 1", len(body.Photos))
}
if body.Photos[0].OriginalURL == "" {
t.Fatalf("empty original url")
}
verify(t, router, body.Photos[0].OriginalURL)
}
func TestShareFlowWithFS(t *testing.T) {
signer, err := storage.NewSigner("http://app.test", "secret", time.Hour)
if err != nil {
t.Fatalf("signer: %v", err)
}
fs, err := storage.NewFS(t.TempDir(), signer)
if err != nil {
t.Fatalf("new fs: %v", err)
}
runShareFlow(t, fs, func(t *testing.T, router *gin.Engine, raw string) {
if !strings.Contains(raw, "/api/objects/") {
t.Fatalf("expected fs signed url, got %q", raw)
}
u, err := url.Parse(raw)
if err != nil {
t.Fatalf("parse url: %v", err)
}
rec := httptest.NewRecorder()
router.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, u.Path+"?"+u.RawQuery, nil))
if rec.Code != http.StatusOK {
t.Fatalf("fs object fetch status = %d", rec.Code)
}
})
}
func TestShareFlowWithS3(t *testing.T) {
endpoint := os.Getenv("S3_ENDPOINT")
if endpoint == "" {
t.Skip("S3_ENDPOINT not set")
}
signer, err := storage.NewSigner("http://app.test", "secret", time.Hour)
if err != nil {
t.Skipf("signer init: %v", err)
}
cl, err := storage.NewS3(context.Background(), config.S3{
Endpoint: endpoint,
Region: os.Getenv("S3_REGION"),
AccessKey: os.Getenv("S3_ACCESS_KEY"),
SecretKey: os.Getenv("S3_SECRET_KEY"),
Bucket: os.Getenv("S3_BUCKET"),
UsePathStyle: true,
}, signer)
if err != nil {
t.Skipf("storage init: %v", err)
}
runShareFlow(t, cl, func(t *testing.T, router *gin.Engine, raw string) {
if !strings.Contains(raw, "/api/objects/") {
t.Fatalf("expected backend url, got %q", raw)
}
if strings.Contains(raw, "X-Amz-Signature") {
t.Fatalf("presigned url leaked to client: %q", raw)
}
u, err := url.Parse(raw)
if err != nil {
t.Fatalf("parse url: %v", err)
}
rec := httptest.NewRecorder()
router.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, u.Path+"?"+u.RawQuery, nil))
if rec.Code != http.StatusOK {
t.Fatalf("s3 object fetch status = %d", rec.Code)
}
})
}
+19
View File
@@ -0,0 +1,19 @@
package auth
import (
"context"
"shared-album/internal/models"
)
// Repository is the persistence contract for the auth domain.
type Repository interface {
CreateUser(ctx context.Context, u *models.User) error
GetUserByEmail(ctx context.Context, email string) (*models.User, error)
GetUserByID(ctx context.Context, id string) (*models.User, error)
UpdatePreferences(ctx context.Context, userID string, raw []byte) error
CreateRefreshToken(ctx context.Context, t *models.RefreshToken) error
GetRefreshToken(ctx context.Context, hash string) (*models.RefreshToken, error)
RevokeRefreshToken(ctx context.Context, id string) error
RevokeAllUserTokens(ctx context.Context, userID string) error
}
+2 -2
View File
@@ -18,11 +18,11 @@ type TokenPair struct {
}
type Service struct {
store *Store
store Repository
tm *TokenManager
}
func NewService(store *Store, tm *TokenManager) *Service {
func NewService(store Repository, tm *TokenManager) *Service {
return &Service{store: store, tm: tm}
}
+111
View File
@@ -0,0 +1,111 @@
package auth
import (
"context"
"database/sql"
"encoding/json"
"errors"
"time"
"shared-album/internal/models"
"shared-album/internal/sqliteutil"
)
// SQLiteStore is the SQLite implementation of Repository.
type SQLiteStore struct {
db *sql.DB
}
func NewSQLiteStore(db *sql.DB) *SQLiteStore { return &SQLiteStore{db: db} }
func (s *SQLiteStore) CreateUser(ctx context.Context, u *models.User) error {
_, err := s.db.ExecContext(ctx,
`INSERT INTO users (id, email, password_hash, name, avatar_url)
VALUES ($1, $2, $3, $4, $5)`,
u.ID, u.Email, u.PasswordHash, u.Name, nullIfEmpty(u.AvatarURL))
return err
}
func (s *SQLiteStore) GetUserByEmail(ctx context.Context, email string) (*models.User, error) {
return s.scanUser(s.db.QueryRowContext(ctx,
`SELECT id, email, password_hash, name, COALESCE(avatar_url, ''),
COALESCE(preferences, '{}'), created_at
FROM users WHERE lower(email) = lower($1)`, email))
}
func (s *SQLiteStore) GetUserByID(ctx context.Context, id string) (*models.User, error) {
return s.scanUser(s.db.QueryRowContext(ctx,
`SELECT id, email, password_hash, name, COALESCE(avatar_url, ''),
COALESCE(preferences, '{}'), created_at
FROM users WHERE id = $1`, id))
}
func (s *SQLiteStore) scanUser(row *sql.Row) (*models.User, error) {
var u models.User
var prefs, createdAt string
if err := row.Scan(&u.ID, &u.Email, &u.PasswordHash, &u.Name, &u.AvatarURL, &prefs, &createdAt); err != nil {
if errors.Is(err, sql.ErrNoRows) {
return nil, ErrNotFound
}
return nil, err
}
u.CreatedAt = sqliteutil.ParseTime(createdAt)
if prefs != "" && prefs != "null" {
_ = json.Unmarshal([]byte(prefs), &u.Preferences)
}
return &u, nil
}
func (s *SQLiteStore) UpdatePreferences(ctx context.Context, userID string, raw []byte) error {
res, err := s.db.ExecContext(ctx,
`UPDATE users SET preferences = $2 WHERE id = $1`, userID, string(raw))
if err != nil {
return err
}
if n, _ := res.RowsAffected(); n == 0 {
return ErrNotFound
}
return nil
}
func (s *SQLiteStore) CreateRefreshToken(ctx context.Context, t *models.RefreshToken) error {
_, err := s.db.ExecContext(ctx,
`INSERT INTO refresh_tokens (id, user_id, token_hash, expires_at)
VALUES ($1, $2, $3, $4)`,
t.ID, t.UserID, t.TokenHash, sqliteutil.FormatTime(t.ExpiresAt))
return err
}
func (s *SQLiteStore) GetRefreshToken(ctx context.Context, hash string) (*models.RefreshToken, error) {
var t models.RefreshToken
var expiresAt, createdAt string
var revokedAt sql.NullString
err := s.db.QueryRowContext(ctx,
`SELECT id, user_id, token_hash, expires_at, revoked_at, created_at
FROM refresh_tokens WHERE token_hash = $1`, hash).
Scan(&t.ID, &t.UserID, &t.TokenHash, &expiresAt, &revokedAt, &createdAt)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return nil, ErrNotFound
}
return nil, err
}
t.ExpiresAt = sqliteutil.ParseTime(expiresAt)
t.CreatedAt = sqliteutil.ParseTime(createdAt)
t.RevokedAt = sqliteutil.ParseNullTime(revokedAt)
return &t, nil
}
func (s *SQLiteStore) RevokeRefreshToken(ctx context.Context, id string) error {
_, err := s.db.ExecContext(ctx,
`UPDATE refresh_tokens SET revoked_at = $2 WHERE id = $1 AND revoked_at IS NULL`,
id, sqliteutil.FormatTime(time.Now()))
return err
}
func (s *SQLiteStore) RevokeAllUserTokens(ctx context.Context, userID string) error {
_, err := s.db.ExecContext(ctx,
`UPDATE refresh_tokens SET revoked_at = $2 WHERE user_id = $1 AND revoked_at IS NULL`,
userID, sqliteutil.FormatTime(time.Now()))
return err
}
+72 -11
View File
@@ -28,12 +28,9 @@ type S3 struct {
SecretKey string
Bucket string
UsePathStyle bool
PublicURL string
PresignTTL time.Duration
}
type Auth struct {
JWTSecret string
AccessTTL time.Duration
RefreshTTL time.Duration
}
@@ -48,12 +45,33 @@ type SSE struct {
ClientBuffer int
}
type Worker struct {
Interval time.Duration
Batch int
PendingTTL time.Duration
}
type Storage struct {
Backend string
FSRoot string
URLSecret string
URLTTL time.Duration
}
type Database struct {
Backend string
SQLitePath string
}
type Config struct {
Postgres Postgres
S3 S3
Storage Storage
Database Database
Auth Auth
Server Server
SSE SSE
Worker Worker
}
func Load() (*Config, error) {
@@ -72,9 +90,9 @@ func Load() (*Config, error) {
return nil, fmt.Errorf("invalid JWT_REFRESH_TTL: %w", err)
}
presignTTL, err := time.ParseDuration(getEnv("S3_PRESIGN_TTL", "1h"))
urlTTL, err := time.ParseDuration(getEnv("OBJECT_URL_TTL", "1h"))
if err != nil {
return nil, fmt.Errorf("invalid S3_PRESIGN_TTL: %w", err)
return nil, fmt.Errorf("invalid OBJECT_URL_TTL: %w", err)
}
heartbeat, err := time.ParseDuration(getEnv("SSE_HEARTBEAT_INTERVAL", "25s"))
@@ -92,6 +110,21 @@ func Load() (*Config, error) {
return nil, fmt.Errorf("invalid S3_USE_PATH_STYLE: %w", err)
}
workerInterval, err := time.ParseDuration(getEnv("WORKER_INTERVAL", "5s"))
if err != nil {
return nil, fmt.Errorf("invalid WORKER_INTERVAL: %w", err)
}
workerBatch, err := strconv.Atoi(getEnv("WORKER_BATCH", "10"))
if err != nil {
return nil, fmt.Errorf("invalid WORKER_BATCH: %w", err)
}
workerPendingTTL, err := time.ParseDuration(getEnv("WORKER_PENDING_TTL", "10m"))
if err != nil {
return nil, fmt.Errorf("invalid WORKER_PENDING_TTL: %w", err)
}
cfg := &Config{
Postgres: Postgres{
Host: getEnv("PG_HOST", "pg.localnet"),
@@ -108,11 +141,18 @@ func Load() (*Config, error) {
SecretKey: getEnv("S3_SECRET_KEY", ""),
Bucket: getEnv("S3_BUCKET", "albums"),
UsePathStyle: pathStyle,
PublicURL: getEnv("S3_PUBLIC_URL", ""),
PresignTTL: presignTTL,
},
Storage: Storage{
Backend: getEnv("STORAGE_BACKEND", "fs"),
FSRoot: getEnv("FS_ROOT", "./data"),
URLSecret: getEnv("URL_SECRET", ""),
URLTTL: urlTTL,
},
Database: Database{
Backend: getEnv("DB_BACKEND", "sqlite"),
SQLitePath: getEnv("SQLITE_PATH", "./galo.db"),
},
Auth: Auth{
JWTSecret: getEnv("JWT_SECRET", ""),
AccessTTL: accessTTL,
RefreshTTL: refreshTTL,
},
@@ -124,14 +164,35 @@ func Load() (*Config, error) {
HeartbeatInterval: heartbeat,
ClientBuffer: bufSize,
},
Worker: Worker{
Interval: workerInterval,
Batch: workerBatch,
PendingTTL: workerPendingTTL,
},
}
if cfg.Auth.JWTSecret == "" {
return nil, fmt.Errorf("JWT_SECRET is required")
}
switch cfg.Database.Backend {
case "postgres":
if cfg.Postgres.Password == "" {
return nil, fmt.Errorf("PG_PASSWORD is required")
}
case "sqlite":
if cfg.Database.SQLitePath == "" {
return nil, fmt.Errorf("SQLITE_PATH is required when DB_BACKEND=sqlite")
}
default:
return nil, fmt.Errorf("invalid DB_BACKEND %q (want postgres or sqlite)", cfg.Database.Backend)
}
switch cfg.Storage.Backend {
case "s3":
case "fs":
if cfg.Storage.FSRoot == "" {
return nil, fmt.Errorf("FS_ROOT is required when STORAGE_BACKEND=fs")
}
default:
return nil, fmt.Errorf("invalid STORAGE_BACKEND %q (want s3 or fs)", cfg.Storage.Backend)
}
return cfg, nil
}
+134
View File
@@ -0,0 +1,134 @@
package config
import (
"testing"
"time"
)
func setRequiredEnv(t *testing.T) {
t.Helper()
t.Setenv("JWT_SECRET", "test-secret")
t.Setenv("PG_PASSWORD", "test-pass")
}
func TestLoadStorageBackend(t *testing.T) {
t.Run("defaults to fs", func(t *testing.T) {
setRequiredEnv(t)
t.Setenv("STORAGE_BACKEND", "")
cfg, err := Load()
if err != nil {
t.Fatalf("load: %v", err)
}
if cfg.Storage.Backend != "fs" || cfg.Storage.FSRoot != "./data" {
t.Fatalf("storage = %+v", cfg.Storage)
}
})
t.Run("fs with root", func(t *testing.T) {
setRequiredEnv(t)
t.Setenv("STORAGE_BACKEND", "fs")
t.Setenv("FS_ROOT", "/var/lib/galo/objects")
cfg, err := Load()
if err != nil {
t.Fatalf("load: %v", err)
}
if cfg.Storage.Backend != "fs" || cfg.Storage.FSRoot != "/var/lib/galo/objects" {
t.Fatalf("storage = %+v", cfg.Storage)
}
})
t.Run("fs empty root falls back to default", func(t *testing.T) {
setRequiredEnv(t)
t.Setenv("STORAGE_BACKEND", "fs")
t.Setenv("FS_ROOT", "")
t.Setenv("URL_SECRET", "")
cfg, err := Load()
if err != nil {
t.Fatalf("load: %v", err)
}
if cfg.Storage.FSRoot != "./data" {
t.Fatalf("FSRoot = %q, want ./data", cfg.Storage.FSRoot)
}
})
t.Run("invalid backend", func(t *testing.T) {
setRequiredEnv(t)
t.Setenv("STORAGE_BACKEND", "gcs")
if _, err := Load(); err == nil {
t.Fatal("expected error for invalid backend")
}
})
t.Run("object url ttl", func(t *testing.T) {
setRequiredEnv(t)
t.Setenv("STORAGE_BACKEND", "")
t.Setenv("OBJECT_URL_TTL", "30m")
cfg, err := Load()
if err != nil {
t.Fatalf("load: %v", err)
}
if cfg.Storage.URLTTL != 30*time.Minute {
t.Fatalf("URLTTL = %v, want 30m", cfg.Storage.URLTTL)
}
})
}
func TestLoadDatabaseBackend(t *testing.T) {
t.Run("defaults to sqlite", func(t *testing.T) {
setRequiredEnv(t)
t.Setenv("DB_BACKEND", "")
cfg, err := Load()
if err != nil {
t.Fatalf("load: %v", err)
}
if cfg.Database.Backend != "sqlite" || cfg.Database.SQLitePath != "./galo.db" {
t.Fatalf("database = %+v", cfg.Database)
}
})
t.Run("postgres requires password", func(t *testing.T) {
t.Setenv("JWT_SECRET", "s")
t.Setenv("PG_PASSWORD", "")
t.Setenv("DB_BACKEND", "postgres")
if _, err := Load(); err == nil {
t.Fatal("expected error for postgres without PG_PASSWORD")
}
})
t.Run("sqlite with path", func(t *testing.T) {
t.Setenv("JWT_SECRET", "s")
t.Setenv("PG_PASSWORD", "")
t.Setenv("DB_BACKEND", "sqlite")
t.Setenv("SQLITE_PATH", "/data/galo.db")
cfg, err := Load()
if err != nil {
t.Fatalf("load: %v", err)
}
if cfg.Database.Backend != "sqlite" || cfg.Database.SQLitePath != "/data/galo.db" {
t.Fatalf("database = %+v", cfg.Database)
}
})
t.Run("sqlite empty path falls back to default", func(t *testing.T) {
t.Setenv("JWT_SECRET", "s")
t.Setenv("PG_PASSWORD", "")
t.Setenv("DB_BACKEND", "sqlite")
t.Setenv("SQLITE_PATH", "")
cfg, err := Load()
if err != nil {
t.Fatalf("load: %v", err)
}
if cfg.Database.SQLitePath != "./galo.db" {
t.Fatalf("SQLitePath = %q, want ./galo.db", cfg.Database.SQLitePath)
}
})
t.Run("invalid backend", func(t *testing.T) {
t.Setenv("JWT_SECRET", "s")
t.Setenv("PG_PASSWORD", "p")
t.Setenv("DB_BACKEND", "mysql")
if _, err := Load(); err == nil {
t.Fatal("expected error for invalid backend")
}
})
}
+31
View File
@@ -8,8 +8,10 @@ import (
"database/sql"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database/pgx/v5"
sqlitemig "github.com/golang-migrate/migrate/v4/database/sqlite"
"github.com/golang-migrate/migrate/v4/source/iofs"
_ "github.com/jackc/pgx/v5/stdlib"
_ "modernc.org/sqlite"
)
// Migrate applies all up migrations from the given fs (root contains *.sql files).
@@ -39,3 +41,32 @@ func Migrate(dsn string, migrations fs.FS) error {
}
return nil
}
// MigrateSQLite applies all up migrations from the given fs to a SQLite database
// at path (creating it if needed).
func MigrateSQLite(path string, migrations fs.FS) error {
source, err := iofs.New(migrations, ".")
if err != nil {
return fmt.Errorf("migrations source: %w", err)
}
sqlDB, err := sql.Open("sqlite", path)
if err != nil {
return fmt.Errorf("open sql db: %w", err)
}
defer sqlDB.Close()
driver, err := sqlitemig.WithInstance(sqlDB, &sqlitemig.Config{})
if err != nil {
return fmt.Errorf("migrate driver: %w", err)
}
m, err := migrate.NewWithInstance("iofs", source, "sqlite", driver)
if err != nil {
return fmt.Errorf("migrate instance: %w", err)
}
if err := m.Up(); err != nil && !errors.Is(err, migrate.ErrNoChange) {
return fmt.Errorf("migrate up: %w", err)
}
return nil
}
@@ -0,0 +1,37 @@
package db
import (
"io/fs"
"path/filepath"
"testing"
assets "shared-album"
)
func TestMigrateSQLite(t *testing.T) {
path := filepath.Join(t.TempDir(), "meta.db")
migFS, err := fs.Sub(assets.Migrations, "migrations/sqlite")
if err != nil {
t.Fatalf("migrations fs: %v", err)
}
if err := MigrateSQLite(path, migFS); err != nil {
t.Fatalf("migrate: %v", err)
}
if err := MigrateSQLite(path, migFS); err != nil {
t.Fatalf("migrate again: %v", err)
}
db, err := OpenSQLite(path)
if err != nil {
t.Fatalf("open: %v", err)
}
defer db.Close()
var n int
if err := db.QueryRow(`SELECT count(*) FROM sqlite_master WHERE type='table' AND name='objects'`).Scan(&n); err != nil {
t.Fatalf("query: %v", err)
}
if n != 1 {
t.Fatalf("objects table missing after migration")
}
}
+31
View File
@@ -0,0 +1,31 @@
package db
import (
"database/sql"
"fmt"
"strings"
_ "modernc.org/sqlite"
)
// OpenSQLite opens (creating if needed) a SQLite database tuned for a single
// writer with concurrent readers: WAL journalling, a busy timeout so writers
// wait instead of failing, and enforced foreign keys.
func OpenSQLite(path string) (*sql.DB, error) {
if strings.TrimSpace(path) == "" {
return nil, fmt.Errorf("sqlite path is required")
}
dsn := fmt.Sprintf(
"file:%s?_pragma=journal_mode(WAL)&_pragma=busy_timeout(5000)&_pragma=foreign_keys(1)&_pragma=synchronous(NORMAL)&_txlock=immediate",
path,
)
db, err := sql.Open("sqlite", dsn)
if err != nil {
return nil, fmt.Errorf("open sqlite: %w", err)
}
if err := db.Ping(); err != nil {
_ = db.Close()
return nil, fmt.Errorf("ping sqlite: %w", err)
}
return db, nil
}
+45
View File
@@ -0,0 +1,45 @@
package db
import (
"context"
"path/filepath"
"strings"
"testing"
)
func TestOpenSQLite(t *testing.T) {
ctx := context.Background()
db, err := OpenSQLite(filepath.Join(t.TempDir(), "test.db"))
if err != nil {
t.Fatalf("open: %v", err)
}
defer db.Close()
if err := db.PingContext(ctx); err != nil {
t.Fatalf("ping: %v", err)
}
var mode string
if err := db.QueryRowContext(ctx, "PRAGMA journal_mode").Scan(&mode); err != nil {
t.Fatalf("journal_mode: %v", err)
}
if !strings.EqualFold(mode, "wal") {
t.Fatalf("journal_mode = %q, want wal", mode)
}
var fk int
if err := db.QueryRowContext(ctx, "PRAGMA foreign_keys").Scan(&fk); err != nil {
t.Fatalf("foreign_keys: %v", err)
}
if fk != 1 {
t.Fatalf("foreign_keys = %d, want 1", fk)
}
var busy int
if err := db.QueryRowContext(ctx, "PRAGMA busy_timeout").Scan(&busy); err != nil {
t.Fatalf("busy_timeout: %v", err)
}
if busy != 5000 {
t.Fatalf("busy_timeout = %d, want 5000", busy)
}
}
+198
View File
@@ -0,0 +1,198 @@
package e2e
import (
"bytes"
"context"
"database/sql"
"image"
"image/color"
"image/png"
"io/fs"
"path/filepath"
"testing"
"time"
assets "shared-album"
"shared-album/internal/albums"
"shared-album/internal/auth"
"shared-album/internal/db"
"shared-album/internal/photos"
"shared-album/internal/settings"
"shared-album/internal/sse"
"shared-album/internal/storage"
"shared-album/internal/store"
)
func makePNG(t *testing.T) []byte {
t.Helper()
img := image.NewRGBA(image.Rect(0, 0, 4, 4))
for x := 0; x < 4; x++ {
for y := 0; y < 4; y++ {
img.Set(x, y, color.RGBA{R: uint8(x * 40), G: uint8(y * 40), B: 128, A: 255})
}
}
var buf bytes.Buffer
if err := png.Encode(&buf, img); err != nil {
t.Fatalf("encode png: %v", err)
}
return buf.Bytes()
}
type stack struct {
db *sql.DB
photos *photos.Service
albums *albums.Service
auth *auth.Service
}
// openStack opens a migrated SQLite database at path and builds the services.
func openStack(t *testing.T, path, blobRoot string) (*stack, func()) {
t.Helper()
migFS, err := fs.Sub(assets.Migrations, "migrations/sqlite")
if err != nil {
t.Fatalf("migrations fs: %v", err)
}
if err := db.MigrateSQLite(path, migFS); err != nil {
t.Fatalf("migrate: %v", err)
}
database, err := db.OpenSQLite(path)
if err != nil {
t.Fatalf("open: %v", err)
}
repos := store.NewSQLite(database)
hub := sse.NewHub(1)
signer, err := storage.NewSigner("http://app.test", "secret", time.Hour)
if err != nil {
t.Fatalf("signer: %v", err)
}
blob, err := storage.NewFS(blobRoot, signer)
if err != nil {
t.Fatalf("fs: %v", err)
}
authSvc := auth.NewService(repos.Auth, auth.NewTokenManager("secret", 15*time.Minute, time.Hour))
albumsSvc := albums.NewService(repos.Albums, repos.Members, hub, blob)
photosSvc := photos.NewService(repos.Photos, albumsSvc, blob, hub)
return &stack{db: database, photos: photosSvc, albums: albumsSvc, auth: authSvc}, func() { database.Close() }
}
func TestSQLiteEndToEndWithRestart(t *testing.T) {
ctx := context.Background()
path := filepath.Join(t.TempDir(), "galo.db")
blobs := t.TempDir()
st, closeDB := openStack(t, path, blobs)
worker := photos.NewWorker(st.photos, 20*time.Millisecond, 10, time.Minute)
wctx, cancel := context.WithCancel(ctx)
go worker.Run(wctx)
user, _, err := st.auth.Register(ctx, "e2e@test.local", "password123", "E2E")
if err != nil {
t.Fatalf("register: %v", err)
}
album, err := st.albums.Create(ctx, user.ID, albums.CreateInput{Title: "Trip"})
if err != nil {
t.Fatalf("create album: %v", err)
}
p, err := st.photos.Upload(ctx, photos.UploadInput{AlbumID: album.ID, UserID: user.ID, Filename: "a.png", Data: makePNG(t)})
if err != nil {
t.Fatalf("upload: %v", err)
}
waitFor(t, "variants of first photo", func() bool {
got, err := st.photos.Get(ctx, p.ID, user.ID)
return err == nil && got.ThumbURL != "" && got.PreviewURL != ""
})
if err := st.photos.PermanentDelete(ctx, p.ID, user.ID); err != nil {
t.Fatalf("permanent delete: %v", err)
}
waitFor(t, "garbage collection", func() bool {
var n int
return st.db.QueryRowContext(ctx, `SELECT count(*) FROM objects`).Scan(&n) == nil && n == 0
})
// Stop the worker, then queue another upload whose variant job must survive
// the restart and be processed by a fresh worker over the same file.
cancel()
time.Sleep(50 * time.Millisecond)
p2, err := st.photos.Upload(ctx, photos.UploadInput{AlbumID: album.ID, UserID: user.ID, Filename: "b.png", Data: makePNG(t)})
if err != nil {
t.Fatalf("upload p2: %v", err)
}
got2, _ := st.photos.Get(ctx, p2.ID, user.ID)
if got2.PreviewURL != "" {
t.Fatalf("variants generated before restart")
}
closeDB()
st2, close2 := openStack(t, path, blobs)
defer close2()
worker2 := photos.NewWorker(st2.photos, 20*time.Millisecond, 10, time.Minute)
wctx2, cancel2 := context.WithCancel(ctx)
defer cancel2()
go worker2.Run(wctx2)
waitFor(t, "variants of second photo after restart", func() bool {
got, err := st2.photos.Get(ctx, p2.ID, user.ID)
return err == nil && got.ThumbURL != "" && got.PreviewURL != ""
})
}
func TestJWTSecretPersistsAcrossRestart(t *testing.T) {
ctx := context.Background()
path := filepath.Join(t.TempDir(), "galo.db")
migFS, err := fs.Sub(assets.Migrations, "migrations/sqlite")
if err != nil {
t.Fatalf("migrations fs: %v", err)
}
open := func() (*sql.DB, store.Repos) {
t.Helper()
if err := db.MigrateSQLite(path, migFS); err != nil {
t.Fatalf("migrate: %v", err)
}
database, err := db.OpenSQLite(path)
if err != nil {
t.Fatalf("open: %v", err)
}
return database, store.NewSQLite(database)
}
db1, repos1 := open()
secret1, err := repos1.Settings.Ensure(ctx, settings.JWTSecretKey, settings.GenerateSecret)
if err != nil || secret1 == "" {
t.Fatalf("ensure secret: %v %q", err, secret1)
}
tm1 := auth.NewTokenManager(secret1, 15*time.Minute, time.Hour)
token, err := tm1.GenerateAccess("user-1", "u@test.local")
if err != nil {
t.Fatalf("generate: %v", err)
}
_ = db1.Close()
db2, repos2 := open()
defer db2.Close()
secret2, err := repos2.Settings.Ensure(ctx, settings.JWTSecretKey, settings.GenerateSecret)
if err != nil {
t.Fatalf("ensure after restart: %v", err)
}
if secret2 != secret1 {
t.Fatalf("secret changed across restart")
}
tm2 := auth.NewTokenManager(secret2, 15*time.Minute, time.Hour)
if _, err := tm2.ParseAccess(token); err != nil {
t.Fatalf("token issued before restart not valid after: %v", err)
}
}
func waitFor(t *testing.T, what string, cond func() bool) {
t.Helper()
deadline := time.Now().Add(10 * time.Second)
for time.Now().Before(deadline) {
if cond() {
return
}
time.Sleep(20 * time.Millisecond)
}
t.Fatalf("timed out waiting for %s", what)
}
+19
View File
@@ -0,0 +1,19 @@
package members
import (
"context"
"shared-album/internal/models"
)
// Repository is the persistence contract for the members domain.
type Repository interface {
Role(ctx context.Context, albumID, userID string) (string, error)
List(ctx context.Context, albumID string) ([]models.AlbumMember, error)
Add(ctx context.Context, albumID, userID, role string, invitedBy *string) error
UpdateRole(ctx context.Context, albumID, userID, role string) error
Remove(ctx context.Context, albumID, userID string) error
CreateInvite(ctx context.Context, inv *models.Invite) error
GetInviteByToken(ctx context.Context, token string) (*models.Invite, error)
IncrementUses(ctx context.Context, id string) error
}
+2 -2
View File
@@ -21,13 +21,13 @@ type Notifier interface {
}
type Service struct {
store *Store
store Repository
albums AlbumReader
hub *sse.Hub
notify Notifier
}
func NewService(store *Store, albums AlbumReader, hub *sse.Hub, notify Notifier) *Service {
func NewService(store Repository, albums AlbumReader, hub *sse.Hub, notify Notifier) *Service {
return &Service{store: store, albums: albums, hub: hub, notify: notify}
}
+133
View File
@@ -0,0 +1,133 @@
package members
import (
"context"
"database/sql"
"errors"
"shared-album/internal/models"
"shared-album/internal/sqliteutil"
)
// SQLiteStore is the SQLite implementation of Repository.
type SQLiteStore struct {
db *sql.DB
}
func NewSQLiteStore(db *sql.DB) *SQLiteStore { return &SQLiteStore{db: db} }
func (s *SQLiteStore) Role(ctx context.Context, albumID, userID string) (string, error) {
var role string
err := s.db.QueryRowContext(ctx,
`SELECT role FROM album_members WHERE album_id = $1 AND user_id = $2`,
albumID, userID).Scan(&role)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return "", nil
}
return "", err
}
return role, nil
}
func (s *SQLiteStore) List(ctx context.Context, albumID string) ([]models.AlbumMember, error) {
rows, err := s.db.QueryContext(ctx,
`SELECT m.album_id, m.user_id, m.role, m.invited_by, m.joined_at,
u.id, u.email, u.name, COALESCE(u.avatar_url, '')
FROM album_members m
JOIN users u ON u.id = m.user_id
WHERE m.album_id = $1
ORDER BY m.joined_at`, albumID)
if err != nil {
return nil, err
}
defer rows.Close()
out := []models.AlbumMember{}
for rows.Next() {
var m models.AlbumMember
var invitedBy sql.NullString
var joinedAt string
if err := rows.Scan(&m.AlbumID, &m.UserID, &m.Role, &invitedBy, &joinedAt,
&m.User.ID, &m.User.Email, &m.User.Name, &m.User.AvatarURL); err != nil {
return nil, err
}
if invitedBy.Valid {
m.InvitedBy = &invitedBy.String
}
m.JoinedAt = sqliteutil.ParseTime(joinedAt)
out = append(out, m)
}
return out, rows.Err()
}
func (s *SQLiteStore) Add(ctx context.Context, albumID, userID, role string, invitedBy *string) error {
_, err := s.db.ExecContext(ctx,
`INSERT INTO album_members (album_id, user_id, role, invited_by)
VALUES ($1, $2, $3, $4)
ON CONFLICT (album_id, user_id) DO UPDATE SET role = excluded.role`,
albumID, userID, role, invitedBy)
return err
}
func (s *SQLiteStore) UpdateRole(ctx context.Context, albumID, userID, role string) error {
res, err := s.db.ExecContext(ctx,
`UPDATE album_members SET role = $3 WHERE album_id = $1 AND user_id = $2`,
albumID, userID, role)
if err != nil {
return err
}
if n, _ := res.RowsAffected(); n == 0 {
return ErrNotFound
}
return nil
}
func (s *SQLiteStore) Remove(ctx context.Context, albumID, userID string) error {
res, err := s.db.ExecContext(ctx,
`DELETE FROM album_members WHERE album_id = $1 AND user_id = $2`, albumID, userID)
if err != nil {
return err
}
if n, _ := res.RowsAffected(); n == 0 {
return ErrNotFound
}
return nil
}
func (s *SQLiteStore) CreateInvite(ctx context.Context, inv *models.Invite) error {
var createdAt string
err := s.db.QueryRowContext(ctx,
`INSERT INTO invites (id, album_id, token, role, created_by, expires_at, max_uses)
VALUES ($1, $2, $3, $4, $5, $6, $7)
RETURNING created_at`,
inv.ID, inv.AlbumID, inv.Token, inv.Role, inv.CreatedBy, sqliteutil.FormatTime(inv.ExpiresAt), inv.MaxUses,
).Scan(&createdAt)
if err != nil {
return err
}
inv.CreatedAt = sqliteutil.ParseTime(createdAt)
return nil
}
func (s *SQLiteStore) GetInviteByToken(ctx context.Context, token string) (*models.Invite, error) {
var inv models.Invite
var expiresAt, createdAt string
err := s.db.QueryRowContext(ctx,
`SELECT id, album_id, token, role, created_by, expires_at, max_uses, uses, created_at
FROM invites WHERE token = $1`, token).
Scan(&inv.ID, &inv.AlbumID, &inv.Token, &inv.Role, &inv.CreatedBy, &expiresAt, &inv.MaxUses, &inv.Uses, &createdAt)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return nil, ErrNotFound
}
return nil, err
}
inv.ExpiresAt = sqliteutil.ParseTime(expiresAt)
inv.CreatedAt = sqliteutil.ParseTime(createdAt)
return &inv, nil
}
func (s *SQLiteStore) IncrementUses(ctx context.Context, id string) error {
_, err := s.db.ExecContext(ctx, `UPDATE invites SET uses = uses + 1 WHERE id = $1`, id)
return err
}
@@ -0,0 +1,14 @@
package notifications
import (
"context"
"shared-album/internal/models"
)
// Repository is the persistence contract for the notifications domain.
type Repository interface {
Create(ctx context.Context, n *models.Notification) error
List(ctx context.Context, userID string, limit int) ([]models.Notification, error)
MarkRead(ctx context.Context, id, userID string) error
}
+2 -2
View File
@@ -10,11 +10,11 @@ import (
)
type Service struct {
store *Store
store Repository
hub *sse.Hub
}
func NewService(store *Store, hub *sse.Hub) *Service {
func NewService(store Repository, hub *sse.Hub) *Service {
return &Service{store: store, hub: hub}
}
@@ -0,0 +1,88 @@
package notifications
import (
"context"
"database/sql"
"encoding/json"
"errors"
"time"
"shared-album/internal/models"
"shared-album/internal/sqliteutil"
)
// SQLiteStore is the SQLite implementation of Repository.
type SQLiteStore struct {
db *sql.DB
}
func NewSQLiteStore(db *sql.DB) *SQLiteStore { return &SQLiteStore{db: db} }
func (s *SQLiteStore) Create(ctx context.Context, n *models.Notification) error {
payload, err := json.Marshal(n.PayloadJSON)
if err != nil {
return err
}
var createdAt string
err = s.db.QueryRowContext(ctx,
`INSERT INTO notifications (id, user_id, type, payload_json)
VALUES ($1, $2, $3, $4)
RETURNING created_at`,
n.ID, n.UserID, n.Type, string(payload)).Scan(&createdAt)
if err != nil {
return err
}
n.CreatedAt = sqliteutil.ParseTime(createdAt)
return nil
}
func (s *SQLiteStore) List(ctx context.Context, userID string, limit int) ([]models.Notification, error) {
if limit <= 0 || limit > 200 {
limit = 100
}
rows, err := s.db.QueryContext(ctx,
`SELECT id, user_id, type, payload_json, read_at, created_at
FROM notifications
WHERE user_id = $1
ORDER BY created_at DESC
LIMIT $2`, userID, limit)
if err != nil {
return nil, err
}
defer rows.Close()
out := []models.Notification{}
for rows.Next() {
var n models.Notification
var payload, createdAt string
var readAt sql.NullString
if err := rows.Scan(&n.ID, &n.UserID, &n.Type, &payload, &readAt, &createdAt); err != nil {
return nil, err
}
_ = json.Unmarshal([]byte(payload), &n.PayloadJSON)
n.ReadAt = sqliteutil.ParseNullTime(readAt)
n.CreatedAt = sqliteutil.ParseTime(createdAt)
out = append(out, n)
}
return out, rows.Err()
}
func (s *SQLiteStore) MarkRead(ctx context.Context, id, userID string) error {
res, err := s.db.ExecContext(ctx,
`UPDATE notifications SET read_at = $3 WHERE id = $1 AND user_id = $2 AND read_at IS NULL`,
id, userID, sqliteutil.FormatTime(time.Now()))
if err != nil {
return err
}
if n, _ := res.RowsAffected(); n == 0 {
var exists int
err := s.db.QueryRowContext(ctx,
`SELECT 1 FROM notifications WHERE id = $1 AND user_id = $2`, id, userID).Scan(&exists)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return ErrNotFound
}
return err
}
}
return nil
}
+92
View File
@@ -0,0 +1,92 @@
package photos
import (
"context"
"testing"
"time"
)
func TestCopySharesStorageObjects(t *testing.T) {
pool := testPool(t)
ctx := context.Background()
svc, cl := newTestService(t, pool)
user := createUser(t, ctx, pool)
albumA := createAlbum(t, ctx, pool, user)
albumB := createAlbum(t, ctx, pool, user)
p, err := svc.Upload(ctx, UploadInput{
AlbumID: albumA, UserID: user, Filename: "p.png", MimeType: "image/png", Data: makePNG(t),
})
if err != nil {
t.Fatalf("upload: %v", err)
}
t.Cleanup(func() { cleanupPhoto(t, ctx, pool, cl, p.ID) })
w := NewWorker(svc, time.Second, 10, time.Minute)
w.processVariantJobs(ctx)
srcIDs, err := svc.store.PhotoObjectIDs(ctx, p.ID)
if err != nil || len(srcIDs) != 3 {
t.Fatalf("source objects = %v err=%v", srcIDs, err)
}
clones, err := svc.CopyBatch(ctx, []string{p.ID}, user, albumB)
if err != nil || len(clones) != 1 {
t.Fatalf("copy: %v len=%d", err, len(clones))
}
clone := clones[0]
t.Cleanup(func() { cleanupPhoto(t, ctx, pool, cl, clone.ID) })
cloneIDs, err := svc.store.PhotoObjectIDs(ctx, clone.ID)
if err != nil || len(cloneIDs) != 3 {
t.Fatalf("clone objects = %v err=%v", cloneIDs, err)
}
if !sameSet(srcIDs, cloneIDs) {
t.Fatalf("clone does not share objects: src=%v clone=%v", srcIDs, cloneIDs)
}
for _, id := range cloneIDs {
if n, err := svc.store.ObjectRefCount(ctx, id); err != nil || n != 2 {
t.Fatalf("object %s ref count = %d err=%v, want 2", id, n, err)
}
}
// Move the source into the same album; the storage key and shared objects
// must be untouched.
beforeKey := p.StorageKey
moved, err := svc.MoveBatch(ctx, []string{p.ID}, user, albumB)
if err != nil || moved != 1 {
t.Fatalf("move: %v moved=%d", err, moved)
}
after, err := svc.store.GetByID(ctx, p.ID, "")
if err != nil {
t.Fatalf("get after move: %v", err)
}
if after.AlbumID != albumB {
t.Fatalf("photo not moved: album=%q", after.AlbumID)
}
if after.StorageKey != beforeKey {
t.Fatalf("move changed storage key: %q -> %q", beforeKey, after.StorageKey)
}
for _, id := range cloneIDs {
if n, err := svc.store.ObjectRefCount(ctx, id); err != nil || n != 2 {
t.Fatalf("object %s ref count after move = %d err=%v, want 2", id, n, err)
}
}
}
func sameSet(a, b []string) bool {
if len(a) != len(b) {
return false
}
m := make(map[string]int, len(a))
for _, v := range a {
m[v]++
}
for _, v := range b {
m[v]--
if m[v] < 0 {
return false
}
}
return true
}
+167
View File
@@ -0,0 +1,167 @@
package photos
import (
"bytes"
"context"
"image"
"image/color"
"image/png"
"os"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"shared-album/internal/auth"
"shared-album/internal/config"
"shared-album/internal/models"
"shared-album/internal/roles"
"shared-album/internal/sse"
"shared-album/internal/storage"
)
func testPool(t *testing.T) *pgxpool.Pool {
t.Helper()
dsn := os.Getenv("TEST_DATABASE_URL")
if dsn == "" {
t.Skip("TEST_DATABASE_URL not set")
}
pool, err := pgxpool.New(context.Background(), dsn)
if err != nil {
t.Skipf("cannot connect: %v", err)
}
if err := pool.Ping(context.Background()); err != nil {
t.Skipf("cannot ping: %v", err)
}
t.Cleanup(pool.Close)
return pool
}
func testEnvOr(key, fallback string) string {
if v := os.Getenv(key); v != "" {
return v
}
return fallback
}
type stubAlbums struct{}
func (stubAlbums) Access(ctx context.Context, albumID, userID string) (*models.Album, string, error) {
return &models.Album{ID: albumID}, roles.Owner, nil
}
func createUser(t *testing.T, ctx context.Context, pool *pgxpool.Pool) string {
t.Helper()
id := auth.NewID()
if _, err := pool.Exec(ctx,
`INSERT INTO users (id, email, password_hash, name) VALUES ($1, $2, 'x', 't')`,
id, id+"@test.local"); err != nil {
t.Fatalf("create user: %v", err)
}
t.Cleanup(func() { _, _ = pool.Exec(ctx, `DELETE FROM users WHERE id = $1`, id) })
return id
}
func createAlbum(t *testing.T, ctx context.Context, pool *pgxpool.Pool, ownerID string) string {
t.Helper()
id := auth.NewID()
if _, err := pool.Exec(ctx,
`INSERT INTO albums (id, owner_id, title) VALUES ($1, $2, 't')`, id, ownerID); err != nil {
t.Fatalf("create album: %v", err)
}
t.Cleanup(func() { _, _ = pool.Exec(ctx, `DELETE FROM albums WHERE id = $1`, id) })
return id
}
func createPhoto(t *testing.T, ctx context.Context, pool *pgxpool.Pool, uploaderID string) string {
t.Helper()
id := auth.NewID()
if _, err := pool.Exec(ctx,
`INSERT INTO photos (id, album_id, uploader_id) VALUES ($1, NULL, $2)`, id, uploaderID); err != nil {
t.Fatalf("create photo: %v", err)
}
return id
}
func makePNG(t *testing.T) []byte {
t.Helper()
img := image.NewRGBA(image.Rect(0, 0, 4, 4))
for x := 0; x < 4; x++ {
for y := 0; y < 4; y++ {
img.Set(x, y, color.RGBA{R: uint8(x * 40), G: uint8(y * 40), B: 128, A: 255})
}
}
var buf bytes.Buffer
if err := png.Encode(&buf, img); err != nil {
t.Fatalf("encode png: %v", err)
}
return buf.Bytes()
}
// newTestService builds a service backed by the real dev storage. It skips when
// S3 env is absent so unit runs stay hermetic.
func newTestService(t *testing.T, pool *pgxpool.Pool) (*Service, storage.Backend) {
t.Helper()
endpoint := os.Getenv("S3_ENDPOINT")
if endpoint == "" {
t.Skip("S3_ENDPOINT not set")
}
signer, err := storage.NewSigner("http://app.test", "test-secret", time.Hour)
if err != nil {
t.Skipf("signer init: %v", err)
}
cl, err := storage.NewS3(context.Background(), config.S3{
Endpoint: endpoint,
Region: testEnvOr("S3_REGION", "garage"),
AccessKey: os.Getenv("S3_ACCESS_KEY"),
SecretKey: os.Getenv("S3_SECRET_KEY"),
Bucket: testEnvOr("S3_BUCKET", "galo-data"),
UsePathStyle: testEnvOr("S3_USE_PATH_STYLE", "true") != "false",
}, signer)
if err != nil {
t.Skipf("storage init: %v", err)
}
return NewService(NewStore(pool), stubAlbums{}, cl, sse.NewHub(1)), cl
}
// newTestServiceFS builds a service backed by a temporary local file-system
// storage backend so the full flow can run without external infrastructure.
func newTestServiceFS(t *testing.T, pool *pgxpool.Pool) (*Service, storage.Backend) {
t.Helper()
signer, err := storage.NewSigner("http://app.test", "test-secret", time.Hour)
if err != nil {
t.Fatalf("storage init: %v", err)
}
fs, err := storage.NewFS(t.TempDir(), signer)
if err != nil {
t.Fatalf("storage init: %v", err)
}
return NewService(NewStore(pool), stubAlbums{}, fs, sse.NewHub(1)), fs
}
// cleanupPhoto removes a photo, its objects (rows and blobs) and bindings.
func cleanupPhoto(t *testing.T, ctx context.Context, pool *pgxpool.Pool, cl storage.Backend, photoID string) {
t.Helper()
rows, err := pool.Query(ctx,
`SELECT o.id, o.storage_key FROM photo_objects po JOIN objects o ON o.id = po.object_id
WHERE po.photo_id = $1`, photoID)
if err != nil {
return
}
var ids, keys []string
for rows.Next() {
var id, key string
if rows.Scan(&id, &key) == nil {
ids = append(ids, id)
keys = append(keys, key)
}
}
rows.Close()
_, _ = pool.Exec(ctx, `DELETE FROM photos WHERE id = $1`, photoID)
for _, id := range ids {
_, _ = pool.Exec(ctx, `DELETE FROM objects WHERE id = $1`, id)
}
if cl != nil && len(keys) > 0 {
_ = cl.Delete(ctx, keys...)
}
}
+17
View File
@@ -0,0 +1,17 @@
package photos
import (
"fmt"
"strings"
)
// objectKey builds an opaque, album-free storage key for a physical object.
// The key never encodes album, photo or any other metadata entity; the mapping
// between a key and its photo binding lives only in the metadata layer. The
// role segment is cosmetic (for debugging) and is not authoritative.
func objectKey(objectID, role, ext string) string {
if ext != "" && !strings.HasPrefix(ext, ".") {
ext = "." + ext
}
return fmt.Sprintf("objects/%s/%s%s", objectID, role, ext)
}
+32
View File
@@ -0,0 +1,32 @@
package photos
import (
"strings"
"testing"
)
func TestObjectKeyIsAlbumFree(t *testing.T) {
cases := []struct {
name string
id string
role string
ext string
want string
}{
{"original with dot", "obj1", RoleOriginal, ".jpg", "objects/obj1/original.jpg"},
{"thumb without dot", "obj1", RoleThumb, "webp", "objects/obj1/thumb.webp"},
{"preview", "obj2", RolePreview, ".webp", "objects/obj2/preview.webp"},
{"empty ext", "obj3", RoleOriginal, "", "objects/obj3/original"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got := objectKey(tc.id, tc.role, tc.ext)
if got != tc.want {
t.Fatalf("objectKey = %q, want %q", got, tc.want)
}
if strings.Contains(got, "album") {
t.Fatalf("key must not encode an album entity: %q", got)
}
})
}
}
+388
View File
@@ -0,0 +1,388 @@
package photos
import (
"context"
"errors"
"time"
"github.com/jackc/pgx/v5"
)
// Object lifecycle states. The DB row is the ledger entry that governs the
// lifecycle of the corresponding blob in the storage layer.
const (
ObjectPending = "pending"
ObjectReady = "ready"
ObjectDeleting = "deleting"
)
// Object roles as bound to a photo. The role is a property of the binding.
const (
RoleOriginal = "original"
RoleThumb = "thumb"
RolePreview = "preview"
)
var (
// ErrObjectNotFound is returned when an object row does not exist.
ErrObjectNotFound = errors.New("object not found")
// ErrObjectNotReady is returned when binding an object that is not ready.
ErrObjectNotReady = errors.New("object is not ready")
)
// Object is a physical file in the storage layer, described by metadata.
type Object struct {
ID string
StorageKey string
MimeType string
Size int64
Width int
Height int
Checksum string
State string
CreatedAt time.Time
}
// CreateObject inserts a new object row in the pending state.
func (s *Store) CreateObject(ctx context.Context, o *Object) error {
if o.State == "" {
o.State = ObjectPending
}
return s.pool.QueryRow(ctx,
`INSERT INTO objects (id, storage_key, mime_type, size, width, height, checksum, state)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
RETURNING created_at`,
o.ID, o.StorageKey, o.MimeType, o.Size, o.Width, o.Height, o.Checksum, o.State).Scan(&o.CreatedAt)
}
// SetObjectState updates an object's lifecycle state.
func (s *Store) SetObjectState(ctx context.Context, id, state string) error {
tag, err := s.pool.Exec(ctx, `UPDATE objects SET state = $2 WHERE id = $1`, id, state)
if err != nil {
return err
}
if tag.RowsAffected() == 0 {
return ErrObjectNotFound
}
return nil
}
// ActivateObject marks an object ready after its bytes have been stored.
func (s *Store) ActivateObject(ctx context.Context, id string) error {
return s.SetObjectState(ctx, id, ObjectReady)
}
// MarkObjectDeleting marks an object as pending removal from storage.
func (s *Store) MarkObjectDeleting(ctx context.Context, id string) error {
return s.SetObjectState(ctx, id, ObjectDeleting)
}
// DeleteObjectRow permanently removes an object row (after its blob is gone).
func (s *Store) DeleteObjectRow(ctx context.Context, id string) error {
tag, err := s.pool.Exec(ctx, `DELETE FROM objects WHERE id = $1`, id)
if err != nil {
return err
}
if tag.RowsAffected() == 0 {
return ErrObjectNotFound
}
return nil
}
// BindObject attaches a ready object to a photo under the given role. It locks
// the object row and refuses objects that are not ready, so an object being
// garbage-collected can never be bound.
func (s *Store) BindObject(ctx context.Context, photoID, objectID, role string) error {
tx, err := s.pool.Begin(ctx)
if err != nil {
return err
}
defer func() { _ = tx.Rollback(ctx) }()
var state string
if err := tx.QueryRow(ctx, `SELECT state FROM objects WHERE id = $1 FOR UPDATE`, objectID).Scan(&state); err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return ErrObjectNotFound
}
return err
}
if state != ObjectReady {
return ErrObjectNotReady
}
if _, err := tx.Exec(ctx,
`INSERT INTO photo_objects (photo_id, object_id, role) VALUES ($1, $2, $3)
ON CONFLICT (photo_id, role) DO UPDATE SET object_id = EXCLUDED.object_id`,
photoID, objectID, role); err != nil {
return err
}
return tx.Commit(ctx)
}
// PhotoObjectIDs returns the object ids currently bound to a photo.
func (s *Store) PhotoObjectIDs(ctx context.Context, photoID string) ([]string, error) {
rows, err := s.pool.Query(ctx, `SELECT object_id FROM photo_objects WHERE photo_id = $1`, photoID)
if err != nil {
return nil, err
}
defer rows.Close()
var ids []string
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
return nil, err
}
ids = append(ids, id)
}
return ids, rows.Err()
}
// ObjectRefCount counts how many photo bindings reference an object.
func (s *Store) ObjectRefCount(ctx context.Context, objectID string) (int, error) {
var n int
err := s.pool.QueryRow(ctx, `SELECT count(*) FROM photo_objects WHERE object_id = $1`, objectID).Scan(&n)
return n, err
}
// OriginalObject returns the original object bound to a photo.
func (s *Store) OriginalObject(ctx context.Context, photoID string) (*Object, error) {
var o Object
err := s.pool.QueryRow(ctx,
`SELECT o.id, o.storage_key, o.mime_type, o.size, o.width, o.height, o.checksum, o.state, o.created_at
FROM photo_objects po JOIN objects o ON o.id = po.object_id
WHERE po.photo_id = $1 AND po.role = $2`, photoID, RoleOriginal).
Scan(&o.ID, &o.StorageKey, &o.MimeType, &o.Size, &o.Width, &o.Height, &o.Checksum, &o.State, &o.CreatedAt)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrObjectNotFound
}
return nil, err
}
return &o, nil
}
// PhotosByOriginalObject returns the ids of all photos bound to the given
// original object, so generated variants can be shared between copies.
func (s *Store) PhotosByOriginalObject(ctx context.Context, originalObjectID string) ([]string, error) {
rows, err := s.pool.Query(ctx,
`SELECT photo_id FROM photo_objects WHERE object_id = $1 AND role = $2`, originalObjectID, RoleOriginal)
if err != nil {
return nil, err
}
defer rows.Close()
var ids []string
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
return nil, err
}
ids = append(ids, id)
}
return ids, rows.Err()
}
// ClaimGarbage claims collectible objects: ready objects with no bindings and
// objects already marked deleting. Rows are locked with SKIP LOCKED so
// concurrent workers do not contend and a concurrent bind cannot slip in.
func (s *Store) ClaimGarbage(ctx context.Context, limit int) ([]Object, error) {
if limit <= 0 {
limit = 10
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return nil, err
}
defer func() { _ = tx.Rollback(ctx) }()
rows, err := tx.Query(ctx,
`SELECT o.id, o.storage_key FROM objects o
WHERE o.state = ANY($1)
AND NOT EXISTS (SELECT 1 FROM photo_objects po WHERE po.object_id = o.id)
ORDER BY o.created_at
LIMIT $2
FOR UPDATE SKIP LOCKED`, []string{ObjectReady, ObjectDeleting}, limit)
if err != nil {
return nil, err
}
var objs []Object
for rows.Next() {
var o Object
if err := rows.Scan(&o.ID, &o.StorageKey); err != nil {
rows.Close()
return nil, err
}
objs = append(objs, o)
}
rows.Close()
if err := rows.Err(); err != nil {
return nil, err
}
if len(objs) == 0 {
return nil, tx.Commit(ctx)
}
ids := make([]string, len(objs))
for i, o := range objs {
ids[i] = o.ID
}
if _, err := tx.Exec(ctx, `UPDATE objects SET state = $2 WHERE id = ANY($1)`, ids, ObjectDeleting); err != nil {
return nil, err
}
if err := tx.Commit(ctx); err != nil {
return nil, err
}
return objs, nil
}
// ClaimStalePending marks abandoned pending objects (whose upload never
// completed) as deleting and returns them for cleanup.
func (s *Store) ClaimStalePending(ctx context.Context, olderThan time.Duration, limit int) ([]Object, error) {
if limit <= 0 {
limit = 10
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return nil, err
}
defer func() { _ = tx.Rollback(ctx) }()
rows, err := tx.Query(ctx,
`SELECT id, storage_key FROM objects
WHERE state = $1 AND created_at < $2
ORDER BY created_at
LIMIT $3
FOR UPDATE SKIP LOCKED`, ObjectPending, time.Now().Add(-olderThan), limit)
if err != nil {
return nil, err
}
var objs []Object
for rows.Next() {
var o Object
if err := rows.Scan(&o.ID, &o.StorageKey); err != nil {
rows.Close()
return nil, err
}
objs = append(objs, o)
}
rows.Close()
if err := rows.Err(); err != nil {
return nil, err
}
if len(objs) == 0 {
return nil, tx.Commit(ctx)
}
ids := make([]string, len(objs))
for i, o := range objs {
ids[i] = o.ID
}
if _, err := tx.Exec(ctx, `UPDATE objects SET state = $2 WHERE id = ANY($1)`, ids, ObjectDeleting); err != nil {
return nil, err
}
if err := tx.Commit(ctx); err != nil {
return nil, err
}
return objs, nil
}
// ClaimVariantJobs leases due variant jobs and returns their photo ids. The
// lease pushes next_attempt_at forward so the same job is not picked up twice
// while it is being processed.
func (s *Store) ClaimVariantJobs(ctx context.Context, limit int, lease time.Duration) ([]string, error) {
if limit <= 0 {
limit = 10
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return nil, err
}
defer func() { _ = tx.Rollback(ctx) }()
rows, err := tx.Query(ctx,
`SELECT photo_id FROM variant_jobs
WHERE next_attempt_at <= now()
ORDER BY next_attempt_at
LIMIT $1
FOR UPDATE SKIP LOCKED`, limit)
if err != nil {
return nil, err
}
var ids []string
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
rows.Close()
return nil, err
}
ids = append(ids, id)
}
rows.Close()
if err := rows.Err(); err != nil {
return nil, err
}
if len(ids) == 0 {
return nil, tx.Commit(ctx)
}
if _, err := tx.Exec(ctx,
`UPDATE variant_jobs SET next_attempt_at = $2 WHERE photo_id = ANY($1)`,
ids, time.Now().Add(lease)); err != nil {
return nil, err
}
if err := tx.Commit(ctx); err != nil {
return nil, err
}
return ids, nil
}
// CompleteVariantJob removes a finished job.
func (s *Store) CompleteVariantJob(ctx context.Context, photoID string) error {
_, err := s.pool.Exec(ctx, `DELETE FROM variant_jobs WHERE photo_id = $1`, photoID)
return err
}
// FailVariantJob records a failure and schedules a retry with exponential
// backoff. After maxAttempts the job is abandoned so a permanently broken photo
// cannot loop forever.
func (s *Store) FailVariantJob(ctx context.Context, photoID, errMsg string, maxAttempts int) error {
tx, err := s.pool.Begin(ctx)
if err != nil {
return err
}
defer func() { _ = tx.Rollback(ctx) }()
var attempts int
err = tx.QueryRow(ctx,
`UPDATE variant_jobs
SET attempts = attempts + 1,
next_attempt_at = now() + make_interval(secs => least(300, 10 * power(2, attempts))),
last_error = $2
WHERE photo_id = $1
RETURNING attempts`, photoID, errMsg).Scan(&attempts)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return tx.Commit(ctx)
}
return err
}
if maxAttempts > 0 && attempts >= maxAttempts {
if _, err := tx.Exec(ctx, `DELETE FROM variant_jobs WHERE photo_id = $1`, photoID); err != nil {
return err
}
}
return tx.Commit(ctx)
}
// AllStorageKeys returns every key registered in the object journal.
func (s *Store) AllStorageKeys(ctx context.Context) ([]string, error) {
rows, err := s.pool.Query(ctx, `SELECT storage_key FROM objects`)
if err != nil {
return nil, err
}
defer rows.Close()
var keys []string
for rows.Next() {
var k string
if err := rows.Scan(&k); err != nil {
return nil, err
}
keys = append(keys, k)
}
return keys, rows.Err()
}
+187
View File
@@ -0,0 +1,187 @@
package photos
import (
"context"
"errors"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"shared-album/internal/auth"
)
func TestObjectStateTransitions(t *testing.T) {
pool := testPool(t)
ctx := context.Background()
st := NewStore(pool)
objID := auth.NewID()
obj := &Object{ID: objID, StorageKey: "objects/" + objID + "/original.jpg", MimeType: "image/jpeg", Size: 42}
t.Cleanup(func() { _, _ = pool.Exec(ctx, `DELETE FROM objects WHERE id = $1`, objID) })
if err := st.CreateObject(ctx, obj); err != nil {
t.Fatalf("create object: %v", err)
}
if obj.State != ObjectPending {
t.Fatalf("new object state = %q, want %q", obj.State, ObjectPending)
}
if obj.CreatedAt.IsZero() {
t.Fatalf("created_at not populated")
}
if err := st.ActivateObject(ctx, objID); err != nil {
t.Fatalf("activate: %v", err)
}
if got := objectState(t, ctx, pool, objID); got != ObjectReady {
t.Fatalf("state after activate = %q", got)
}
if err := st.MarkObjectDeleting(ctx, objID); err != nil {
t.Fatalf("mark deleting: %v", err)
}
if got := objectState(t, ctx, pool, objID); got != ObjectDeleting {
t.Fatalf("state after mark = %q", got)
}
if err := st.DeleteObjectRow(ctx, objID); err != nil {
t.Fatalf("delete row: %v", err)
}
if err := st.DeleteObjectRow(ctx, objID); !errors.Is(err, ErrObjectNotFound) {
t.Fatalf("second delete = %v, want ErrObjectNotFound", err)
}
}
func TestClaimStalePending(t *testing.T) {
pool := testPool(t)
ctx := context.Background()
st := NewStore(pool)
objID := auth.NewID()
obj := &Object{ID: objID, StorageKey: "objects/" + objID + "/original.jpg"}
t.Cleanup(func() { _, _ = pool.Exec(ctx, `DELETE FROM objects WHERE id = $1`, objID) })
if err := st.CreateObject(ctx, obj); err != nil {
t.Fatalf("create object: %v", err)
}
if _, err := pool.Exec(ctx, `UPDATE objects SET created_at = now() - interval '1 hour' WHERE id = $1`, objID); err != nil {
t.Fatalf("age object: %v", err)
}
objs, err := st.ClaimStalePending(ctx, 10*time.Minute, 100)
if err != nil {
t.Fatalf("claim stale: %v", err)
}
if !containsObject(objs, objID) {
t.Fatalf("stale pending object not claimed: %+v", objs)
}
if got := objectState(t, ctx, pool, objID); got != ObjectDeleting {
t.Fatalf("claimed stale object state = %q, want deleting", got)
}
}
func TestBindObjectRequiresReadyAndShares(t *testing.T) {
pool := testPool(t)
ctx := context.Background()
st := NewStore(pool)
user := createUser(t, ctx, pool)
p1 := createPhoto(t, ctx, pool, user)
p2 := createPhoto(t, ctx, pool, user)
objID := auth.NewID()
obj := &Object{ID: objID, StorageKey: "objects/" + objID + "/original.jpg"}
t.Cleanup(func() {
_, _ = pool.Exec(ctx, `DELETE FROM photo_objects WHERE object_id = $1`, objID)
_, _ = pool.Exec(ctx, `DELETE FROM objects WHERE id = $1`, objID)
})
if err := st.CreateObject(ctx, obj); err != nil {
t.Fatalf("create object: %v", err)
}
if err := st.BindObject(ctx, p1, objID, RoleOriginal); !errors.Is(err, ErrObjectNotReady) {
t.Fatalf("bind pending object = %v, want ErrObjectNotReady", err)
}
if err := st.ActivateObject(ctx, objID); err != nil {
t.Fatalf("activate: %v", err)
}
if err := st.BindObject(ctx, p1, objID, RoleOriginal); err != nil {
t.Fatalf("bind p1: %v", err)
}
if err := st.BindObject(ctx, p2, objID, RoleOriginal); err != nil {
t.Fatalf("bind p2: %v", err)
}
if n, err := st.ObjectRefCount(ctx, objID); err != nil || n != 2 {
t.Fatalf("ref count = %d err=%v, want 2", n, err)
}
siblings, err := st.PhotosByOriginalObject(ctx, objID)
if err != nil || len(siblings) != 2 {
t.Fatalf("siblings = %v err=%v, want 2", siblings, err)
}
}
func TestClaimGarbageRespectsReferences(t *testing.T) {
pool := testPool(t)
ctx := context.Background()
st := NewStore(pool)
user := createUser(t, ctx, pool)
photo := createPhoto(t, ctx, pool, user)
boundID := auth.NewID()
freeID := auth.NewID()
t.Cleanup(func() {
_, _ = pool.Exec(ctx, `DELETE FROM photo_objects WHERE object_id IN ($1, $2)`, boundID, freeID)
_, _ = pool.Exec(ctx, `DELETE FROM objects WHERE id IN ($1, $2)`, boundID, freeID)
})
for _, id := range []string{boundID, freeID} {
if err := st.CreateObject(ctx, &Object{ID: id, StorageKey: "objects/" + id + "/original.jpg"}); err != nil {
t.Fatalf("create object: %v", err)
}
if err := st.ActivateObject(ctx, id); err != nil {
t.Fatalf("activate: %v", err)
}
}
if err := st.BindObject(ctx, photo, boundID, RoleOriginal); err != nil {
t.Fatalf("bind: %v", err)
}
claimed, err := st.ClaimGarbage(ctx, 100)
if err != nil {
t.Fatalf("claim garbage: %v", err)
}
if !containsObject(claimed, freeID) {
t.Fatalf("unreferenced object not claimed: %+v", claimed)
}
if containsObject(claimed, boundID) {
t.Fatalf("referenced object was claimed for deletion")
}
// Removing the last reference makes both collectible.
if _, err := pool.Exec(ctx, `DELETE FROM photos WHERE id = $1`, photo); err != nil {
t.Fatalf("delete photo: %v", err)
}
claimed, err = st.ClaimGarbage(ctx, 100)
if err != nil {
t.Fatalf("claim garbage 2: %v", err)
}
if !containsObject(claimed, boundID) {
t.Fatalf("object with no remaining references not claimed: %+v", claimed)
}
}
func objectState(t *testing.T, ctx context.Context, pool *pgxpool.Pool, id string) string {
t.Helper()
var state string
if err := pool.QueryRow(ctx, `SELECT state FROM objects WHERE id = $1`, id).Scan(&state); err != nil {
t.Fatalf("read state: %v", err)
}
return state
}
func containsObject(objs []Object, id string) bool {
for _, o := range objs {
if o.ID == id {
return true
}
}
return false
}
+8
View File
@@ -2,6 +2,8 @@ package photos
import (
"bytes"
"crypto/sha256"
"encoding/hex"
"fmt"
"image"
"strconv"
@@ -138,3 +140,9 @@ func buildVariants(data []byte) (thumb, preview []byte, err error) {
}
return thumbBuf.Bytes(), previewBuf.Bytes(), nil
}
// checksum returns the hex-encoded SHA-256 of the given bytes.
func checksum(data []byte) string {
sum := sha256.Sum256(data)
return hex.EncodeToString(sum[:])
}
+52
View File
@@ -0,0 +1,52 @@
package photos
import (
"context"
"time"
"shared-album/internal/models"
)
// Repository is the persistence contract for the photos domain, including the
// object ledger, variant queue and garbage collection.
type Repository interface {
CreateWithOriginal(ctx context.Context, p *models.Photo, obj *Object) error
DeleteUpload(ctx context.Context, photoID, objectID string) error
GetByID(ctx context.Context, id, viewerID string) (*models.Photo, error)
List(ctx context.Context, in ListParams) ([]models.Photo, error)
SetDeleted(ctx context.Context, id string, deleted bool) error
UpdateAlbum(ctx context.Context, id, albumID string) error
Clone(ctx context.Context, src *models.Photo, albumID, uploaderID, id string) error
HardDelete(ctx context.Context, id string) error
ListTrash(ctx context.Context, userID string) ([]models.Photo, error)
TrashCount(ctx context.Context, userID string) (int, error)
RestoreToAlbum(ctx context.Context, id, albumID string) error
AlbumExists(ctx context.Context, albumID string) (bool, error)
ListTags(ctx context.Context, userID, q string) ([]models.Tag, error)
EnsureTag(ctx context.Context, userID, name string) (*models.Tag, error)
AttachTag(ctx context.Context, photoID, tagID string) error
DetachTag(ctx context.Context, photoID, tagID string) error
TagOwnedBy(ctx context.Context, tagID, userID string) (bool, error)
SetCoverIfEmpty(ctx context.Context, albumID, photoID string)
UpdateFilename(ctx context.Context, id, filename string) error
BackfillEXIF(ctx context.Context, fetch func(context.Context, string) ([]byte, error)) (int, error)
CreateObject(ctx context.Context, o *Object) error
SetObjectState(ctx context.Context, id, state string) error
ActivateObject(ctx context.Context, id string) error
MarkObjectDeleting(ctx context.Context, id string) error
DeleteObjectRow(ctx context.Context, id string) error
BindObject(ctx context.Context, photoID, objectID, role string) error
PhotoObjectIDs(ctx context.Context, photoID string) ([]string, error)
ObjectRefCount(ctx context.Context, objectID string) (int, error)
OriginalObject(ctx context.Context, photoID string) (*Object, error)
PhotosByOriginalObject(ctx context.Context, originalObjectID string) ([]string, error)
ClaimGarbage(ctx context.Context, limit int) ([]Object, error)
ClaimStalePending(ctx context.Context, olderThan time.Duration, limit int) ([]Object, error)
ClaimVariantJobs(ctx context.Context, limit int, lease time.Duration) ([]string, error)
CompleteVariantJob(ctx context.Context, photoID string) error
FailVariantJob(ctx context.Context, photoID, errMsg string, maxAttempts int) error
// AllStorageKeys returns every key registered in the object journal.
AllStorageKeys(ctx context.Context) ([]string, error)
}
+43 -56
View File
@@ -4,7 +4,6 @@ import (
"bytes"
"context"
"errors"
"fmt"
"net/http"
"path/filepath"
"strings"
@@ -23,13 +22,13 @@ type AlbumAccess interface {
}
type Service struct {
store *Store
store Repository
albums AlbumAccess
storage *storage.Client
storage storage.Backend
hub *sse.Hub
}
func NewService(store *Store, albums AlbumAccess, storage *storage.Client, hub *sse.Hub) *Service {
func NewService(store Repository, albums AlbumAccess, storage storage.Backend, hub *sse.Hub) *Service {
return &Service{store: store, albums: albums, storage: storage, hub: hub}
}
@@ -64,22 +63,20 @@ func (s *Service) Upload(ctx context.Context, in UploadInput) (*models.Photo, er
}
meta := extractEXIF(in.Data)
id := auth.NewID()
ext := strings.ToLower(filepath.Ext(in.Filename))
if ext == "" {
ext = ".bin"
}
storageKey := fmt.Sprintf("albums/%s/original/%s%s", in.AlbumID, id, ext)
if err := s.storage.Put(ctx, storageKey, bytes.NewReader(in.Data), int64(len(in.Data)), mime); err != nil {
return nil, err
}
photoID := auth.NewID()
objectID := auth.NewID()
key := objectKey(objectID, RoleOriginal, ext)
p := &models.Photo{
ID: id,
ID: photoID,
AlbumID: in.AlbumID,
UploaderID: in.UserID,
StorageKey: storageKey,
StorageKey: key,
OriginalFilename: in.Filename,
Size: int64(len(in.Data)),
Width: width,
@@ -90,12 +87,28 @@ func (s *Service) Upload(ctx context.Context, in UploadInput) (*models.Photo, er
Longitude: meta.Longitude,
EXIF: meta.Attributes,
}
if err := s.store.Create(ctx, p); err != nil {
return nil, err
obj := &Object{
ID: objectID,
StorageKey: key,
MimeType: mime,
Size: int64(len(in.Data)),
Width: width,
Height: height,
Checksum: checksum(in.Data),
}
data := in.Data
go s.processVariants(p.ID, in.AlbumID, data)
// The metadata layer is the ledger of record: the row is written first,
// then the bytes are stored, then the object is activated.
if err := s.store.CreateWithOriginal(ctx, p, obj); err != nil {
return nil, err
}
if err := s.storage.Put(ctx, key, bytes.NewReader(in.Data), int64(len(in.Data)), mime); err != nil {
_ = s.store.DeleteUpload(ctx, photoID, objectID)
return nil, err
}
if err := s.store.ActivateObject(ctx, objectID); err != nil {
return nil, err
}
if err := s.attachURLs(ctx, p); err != nil {
return nil, err
@@ -103,36 +116,9 @@ func (s *Service) Upload(ctx context.Context, in UploadInput) (*models.Photo, er
return p, nil
}
func (s *Service) processVariants(photoID, albumID string, data []byte) {
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
thumb, preview, err := buildVariants(data)
if err != nil {
return
}
thumbKey := fmt.Sprintf("albums/%s/thumb/%s.webp", albumID, photoID)
previewKey := fmt.Sprintf("albums/%s/preview/%s.webp", albumID, photoID)
if err := s.storage.Put(ctx, thumbKey, bytes.NewReader(thumb), int64(len(thumb)), "image/webp"); err != nil {
return
}
if err := s.storage.Put(ctx, previewKey, bytes.NewReader(preview), int64(len(preview)), "image/webp"); err != nil {
return
}
if err := s.store.SetVariants(ctx, photoID, thumbKey, previewKey); err != nil {
return
}
s.store.SetCoverIfEmpty(ctx, albumID, photoID)
if p, err := s.store.GetByID(ctx, photoID, ""); err == nil {
if err := s.attachURLs(ctx, p); err == nil {
s.hub.Publish(albumID, "photo.created", p)
}
}
}
func (s *Service) attachURLs(ctx context.Context, p *models.Photo) error {
if p.StorageKey != "" {
url, err := s.storage.PresignGet(ctx, p.StorageKey)
url, err := s.storage.URL(ctx, p.StorageKey)
if err != nil {
return err
}
@@ -140,12 +126,12 @@ func (s *Service) attachURLs(ctx context.Context, p *models.Photo) error {
p.DownloadURL = url
}
if p.ThumbKey != "" {
if url, err := s.storage.PresignGet(ctx, p.ThumbKey); err == nil {
if url, err := s.storage.URL(ctx, p.ThumbKey); err == nil {
p.ThumbURL = url
}
}
if p.PreviewKey != "" {
if url, err := s.storage.PresignGet(ctx, p.PreviewKey); err == nil {
if url, err := s.storage.URL(ctx, p.PreviewKey); err == nil {
p.PreviewURL = url
}
}
@@ -216,13 +202,13 @@ func (s *Service) GetPublic(ctx context.Context, photoID string) (*models.Photo,
return p, nil
}
// DownloadPublic returns a presigned original URL for a share link.
// DownloadPublic returns a signed original URL for a share link.
func (s *Service) DownloadPublic(ctx context.Context, photoID string) (string, error) {
p, err := s.GetPublic(ctx, photoID)
if err != nil {
return "", err
}
return s.storage.PresignGet(ctx, p.StorageKey)
return s.storage.URL(ctx, p.StorageKey)
}
func (s *Service) Get(ctx context.Context, photoID, userID string) (*models.Photo, error) {
@@ -250,7 +236,7 @@ func (s *Service) Download(ctx context.Context, photoID, userID string) (string,
if err != nil {
return "", err
}
url, err := s.storage.PresignGet(ctx, p.StorageKey)
url, err := s.storage.URL(ctx, p.StorageKey)
if err != nil {
return "", err
}
@@ -416,21 +402,22 @@ func (s *Service) PermanentDelete(ctx context.Context, photoID, userID string) e
if !canDeletePhoto(role, p.UploaderID == userID) {
return httpx.Forbidden("forbidden", "you cannot delete this photo")
}
objectIDs, err := s.store.PhotoObjectIDs(ctx, photoID)
if err != nil {
return err
}
if err := s.store.HardDelete(ctx, photoID); err != nil {
return err
}
if p.AlbumID != "" {
s.publishDeleted(p)
}
if n, err := s.store.StorageKeyRefCount(ctx, p.StorageKey); err == nil && n == 0 {
keys := []string{p.StorageKey}
if p.ThumbKey != "" {
keys = append(keys, p.ThumbKey)
// Mark objects that no longer have any binding for garbage collection.
// The background worker removes the blobs from storage.
for _, oid := range objectIDs {
if n, err := s.store.ObjectRefCount(ctx, oid); err == nil && n == 0 {
_ = s.store.MarkObjectDeleting(ctx, oid)
}
if p.PreviewKey != "" {
keys = append(keys, p.PreviewKey)
}
_ = s.storage.Delete(ctx, keys...)
}
return nil
}
+30
View File
@@ -0,0 +1,30 @@
package photos
import (
"time"
"shared-album/internal/sqliteutil"
)
func nullIfEmpty(s string) any {
if s == "" {
return nil
}
return s
}
func nullTime(t *time.Time) any {
if t == nil {
return nil
}
return sqliteutil.FormatTime(*t)
}
func nullFloat(f *float64) any {
if f == nil {
return nil
}
return *f
}
func nowUTC() time.Time { return time.Now().UTC() }
+365
View File
@@ -0,0 +1,365 @@
package photos
import (
"context"
"database/sql"
"errors"
"time"
"shared-album/internal/sqliteutil"
)
// CreateObject inserts a new object row in the pending state.
func (s *SQLiteStore) CreateObject(ctx context.Context, o *Object) error {
if o.State == "" {
o.State = ObjectPending
}
var createdAt string
err := s.db.QueryRowContext(ctx,
`INSERT INTO objects (id, storage_key, mime_type, size, width, height, checksum, state)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
RETURNING created_at`,
o.ID, o.StorageKey, o.MimeType, o.Size, o.Width, o.Height, o.Checksum, o.State).Scan(&createdAt)
if err != nil {
return err
}
o.CreatedAt = sqliteutil.ParseTime(createdAt)
return nil
}
// SetObjectState updates an object's lifecycle state.
func (s *SQLiteStore) SetObjectState(ctx context.Context, id, state string) error {
res, err := s.db.ExecContext(ctx, `UPDATE objects SET state = $2 WHERE id = $1`, id, state)
if err != nil {
return err
}
if n, _ := res.RowsAffected(); n == 0 {
return ErrObjectNotFound
}
return nil
}
// ActivateObject marks an object ready after its bytes have been stored.
func (s *SQLiteStore) ActivateObject(ctx context.Context, id string) error {
return s.SetObjectState(ctx, id, ObjectReady)
}
// MarkObjectDeleting marks an object as pending removal from storage.
func (s *SQLiteStore) MarkObjectDeleting(ctx context.Context, id string) error {
return s.SetObjectState(ctx, id, ObjectDeleting)
}
// DeleteObjectRow permanently removes an object row (after its blob is gone).
func (s *SQLiteStore) DeleteObjectRow(ctx context.Context, id string) error {
res, err := s.db.ExecContext(ctx, `DELETE FROM objects WHERE id = $1`, id)
if err != nil {
return err
}
if n, _ := res.RowsAffected(); n == 0 {
return ErrObjectNotFound
}
return nil
}
// BindObject attaches a ready object to a photo under the given role. The write
// lock of the immediate transaction serializes it against garbage collection.
func (s *SQLiteStore) BindObject(ctx context.Context, photoID, objectID, role string) error {
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer func() { _ = tx.Rollback() }()
var state string
if err := tx.QueryRowContext(ctx, `SELECT state FROM objects WHERE id = $1`, objectID).Scan(&state); err != nil {
if errors.Is(err, sql.ErrNoRows) {
return ErrObjectNotFound
}
return err
}
if state != ObjectReady {
return ErrObjectNotReady
}
if _, err := tx.ExecContext(ctx,
`INSERT INTO photo_objects (photo_id, object_id, role) VALUES ($1, $2, $3)
ON CONFLICT (photo_id, role) DO UPDATE SET object_id = excluded.object_id`,
photoID, objectID, role); err != nil {
return err
}
return tx.Commit()
}
// PhotoObjectIDs returns the object ids currently bound to a photo.
func (s *SQLiteStore) PhotoObjectIDs(ctx context.Context, photoID string) ([]string, error) {
rows, err := s.db.QueryContext(ctx, `SELECT object_id FROM photo_objects WHERE photo_id = $1`, photoID)
if err != nil {
return nil, err
}
defer rows.Close()
var ids []string
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
return nil, err
}
ids = append(ids, id)
}
return ids, rows.Err()
}
// ObjectRefCount counts how many photo bindings reference an object.
func (s *SQLiteStore) ObjectRefCount(ctx context.Context, objectID string) (int, error) {
var n int
err := s.db.QueryRowContext(ctx, `SELECT count(*) FROM photo_objects WHERE object_id = $1`, objectID).Scan(&n)
return n, err
}
// OriginalObject returns the original object bound to a photo.
func (s *SQLiteStore) OriginalObject(ctx context.Context, photoID string) (*Object, error) {
var o Object
var createdAt string
err := s.db.QueryRowContext(ctx,
`SELECT o.id, o.storage_key, o.mime_type, o.size, o.width, o.height, o.checksum, o.state, o.created_at
FROM photo_objects po JOIN objects o ON o.id = po.object_id
WHERE po.photo_id = $1 AND po.role = $2`, photoID, RoleOriginal).
Scan(&o.ID, &o.StorageKey, &o.MimeType, &o.Size, &o.Width, &o.Height, &o.Checksum, &o.State, &createdAt)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return nil, ErrObjectNotFound
}
return nil, err
}
o.CreatedAt = sqliteutil.ParseTime(createdAt)
return &o, nil
}
// PhotosByOriginalObject returns the ids of all photos bound to the given
// original object, so generated variants can be shared between copies.
func (s *SQLiteStore) PhotosByOriginalObject(ctx context.Context, originalObjectID string) ([]string, error) {
rows, err := s.db.QueryContext(ctx,
`SELECT photo_id FROM photo_objects WHERE object_id = $1 AND role = $2`, originalObjectID, RoleOriginal)
if err != nil {
return nil, err
}
defer rows.Close()
var ids []string
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
return nil, err
}
ids = append(ids, id)
}
return ids, rows.Err()
}
// ClaimGarbage claims collectible objects: ready objects with no bindings and
// objects already marked deleting.
func (s *SQLiteStore) ClaimGarbage(ctx context.Context, limit int) ([]Object, error) {
if limit <= 0 {
limit = 10
}
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return nil, err
}
defer func() { _ = tx.Rollback() }()
rows, err := tx.QueryContext(ctx,
`SELECT o.id, o.storage_key FROM objects o
WHERE o.state IN ('ready', 'deleting')
AND NOT EXISTS (SELECT 1 FROM photo_objects po WHERE po.object_id = o.id)
ORDER BY o.created_at
LIMIT $1`, limit)
if err != nil {
return nil, err
}
objs, err := scanObjectKeys(rows)
if err != nil {
return nil, err
}
if len(objs) == 0 {
return nil, tx.Commit()
}
if err := markDeleting(ctx, tx, objs); err != nil {
return nil, err
}
if err := tx.Commit(); err != nil {
return nil, err
}
return objs, nil
}
// ClaimStalePending marks abandoned pending objects as deleting and returns them.
func (s *SQLiteStore) ClaimStalePending(ctx context.Context, olderThan time.Duration, limit int) ([]Object, error) {
if limit <= 0 {
limit = 10
}
threshold := sqliteutil.FormatTime(time.Now().UTC().Add(-olderThan))
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return nil, err
}
defer func() { _ = tx.Rollback() }()
rows, err := tx.QueryContext(ctx,
`SELECT id, storage_key FROM objects
WHERE state = 'pending' AND created_at < $1
ORDER BY created_at
LIMIT $2`, threshold, limit)
if err != nil {
return nil, err
}
objs, err := scanObjectKeys(rows)
if err != nil {
return nil, err
}
if len(objs) == 0 {
return nil, tx.Commit()
}
if err := markDeleting(ctx, tx, objs); err != nil {
return nil, err
}
if err := tx.Commit(); err != nil {
return nil, err
}
return objs, nil
}
// ClaimVariantJobs leases due variant jobs and returns their photo ids.
func (s *SQLiteStore) ClaimVariantJobs(ctx context.Context, limit int, lease time.Duration) ([]string, error) {
if limit <= 0 {
limit = 10
}
now := sqliteutil.FormatTime(time.Now().UTC())
next := sqliteutil.FormatTime(time.Now().UTC().Add(lease))
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return nil, err
}
defer func() { _ = tx.Rollback() }()
rows, err := tx.QueryContext(ctx,
`SELECT photo_id FROM variant_jobs
WHERE next_attempt_at <= $1
ORDER BY next_attempt_at
LIMIT $2`, now, limit)
if err != nil {
return nil, err
}
var ids []string
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
rows.Close()
return nil, err
}
ids = append(ids, id)
}
rows.Close()
if err := rows.Err(); err != nil {
return nil, err
}
if len(ids) == 0 {
return nil, tx.Commit()
}
args := make([]any, 0, len(ids)+1)
args = append(args, next)
for _, id := range ids {
args = append(args, id)
}
if _, err := tx.ExecContext(ctx,
`UPDATE variant_jobs SET next_attempt_at = $1 WHERE photo_id IN (`+sqliteutil.Placeholders(2, len(ids))+`)`,
args...); err != nil {
return nil, err
}
if err := tx.Commit(); err != nil {
return nil, err
}
return ids, nil
}
// CompleteVariantJob removes a finished job.
func (s *SQLiteStore) CompleteVariantJob(ctx context.Context, photoID string) error {
_, err := s.db.ExecContext(ctx, `DELETE FROM variant_jobs WHERE photo_id = $1`, photoID)
return err
}
// FailVariantJob records a failure and schedules a retry with exponential
// backoff, abandoning the job after maxAttempts.
func (s *SQLiteStore) FailVariantJob(ctx context.Context, photoID, errMsg string, maxAttempts int) error {
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer func() { _ = tx.Rollback() }()
var attempts int
err = tx.QueryRowContext(ctx, `SELECT attempts FROM variant_jobs WHERE photo_id = $1`, photoID).Scan(&attempts)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return tx.Commit()
}
return err
}
newAttempts := attempts + 1
if maxAttempts > 0 && newAttempts >= maxAttempts {
if _, err := tx.ExecContext(ctx, `DELETE FROM variant_jobs WHERE photo_id = $1`, photoID); err != nil {
return err
}
} else {
secs := 10 * (1 << attempts)
if secs > 300 {
secs = 300
}
next := sqliteutil.FormatTime(time.Now().UTC().Add(time.Duration(secs) * time.Second))
if _, err := tx.ExecContext(ctx,
`UPDATE variant_jobs SET attempts = $2, next_attempt_at = $3, last_error = $4 WHERE photo_id = $1`,
photoID, newAttempts, next, errMsg); err != nil {
return err
}
}
return tx.Commit()
}
func scanObjectKeys(rows *sql.Rows) ([]Object, error) {
var objs []Object
for rows.Next() {
var o Object
if err := rows.Scan(&o.ID, &o.StorageKey); err != nil {
rows.Close()
return nil, err
}
objs = append(objs, o)
}
rows.Close()
return objs, rows.Err()
}
func markDeleting(ctx context.Context, tx *sql.Tx, objs []Object) error {
args := make([]any, len(objs))
for i, o := range objs {
args[i] = o.ID
}
_, err := tx.ExecContext(ctx,
`UPDATE objects SET state = 'deleting' WHERE id IN (`+sqliteutil.Placeholders(1, len(objs))+`)`, args...)
return err
}
// AllStorageKeys returns every key registered in the object journal.
func (s *SQLiteStore) AllStorageKeys(ctx context.Context) ([]string, error) {
rows, err := s.db.QueryContext(ctx, `SELECT storage_key FROM objects`)
if err != nil {
return nil, err
}
defer rows.Close()
var keys []string
for rows.Next() {
var k string
if err := rows.Scan(&k); err != nil {
return nil, err
}
keys = append(keys, k)
}
return keys, rows.Err()
}
+470
View File
@@ -0,0 +1,470 @@
package photos
import (
"context"
"database/sql"
"encoding/json"
"errors"
"strings"
"shared-album/internal/auth"
"shared-album/internal/models"
"shared-album/internal/sqliteutil"
)
// SQLiteStore is the SQLite implementation of Repository.
type SQLiteStore struct {
db *sql.DB
}
func NewSQLiteStore(db *sql.DB) *SQLiteStore { return &SQLiteStore{db: db} }
const sqlitePhotoColumns = `p.id, COALESCE(p.album_id, ''), p.uploader_id,
COALESCE(o_o.storage_key, ''),
COALESCE(o_t.storage_key, ''),
COALESCE(o_p.storage_key, ''),
p.original_filename, COALESCE(o_o.size, 0), COALESCE(o_o.width, 0), COALESCE(o_o.height, 0), COALESCE(o_o.mime_type, ''),
p.taken_at, p.latitude, p.longitude, COALESCE(p.caption, ''), p.created_at,
u.id, u.name, COALESCE(u.avatar_url, ''),
(SELECT count(*) FROM likes l WHERE l.photo_id = p.id),
(SELECT count(*) FROM comments c WHERE c.photo_id = p.id),
EXISTS(SELECT 1 FROM likes l WHERE l.photo_id = p.id AND l.user_id = $2),
COALESCE(p.exif, '{}'), p.deleted_at,
COALESCE((SELECT json_group_array(json_object('id', t.id, 'name', t.name))
FROM (SELECT t.id, t.name FROM photo_tags pt JOIN tags t ON t.id = pt.tag_id
WHERE pt.photo_id = p.id AND t.user_id = $2
ORDER BY lower(t.name)) t), '[]')`
const sqlitePhotoJoins = `
LEFT JOIN photo_objects po_o ON po_o.photo_id = p.id AND po_o.role = 'original'
LEFT JOIN objects o_o ON o_o.id = po_o.object_id
LEFT JOIN photo_objects po_t ON po_t.photo_id = p.id AND po_t.role = 'thumb'
LEFT JOIN objects o_t ON o_t.id = po_t.object_id
LEFT JOIN photo_objects po_p ON po_p.photo_id = p.id AND po_p.role = 'preview'
LEFT JOIN objects o_p ON o_p.id = po_p.object_id`
func scanSQLitePhoto(scan func(...any) error) (*models.Photo, error) {
var p models.Photo
var exifText, tagsText, createdAt string
var takenAt, deletedAt sql.NullString
var lat, long sql.NullFloat64
var liked int
if err := scan(&p.ID, &p.AlbumID, &p.UploaderID, &p.StorageKey,
&p.ThumbKey, &p.PreviewKey,
&p.OriginalFilename, &p.Size, &p.Width, &p.Height, &p.MimeType,
&takenAt, &lat, &long, &p.Caption, &createdAt,
&p.Uploader.ID, &p.Uploader.Name, &p.Uploader.AvatarURL,
&p.LikesCount, &p.CommentsCount, &liked, &exifText, &deletedAt, &tagsText); err != nil {
return nil, err
}
p.TakenAt = sqliteutil.ParseNullTime(takenAt)
if lat.Valid {
p.Latitude = &lat.Float64
}
if long.Valid {
p.Longitude = &long.Float64
}
p.CreatedAt = sqliteutil.ParseTime(createdAt)
p.DeletedAt = sqliteutil.ParseNullTime(deletedAt)
p.LikedByMe = liked != 0
setExif(&p, exifText)
setTags(&p, tagsText)
return &p, nil
}
func (s *SQLiteStore) CreateWithOriginal(ctx context.Context, p *models.Photo, obj *Object) error {
exifJSON, err := json.Marshal(p.EXIF)
if err != nil || len(exifJSON) == 0 {
exifJSON = []byte("{}")
}
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer func() { _ = tx.Rollback() }()
if _, err := tx.ExecContext(ctx,
`INSERT INTO objects (id, storage_key, mime_type, size, width, height, checksum, state)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)`,
obj.ID, obj.StorageKey, obj.MimeType, obj.Size, obj.Width, obj.Height, obj.Checksum, ObjectPending); err != nil {
return err
}
var createdAt string
if err := tx.QueryRowContext(ctx,
`INSERT INTO photos (id, album_id, uploader_id, original_filename, taken_at, latitude, longitude, caption, exif)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)
RETURNING created_at`,
p.ID, nullIfEmpty(p.AlbumID), p.UploaderID, p.OriginalFilename,
nullTime(p.TakenAt), nullFloat(p.Latitude), nullFloat(p.Longitude), p.Caption, string(exifJSON),
).Scan(&createdAt); err != nil {
return err
}
p.CreatedAt = sqliteutil.ParseTime(createdAt)
if _, err := tx.ExecContext(ctx,
`INSERT INTO photo_objects (photo_id, object_id, role) VALUES ($1, $2, $3)`,
p.ID, obj.ID, RoleOriginal); err != nil {
return err
}
if _, err := tx.ExecContext(ctx,
`INSERT INTO variant_jobs (photo_id) VALUES ($1) ON CONFLICT (photo_id) DO NOTHING`, p.ID); err != nil {
return err
}
return tx.Commit()
}
func (s *SQLiteStore) DeleteUpload(ctx context.Context, photoID, objectID string) error {
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer func() { _ = tx.Rollback() }()
if _, err := tx.ExecContext(ctx, `DELETE FROM photos WHERE id = $1`, photoID); err != nil {
return err
}
if _, err := tx.ExecContext(ctx, `DELETE FROM objects WHERE id = $1`, objectID); err != nil {
return err
}
return tx.Commit()
}
func (s *SQLiteStore) GetByID(ctx context.Context, id, viewerID string) (*models.Photo, error) {
row := s.db.QueryRowContext(ctx,
`SELECT `+sqlitePhotoColumns+`
FROM photos p JOIN users u ON u.id = p.uploader_id`+sqlitePhotoJoins+`
WHERE p.id = $1`, id, viewerID)
p, err := scanSQLitePhoto(row.Scan)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return nil, ErrNotFound
}
return nil, err
}
return p, nil
}
func (s *SQLiteStore) List(ctx context.Context, in ListParams) ([]models.Photo, error) {
if in.Limit <= 0 || in.Limit > 100 {
in.Limit = 50
}
var since any
if in.Since != nil {
since = sqliteutil.FormatTime(*in.Since)
}
rows, err := s.db.QueryContext(ctx,
`SELECT `+sqlitePhotoColumns+`
FROM photos p JOIN users u ON u.id = p.uploader_id`+sqlitePhotoJoins+`
WHERE p.album_id = $1
AND p.deleted_at IS NULL
AND ($3 = '' OR (p.created_at, p.id) < (SELECT created_at, id FROM photos WHERE id = $3))
AND ($4 IS NULL OR p.created_at > $4)
ORDER BY p.created_at DESC, p.id DESC
LIMIT $5`,
in.AlbumID, in.Viewer, in.Cursor, since, in.Limit)
if err != nil {
return nil, err
}
defer rows.Close()
out := []models.Photo{}
for rows.Next() {
p, err := scanSQLitePhoto(rows.Scan)
if err != nil {
return nil, err
}
out = append(out, *p)
}
return out, rows.Err()
}
func (s *SQLiteStore) SetDeleted(ctx context.Context, id string, deleted bool) error {
var res sql.Result
var err error
if deleted {
res, err = s.db.ExecContext(ctx,
`UPDATE photos SET deleted_at = $2 WHERE id = $1 AND deleted_at IS NULL`,
id, sqliteutil.FormatTime(nowUTC()))
} else {
res, err = s.db.ExecContext(ctx, `UPDATE photos SET deleted_at = NULL WHERE id = $1`, id)
}
if err != nil {
return err
}
if n, _ := res.RowsAffected(); n == 0 {
return ErrNotFound
}
return nil
}
func (s *SQLiteStore) UpdateAlbum(ctx context.Context, id, albumID string) error {
res, err := s.db.ExecContext(ctx,
`UPDATE photos SET album_id = $2 WHERE id = $1 AND deleted_at IS NULL`, id, albumID)
if err != nil {
return err
}
if n, _ := res.RowsAffected(); n == 0 {
return ErrNotFound
}
return nil
}
func (s *SQLiteStore) Clone(ctx context.Context, src *models.Photo, albumID, uploaderID, id string) error {
exifJSON, err := json.Marshal(src.EXIF)
if err != nil || len(exifJSON) == 0 {
exifJSON = []byte("{}")
}
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer func() { _ = tx.Rollback() }()
if _, err := tx.ExecContext(ctx,
`INSERT INTO photos (id, album_id, uploader_id, original_filename, taken_at, latitude, longitude, caption, exif)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)`,
id, nullIfEmpty(albumID), uploaderID, src.OriginalFilename,
nullTime(src.TakenAt), nullFloat(src.Latitude), nullFloat(src.Longitude), src.Caption, string(exifJSON)); err != nil {
return err
}
if _, err := tx.ExecContext(ctx,
`INSERT INTO photo_objects (photo_id, object_id, role)
SELECT $1, object_id, role FROM photo_objects WHERE photo_id = $2`,
id, src.ID); err != nil {
return err
}
return tx.Commit()
}
func (s *SQLiteStore) HardDelete(ctx context.Context, id string) error {
res, err := s.db.ExecContext(ctx, `DELETE FROM photos WHERE id = $1`, id)
if err != nil {
return err
}
if n, _ := res.RowsAffected(); n == 0 {
return ErrNotFound
}
return nil
}
func (s *SQLiteStore) ListTrash(ctx context.Context, userID string) ([]models.Photo, error) {
rows, err := s.db.QueryContext(ctx,
`SELECT `+sqlitePhotoColumns+`, COALESCE(a.title, '')
FROM photos p
JOIN users u ON u.id = p.uploader_id
LEFT JOIN albums a ON a.id = p.album_id`+sqlitePhotoJoins+`
WHERE p.deleted_at IS NOT NULL
AND (
(a.id IS NOT NULL AND (a.owner_id = $1 OR EXISTS(
SELECT 1 FROM album_members m WHERE m.album_id = a.id AND m.user_id = $1)))
OR (p.album_id IS NULL AND p.uploader_id = $1)
)
ORDER BY p.deleted_at DESC
LIMIT 200`,
userID, userID)
if err != nil {
return nil, err
}
defer rows.Close()
out := []models.Photo{}
for rows.Next() {
var p models.Photo
var exifText, tagsText, createdAt, albumTitle string
var takenAt, deletedAt sql.NullString
var lat, long sql.NullFloat64
var liked int
if err := rows.Scan(&p.ID, &p.AlbumID, &p.UploaderID, &p.StorageKey,
&p.ThumbKey, &p.PreviewKey,
&p.OriginalFilename, &p.Size, &p.Width, &p.Height, &p.MimeType,
&takenAt, &lat, &long, &p.Caption, &createdAt,
&p.Uploader.ID, &p.Uploader.Name, &p.Uploader.AvatarURL,
&p.LikesCount, &p.CommentsCount, &liked, &exifText, &deletedAt, &albumTitle, &tagsText); err != nil {
return nil, err
}
p.TakenAt = sqliteutil.ParseNullTime(takenAt)
if lat.Valid {
p.Latitude = &lat.Float64
}
if long.Valid {
p.Longitude = &long.Float64
}
p.CreatedAt = sqliteutil.ParseTime(createdAt)
p.DeletedAt = sqliteutil.ParseNullTime(deletedAt)
p.AlbumTitle = albumTitle
p.LikedByMe = liked != 0
setExif(&p, exifText)
setTags(&p, tagsText)
out = append(out, p)
}
return out, rows.Err()
}
func (s *SQLiteStore) TrashCount(ctx context.Context, userID string) (int, error) {
var n int
err := s.db.QueryRowContext(ctx,
`SELECT count(*) FROM photos p
LEFT JOIN albums a ON a.id = p.album_id
WHERE p.deleted_at IS NOT NULL
AND (
(a.id IS NOT NULL AND (a.owner_id = $1 OR EXISTS(
SELECT 1 FROM album_members m WHERE m.album_id = a.id AND m.user_id = $1)))
OR (p.album_id IS NULL AND p.uploader_id = $1)
)`, userID).Scan(&n)
return n, err
}
func (s *SQLiteStore) RestoreToAlbum(ctx context.Context, id, albumID string) error {
res, err := s.db.ExecContext(ctx,
`UPDATE photos SET album_id = $2, deleted_at = NULL WHERE id = $1`, id, albumID)
if err != nil {
return err
}
if n, _ := res.RowsAffected(); n == 0 {
return ErrNotFound
}
return nil
}
func (s *SQLiteStore) AlbumExists(ctx context.Context, albumID string) (bool, error) {
var ok int
err := s.db.QueryRowContext(ctx, `SELECT EXISTS(SELECT 1 FROM albums WHERE id = $1)`, albumID).Scan(&ok)
return ok != 0, err
}
func (s *SQLiteStore) ListTags(ctx context.Context, userID, q string) ([]models.Tag, error) {
rows, err := s.db.QueryContext(ctx,
`SELECT id, name FROM tags
WHERE user_id = $1 AND ($2 = '' OR name LIKE '%' || $2 || '%')
ORDER BY lower(name) LIMIT 50`, userID, q)
if err != nil {
return nil, err
}
defer rows.Close()
out := []models.Tag{}
for rows.Next() {
var t models.Tag
if err := rows.Scan(&t.ID, &t.Name); err != nil {
return nil, err
}
out = append(out, t)
}
return out, rows.Err()
}
func (s *SQLiteStore) EnsureTag(ctx context.Context, userID, name string) (*models.Tag, error) {
name = strings.TrimSpace(name)
if name == "" || len(name) > 64 {
return nil, ErrNotFound
}
var t models.Tag
err := s.db.QueryRowContext(ctx,
`SELECT id, name FROM tags WHERE user_id = $1 AND lower(name) = lower($2)`, userID, name).
Scan(&t.ID, &t.Name)
if err == nil {
return &t, nil
}
if !errors.Is(err, sql.ErrNoRows) {
return nil, err
}
id := auth.NewID()
err = s.db.QueryRowContext(ctx,
`INSERT INTO tags (id, user_id, name) VALUES ($1, $2, $3) RETURNING id, name`, id, userID, name).
Scan(&t.ID, &t.Name)
if err != nil {
if e2 := s.db.QueryRowContext(ctx,
`SELECT id, name FROM tags WHERE user_id = $1 AND lower(name) = lower($2)`, userID, name).
Scan(&t.ID, &t.Name); e2 == nil {
return &t, nil
}
return nil, err
}
return &t, nil
}
func (s *SQLiteStore) AttachTag(ctx context.Context, photoID, tagID string) error {
_, err := s.db.ExecContext(ctx,
`INSERT INTO photo_tags (photo_id, tag_id) VALUES ($1, $2) ON CONFLICT DO NOTHING`, photoID, tagID)
return err
}
func (s *SQLiteStore) DetachTag(ctx context.Context, photoID, tagID string) error {
_, err := s.db.ExecContext(ctx,
`DELETE FROM photo_tags WHERE photo_id = $1 AND tag_id = $2`, photoID, tagID)
return err
}
func (s *SQLiteStore) TagOwnedBy(ctx context.Context, tagID, userID string) (bool, error) {
var ok int
err := s.db.QueryRowContext(ctx,
`SELECT EXISTS(SELECT 1 FROM tags WHERE id = $1 AND user_id = $2)`, tagID, userID).Scan(&ok)
return ok != 0, err
}
func (s *SQLiteStore) SetCoverIfEmpty(ctx context.Context, albumID, photoID string) {
_, _ = s.db.ExecContext(ctx,
`UPDATE albums SET cover_photo_id = $2 WHERE id = $1 AND cover_photo_id IS NULL`,
albumID, photoID)
}
func (s *SQLiteStore) UpdateFilename(ctx context.Context, id, filename string) error {
res, err := s.db.ExecContext(ctx, `UPDATE photos SET original_filename = $2 WHERE id = $1`, id, filename)
if err != nil {
return err
}
if n, _ := res.RowsAffected(); n == 0 {
return ErrNotFound
}
return nil
}
// BackfillEXIF mirrors the PostgreSQL implementation, reading the original blob
// of every photo with empty exif.
func (s *SQLiteStore) BackfillEXIF(ctx context.Context, fetch func(context.Context, string) ([]byte, error)) (int, error) {
rows, err := s.db.QueryContext(ctx,
`SELECT p.id, o.storage_key
FROM photos p
JOIN photo_objects po ON po.photo_id = p.id AND po.role = 'original'
JOIN objects o ON o.id = po.object_id
WHERE p.exif IS NULL OR p.exif = '{}'
ORDER BY p.created_at`)
if err != nil {
return 0, err
}
var pending []backfillRow
for rows.Next() {
var r backfillRow
if err := rows.Scan(&r.ID, &r.StorageKey); err != nil {
rows.Close()
return 0, err
}
pending = append(pending, r)
}
rows.Close()
if err := rows.Err(); err != nil {
return 0, err
}
updated := 0
for _, r := range pending {
data, err := fetch(ctx, r.StorageKey)
if err != nil {
continue
}
meta := extractEXIF(data)
if len(meta.Attributes) == 0 && meta.TakenAt == nil && meta.Latitude == nil && meta.Longitude == nil {
continue
}
attrs, err := json.Marshal(meta.Attributes)
if err != nil || len(attrs) == 0 {
attrs = []byte("{}")
}
if _, err := s.db.ExecContext(ctx,
`UPDATE photos SET exif = $2,
taken_at = COALESCE(taken_at, $3),
latitude = COALESCE(latitude, $4),
longitude = COALESCE(longitude, $5)
WHERE id = $1`,
r.ID, string(attrs), nullTime(meta.TakenAt), nullFloat(meta.Latitude), nullFloat(meta.Longitude)); err != nil {
continue
}
updated++
}
return updated, nil
}
+97 -43
View File
@@ -23,9 +23,14 @@ type Store struct {
func NewStore(pool *pgxpool.Pool) *Store { return &Store{pool: pool} }
const photoColumns = `p.id, COALESCE(p.album_id, ''), p.uploader_id, p.storage_key,
COALESCE(p.thumb_key, ''), COALESCE(p.preview_key, ''),
p.original_filename, p.size, p.width, p.height, p.mime_type,
// photoColumns selects photo rows together with the keys and physical facts of
// their bound objects (original, thumb, preview) and the original's size,
// dimensions and mime type.
const photoColumns = `p.id, COALESCE(p.album_id, ''), p.uploader_id,
COALESCE(o_o.storage_key, ''),
COALESCE(o_t.storage_key, ''),
COALESCE(o_p.storage_key, ''),
p.original_filename, COALESCE(o_o.size, 0), COALESCE(o_o.width, 0), COALESCE(o_o.height, 0), COALESCE(o_o.mime_type, ''),
p.taken_at, p.latitude, p.longitude, COALESCE(p.caption, ''), p.created_at,
u.id, u.name, COALESCE(u.avatar_url, ''),
(SELECT count(*) FROM likes l WHERE l.photo_id = p.id),
@@ -36,6 +41,14 @@ const photoColumns = `p.id, COALESCE(p.album_id, ''), p.uploader_id, p.storage_k
FROM photo_tags pt JOIN tags t ON t.id = pt.tag_id
WHERE pt.photo_id = p.id AND t.user_id = $2), '[]')::text`
const photoJoins = `
LEFT JOIN photo_objects po_o ON po_o.photo_id = p.id AND po_o.role = 'original'
LEFT JOIN objects o_o ON o_o.id = po_o.object_id
LEFT JOIN photo_objects po_t ON po_t.photo_id = p.id AND po_t.role = 'thumb'
LEFT JOIN objects o_t ON o_t.id = po_t.object_id
LEFT JOIN photo_objects po_p ON po_p.photo_id = p.id AND po_p.role = 'preview'
LEFT JOIN objects o_p ON o_p.id = po_p.object_id`
func scanPhoto(row pgx.Row) (*models.Photo, error) {
var p models.Photo
var exifText, tagsText string
@@ -76,31 +89,68 @@ func setExif(p *models.Photo, raw string) {
}
}
func (s *Store) Create(ctx context.Context, p *models.Photo) error {
// CreateWithOriginal records a new photo together with its pending original
// object, the original binding and a variant job — all in one transaction. The
// caller stores the bytes and then activates the object. The metadata layer is
// the ledger of record; the storage layer follows.
func (s *Store) CreateWithOriginal(ctx context.Context, p *models.Photo, obj *Object) error {
exifJSON, err := json.Marshal(p.EXIF)
if err != nil || len(exifJSON) == 0 {
exifJSON = []byte("{}")
}
return s.pool.QueryRow(ctx,
`INSERT INTO photos (id, album_id, uploader_id, storage_key, original_filename,
size, width, height, mime_type, taken_at, latitude, longitude, caption, exif)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14::jsonb)
tx, err := s.pool.Begin(ctx)
if err != nil {
return err
}
defer func() { _ = tx.Rollback(ctx) }()
if _, err := tx.Exec(ctx,
`INSERT INTO objects (id, storage_key, mime_type, size, width, height, checksum, state)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)`,
obj.ID, obj.StorageKey, obj.MimeType, obj.Size, obj.Width, obj.Height, obj.Checksum, ObjectPending); err != nil {
return err
}
if err := tx.QueryRow(ctx,
`INSERT INTO photos (id, album_id, uploader_id, original_filename, taken_at, latitude, longitude, caption, exif)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9::jsonb)
RETURNING created_at`,
p.ID, p.AlbumID, p.UploaderID, p.StorageKey, p.OriginalFilename,
p.Size, p.Width, p.Height, p.MimeType, p.TakenAt, p.Latitude, p.Longitude, p.Caption, string(exifJSON),
).Scan(&p.CreatedAt)
p.ID, p.AlbumID, p.UploaderID, p.OriginalFilename, p.TakenAt, p.Latitude, p.Longitude, p.Caption, string(exifJSON),
).Scan(&p.CreatedAt); err != nil {
return err
}
if _, err := tx.Exec(ctx,
`INSERT INTO photo_objects (photo_id, object_id, role) VALUES ($1, $2, $3)`,
p.ID, obj.ID, RoleOriginal); err != nil {
return err
}
if _, err := tx.Exec(ctx,
`INSERT INTO variant_jobs (photo_id) VALUES ($1) ON CONFLICT (photo_id) DO NOTHING`, p.ID); err != nil {
return err
}
return tx.Commit(ctx)
}
func (s *Store) SetVariants(ctx context.Context, id, thumbKey, previewKey string) error {
_, err := s.pool.Exec(ctx,
`UPDATE photos SET thumb_key = $2, preview_key = $3 WHERE id = $1`, id, thumbKey, previewKey)
// DeleteUpload rolls back a failed upload: the photo (and its bindings and job)
// plus the pending object row are removed.
func (s *Store) DeleteUpload(ctx context.Context, photoID, objectID string) error {
tx, err := s.pool.Begin(ctx)
if err != nil {
return err
}
defer func() { _ = tx.Rollback(ctx) }()
if _, err := tx.Exec(ctx, `DELETE FROM photos WHERE id = $1`, photoID); err != nil {
return err
}
if _, err := tx.Exec(ctx, `DELETE FROM objects WHERE id = $1`, objectID); err != nil {
return err
}
return tx.Commit(ctx)
}
func (s *Store) GetByID(ctx context.Context, id, viewerID string) (*models.Photo, error) {
return scanPhoto(s.pool.QueryRow(ctx,
`SELECT `+photoColumns+`
FROM photos p JOIN users u ON u.id = p.uploader_id
FROM photos p JOIN users u ON u.id = p.uploader_id`+photoJoins+`
WHERE p.id = $1`, id, viewerID))
}
@@ -118,7 +168,7 @@ func (s *Store) List(ctx context.Context, in ListParams) ([]models.Photo, error)
}
rows, err := s.pool.Query(ctx,
`SELECT `+photoColumns+`
FROM photos p JOIN users u ON u.id = p.uploader_id
FROM photos p JOIN users u ON u.id = p.uploader_id`+photoJoins+`
WHERE p.album_id = $1
AND p.deleted_at IS NULL
AND ($3 = '' OR (p.created_at, p.id) < (SELECT created_at, id FROM photos WHERE id = $3))
@@ -181,30 +231,36 @@ func (s *Store) UpdateAlbum(ctx context.Context, id, albumID string) error {
return nil
}
// Clone creates a new photo row that references the same stored objects as src.
// Clone creates a new photo row in albumID that references the same objects as
// src. No storage operation is performed: the clone shares the physical files.
func (s *Store) Clone(ctx context.Context, src *models.Photo, albumID, uploaderID, id string) error {
exifJSON, err := json.Marshal(src.EXIF)
if err != nil || len(exifJSON) == 0 {
exifJSON = []byte("{}")
}
_, err = s.pool.Exec(ctx,
`INSERT INTO photos (id, album_id, uploader_id, storage_key, thumb_key, preview_key,
original_filename, size, width, height, mime_type, taken_at, latitude, longitude, caption, exif)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16::jsonb)`,
id, albumID, uploaderID, src.StorageKey, nullIfEmpty(src.ThumbKey), nullIfEmpty(src.PreviewKey),
src.OriginalFilename, src.Size, src.Width, src.Height, src.MimeType,
src.TakenAt, src.Latitude, src.Longitude, src.Caption, string(exifJSON))
tx, err := s.pool.Begin(ctx)
if err != nil {
return err
}
defer func() { _ = tx.Rollback(ctx) }()
if _, err := tx.Exec(ctx,
`INSERT INTO photos (id, album_id, uploader_id, original_filename, taken_at, latitude, longitude, caption, exif)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9::jsonb)`,
id, albumID, uploaderID, src.OriginalFilename, src.TakenAt, src.Latitude, src.Longitude, src.Caption, string(exifJSON)); err != nil {
return err
}
if _, err := tx.Exec(ctx,
`INSERT INTO photo_objects (photo_id, object_id, role)
SELECT $1, object_id, role FROM photo_objects WHERE photo_id = $2`,
id, src.ID); err != nil {
return err
}
return tx.Commit(ctx)
}
// StorageKeyRefCount counts photo rows (any state) referencing the stored object.
func (s *Store) StorageKeyRefCount(ctx context.Context, storageKey string) (int, error) {
var n int
err := s.pool.QueryRow(ctx, `SELECT count(*) FROM photos WHERE storage_key = $1`, storageKey).Scan(&n)
return n, err
}
// HardDelete permanently removes the photo row.
// HardDelete permanently removes the photo row. Bound objects are not touched
// here; the caller marks orphaned objects for collection.
func (s *Store) HardDelete(ctx context.Context, id string) error {
tag, err := s.pool.Exec(ctx, `DELETE FROM photos WHERE id = $1`, id)
if err != nil {
@@ -223,7 +279,7 @@ func (s *Store) ListTrash(ctx context.Context, userID string) ([]models.Photo, e
`SELECT `+photoColumns+`, COALESCE(a.title, '')
FROM photos p
JOIN users u ON u.id = p.uploader_id
LEFT JOIN albums a ON a.id = p.album_id
LEFT JOIN albums a ON a.id = p.album_id`+photoJoins+`
WHERE p.deleted_at IS NOT NULL
AND (
(a.id IS NOT NULL AND (a.owner_id = $1 OR EXISTS(
@@ -362,13 +418,6 @@ func (s *Store) TagOwnedBy(ctx context.Context, tagID, userID string) (bool, err
return ok, err
}
func nullIfEmpty(s string) *string {
if s == "" {
return nil
}
return &s
}
func (s *Store) SetCoverIfEmpty(ctx context.Context, albumID, photoID string) {
_, _ = s.pool.Exec(ctx,
`UPDATE albums SET cover_photo_id = $2 WHERE id = $1 AND cover_photo_id IS NULL`,
@@ -391,11 +440,16 @@ type backfillRow struct {
StorageKey string
}
// BackfillEXIF re-reads the original object of every photo with empty exif and
// fills the exif/geo fields on the photo.
func (s *Store) BackfillEXIF(ctx context.Context, fetch func(context.Context, string) ([]byte, error)) (int, error) {
rows, err := s.pool.Query(ctx,
`SELECT id, storage_key FROM photos
WHERE exif IS NULL OR exif = '{}'::jsonb
ORDER BY created_at`)
`SELECT p.id, o.storage_key
FROM photos p
JOIN photo_objects po ON po.photo_id = p.id AND po.role = 'original'
JOIN objects o ON o.id = po.object_id
WHERE p.exif IS NULL OR p.exif = '{}'::jsonb
ORDER BY p.created_at`)
if err != nil {
return 0, err
}
+70
View File
@@ -0,0 +1,70 @@
package photos
import (
"context"
"strings"
"testing"
)
func TestUploadCreatesLedgerAndAlbumFreeKey(t *testing.T) {
pool := testPool(t)
ctx := context.Background()
svc, cl := newTestService(t, pool)
user := createUser(t, ctx, pool)
album := createAlbum(t, ctx, pool, user)
p, err := svc.Upload(ctx, UploadInput{
AlbumID: album,
UserID: user,
Filename: "photo.png",
MimeType: "image/png",
Data: makePNG(t),
})
if err != nil {
t.Fatalf("upload: %v", err)
}
t.Cleanup(func() { cleanupPhoto(t, ctx, pool, cl, p.ID) })
if !strings.HasPrefix(p.StorageKey, "objects/") {
t.Fatalf("storage key %q is not an object key", p.StorageKey)
}
if strings.Contains(p.StorageKey, album) {
t.Fatalf("storage key leaks album id: %q", p.StorageKey)
}
if p.OriginalURL == "" {
t.Fatalf("original url not set")
}
if p.ThumbURL != "" || p.PreviewURL != "" {
t.Fatalf("variants should be pending, got thumb=%q preview=%q", p.ThumbURL, p.PreviewURL)
}
orig, err := svc.store.OriginalObject(ctx, p.ID)
if err != nil {
t.Fatalf("original object: %v", err)
}
if orig.State != ObjectReady {
t.Fatalf("original object state = %q, want ready", orig.State)
}
if n, err := svc.store.ObjectRefCount(ctx, orig.ID); err != nil || n != 1 {
t.Fatalf("original ref count = %d err=%v, want 1", n, err)
}
var jobs int
if err := pool.QueryRow(ctx, `SELECT count(*) FROM variant_jobs WHERE photo_id = $1`, p.ID).Scan(&jobs); err != nil {
t.Fatalf("count jobs: %v", err)
}
if jobs != 1 {
t.Fatalf("variant jobs = %d, want 1", jobs)
}
list, _, err := svc.List(ctx, album, user, "", nil, 50)
if err != nil || len(list) != 1 {
t.Fatalf("list: %v len=%d", err, len(list))
}
if list[0].MimeType != "image/png" || list[0].Width != 4 || list[0].Height != 4 {
t.Fatalf("derived fields wrong: mime=%q %dx%d", list[0].MimeType, list[0].Width, list[0].Height)
}
if list[0].OriginalURL == "" {
t.Fatalf("list original url not set")
}
}
+210
View File
@@ -0,0 +1,210 @@
package photos
import (
"bytes"
"context"
"errors"
"log/slog"
"time"
"shared-album/internal/auth"
)
const maxVariantAttempts = 5
// Worker is the background reconciler. It drives the variant generation queue
// and the object garbage collector entirely from the state recorded in the
// metadata layer, so it resumes correctly after a restart.
type Worker struct {
svc *Service
interval time.Duration
batch int
pendingTTL time.Duration
lease time.Duration
}
// NewWorker builds a worker. Parameters are clamped to sane defaults.
func NewWorker(svc *Service, interval time.Duration, batch int, pendingTTL time.Duration) *Worker {
if interval <= 0 {
interval = 5 * time.Second
}
if batch <= 0 {
batch = 10
}
if pendingTTL <= 0 {
pendingTTL = 10 * time.Minute
}
return &Worker{
svc: svc,
interval: interval,
batch: batch,
pendingTTL: pendingTTL,
lease: interval * 4,
}
}
// Run loops until the context is cancelled.
func (w *Worker) Run(ctx context.Context) {
slog.Info("background worker started", "interval", w.interval)
defer slog.Info("background worker stopped")
ticker := time.NewTicker(w.interval)
defer ticker.Stop()
for {
w.tick(ctx)
select {
case <-ctx.Done():
return
case <-ticker.C:
}
}
}
func (w *Worker) tick(ctx context.Context) {
w.processVariantJobs(ctx)
w.collectGarbage(ctx)
w.reapStalePending(ctx)
}
func (w *Worker) processVariantJobs(ctx context.Context) {
ids, err := w.svc.store.ClaimVariantJobs(ctx, w.batch, w.lease)
if err != nil {
slog.Error("claim variant jobs", "error", err)
return
}
for _, id := range ids {
w.generateVariants(ctx, id)
}
}
func (w *Worker) generateVariants(ctx context.Context, photoID string) {
orig, err := w.svc.store.OriginalObject(ctx, photoID)
if err != nil {
// The photo or its original binding is gone; nothing to do.
_ = w.svc.store.CompleteVariantJob(ctx, photoID)
return
}
if orig.State != ObjectReady {
w.fail(ctx, photoID, "original object is not ready")
return
}
data, err := w.svc.storage.Get(ctx, orig.StorageKey)
if err != nil {
w.fail(ctx, photoID, "fetch original: "+err.Error())
return
}
thumb, preview, err := buildVariants(data)
if err != nil {
w.fail(ctx, photoID, "build variants: "+err.Error())
return
}
siblings, err := w.svc.store.PhotosByOriginalObject(ctx, orig.ID)
if err != nil || len(siblings) == 0 {
siblings = []string{photoID}
}
if err := w.createVariant(ctx, siblings, RoleThumb, thumb); err != nil {
w.fail(ctx, photoID, "store thumb: "+err.Error())
return
}
if err := w.createVariant(ctx, siblings, RolePreview, preview); err != nil {
w.fail(ctx, photoID, "store preview: "+err.Error())
return
}
_ = w.svc.store.CompleteVariantJob(ctx, photoID)
for _, pid := range siblings {
w.notify(ctx, pid)
}
}
// createVariant stores one derived object and binds it to every photo sharing
// the original, so copies reuse the same physical variant.
func (w *Worker) createVariant(ctx context.Context, photoIDs []string, role string, data []byte) error {
objID := auth.NewID()
key := objectKey(objID, role, ".webp")
obj := &Object{
ID: objID,
StorageKey: key,
MimeType: "image/webp",
Size: int64(len(data)),
Checksum: checksum(data),
}
if err := w.svc.store.CreateObject(ctx, obj); err != nil {
return err
}
if err := w.svc.storage.Put(ctx, key, bytes.NewReader(data), int64(len(data)), "image/webp"); err != nil {
_ = w.svc.store.DeleteObjectRow(ctx, objID)
return err
}
if err := w.svc.store.ActivateObject(ctx, objID); err != nil {
return err
}
bound := false
for _, pid := range photoIDs {
if err := w.svc.store.BindObject(ctx, pid, objID, role); err != nil {
if errors.Is(err, ErrObjectNotReady) || errors.Is(err, ErrObjectNotFound) {
continue
}
return err
}
bound = true
}
if !bound {
// No photo could take the variant; let the collector reclaim it.
_ = w.svc.store.MarkObjectDeleting(ctx, objID)
}
return nil
}
func (w *Worker) notify(ctx context.Context, photoID string) {
p, err := w.svc.store.GetByID(ctx, photoID, "")
if err != nil {
return
}
if err := w.svc.attachURLs(ctx, p); err != nil {
return
}
w.svc.hub.Publish(p.AlbumID, "photo.created", p)
}
func (w *Worker) fail(ctx context.Context, photoID, msg string) {
slog.Warn("variant job failed", "photo", photoID, "error", msg)
_ = w.svc.store.FailVariantJob(ctx, photoID, msg, maxVariantAttempts)
}
func (w *Worker) collectGarbage(ctx context.Context) {
objs, err := w.svc.store.ClaimGarbage(ctx, w.batch)
if err != nil {
slog.Error("claim garbage", "error", err)
return
}
for _, o := range objs {
w.deleteObject(ctx, o)
}
}
func (w *Worker) reapStalePending(ctx context.Context) {
objs, err := w.svc.store.ClaimStalePending(ctx, w.pendingTTL, w.batch)
if err != nil {
slog.Error("claim stale pending", "error", err)
return
}
for _, o := range objs {
w.deleteObject(ctx, o)
}
}
// deleteObject removes an object's blob then its row. If a binding appeared
// meanwhile the object is returned to ready; if the storage delete fails the
// row stays deleting and is retried on the next pass.
func (w *Worker) deleteObject(ctx context.Context, o Object) {
if n, err := w.svc.store.ObjectRefCount(ctx, o.ID); err == nil && n > 0 {
_ = w.svc.store.SetObjectState(ctx, o.ID, ObjectReady)
return
}
if err := w.svc.storage.Delete(ctx, o.StorageKey); err != nil {
slog.Warn("delete object blob", "object", o.ID, "error", err)
return
}
if err := w.svc.store.DeleteObjectRow(ctx, o.ID); err != nil && !errors.Is(err, ErrObjectNotFound) {
slog.Warn("delete object row", "object", o.ID, "error", err)
}
}
+92
View File
@@ -0,0 +1,92 @@
package photos
import (
"context"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"shared-album/internal/storage"
)
func TestWorkerGeneratesVariantsAndCollects(t *testing.T) {
pool := testPool(t)
svc, cl := newTestService(t, pool)
runWorkerFlow(t, pool, svc, cl)
}
func TestWorkerGeneratesVariantsWithFS(t *testing.T) {
pool := testPool(t)
svc, cl := newTestServiceFS(t, pool)
runWorkerFlow(t, pool, svc, cl)
}
func runWorkerFlow(t *testing.T, pool *pgxpool.Pool, svc *Service, cl storage.Backend) {
t.Helper()
ctx := context.Background()
user := createUser(t, ctx, pool)
album := createAlbum(t, ctx, pool, user)
p, err := svc.Upload(ctx, UploadInput{
AlbumID: album,
UserID: user,
Filename: "photo.png",
MimeType: "image/png",
Data: makePNG(t),
})
if err != nil {
t.Fatalf("upload: %v", err)
}
t.Cleanup(func() { cleanupPhoto(t, ctx, pool, cl, p.ID) })
w := NewWorker(svc, time.Second, 10, time.Minute)
w.processVariantJobs(ctx)
refreshed, err := svc.store.GetByID(ctx, p.ID, "")
if err != nil {
t.Fatalf("get refreshed: %v", err)
}
if refreshed.ThumbKey == "" || refreshed.PreviewKey == "" {
t.Fatalf("variants not bound: thumb=%q preview=%q", refreshed.ThumbKey, refreshed.PreviewKey)
}
if err := svc.attachURLs(ctx, refreshed); err != nil {
t.Fatalf("attach urls: %v", err)
}
if refreshed.ThumbURL == "" || refreshed.PreviewURL == "" {
t.Fatalf("variant urls not set")
}
var jobs int
if err := pool.QueryRow(ctx, `SELECT count(*) FROM variant_jobs WHERE photo_id = $1`, p.ID).Scan(&jobs); err != nil {
t.Fatalf("count jobs: %v", err)
}
if jobs != 0 {
t.Fatalf("variant job not completed: %d left", jobs)
}
objectIDs, err := svc.store.PhotoObjectIDs(ctx, p.ID)
if err != nil || len(objectIDs) != 3 {
t.Fatalf("expected 3 bound objects, got %v err=%v", objectIDs, err)
}
if err := svc.PermanentDelete(ctx, p.ID, user); err != nil {
t.Fatalf("permanent delete: %v", err)
}
// Run the collector until the objects are gone (bounded to avoid hangs).
for attempt := 0; attempt < 100 && countObjects(t, ctx, pool, objectIDs) > 0; attempt++ {
w.collectGarbage(ctx)
}
if n := countObjects(t, ctx, pool, objectIDs); n != 0 {
t.Fatalf("%d objects were not collected", n)
}
}
func countObjects(t *testing.T, ctx context.Context, pool *pgxpool.Pool, ids []string) int {
t.Helper()
var n int
if err := pool.QueryRow(ctx, `SELECT count(*) FROM objects WHERE id = ANY($1)`, ids).Scan(&n); err != nil {
t.Fatalf("count objects: %v", err)
}
return n
}
+74
View File
@@ -0,0 +1,74 @@
// Package reconcile computes the difference between the object storage and the
// metadata journal so orphaned blobs can be reported and, on request, removed.
package reconcile
import (
"sort"
"strings"
"time"
"shared-album/internal/storage"
)
// Result holds the two kinds of divergence between storage and the journal.
type Result struct {
// Orphans are keys present in storage but absent from the journal; they are
// candidates for deletion.
Orphans []string
// Missing are keys present in the journal but absent from storage; they are
// reported only and never modified.
Missing []string
}
// Classify compares the listed storage objects with the journaled keys.
func Classify(objects []storage.ObjectInfo, journaled []string) Result {
have := make(map[string]struct{}, len(journaled))
for _, k := range journaled {
have[k] = struct{}{}
}
seen := make(map[string]struct{}, len(objects))
var orphans []string
for _, o := range objects {
if _, dup := seen[o.Key]; dup {
continue
}
seen[o.Key] = struct{}{}
if _, ok := have[o.Key]; !ok {
orphans = append(orphans, o.Key)
}
}
var missing []string
for _, k := range journaled {
if _, ok := seen[k]; !ok {
missing = append(missing, k)
}
}
sort.Strings(orphans)
sort.Strings(missing)
return Result{Orphans: orphans, Missing: missing}
}
// Deletable selects orphaned keys that are safe to remove: within prefix and
// older than minAge (a guard against racing an in-flight upload whose row is
// not written yet).
func Deletable(objects []storage.ObjectInfo, res Result, prefix string, minAge time.Duration, now time.Time) []string {
orphans := make(map[string]struct{}, len(res.Orphans))
for _, k := range res.Orphans {
orphans[k] = struct{}{}
}
var out []string
for _, o := range objects {
if _, ok := orphans[o.Key]; !ok {
continue
}
if prefix != "" && !strings.HasPrefix(o.Key, prefix) {
continue
}
if now.Sub(o.ModTime) < minAge {
continue
}
out = append(out, o.Key)
}
sort.Strings(out)
return out
}
@@ -0,0 +1,41 @@
package reconcile
import (
"reflect"
"testing"
"time"
"shared-album/internal/storage"
)
func TestClassify(t *testing.T) {
now := time.Now()
objects := []storage.ObjectInfo{
{Key: "objects/a/original.jpg", ModTime: now},
{Key: "objects/b/original.jpg", ModTime: now},
}
journaled := []string{"objects/b/original.jpg", "objects/c/original.jpg"}
got := Classify(objects, journaled)
if !reflect.DeepEqual(got.Orphans, []string{"objects/a/original.jpg"}) {
t.Fatalf("orphans = %v", got.Orphans)
}
if !reflect.DeepEqual(got.Missing, []string{"objects/c/original.jpg"}) {
t.Fatalf("missing = %v", got.Missing)
}
}
func TestDeletableRespectsAgeAndPrefix(t *testing.T) {
now := time.Now()
objects := []storage.ObjectInfo{
{Key: "objects/old/original.jpg", ModTime: now.Add(-2 * time.Hour)},
{Key: "objects/new/original.jpg", ModTime: now.Add(-1 * time.Minute)},
{Key: "misc/other.bin", ModTime: now.Add(-2 * time.Hour)},
}
res := Result{Orphans: []string{"objects/old/original.jpg", "objects/new/original.jpg", "misc/other.bin"}}
got := Deletable(objects, res, "objects/", time.Hour, now)
if !reflect.DeepEqual(got, []string{"objects/old/original.jpg"}) {
t.Fatalf("deletable = %v, want only the old in-prefix orphan", got)
}
}
+560
View File
@@ -0,0 +1,560 @@
// Package contract holds a backend-agnostic test suite for the metadata
// repositories. The same suite runs against PostgreSQL and SQLite so both
// implementations are held to identical observable behaviour.
package contract
import (
"context"
"errors"
"testing"
"time"
"shared-album/internal/albums"
"shared-album/internal/auth"
"shared-album/internal/members"
"shared-album/internal/models"
"shared-album/internal/notifications"
"shared-album/internal/photos"
"shared-album/internal/settings"
"shared-album/internal/sharelinks"
"shared-album/internal/social"
)
// Repos bundles one repository per domain plus a Reset hook that clears all
// metadata so each subtest starts from an empty state.
type Repos struct {
Auth auth.Repository
Albums albums.Repository
Members members.Repository
Photos photos.Repository
Social social.Repository
Notifications notifications.Repository
ShareLinks sharelinks.Repository
Settings settings.Repository
Reset func(ctx context.Context) error
}
// Run executes the whole contract suite against the given repositories.
func Run(t *testing.T, r Repos) {
t.Helper()
cases := []struct {
name string
fn func(*testing.T, context.Context, Repos)
}{
{"Auth", testAuth},
{"Albums", testAlbums},
{"Members", testMembers},
{"Notifications", testNotifications},
{"ShareLinks", testShareLinks},
{"Social", testSocial},
{"Photos", testPhotos},
{"Objects", testObjects},
{"Settings", testSettings},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if err := r.Reset(context.Background()); err != nil {
t.Fatalf("reset: %v", err)
}
tc.fn(t, context.Background(), r)
})
}
}
func newID() string { return auth.NewID() }
func seedUser(t *testing.T, ctx context.Context, r Repos, name string) *models.User {
t.Helper()
u := &models.User{ID: newID(), Email: newID()[:10] + "@test.local", PasswordHash: "x", Name: name}
if err := r.Auth.CreateUser(ctx, u); err != nil {
t.Fatalf("create user: %v", err)
}
return u
}
func seedAlbum(t *testing.T, ctx context.Context, r Repos, ownerID string) *models.Album {
t.Helper()
a := &models.Album{ID: newID(), OwnerID: ownerID, Title: "Album"}
if err := r.Albums.Create(ctx, a); err != nil {
t.Fatalf("create album: %v", err)
}
return a
}
func seedPhoto(t *testing.T, ctx context.Context, r Repos, uploaderID, albumID string) *models.Photo {
t.Helper()
obj := &photos.Object{ID: newID(), StorageKey: "objects/" + newID(), State: photos.ObjectPending}
p := &models.Photo{ID: newID(), AlbumID: albumID, UploaderID: uploaderID, OriginalFilename: "p.jpg"}
if err := r.Photos.CreateWithOriginal(ctx, p, obj); err != nil {
t.Fatalf("create photo: %v", err)
}
if err := r.Photos.ActivateObject(ctx, obj.ID); err != nil {
t.Fatalf("activate object: %v", err)
}
return p
}
func testAuth(t *testing.T, ctx context.Context, r Repos) {
u := seedUser(t, ctx, r, "Tester")
byEmail, err := r.Auth.GetUserByEmail(ctx, u.Email)
if err != nil || byEmail.ID != u.ID {
t.Fatalf("get by email: %v id=%v", err, byEmail)
}
if _, err := r.Auth.GetUserByID(ctx, u.ID); err != nil {
t.Fatalf("get by id: %v", err)
}
if _, err := r.Auth.GetUserByEmail(ctx, "missing@test.local"); !errors.Is(err, auth.ErrNotFound) {
t.Fatalf("missing email err = %v, want ErrNotFound", err)
}
if err := r.Auth.UpdatePreferences(ctx, u.ID, []byte(`{"theme":"dark"}`)); err != nil {
t.Fatalf("update preferences: %v", err)
}
got, _ := r.Auth.GetUserByID(ctx, u.ID)
if got.Preferences["theme"] != "dark" {
t.Fatalf("preferences not applied: %+v", got.Preferences)
}
if err := r.Auth.UpdatePreferences(ctx, "missing", []byte(`{}`)); !errors.Is(err, auth.ErrNotFound) {
t.Fatalf("update missing prefs err = %v, want ErrNotFound", err)
}
hash := newID()
rt := &models.RefreshToken{ID: newID(), UserID: u.ID, TokenHash: hash, ExpiresAt: time.Now().Add(time.Hour)}
if err := r.Auth.CreateRefreshToken(ctx, rt); err != nil {
t.Fatalf("create refresh: %v", err)
}
fetched, err := r.Auth.GetRefreshToken(ctx, hash)
if err != nil || fetched.ID != rt.ID {
t.Fatalf("get refresh: %v id=%v", err, fetched)
}
if err := r.Auth.RevokeRefreshToken(ctx, rt.ID); err != nil {
t.Fatalf("revoke refresh: %v", err)
}
fetched, _ = r.Auth.GetRefreshToken(ctx, hash)
if fetched.RevokedAt == nil {
t.Fatalf("refresh not revoked")
}
if err := r.Auth.RevokeAllUserTokens(ctx, u.ID); err != nil {
t.Fatalf("revoke all: %v", err)
}
}
func testAlbums(t *testing.T, ctx context.Context, r Repos) {
owner := seedUser(t, ctx, r, "Owner")
a := seedAlbum(t, ctx, r, owner.ID)
if a.CreatedAt.IsZero() || a.UpdatedAt.IsZero() {
t.Fatalf("album timestamps not returned: %+v", a)
}
got, err := r.Albums.GetByID(ctx, a.ID)
if err != nil || got.Title != "Album" {
t.Fatalf("get: %v %+v", err, got)
}
if oid, err := r.Albums.OwnerID(ctx, a.ID); err != nil || oid != owner.ID {
t.Fatalf("owner id: %v %v", oid, err)
}
list, err := r.Albums.ListForUser(ctx, owner.ID)
if err != nil || len(list) != 1 {
t.Fatalf("list: %v len=%d", err, len(list))
}
a.Title = "Renamed"
if err := r.Albums.Update(ctx, a); err != nil {
t.Fatalf("update: %v", err)
}
got, _ = r.Albums.GetByID(ctx, a.ID)
if got.Title != "Renamed" {
t.Fatalf("update not applied: %q", got.Title)
}
if err := r.Albums.Delete(ctx, a.ID); err != nil {
t.Fatalf("delete: %v", err)
}
if _, err := r.Albums.GetByID(ctx, a.ID); !errors.Is(err, albums.ErrNotFound) {
t.Fatalf("after delete err = %v, want ErrNotFound", err)
}
}
func testMembers(t *testing.T, ctx context.Context, r Repos) {
owner := seedUser(t, ctx, r, "Owner")
member := seedUser(t, ctx, r, "Member")
a := seedAlbum(t, ctx, r, owner.ID)
if role, _ := r.Members.Role(ctx, a.ID, member.ID); role != "" {
t.Fatalf("role before add = %q", role)
}
if err := r.Members.Add(ctx, a.ID, member.ID, "member", &owner.ID); err != nil {
t.Fatalf("add: %v", err)
}
if role, err := r.Members.Role(ctx, a.ID, member.ID); err != nil || role != "member" {
t.Fatalf("role = %q err=%v", role, err)
}
list, err := r.Members.List(ctx, a.ID)
if err != nil || len(list) != 1 || list[0].User.Name != "Member" {
t.Fatalf("list: %v %+v", err, list)
}
if err := r.Members.UpdateRole(ctx, a.ID, member.ID, "editor"); err != nil {
t.Fatalf("update role: %v", err)
}
if role, _ := r.Members.Role(ctx, a.ID, member.ID); role != "editor" {
t.Fatalf("role not updated: %q", role)
}
inv := &models.Invite{ID: newID(), AlbumID: a.ID, Token: newID(), Role: "viewer", CreatedBy: owner.ID, ExpiresAt: time.Now().Add(time.Hour), MaxUses: 2}
if err := r.Members.CreateInvite(ctx, inv); err != nil {
t.Fatalf("create invite: %v", err)
}
if inv.CreatedAt.IsZero() {
t.Fatalf("invite created_at not returned")
}
got, err := r.Members.GetInviteByToken(ctx, inv.Token)
if err != nil || got.Role != "viewer" || got.MaxUses != 2 {
t.Fatalf("get invite: %v %+v", err, got)
}
if err := r.Members.IncrementUses(ctx, inv.ID); err != nil {
t.Fatalf("increment: %v", err)
}
got, _ = r.Members.GetInviteByToken(ctx, inv.Token)
if got.Uses != 1 {
t.Fatalf("uses = %d, want 1", got.Uses)
}
if _, err := r.Members.GetInviteByToken(ctx, "missing"); !errors.Is(err, members.ErrNotFound) {
t.Fatalf("missing invite err = %v, want ErrNotFound", err)
}
if err := r.Members.Remove(ctx, a.ID, member.ID); err != nil {
t.Fatalf("remove: %v", err)
}
if role, _ := r.Members.Role(ctx, a.ID, member.ID); role != "" {
t.Fatalf("member not removed: %q", role)
}
if err := r.Members.Remove(ctx, a.ID, member.ID); !errors.Is(err, members.ErrNotFound) {
t.Fatalf("remove missing err = %v, want ErrNotFound", err)
}
}
func testNotifications(t *testing.T, ctx context.Context, r Repos) {
u := seedUser(t, ctx, r, "Notify")
n := &models.Notification{ID: newID(), UserID: u.ID, Type: "test", PayloadJSON: map[string]any{"k": "v"}}
if err := r.Notifications.Create(ctx, n); err != nil {
t.Fatalf("create: %v", err)
}
if n.CreatedAt.IsZero() {
t.Fatalf("notification created_at not returned")
}
list, err := r.Notifications.List(ctx, u.ID, 10)
if err != nil || len(list) != 1 || list[0].PayloadJSON["k"] != "v" {
t.Fatalf("list: %v %+v", err, list)
}
if err := r.Notifications.MarkRead(ctx, n.ID, u.ID); err != nil {
t.Fatalf("mark read: %v", err)
}
list, _ = r.Notifications.List(ctx, u.ID, 10)
if list[0].ReadAt == nil {
t.Fatalf("notification not marked read")
}
if err := r.Notifications.MarkRead(ctx, newID(), u.ID); !errors.Is(err, notifications.ErrNotFound) {
t.Fatalf("mark missing err = %v, want ErrNotFound", err)
}
}
func testShareLinks(t *testing.T, ctx context.Context, r Repos) {
owner := seedUser(t, ctx, r, "Owner")
a := seedAlbum(t, ctx, r, owner.ID)
l := &models.ShareLink{ID: newID(), AlbumID: a.ID, Token: newID(), CreatedBy: owner.ID, ExpiresAt: time.Now().Add(time.Hour)}
if err := r.ShareLinks.Create(ctx, l); err != nil {
t.Fatalf("create: %v", err)
}
if l.CreatedAt.IsZero() {
t.Fatalf("share link created_at not returned")
}
got, err := r.ShareLinks.GetByToken(ctx, l.Token)
if err != nil || !got.Active(time.Now()) {
t.Fatalf("get token: %v %+v", err, got)
}
list, err := r.ShareLinks.ListByAlbum(ctx, a.ID)
if err != nil || len(list) != 1 {
t.Fatalf("list: %v len=%d", err, len(list))
}
if err := r.ShareLinks.Revoke(ctx, l.ID, a.ID); err != nil {
t.Fatalf("revoke: %v", err)
}
got, _ = r.ShareLinks.GetByToken(ctx, l.Token)
if got.Active(time.Now()) {
t.Fatalf("link still active after revoke")
}
if err := r.ShareLinks.Revoke(ctx, l.ID, a.ID); !errors.Is(err, sharelinks.ErrNotFound) {
t.Fatalf("revoke again err = %v, want ErrNotFound", err)
}
if _, err := r.ShareLinks.GetByToken(ctx, "missing"); !errors.Is(err, sharelinks.ErrNotFound) {
t.Fatalf("missing token err = %v, want ErrNotFound", err)
}
}
func testSocial(t *testing.T, ctx context.Context, r Repos) {
owner := seedUser(t, ctx, r, "Owner")
a := seedAlbum(t, ctx, r, owner.ID)
p := seedPhoto(t, ctx, r, owner.ID, a.ID)
if err := r.Social.Like(ctx, p.ID, owner.ID); err != nil {
t.Fatalf("like: %v", err)
}
if err := r.Social.Like(ctx, p.ID, owner.ID); err != nil {
t.Fatalf("like twice: %v", err)
}
if err := r.Social.Unlike(ctx, p.ID, owner.ID); err != nil {
t.Fatalf("unlike: %v", err)
}
c := &models.Comment{ID: newID(), PhotoID: p.ID, UserID: owner.ID, Text: "hi"}
if err := r.Social.CreateComment(ctx, c); err != nil {
t.Fatalf("create comment: %v", err)
}
if c.CreatedAt.IsZero() || c.UpdatedAt.IsZero() {
t.Fatalf("comment timestamps not returned: %+v", c)
}
list, err := r.Social.ListComments(ctx, p.ID)
if err != nil || len(list) != 1 || list[0].Text != "hi" {
t.Fatalf("list comments: %v %+v", err, list)
}
got, err := r.Social.GetComment(ctx, c.ID)
if err != nil || got.Text != "hi" {
t.Fatalf("get comment: %v %+v", err, got)
}
if err := r.Social.DeleteComment(ctx, c.ID); err != nil {
t.Fatalf("delete comment: %v", err)
}
if _, err := r.Social.GetComment(ctx, c.ID); !errors.Is(err, social.ErrNotFound) {
t.Fatalf("get deleted err = %v, want ErrNotFound", err)
}
if err := r.Social.DeleteComment(ctx, c.ID); !errors.Is(err, social.ErrNotFound) {
t.Fatalf("delete again err = %v, want ErrNotFound", err)
}
}
func testPhotos(t *testing.T, ctx context.Context, r Repos) {
owner := seedUser(t, ctx, r, "Owner")
a := seedAlbum(t, ctx, r, owner.ID)
p := seedPhoto(t, ctx, r, owner.ID, a.ID)
if p.CreatedAt.IsZero() {
t.Fatalf("photo created_at not returned")
}
got, err := r.Photos.GetByID(ctx, p.ID, "")
if err != nil || got.StorageKey == "" {
t.Fatalf("get photo: %v %+v", err, got)
}
list, err := r.Photos.List(ctx, photos.ListParams{AlbumID: a.ID, Limit: 10})
if err != nil || len(list) != 1 {
t.Fatalf("list: %v len=%d", err, len(list))
}
tag, err := r.Photos.EnsureTag(ctx, owner.ID, "Vacation")
if err != nil || tag.Name != "Vacation" {
t.Fatalf("ensure tag: %v %+v", err, tag)
}
tag2, err := r.Photos.EnsureTag(ctx, owner.ID, "vacation")
if err != nil || tag2.ID != tag.ID {
t.Fatalf("ensure tag case-insensitive: %v %+v", err, tag2)
}
if err := r.Photos.AttachTag(ctx, p.ID, tag.ID); err != nil {
t.Fatalf("attach tag: %v", err)
}
got, _ = r.Photos.GetByID(ctx, p.ID, owner.ID)
if len(got.Tags) != 1 || got.Tags[0].Name != "Vacation" {
t.Fatalf("photo tags = %+v", got.Tags)
}
if owned, _ := r.Photos.TagOwnedBy(ctx, tag.ID, owner.ID); !owned {
t.Fatalf("tag not owned")
}
tags, err := r.Photos.ListTags(ctx, owner.ID, "vac")
if err != nil || len(tags) != 1 {
t.Fatalf("list tags: %v len=%d", err, len(tags))
}
if err := r.Photos.DetachTag(ctx, p.ID, tag.ID); err != nil {
t.Fatalf("detach tag: %v", err)
}
r.Photos.SetCoverIfEmpty(ctx, a.ID, p.ID)
al, _ := r.Albums.GetByID(ctx, a.ID)
if al.CoverPhotoID == nil || *al.CoverPhotoID != p.ID {
t.Fatalf("cover not set: %+v", al.CoverPhotoID)
}
if err := r.Photos.SetDeleted(ctx, p.ID, true); err != nil {
t.Fatalf("soft delete: %v", err)
}
if n, _ := r.Photos.TrashCount(ctx, owner.ID); n != 1 {
t.Fatalf("trash count = %d, want 1", n)
}
trash, err := r.Photos.ListTrash(ctx, owner.ID)
if err != nil || len(trash) != 1 {
t.Fatalf("trash: %v len=%d", err, len(trash))
}
if err := r.Photos.RestoreToAlbum(ctx, p.ID, a.ID); err != nil {
t.Fatalf("restore: %v", err)
}
if n, _ := r.Photos.TrashCount(ctx, owner.ID); n != 0 {
t.Fatalf("trash count after restore = %d", n)
}
if err := r.Photos.UpdateFilename(ctx, p.ID, "renamed.jpg"); err != nil {
t.Fatalf("update filename: %v", err)
}
got, _ = r.Photos.GetByID(ctx, p.ID, "")
if got.OriginalFilename != "renamed.jpg" {
t.Fatalf("filename = %q", got.OriginalFilename)
}
if ok, _ := r.Photos.AlbumExists(ctx, a.ID); !ok {
t.Fatalf("album should exist")
}
if err := r.Photos.HardDelete(ctx, p.ID); err != nil {
t.Fatalf("hard delete: %v", err)
}
if _, err := r.Photos.GetByID(ctx, p.ID, ""); !errors.Is(err, photos.ErrNotFound) {
t.Fatalf("get deleted err = %v, want ErrNotFound", err)
}
}
func testObjects(t *testing.T, ctx context.Context, r Repos) {
owner := seedUser(t, ctx, r, "Owner")
a := seedAlbum(t, ctx, r, owner.ID)
p := seedPhoto(t, ctx, r, owner.ID, a.ID)
orig, err := r.Photos.OriginalObject(ctx, p.ID)
if err != nil || orig.State != photos.ObjectReady {
t.Fatalf("original object: %v %+v", err, orig)
}
keys, err := r.Photos.AllStorageKeys(ctx)
if err != nil || !contains(keys, orig.StorageKey) {
t.Fatalf("AllStorageKeys = %v err=%v, want %s", keys, err, orig.StorageKey)
}
if refs, _ := r.Photos.ObjectRefCount(ctx, orig.ID); refs != 1 {
t.Fatalf("refs = %d, want 1", refs)
}
sibs, _ := r.Photos.PhotosByOriginalObject(ctx, orig.ID)
if len(sibs) != 1 || sibs[0] != p.ID {
t.Fatalf("siblings = %+v", sibs)
}
thumb := &photos.Object{ID: newID(), StorageKey: "objects/" + newID(), MimeType: "image/webp", State: photos.ObjectPending}
if err := r.Photos.CreateObject(ctx, thumb); err != nil {
t.Fatalf("create object: %v", err)
}
if err := r.Photos.BindObject(ctx, p.ID, thumb.ID, photos.RoleThumb); !errors.Is(err, photos.ErrObjectNotReady) {
t.Fatalf("bind pending err = %v, want ErrObjectNotReady", err)
}
if err := r.Photos.ActivateObject(ctx, thumb.ID); err != nil {
t.Fatalf("activate: %v", err)
}
if err := r.Photos.BindObject(ctx, p.ID, thumb.ID, photos.RoleThumb); err != nil {
t.Fatalf("bind thumb: %v", err)
}
if refs, _ := r.Photos.ObjectRefCount(ctx, thumb.ID); refs != 1 {
t.Fatalf("thumb refs = %d, want 1", refs)
}
// After the photo is gone, its objects have no bindings and are collectible.
if err := r.Photos.HardDelete(ctx, p.ID); err != nil {
t.Fatalf("hard delete: %v", err)
}
claimed, err := r.Photos.ClaimGarbage(ctx, 10)
if err != nil {
t.Fatalf("claim garbage: %v", err)
}
got := map[string]bool{}
for _, o := range claimed {
got[o.ID] = true
}
if !got[orig.ID] || !got[thumb.ID] {
t.Fatalf("claimed = %+v, want original and thumb", got)
}
for _, o := range claimed {
if err := r.Photos.DeleteObjectRow(ctx, o.ID); err != nil {
t.Fatalf("delete object row: %v", err)
}
}
if again, _ := r.Photos.ClaimGarbage(ctx, 10); len(again) != 0 {
t.Fatalf("garbage reclaimed: %+v", again)
}
// Stale pending objects are reaped. A negative age puts the threshold in
// the future so the mechanism is exercised without depending on clock skew
// between the test process and the database.
stale := &photos.Object{ID: newID(), StorageKey: "objects/" + newID(), State: photos.ObjectPending}
if err := r.Photos.CreateObject(ctx, stale); err != nil {
t.Fatalf("create stale: %v", err)
}
staleClaimed, err := r.Photos.ClaimStalePending(ctx, -time.Hour, 10)
if err != nil || len(staleClaimed) == 0 {
t.Fatalf("claim stale: %v %+v", err, staleClaimed)
}
// Variant queue: a fresh photo has one job.
p2 := seedPhoto(t, ctx, r, owner.ID, a.ID)
jobs, err := r.Photos.ClaimVariantJobs(ctx, 10, time.Minute)
if err != nil {
t.Fatalf("claim jobs: %v", err)
}
if !contains(jobs, p2.ID) {
t.Fatalf("jobs = %+v, want %s", jobs, p2.ID)
}
if err := r.Photos.CompleteVariantJob(ctx, p2.ID); err != nil {
t.Fatalf("complete job: %v", err)
}
if jobs, _ = r.Photos.ClaimVariantJobs(ctx, 10, time.Minute); contains(jobs, p2.ID) {
t.Fatalf("completed job still claimed: %+v", jobs)
}
// Failure schedules a retry and eventually abandons the job.
p3 := seedPhoto(t, ctx, r, owner.ID, a.ID)
if err := r.Photos.FailVariantJob(ctx, p3.ID, "boom", 3); err != nil {
t.Fatalf("fail job: %v", err)
}
if jobs, _ = r.Photos.ClaimVariantJobs(ctx, 10, time.Minute); contains(jobs, p3.ID) {
t.Fatalf("failed job retried immediately: %+v", jobs)
}
for i := 0; i < 3; i++ {
if err := r.Photos.FailVariantJob(ctx, p3.ID, "boom", 3); err != nil {
t.Fatalf("fail job again: %v", err)
}
}
if jobs, _ = r.Photos.ClaimVariantJobs(ctx, 10, time.Minute); contains(jobs, p3.ID) {
t.Fatalf("abandoned job still present: %+v", jobs)
}
}
func testSettings(t *testing.T, ctx context.Context, r Repos) {
v, err := r.Settings.Ensure(ctx, settings.JWTSecretKey, func() (string, error) { return "generated-1", nil })
if err != nil || v != "generated-1" {
t.Fatalf("ensure first: %v %q", err, v)
}
v2, err := r.Settings.Ensure(ctx, settings.JWTSecretKey, func() (string, error) { return "generated-2", nil })
if err != nil || v2 != "generated-1" {
t.Fatalf("ensure not idempotent: %v %q", err, v2)
}
got, ok, err := r.Settings.Get(ctx, settings.JWTSecretKey)
if err != nil || !ok || got != "generated-1" {
t.Fatalf("get: %v ok=%v %q", err, ok, got)
}
if _, ok, _ := r.Settings.Get(ctx, "missing"); ok {
t.Fatalf("missing key reported present")
}
if err := r.Settings.Set(ctx, settings.JWTSecretKey, "rotated"); err != nil {
t.Fatalf("set: %v", err)
}
if got, _, _ := r.Settings.Get(ctx, settings.JWTSecretKey); got != "rotated" {
t.Fatalf("set not applied: %q", got)
}
}
func contains(list []string, v string) bool {
for _, x := range list {
if x == v {
return true
}
}
return false
}
@@ -0,0 +1,80 @@
package contract
import (
"context"
"fmt"
"io/fs"
"os"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
assets "shared-album"
"shared-album/internal/albums"
"shared-album/internal/auth"
"shared-album/internal/db"
"shared-album/internal/members"
"shared-album/internal/notifications"
"shared-album/internal/photos"
"shared-album/internal/settings"
"shared-album/internal/sharelinks"
"shared-album/internal/social"
)
// TestPostgresContract runs the shared suite against PostgreSQL inside its own
// schema, so it never clashes with other packages that use the same database.
func TestPostgresContract(t *testing.T) {
base := os.Getenv("TEST_DATABASE_URL")
if base == "" {
t.Skip("TEST_DATABASE_URL not set")
}
ctx := context.Background()
admin, err := pgxpool.New(ctx, base)
if err != nil {
t.Skipf("cannot connect: %v", err)
}
if err := admin.Ping(ctx); err != nil {
t.Skipf("cannot ping: %v", err)
}
t.Cleanup(admin.Close)
schema := fmt.Sprintf("ct_%d", time.Now().UnixNano())
if _, err := admin.Exec(ctx, "CREATE SCHEMA "+schema); err != nil {
t.Fatalf("create schema: %v", err)
}
t.Cleanup(func() {
_, _ = admin.Exec(context.Background(), "DROP SCHEMA "+schema+" CASCADE")
})
dsn := base + "&options=-c%20search_path%3D" + schema
migFS, err := fs.Sub(assets.Migrations, "migrations")
if err != nil {
t.Fatalf("migrations fs: %v", err)
}
if err := db.Migrate(dsn, migFS); err != nil {
t.Fatalf("migrate: %v", err)
}
pool, err := pgxpool.New(ctx, dsn)
if err != nil {
t.Fatalf("pool: %v", err)
}
t.Cleanup(pool.Close)
Run(t, Repos{
Auth: auth.NewStore(pool),
Albums: albums.NewStore(pool),
Members: members.NewStore(pool),
Photos: photos.NewStore(pool),
Social: social.NewStore(pool),
Notifications: notifications.NewStore(pool),
ShareLinks: sharelinks.NewStore(pool),
Settings: settings.NewStore(pool),
Reset: func(ctx context.Context) error {
_, err := pool.Exec(ctx, `TRUNCATE users, objects, app_settings CASCADE`)
return err
},
})
}
@@ -0,0 +1,59 @@
package contract
import (
"context"
"io/fs"
"path/filepath"
"testing"
assets "shared-album"
"shared-album/internal/albums"
"shared-album/internal/auth"
"shared-album/internal/db"
"shared-album/internal/members"
"shared-album/internal/notifications"
"shared-album/internal/photos"
"shared-album/internal/settings"
"shared-album/internal/sharelinks"
"shared-album/internal/social"
)
func TestSQLiteContract(t *testing.T) {
path := filepath.Join(t.TempDir(), "meta.db")
migFS, err := fs.Sub(assets.Migrations, "migrations/sqlite")
if err != nil {
t.Fatalf("migrations fs: %v", err)
}
if err := db.MigrateSQLite(path, migFS); err != nil {
t.Fatalf("migrate: %v", err)
}
database, err := db.OpenSQLite(path)
if err != nil {
t.Fatalf("open: %v", err)
}
t.Cleanup(func() { database.Close() })
Run(t, Repos{
Auth: auth.NewSQLiteStore(database),
Albums: albums.NewSQLiteStore(database),
Members: members.NewSQLiteStore(database),
Photos: photos.NewSQLiteStore(database),
Social: social.NewSQLiteStore(database),
Notifications: notifications.NewSQLiteStore(database),
ShareLinks: sharelinks.NewSQLiteStore(database),
Settings: settings.NewSQLiteStore(database),
Reset: func(ctx context.Context) error {
tables := []string{
"variant_jobs", "photo_objects", "objects", "photo_tags", "tags",
"refresh_tokens", "share_links", "notifications", "likes", "comments",
"invites", "album_members", "photos", "albums", "users", "app_settings",
}
for _, tbl := range tables {
if _, err := database.ExecContext(ctx, "DELETE FROM "+tbl); err != nil {
return err
}
}
return nil
},
})
}
+25 -14
View File
@@ -29,6 +29,7 @@ func testPool(t *testing.T) *pgxpool.Pool {
if err := pool.Ping(context.Background()); err != nil {
t.Skipf("cannot ping: %v", err)
}
t.Cleanup(pool.Close)
return pool
}
@@ -48,7 +49,6 @@ func newUser(t *testing.T, ctx context.Context, pool *pgxpool.Pool, name string)
func TestUserRepository(t *testing.T) {
pool := testPool(t)
defer pool.Close()
ctx := context.Background()
store := auth.NewStore(pool)
@@ -82,7 +82,6 @@ func TestUserRepository(t *testing.T) {
func TestAlbumRepository(t *testing.T) {
pool := testPool(t)
defer pool.Close()
ctx := context.Background()
store := albums.NewStore(pool)
@@ -126,7 +125,6 @@ func TestAlbumRepository(t *testing.T) {
func TestAlbumCoverRepository(t *testing.T) {
pool := testPool(t)
defer pool.Close()
ctx := context.Background()
store := albums.NewStore(pool)
@@ -138,18 +136,33 @@ func TestAlbumCoverRepository(t *testing.T) {
t.Fatalf("create album: %v", err)
}
insertPhoto := func(id, thumbKey string) {
insertPhoto := func(id string) string {
t.Helper()
origObj := auth.NewID()
thumbObj := auth.NewID()
thumbKey := "objects/" + thumbObj + "/thumb.webp"
if _, err := pool.Exec(ctx,
`INSERT INTO photos (id, album_id, uploader_id, storage_key, thumb_key)
VALUES ($1, $2, $3, $4, $5)`,
id, a.ID, owner.ID, "albums/"+a.ID+"/original/"+id+".jpg", thumbKey); err != nil {
`INSERT INTO objects (id, storage_key, mime_type, state) VALUES
($1, $2, 'image/jpeg', 'ready'),
($3, $4, 'image/webp', 'ready')`,
origObj, "objects/"+origObj+"/original.jpg", thumbObj, thumbKey); err != nil {
t.Fatalf("insert objects: %v", err)
}
if _, err := pool.Exec(ctx,
`INSERT INTO photos (id, album_id, uploader_id) VALUES ($1, $2, $3)`,
id, a.ID, owner.ID); err != nil {
t.Fatalf("insert photo: %v", err)
}
if _, err := pool.Exec(ctx,
`INSERT INTO photo_objects (photo_id, object_id, role) VALUES ($1, $2, 'original'), ($1, $3, 'thumb')`,
id, origObj, thumbObj); err != nil {
t.Fatalf("insert bindings: %v", err)
}
return thumbKey
}
photo1 := auth.NewID()
insertPhoto(photo1, "albums/"+a.ID+"/thumb/"+photo1+".webp")
thumb1 := insertPhoto(photo1)
got, err := store.GetByID(ctx, a.ID)
if err != nil || got.CoverThumbKey != "" {
@@ -162,12 +175,12 @@ func TestAlbumCoverRepository(t *testing.T) {
if got.CoverPhotoID == nil || *got.CoverPhotoID != photo1 {
t.Fatalf("cover not set: %+v", got.CoverPhotoID)
}
if want := "albums/" + a.ID + "/thumb/" + photo1 + ".webp"; got.CoverThumbKey != want {
t.Fatalf("cover thumb key = %q want %q", got.CoverThumbKey, want)
if got.CoverThumbKey != thumb1 {
t.Fatalf("cover thumb key = %q want %q", got.CoverThumbKey, thumb1)
}
photo2 := auth.NewID()
insertPhoto(photo2, "thumb-2")
thumb2 := insertPhoto(photo2)
ps.SetCoverIfEmpty(ctx, a.ID, photo2)
got, _ = store.GetByID(ctx, a.ID)
if got.CoverPhotoID == nil || *got.CoverPhotoID != photo1 {
@@ -184,14 +197,13 @@ func TestAlbumCoverRepository(t *testing.T) {
ps.SetCoverIfEmpty(ctx, a.ID, photo2)
list, err := store.ListForUser(ctx, owner.ID)
if err != nil || len(list) != 1 || list[0].CoverThumbKey != "thumb-2" {
if err != nil || len(list) != 1 || list[0].CoverThumbKey != thumb2 {
t.Fatalf("list cover key: %v %+v", err, list)
}
}
func TestMemberRepository(t *testing.T) {
pool := testPool(t)
defer pool.Close()
ctx := context.Background()
owner := newUser(t, ctx, pool, "Owner")
@@ -255,7 +267,6 @@ func TestMemberRepository(t *testing.T) {
func TestShareLinkRepository(t *testing.T) {
pool := testPool(t)
defer pool.Close()
ctx := context.Background()
owner := newUser(t, ctx, pool, "Owner")
+15
View File
@@ -0,0 +1,15 @@
package settings
import (
"crypto/rand"
"encoding/base64"
)
// GenerateSecret returns a new random secret suitable for HMAC signing.
func GenerateSecret() (string, error) {
buf := make([]byte, 32)
if _, err := rand.Read(buf); err != nil {
return "", err
}
return base64.RawURLEncoding.EncodeToString(buf), nil
}
+20
View File
@@ -0,0 +1,20 @@
// Package settings persists application-level key/value settings in the
// metadata layer. The main use is the token signing secret, generated on first
// use and reused afterwards.
package settings
import "context"
// JWTSecretKey is the settings key holding the token signing secret.
const JWTSecretKey = "jwt_secret"
// Repository reads and writes application settings.
type Repository interface {
// Get returns the value for key and whether it exists.
Get(ctx context.Context, key string) (string, bool, error)
// Set stores or replaces the value for key.
Set(ctx context.Context, key, value string) error
// Ensure returns the existing value for key, or generates and stores one
// using generate when absent. Concurrent first use yields one value.
Ensure(ctx context.Context, key string, generate func() (string, error)) (string, error)
}
+63
View File
@@ -0,0 +1,63 @@
package settings
import (
"context"
"database/sql"
"errors"
"fmt"
"time"
"shared-album/internal/sqliteutil"
)
// SQLiteStore is the SQLite implementation of Repository.
type SQLiteStore struct {
db *sql.DB
}
func NewSQLiteStore(db *sql.DB) *SQLiteStore { return &SQLiteStore{db: db} }
func (s *SQLiteStore) Get(ctx context.Context, key string) (string, bool, error) {
var v string
err := s.db.QueryRowContext(ctx, `SELECT value FROM app_settings WHERE key = $1`, key).Scan(&v)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return "", false, nil
}
return "", false, err
}
return v, true, nil
}
func (s *SQLiteStore) Set(ctx context.Context, key, value string) error {
_, err := s.db.ExecContext(ctx,
`INSERT INTO app_settings (key, value) VALUES ($1, $2)
ON CONFLICT (key) DO UPDATE SET value = excluded.value, updated_at = $3`,
key, value, sqliteutil.FormatTime(time.Now()))
return err
}
func (s *SQLiteStore) Ensure(ctx context.Context, key string, generate func() (string, error)) (string, error) {
if v, ok, err := s.Get(ctx, key); err != nil {
return "", err
} else if ok {
return v, nil
}
v, err := generate()
if err != nil {
return "", err
}
if _, err := s.db.ExecContext(ctx,
`INSERT INTO app_settings (key, value) VALUES ($1, $2) ON CONFLICT (key) DO NOTHING`,
key, v); err != nil {
return "", err
}
got, ok, err := s.Get(ctx, key)
if err != nil {
return "", err
}
if !ok {
return "", fmt.Errorf("settings: %q missing after insert", key)
}
return got, nil
}
+62
View File
@@ -0,0 +1,62 @@
package settings
import (
"context"
"errors"
"fmt"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
)
// Store is the PostgreSQL implementation of Repository.
type Store struct {
pool *pgxpool.Pool
}
func NewStore(pool *pgxpool.Pool) *Store { return &Store{pool: pool} }
func (s *Store) Get(ctx context.Context, key string) (string, bool, error) {
var v string
err := s.pool.QueryRow(ctx, `SELECT value FROM app_settings WHERE key = $1`, key).Scan(&v)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return "", false, nil
}
return "", false, err
}
return v, true, nil
}
func (s *Store) Set(ctx context.Context, key, value string) error {
_, err := s.pool.Exec(ctx,
`INSERT INTO app_settings (key, value) VALUES ($1, $2)
ON CONFLICT (key) DO UPDATE SET value = EXCLUDED.value, updated_at = now()`,
key, value)
return err
}
func (s *Store) Ensure(ctx context.Context, key string, generate func() (string, error)) (string, error) {
if v, ok, err := s.Get(ctx, key); err != nil {
return "", err
} else if ok {
return v, nil
}
v, err := generate()
if err != nil {
return "", err
}
if _, err := s.pool.Exec(ctx,
`INSERT INTO app_settings (key, value) VALUES ($1, $2) ON CONFLICT (key) DO NOTHING`,
key, v); err != nil {
return "", err
}
got, ok, err := s.Get(ctx, key)
if err != nil {
return "", err
}
if !ok {
return "", fmt.Errorf("settings: %q missing after insert", key)
}
return got, nil
}
+15
View File
@@ -0,0 +1,15 @@
package sharelinks
import (
"context"
"shared-album/internal/models"
)
// Repository is the persistence contract for the share-links domain.
type Repository interface {
Create(ctx context.Context, l *models.ShareLink) error
ListByAlbum(ctx context.Context, albumID string) ([]models.ShareLink, error)
GetByToken(ctx context.Context, token string) (*models.ShareLink, error)
Revoke(ctx context.Context, id, albumID string) error
}
+2 -2
View File
@@ -16,11 +16,11 @@ type AlbumAccess interface {
}
type Service struct {
store *Store
store Repository
albums AlbumAccess
}
func NewService(store *Store, albums AlbumAccess) *Service {
func NewService(store Repository, albums AlbumAccess) *Service {
return &Service{store: store, albums: albums}
}
@@ -0,0 +1,90 @@
package sharelinks
import (
"context"
"database/sql"
"errors"
"time"
"shared-album/internal/models"
"shared-album/internal/sqliteutil"
)
// SQLiteStore is the SQLite implementation of Repository.
type SQLiteStore struct {
db *sql.DB
}
func NewSQLiteStore(db *sql.DB) *SQLiteStore { return &SQLiteStore{db: db} }
func (s *SQLiteStore) Create(ctx context.Context, l *models.ShareLink) error {
var createdAt string
err := s.db.QueryRowContext(ctx,
`INSERT INTO share_links (id, album_id, token, created_by, expires_at)
VALUES ($1, $2, $3, $4, $5)
RETURNING created_at`,
l.ID, l.AlbumID, l.Token, l.CreatedBy, sqliteutil.FormatTime(l.ExpiresAt),
).Scan(&createdAt)
if err != nil {
return err
}
l.CreatedAt = sqliteutil.ParseTime(createdAt)
return nil
}
func (s *SQLiteStore) ListByAlbum(ctx context.Context, albumID string) ([]models.ShareLink, error) {
rows, err := s.db.QueryContext(ctx,
`SELECT id, album_id, token, created_by, created_at, expires_at, revoked_at
FROM share_links WHERE album_id = $1 ORDER BY created_at DESC`, albumID)
if err != nil {
return nil, err
}
defer rows.Close()
out := []models.ShareLink{}
for rows.Next() {
var l models.ShareLink
var createdAt, expiresAt string
var revokedAt sql.NullString
if err := rows.Scan(&l.ID, &l.AlbumID, &l.Token, &l.CreatedBy, &createdAt, &expiresAt, &revokedAt); err != nil {
return nil, err
}
l.CreatedAt = sqliteutil.ParseTime(createdAt)
l.ExpiresAt = sqliteutil.ParseTime(expiresAt)
l.RevokedAt = sqliteutil.ParseNullTime(revokedAt)
out = append(out, l)
}
return out, rows.Err()
}
func (s *SQLiteStore) GetByToken(ctx context.Context, token string) (*models.ShareLink, error) {
var l models.ShareLink
var createdAt, expiresAt string
var revokedAt sql.NullString
err := s.db.QueryRowContext(ctx,
`SELECT id, album_id, token, created_by, created_at, expires_at, revoked_at
FROM share_links WHERE token = $1`, token).
Scan(&l.ID, &l.AlbumID, &l.Token, &l.CreatedBy, &createdAt, &expiresAt, &revokedAt)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return nil, ErrNotFound
}
return nil, err
}
l.CreatedAt = sqliteutil.ParseTime(createdAt)
l.ExpiresAt = sqliteutil.ParseTime(expiresAt)
l.RevokedAt = sqliteutil.ParseNullTime(revokedAt)
return &l, nil
}
func (s *SQLiteStore) Revoke(ctx context.Context, id, albumID string) error {
res, err := s.db.ExecContext(ctx,
`UPDATE share_links SET revoked_at = $3 WHERE id = $1 AND album_id = $2 AND revoked_at IS NULL`,
id, albumID, sqliteutil.FormatTime(time.Now()))
if err != nil {
return err
}
if n, _ := res.RowsAffected(); n == 0 {
return ErrNotFound
}
return nil
}
+17
View File
@@ -0,0 +1,17 @@
package social
import (
"context"
"shared-album/internal/models"
)
// Repository is the persistence contract for the social domain.
type Repository interface {
Like(ctx context.Context, photoID, userID string) error
Unlike(ctx context.Context, photoID, userID string) error
CreateComment(ctx context.Context, c *models.Comment) error
ListComments(ctx context.Context, photoID string) ([]models.Comment, error)
GetComment(ctx context.Context, id string) (*models.Comment, error)
DeleteComment(ctx context.Context, id string) error
}
+2 -2
View File
@@ -25,14 +25,14 @@ type Notifier interface {
}
type Service struct {
store *Store
store Repository
albums AlbumAccess
photos PhotoLookup
hub *sse.Hub
notify Notifier
}
func NewService(store *Store, albums AlbumAccess, photos PhotoLookup, hub *sse.Hub, notify Notifier) *Service {
func NewService(store Repository, albums AlbumAccess, photos PhotoLookup, hub *sse.Hub, notify Notifier) *Service {
return &Service{store: store, albums: albums, photos: photos, hub: hub, notify: notify}
}
+101
View File
@@ -0,0 +1,101 @@
package social
import (
"context"
"database/sql"
"errors"
"shared-album/internal/models"
"shared-album/internal/sqliteutil"
)
// SQLiteStore is the SQLite implementation of Repository.
type SQLiteStore struct {
db *sql.DB
}
func NewSQLiteStore(db *sql.DB) *SQLiteStore { return &SQLiteStore{db: db} }
func (s *SQLiteStore) Like(ctx context.Context, photoID, userID string) error {
_, err := s.db.ExecContext(ctx,
`INSERT INTO likes (photo_id, user_id) VALUES ($1, $2)
ON CONFLICT (photo_id, user_id) DO NOTHING`, photoID, userID)
return err
}
func (s *SQLiteStore) Unlike(ctx context.Context, photoID, userID string) error {
_, err := s.db.ExecContext(ctx,
`DELETE FROM likes WHERE photo_id = $1 AND user_id = $2`, photoID, userID)
return err
}
func (s *SQLiteStore) CreateComment(ctx context.Context, c *models.Comment) error {
var createdAt, updatedAt string
err := s.db.QueryRowContext(ctx,
`INSERT INTO comments (id, photo_id, user_id, text)
VALUES ($1, $2, $3, $4)
RETURNING created_at, updated_at`,
c.ID, c.PhotoID, c.UserID, c.Text,
).Scan(&createdAt, &updatedAt)
if err != nil {
return err
}
c.CreatedAt = sqliteutil.ParseTime(createdAt)
c.UpdatedAt = sqliteutil.ParseTime(updatedAt)
return nil
}
func (s *SQLiteStore) ListComments(ctx context.Context, photoID string) ([]models.Comment, error) {
rows, err := s.db.QueryContext(ctx,
`SELECT c.id, c.photo_id, c.user_id, c.text, c.created_at, c.updated_at,
u.id, u.name, COALESCE(u.avatar_url, '')
FROM comments c
JOIN users u ON u.id = c.user_id
WHERE c.photo_id = $1
ORDER BY c.created_at`, photoID)
if err != nil {
return nil, err
}
defer rows.Close()
out := []models.Comment{}
for rows.Next() {
var c models.Comment
var createdAt, updatedAt string
if err := rows.Scan(&c.ID, &c.PhotoID, &c.UserID, &c.Text, &createdAt, &updatedAt,
&c.User.ID, &c.User.Name, &c.User.AvatarURL); err != nil {
return nil, err
}
c.CreatedAt = sqliteutil.ParseTime(createdAt)
c.UpdatedAt = sqliteutil.ParseTime(updatedAt)
out = append(out, c)
}
return out, rows.Err()
}
func (s *SQLiteStore) GetComment(ctx context.Context, id string) (*models.Comment, error) {
var c models.Comment
var createdAt, updatedAt string
err := s.db.QueryRowContext(ctx,
`SELECT id, photo_id, user_id, text, created_at, updated_at FROM comments WHERE id = $1`, id).
Scan(&c.ID, &c.PhotoID, &c.UserID, &c.Text, &createdAt, &updatedAt)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return nil, ErrNotFound
}
return nil, err
}
c.CreatedAt = sqliteutil.ParseTime(createdAt)
c.UpdatedAt = sqliteutil.ParseTime(updatedAt)
return &c, nil
}
func (s *SQLiteStore) DeleteComment(ctx context.Context, id string) error {
res, err := s.db.ExecContext(ctx, `DELETE FROM comments WHERE id = $1`, id)
if err != nil {
return err
}
if n, _ := res.RowsAffected(); n == 0 {
return ErrNotFound
}
return nil
}
+50
View File
@@ -0,0 +1,50 @@
// Package sqliteutil holds small helpers shared by the SQLite repositories:
// timestamp conversion and dynamic IN-lists.
package sqliteutil
import (
"database/sql"
"strconv"
"strings"
"time"
)
// TimeLayout matches the SQLite column defaults produced by strftime with %f.
const TimeLayout = "2006-01-02T15:04:05.000Z"
// FormatTime renders a time in the canonical TEXT format used by the schema.
func FormatTime(t time.Time) string { return t.UTC().Format(TimeLayout) }
// ParseTime parses a canonical timestamp, tolerating the RFC3339 variants the
// driver may return. An empty or unparseable value yields the zero time.
func ParseTime(s string) time.Time {
if s == "" {
return time.Time{}
}
if t, err := time.Parse(TimeLayout, s); err == nil {
return t
}
if t, err := time.Parse(time.RFC3339Nano, s); err == nil {
return t
}
return time.Time{}
}
// ParseNullTime converts a nullable TEXT timestamp into *time.Time.
func ParseNullTime(ns sql.NullString) *time.Time {
if !ns.Valid || ns.String == "" {
return nil
}
t := ParseTime(ns.String)
return &t
}
// Placeholders returns an IN-list placeholder group of n items starting at
// index start, e.g. Placeholders(3, 2) -> "$3,$4".
func Placeholders(start, n int) string {
parts := make([]string, n)
for i := 0; i < n; i++ {
parts[i] = "$" + strconv.Itoa(start+i)
}
return strings.Join(parts, ",")
}
+233
View File
@@ -0,0 +1,233 @@
package storage
import (
"context"
"errors"
"fmt"
"io"
"io/fs"
"mime"
"net/http"
"net/url"
"os"
"path/filepath"
"strings"
)
// errInvalidKey is returned for object keys that are empty, absolute or attempt
// to escape the storage root.
var errInvalidKey = errors.New("fs: invalid object key")
// FS is a Backend backed by a local directory tree. Blobs live under Root at the
// path formed by the opaque key. Because a local directory has no external URL,
// FS serves blobs through the app using the shared Signer.
type FS struct {
root string
signer *Signer
}
// NewFS builds a local file-system storage backend rooted at root. signer issues
// and validates the app URLs used to serve blobs.
func NewFS(root string, signer *Signer) (*FS, error) {
if strings.TrimSpace(root) == "" {
return nil, errors.New("fs: root is required")
}
if signer == nil {
return nil, errors.New("fs: signer is required")
}
abs, err := filepath.Abs(root)
if err != nil {
return nil, fmt.Errorf("fs: resolve root: %w", err)
}
if err := os.MkdirAll(abs, 0o755); err != nil {
return nil, fmt.Errorf("fs: create root: %w", err)
}
return &FS{root: abs, signer: signer}, nil
}
func (f *FS) Bucket() string { return f.root }
func (f *FS) EnsureBucket(context.Context) error {
return os.MkdirAll(f.root, 0o755)
}
func (f *FS) Ping(context.Context) error {
info, err := os.Stat(f.root)
if err != nil {
return err
}
if !info.IsDir() {
return errors.New("fs: root is not a directory")
}
return nil
}
// objectPath maps an opaque key to an absolute path inside the root, rejecting
// absolute keys and any key that resolves outside the root.
func (f *FS) objectPath(key string) (string, error) {
if key == "" || filepath.IsAbs(key) {
return "", errInvalidKey
}
clean := filepath.Clean(key)
if clean == ".." || strings.HasPrefix(clean, ".."+string(os.PathSeparator)) {
return "", errInvalidKey
}
p := filepath.Join(f.root, clean)
rel, err := filepath.Rel(f.root, p)
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(os.PathSeparator)) {
return "", errInvalidKey
}
return p, nil
}
func (f *FS) Put(_ context.Context, key string, body io.Reader, _ int64, _ string) error {
p, err := f.objectPath(key)
if err != nil {
return err
}
if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
return fmt.Errorf("fs: mkdir: %w", err)
}
tmp, err := os.CreateTemp(filepath.Dir(p), ".upload-*")
if err != nil {
return fmt.Errorf("fs: temp file: %w", err)
}
tmpName := tmp.Name()
defer func() { _ = os.Remove(tmpName) }()
if _, err := io.Copy(tmp, body); err != nil {
_ = tmp.Close()
return fmt.Errorf("fs: write: %w", err)
}
if err := tmp.Close(); err != nil {
return fmt.Errorf("fs: close: %w", err)
}
if err := os.Rename(tmpName, p); err != nil {
return fmt.Errorf("fs: rename: %w", err)
}
return nil
}
func (f *FS) Get(_ context.Context, key string) ([]byte, error) {
p, err := f.objectPath(key)
if err != nil {
return nil, err
}
data, err := os.ReadFile(p)
if err != nil {
if errors.Is(err, os.ErrNotExist) {
return nil, ErrObjectNotFound
}
return nil, err
}
return data, nil
}
func (f *FS) Delete(_ context.Context, keys ...string) error {
var errs []error
for _, key := range keys {
p, err := f.objectPath(key)
if err != nil {
errs = append(errs, err)
continue
}
if err := os.Remove(p); err != nil && !errors.Is(err, os.ErrNotExist) {
errs = append(errs, err)
}
}
return errors.Join(errs...)
}
// URL returns a signed app URL served by this process.
func (f *FS) URL(_ context.Context, key string) (string, error) {
if _, err := f.objectPath(key); err != nil {
return "", err
}
return f.signer.URL(key), nil
}
// List returns every blob under prefix as a key relative to the storage root.
func (f *FS) List(_ context.Context, prefix string) ([]ObjectInfo, error) {
var out []ObjectInfo
err := filepath.WalkDir(f.root, func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() {
return nil
}
rel, err := filepath.Rel(f.root, path)
if err != nil {
return err
}
rel = filepath.ToSlash(rel)
if prefix != "" && !strings.HasPrefix(rel, prefix) {
return nil
}
info, err := d.Info()
if err != nil {
return err
}
out = append(out, ObjectInfo{Key: rel, Size: info.Size(), ModTime: info.ModTime()})
return nil
})
if err != nil {
return nil, err
}
return out, nil
}
// ServeObject validates the signed URL and serves the blob. It supports range
// requests and sets the content type from the key extension. It writes the
// response itself and returns nil on success.
func (f *FS) ServeObject(w http.ResponseWriter, r *http.Request, key string) error {
if unescaped, err := url.PathUnescape(key); err == nil {
key = unescaped
}
if err := f.signer.VerifyRequest(key, r); err != nil {
return err
}
p, err := f.objectPath(key)
if err != nil {
return ErrSignedURLInvalid
}
file, err := os.Open(p)
if err != nil {
if errors.Is(err, os.ErrNotExist) {
return ErrObjectNotFound
}
return err
}
defer file.Close()
info, err := file.Stat()
if err != nil {
return err
}
w.Header().Set("Content-Type", contentTypeForKey(key))
http.ServeContent(w, r, info.Name(), info.ModTime(), file)
return nil
}
var imageContentTypes = map[string]string{
".jpg": "image/jpeg",
".jpeg": "image/jpeg",
".png": "image/png",
".gif": "image/gif",
".webp": "image/webp",
".heic": "image/heic",
".heif": "image/heif",
".tiff": "image/tiff",
".tif": "image/tiff",
".bmp": "image/bmp",
".avif": "image/avif",
}
func contentTypeForKey(key string) string {
ext := strings.ToLower(filepath.Ext(key))
if ct, ok := imageContentTypes[ext]; ok {
return ct
}
if ct := mime.TypeByExtension(ext); ct != "" {
return ct
}
return "application/octet-stream"
}
+204
View File
@@ -0,0 +1,204 @@
package storage
import (
"bytes"
"context"
"errors"
"io"
"net/http"
"net/http/httptest"
"net/url"
"strconv"
"strings"
"testing"
"time"
)
func newTestFS(t *testing.T) *FS {
t.Helper()
signer, err := NewSigner("http://app.test", "test-secret", time.Hour)
if err != nil {
t.Fatalf("signer: %v", err)
}
fs, err := NewFS(t.TempDir(), signer)
if err != nil {
t.Fatalf("new fs: %v", err)
}
return fs
}
func TestFSRoundTrip(t *testing.T) {
ctx := context.Background()
fs := newTestFS(t)
key := "objects/abc/original.jpg"
data := []byte("hello blob")
if err := fs.Put(ctx, key, bytes.NewReader(data), int64(len(data)), "image/jpeg"); err != nil {
t.Fatalf("put: %v", err)
}
got, err := fs.Get(ctx, key)
if err != nil {
t.Fatalf("get: %v", err)
}
if !bytes.Equal(got, data) {
t.Fatalf("get = %q, want %q", got, data)
}
if err := fs.Delete(ctx, key); err != nil {
t.Fatalf("delete: %v", err)
}
if _, err := fs.Get(ctx, key); !errors.Is(err, ErrObjectNotFound) {
t.Fatalf("get after delete = %v, want ErrObjectNotFound", err)
}
}
func TestFSPathTraversalRejected(t *testing.T) {
ctx := context.Background()
fs := newTestFS(t)
for _, key := range []string{"../escape", "../../etc/passwd", "/etc/passwd", ""} {
if err := fs.Put(ctx, key, strings.NewReader("x"), 1, "text/plain"); err == nil {
t.Errorf("put %q: expected rejection", key)
}
if _, err := fs.Get(ctx, key); err == nil {
t.Errorf("get %q: expected rejection", key)
}
if _, err := fs.URL(ctx, key); err == nil {
t.Errorf("url %q: expected rejection", key)
}
}
}
func TestFSURLAndServe(t *testing.T) {
ctx := context.Background()
fs := newTestFS(t)
key := "objects/abc/thumb.webp"
if err := fs.Put(ctx, key, strings.NewReader("webp-bytes"), 10, "image/webp"); err != nil {
t.Fatalf("put: %v", err)
}
raw, err := fs.URL(ctx, key)
if err != nil {
t.Fatalf("url: %v", err)
}
u, err := url.Parse(raw)
if err != nil {
t.Fatalf("parse url: %v", err)
}
if !strings.HasPrefix(u.Path, "/api/objects/objects/abc/thumb.webp") {
t.Fatalf("unexpected path: %s", u.Path)
}
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, raw, nil)
if err := fs.ServeObject(rec, req, strings.TrimPrefix(u.Path, "/api/objects/")); err != nil {
t.Fatalf("serve: %v", err)
}
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200", rec.Code)
}
if ct := rec.Header().Get("Content-Type"); ct != "image/webp" {
t.Fatalf("content-type = %q, want image/webp", ct)
}
if rec.Body.String() != "webp-bytes" {
t.Fatalf("body = %q", rec.Body.String())
}
}
func TestFSServeExpiredAndForged(t *testing.T) {
ctx := context.Background()
fs := newTestFS(t)
key := "objects/abc/original.jpg"
if err := fs.Put(ctx, key, strings.NewReader("x"), 1, "image/jpeg"); err != nil {
t.Fatalf("put: %v", err)
}
past := time.Now().Add(-time.Minute).Unix()
expired := "/api/objects/" + key + "?e=" + strconv.FormatInt(past, 10) + "&s=" + fs.signer.sign(key, past)
rec := httptest.NewRecorder()
err := fs.ServeObject(rec, httptest.NewRequest(http.MethodGet, expired, nil), key)
if !errors.Is(err, ErrSignedURLInvalid) {
t.Fatalf("expired = %v, want ErrSignedURLInvalid", err)
}
future := time.Now().Add(time.Hour).Unix()
forged := "/api/objects/" + key + "?e=" + strconv.FormatInt(future, 10) + "&s=deadbeef"
rec = httptest.NewRecorder()
err = fs.ServeObject(rec, httptest.NewRequest(http.MethodGet, forged, nil), key)
if !errors.Is(err, ErrSignedURLInvalid) {
t.Fatalf("forged = %v, want ErrSignedURLInvalid", err)
}
}
func TestFSServeRangeAndMissing(t *testing.T) {
ctx := context.Background()
fs := newTestFS(t)
key := "objects/abc/original.jpg"
if err := fs.Put(ctx, key, strings.NewReader("0123456789"), 10, "image/jpeg"); err != nil {
t.Fatalf("put: %v", err)
}
exp := time.Now().Add(time.Hour).Unix()
base := "/api/objects/" + key + "?e=" + strconv.FormatInt(exp, 10) + "&s=" + fs.signer.sign(key, exp)
req := httptest.NewRequest(http.MethodGet, base, nil)
req.Header.Set("Range", "bytes=2-5")
rec := httptest.NewRecorder()
if err := fs.ServeObject(rec, req, key); err != nil {
t.Fatalf("serve range: %v", err)
}
if rec.Code != http.StatusPartialContent {
t.Fatalf("status = %d, want 206", rec.Code)
}
if body, _ := io.ReadAll(rec.Body); string(body) != "2345" {
t.Fatalf("range body = %q, want 2345", body)
}
missingKey := "objects/def/original.jpg"
missing := "/api/objects/" + missingKey + "?e=" + strconv.FormatInt(exp, 10) + "&s=" + fs.signer.sign(missingKey, exp)
rec = httptest.NewRecorder()
err := fs.ServeObject(rec, httptest.NewRequest(http.MethodGet, missing, nil), missingKey)
if !errors.Is(err, ErrObjectNotFound) {
t.Fatalf("missing = %v, want ErrObjectNotFound", err)
}
}
func TestContentTypeForKey(t *testing.T) {
cases := map[string]string{
"objects/a/original.jpg": "image/jpeg",
"objects/a/thumb.webp": "image/webp",
"objects/a/odd.unknownext": "application/octet-stream",
}
for key, want := range cases {
if got := contentTypeForKey(key); got != want {
t.Errorf("contentTypeForKey(%q) = %q, want %q", key, got, want)
}
}
}
func TestFSList(t *testing.T) {
ctx := context.Background()
fs := newTestFS(t)
for _, key := range []string{"objects/a/original.jpg", "objects/b/original.jpg", "misc/readme.txt"} {
if err := fs.Put(ctx, key, strings.NewReader("x"), 1, "text/plain"); err != nil {
t.Fatalf("put %s: %v", key, err)
}
}
all, err := fs.List(ctx, "")
if err != nil || len(all) != 3 {
t.Fatalf("list all: %v len=%d", err, len(all))
}
objs, err := fs.List(ctx, "objects/")
if err != nil {
t.Fatalf("list objects: %v", err)
}
if len(objs) != 2 {
t.Fatalf("list objects len=%d, want 2", len(objs))
}
for _, o := range objs {
if !strings.HasPrefix(o.Key, "objects/") {
t.Fatalf("unexpected key %q", o.Key)
}
if o.Size != 1 || o.ModTime.IsZero() {
t.Fatalf("unexpected info %+v", o)
}
}
}
+169
View File
@@ -0,0 +1,169 @@
package storage
import (
"bytes"
"context"
"errors"
"io"
"net/http"
"net/http/httptest"
"os"
"strconv"
"strings"
"testing"
"time"
"shared-album/internal/config"
)
func newTestS3(t *testing.T) (*S3, *Signer) {
t.Helper()
endpoint := os.Getenv("S3_ENDPOINT")
if endpoint == "" {
t.Skip("S3_ENDPOINT not set")
}
signer, err := NewSigner("http://app.test", "secret", time.Hour)
if err != nil {
t.Fatalf("signer: %v", err)
}
cl, err := NewS3(context.Background(), config.S3{
Endpoint: endpoint,
Region: os.Getenv("S3_REGION"),
AccessKey: os.Getenv("S3_ACCESS_KEY"),
SecretKey: os.Getenv("S3_SECRET_KEY"),
Bucket: os.Getenv("S3_BUCKET"),
UsePathStyle: true,
}, signer)
if err != nil {
t.Skipf("storage init: %v", err)
}
return cl, signer
}
func signedQuery(signer *Signer, key string, exp int64) string {
return "?e=" + strconv.FormatInt(exp, 10) + "&s=" + signer.sign(key, exp)
}
func TestS3ServeObject(t *testing.T) {
ctx := context.Background()
cl, signer := newTestS3(t)
key := "objects/" + strconv.FormatInt(time.Now().UnixNano(), 10) + "/original.jpg"
data := []byte("0123456789")
if err := cl.Put(ctx, key, bytes.NewReader(data), int64(len(data)), "image/jpeg"); err != nil {
t.Fatalf("put: %v", err)
}
t.Cleanup(func() { _ = cl.Delete(context.Background(), key) })
exp := time.Now().Add(time.Hour).Unix()
base := "/api/objects/" + key + signedQuery(signer, key, exp)
t.Run("full", func(t *testing.T) {
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, base, nil)
if err := cl.ServeObject(rec, req, key); err != nil {
t.Fatalf("serve: %v", err)
}
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200", rec.Code)
}
if !bytes.Equal(rec.Body.Bytes(), data) {
t.Fatalf("body = %q", rec.Body.String())
}
})
t.Run("range", func(t *testing.T) {
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, base, nil)
req.Header.Set("Range", "bytes=2-5")
if err := cl.ServeObject(rec, req, key); err != nil {
t.Fatalf("serve range: %v", err)
}
if rec.Code != http.StatusPartialContent {
t.Fatalf("status = %d, want 206", rec.Code)
}
if body, _ := io.ReadAll(rec.Body); string(body) != "2345" {
t.Fatalf("range body = %q, want 2345", body)
}
})
t.Run("invalid range", func(t *testing.T) {
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, base, nil)
req.Header.Set("Range", "bytes=1000-2000")
err := cl.ServeObject(rec, req, key)
if !errors.Is(err, ErrRangeNotSatisfiable) {
t.Fatalf("err = %v, want ErrRangeNotSatisfiable", err)
}
})
t.Run("missing", func(t *testing.T) {
missingKey := "objects/" + strconv.FormatInt(time.Now().UnixNano(), 10) + "/original.jpg"
target := "/api/objects/" + missingKey + signedQuery(signer, missingKey, exp)
rec := httptest.NewRecorder()
err := cl.ServeObject(rec, httptest.NewRequest(http.MethodGet, target, nil), missingKey)
if !errors.Is(err, ErrObjectNotFound) {
t.Fatalf("err = %v, want ErrObjectNotFound", err)
}
})
t.Run("expired signature", func(t *testing.T) {
past := time.Now().Add(-time.Minute).Unix()
target := "/api/objects/" + key + signedQuery(signer, key, past)
rec := httptest.NewRecorder()
err := cl.ServeObject(rec, httptest.NewRequest(http.MethodGet, target, nil), key)
if !errors.Is(err, ErrSignedURLInvalid) {
t.Fatalf("err = %v, want ErrSignedURLInvalid", err)
}
})
}
func TestS3List(t *testing.T) {
ctx := context.Background()
cl, _ := newTestS3(t)
prefix := "objects/" + strconv.FormatInt(time.Now().UnixNano(), 10) + "/"
keys := []string{prefix + "original.jpg", prefix + "thumb.webp"}
for _, k := range keys {
if err := cl.Put(ctx, k, strings.NewReader("x"), 1, "image/jpeg"); err != nil {
t.Fatalf("put %s: %v", k, err)
}
}
t.Cleanup(func() { _ = cl.Delete(context.Background(), keys...) })
got, err := cl.List(ctx, prefix)
if err != nil {
t.Fatalf("list: %v", err)
}
found := map[string]bool{}
for _, o := range got {
found[o.Key] = true
}
for _, k := range keys {
if !found[k] {
t.Fatalf("key %s not listed: %+v", k, got)
}
}
}
func TestS3ServeObjectHead(t *testing.T) {
ctx := context.Background()
cl, signer := newTestS3(t)
key := "objects/" + strconv.FormatInt(time.Now().UnixNano(), 10) + "/original.jpg"
data := []byte("head-check")
if err := cl.Put(ctx, key, strings.NewReader(string(data)), int64(len(data)), "image/jpeg"); err != nil {
t.Fatalf("put: %v", err)
}
t.Cleanup(func() { _ = cl.Delete(context.Background(), key) })
exp := time.Now().Add(time.Hour).Unix()
target := "/api/objects/" + key + signedQuery(signer, key, exp)
rec := httptest.NewRecorder()
if err := cl.ServeObject(rec, httptest.NewRequest(http.MethodHead, target, nil), key); err != nil {
t.Fatalf("head: %v", err)
}
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200", rec.Code)
}
if rec.Body.Len() != 0 {
t.Fatalf("HEAD body should be empty, got %d bytes", rec.Body.Len())
}
}
+80
View File
@@ -0,0 +1,80 @@
package storage
import (
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"errors"
"fmt"
"net/http"
"net/url"
"strconv"
"strings"
"time"
)
// Signer produces and verifies the signed app URLs used by the /api/objects
// route. It is shared by every storage backend so delivery is identical: the
// client always talks to the backend, never to object storage directly.
type Signer struct {
secret []byte
baseURL string
ttl time.Duration
}
// NewSigner builds a signer. secret signs the URLs (required); baseURL is the
// public base URL of the app; ttl limits URL lifetime (defaults to one hour).
func NewSigner(baseURL, secret string, ttl time.Duration) (*Signer, error) {
if strings.TrimSpace(secret) == "" {
return nil, errors.New("url secret is required")
}
if ttl <= 0 {
ttl = time.Hour
}
return &Signer{
secret: []byte(secret),
baseURL: strings.TrimRight(baseURL, "/"),
ttl: ttl,
}, nil
}
// URL returns a signed app URL for the given opaque key.
func (s *Signer) URL(key string) string {
exp := time.Now().Add(s.ttl).Unix()
q := url.Values{}
q.Set("e", strconv.FormatInt(exp, 10))
q.Set("s", s.sign(key, exp))
return fmt.Sprintf("%s/api/objects/%s?%s", s.baseURL, escapeKey(key), q.Encode())
}
// VerifyRequest checks the signature and expiry carried by an object request.
// It returns ErrSignedURLInvalid when the request is not authorized.
func (s *Signer) VerifyRequest(key string, r *http.Request) error {
q := r.URL.Query()
exp, err := strconv.ParseInt(q.Get("e"), 10, 64)
if err != nil || !s.verify(key, exp, q.Get("s")) {
return ErrSignedURLInvalid
}
return nil
}
func (s *Signer) sign(key string, exp int64) string {
mac := hmac.New(sha256.New, s.secret)
_, _ = fmt.Fprintf(mac, "%s\n%d", key, exp)
return hex.EncodeToString(mac.Sum(nil))
}
func (s *Signer) verify(key string, exp int64, sig string) bool {
if sig == "" || time.Now().Unix() > exp {
return false
}
return hmac.Equal([]byte(s.sign(key, exp)), []byte(sig))
}
func escapeKey(key string) string {
parts := strings.Split(key, "/")
for i, p := range parts {
parts[i] = url.PathEscape(p)
}
return strings.Join(parts, "/")
}
+203 -37
View File
@@ -5,6 +5,8 @@ import (
"errors"
"fmt"
"io"
"net/http"
"strconv"
"strings"
"time"
@@ -18,14 +20,58 @@ import (
"shared-album/internal/config"
)
type Client struct {
s3 *s3.Client
presign *s3.PresignClient
bucket string
presignTTL time.Duration
// ErrSignedURLInvalid is returned when an app-served object URL has a missing,
// malformed, expired or forged signature.
var ErrSignedURLInvalid = errors.New("invalid or expired object signature")
// ErrObjectNotFound is returned when the requested blob does not exist.
var ErrObjectNotFound = errors.New("object not found")
// ErrRangeNotSatisfiable is returned when a requested byte range is invalid.
var ErrRangeNotSatisfiable = errors.New("requested range not satisfiable")
// ObjectInfo describes one stored blob for inventory and reconciliation.
type ObjectInfo struct {
Key string
Size int64
ModTime time.Time
}
func New(ctx context.Context, cfg config.S3) (*Client, error) {
// Backend is the object storage abstraction. Keys are opaque and independent of
// the backend: every implementation stores and serves the same logical blob.
// Blobs are always delivered through the app.
type Backend interface {
Put(ctx context.Context, key string, body io.Reader, size int64, contentType string) error
Delete(ctx context.Context, keys ...string) error
Get(ctx context.Context, key string) ([]byte, error)
// URL returns a time-limited signed app URL the client can use to fetch the blob.
URL(ctx context.Context, key string) (string, error)
// ServeObject validates the signed URL and writes the blob, including
// Content-Type and range handling, to the response.
ServeObject(w http.ResponseWriter, r *http.Request, key string) error
// List returns the blobs under prefix for inventory and reconciliation.
List(ctx context.Context, prefix string) ([]ObjectInfo, error)
Ping(ctx context.Context) error
EnsureBucket(ctx context.Context) error
// Bucket returns a human-readable identifier for the storage location.
Bucket() string
}
// S3 is a Backend backed by an S3-compatible object store (Garage). Blobs are
// delivered through the app: the client receives a signed app URL and the bytes
// are streamed from S3 over the internal network.
type S3 struct {
s3 *s3.Client
bucket string
signer *Signer
}
// NewS3 builds an S3-compatible storage backend. signer issues the app URLs used
// to serve blobs.
func NewS3(ctx context.Context, cfg config.S3, signer *Signer) (*S3, error) {
if signer == nil {
return nil, errors.New("s3: signer is required")
}
awsCfg, err := awsconfig.LoadDefaultConfig(ctx,
awsconfig.WithRegion(cfg.Region),
awsconfig.WithCredentialsProvider(credentials.NewStaticCredentialsProvider(
@@ -41,17 +87,42 @@ func New(ctx context.Context, cfg config.S3) (*Client, error) {
o.UsePathStyle = cfg.UsePathStyle
})
return &Client{
return &S3{
s3: client,
presign: s3.NewPresignClient(client),
bucket: cfg.Bucket,
presignTTL: cfg.PresignTTL,
signer: signer,
}, nil
}
func (c *Client) Bucket() string { return c.bucket }
func (c *S3) Bucket() string { return c.bucket }
func (c *Client) EnsureBucket(ctx context.Context) error {
// Open builds the storage backend selected by configuration: S3-compatible by
// default, or the local file system when STORAGE_BACKEND=fs. Every backend
// delivers blobs through the app using a shared Signer. jwtSecret is the
// application secret used to sign object URLs when URL_SECRET is not set.
func Open(ctx context.Context, cfg *config.Config, jwtSecret string) (Backend, error) {
secret := cfg.Storage.URLSecret
if secret == "" {
secret = jwtSecret
}
if secret == "" {
return nil, errors.New("object URL secret is not available")
}
signer, err := NewSigner(cfg.Server.BaseURL, secret, cfg.Storage.URLTTL)
if err != nil {
return nil, err
}
switch cfg.Storage.Backend {
case "", "s3":
return NewS3(ctx, cfg.S3, signer)
case "fs":
return NewFS(cfg.Storage.FSRoot, signer)
default:
return nil, fmt.Errorf("unknown storage backend %q", cfg.Storage.Backend)
}
}
func (c *S3) EnsureBucket(ctx context.Context) error {
_, err := c.s3.HeadBucket(ctx, &s3.HeadBucketInput{Bucket: aws.String(c.bucket)})
if err == nil {
return nil
@@ -66,12 +137,12 @@ func (c *Client) EnsureBucket(ctx context.Context) error {
return nil
}
func (c *Client) Ping(ctx context.Context) error {
func (c *S3) Ping(ctx context.Context) error {
_, err := c.s3.HeadBucket(ctx, &s3.HeadBucketInput{Bucket: aws.String(c.bucket)})
return err
}
func (c *Client) Put(ctx context.Context, key string, body io.Reader, size int64, contentType string) error {
func (c *S3) Put(ctx context.Context, key string, body io.Reader, size int64, contentType string) error {
in := &s3.PutObjectInput{
Bucket: aws.String(c.bucket),
Key: aws.String(key),
@@ -87,7 +158,7 @@ func (c *Client) Put(ctx context.Context, key string, body io.Reader, size int64
return nil
}
func (c *Client) Delete(ctx context.Context, keys ...string) error {
func (c *S3) Delete(ctx context.Context, keys ...string) error {
if len(keys) == 0 {
return nil
}
@@ -104,25 +175,7 @@ func (c *Client) Delete(ctx context.Context, keys ...string) error {
return nil
}
func (c *Client) DeletePrefix(ctx context.Context, prefix string) error {
var keys []string
paginator := s3.NewListObjectsV2Paginator(c.s3, &s3.ListObjectsV2Input{
Bucket: aws.String(c.bucket),
Prefix: aws.String(prefix),
})
for paginator.HasMorePages() {
page, err := paginator.NextPage(ctx)
if err != nil {
return fmt.Errorf("list objects: %w", err)
}
for _, o := range page.Contents {
keys = append(keys, aws.ToString(o.Key))
}
}
return c.Delete(ctx, keys...)
}
func (c *Client) Get(ctx context.Context, key string) ([]byte, error) {
func (c *S3) Get(ctx context.Context, key string) ([]byte, error) {
out, err := c.s3.GetObject(ctx, &s3.GetObjectInput{
Bucket: aws.String(c.bucket),
Key: aws.String(key),
@@ -138,14 +191,127 @@ func (c *Client) Get(ctx context.Context, key string) ([]byte, error) {
return data, nil
}
func (c *Client) PresignGet(ctx context.Context, key string) (string, error) { out, err := c.presign.PresignGetObject(ctx, &s3.GetObjectInput{
// URL returns a signed app URL. The blob itself is streamed by ServeObject, so
// the S3 endpoint is never exposed to the client.
func (c *S3) URL(_ context.Context, key string) (string, error) {
return c.signer.URL(key), nil
}
// List returns every object under prefix using paginated ListObjectsV2.
func (c *S3) List(ctx context.Context, prefix string) ([]ObjectInfo, error) {
var out []ObjectInfo
var token *string
for {
in := &s3.ListObjectsV2Input{Bucket: aws.String(c.bucket)}
if prefix != "" {
in.Prefix = aws.String(prefix)
}
if token != nil {
in.ContinuationToken = token
}
res, err := c.s3.ListObjectsV2(ctx, in)
if err != nil {
return nil, fmt.Errorf("list objects: %w", err)
}
for _, o := range res.Contents {
out = append(out, ObjectInfo{
Key: aws.ToString(o.Key),
Size: aws.ToInt64(o.Size),
ModTime: aws.ToTime(o.LastModified),
})
}
if res.IsTruncated == nil || !*res.IsTruncated {
return out, nil
}
token = res.NextContinuationToken
}
}
// ServeObject validates the signed URL and streams the blob from S3. It passes
// the client's Range header through to S3, so byte ranges are handled natively,
// and answers HEAD via HeadObject. It writes the response itself.
func (c *S3) ServeObject(w http.ResponseWriter, r *http.Request, key string) error {
if err := c.signer.VerifyRequest(key, r); err != nil {
return err
}
if r.Method == http.MethodHead {
return c.headObject(w, r, key)
}
in := &s3.GetObjectInput{Bucket: aws.String(c.bucket), Key: aws.String(key)}
if rng := r.Header.Get("Range"); rng != "" {
in.Range = aws.String(rng)
}
out, err := c.s3.GetObject(r.Context(), in)
if err != nil {
if isNoSuchKey(err) {
return ErrObjectNotFound
}
if isInvalidRange(err) {
return ErrRangeNotSatisfiable
}
return err
}
defer out.Body.Close()
if ct := aws.ToString(out.ContentType); ct != "" {
w.Header().Set("Content-Type", ct)
} else {
w.Header().Set("Content-Type", contentTypeForKey(key))
}
if out.ContentLength != nil {
w.Header().Set("Content-Length", strconv.FormatInt(*out.ContentLength, 10))
}
if out.ContentRange != nil {
w.Header().Set("Content-Range", *out.ContentRange)
}
status := http.StatusOK
if out.ContentRange != nil {
status = http.StatusPartialContent
}
w.WriteHeader(status)
if _, err := io.Copy(w, out.Body); err != nil {
return err
}
return nil
}
func (c *S3) headObject(w http.ResponseWriter, r *http.Request, key string) error {
out, err := c.s3.HeadObject(r.Context(), &s3.HeadObjectInput{
Bucket: aws.String(c.bucket),
Key: aws.String(key),
}, s3.WithPresignExpires(c.presignTTL))
})
if err != nil {
return "", fmt.Errorf("presign %s: %w", key, err)
if isNoSuchKey(err) {
return ErrObjectNotFound
}
return out.URL, nil
return err
}
if ct := aws.ToString(out.ContentType); ct != "" {
w.Header().Set("Content-Type", ct)
} else {
w.Header().Set("Content-Type", contentTypeForKey(key))
}
if out.ContentLength != nil {
w.Header().Set("Content-Length", strconv.FormatInt(*out.ContentLength, 10))
}
w.WriteHeader(http.StatusOK)
return nil
}
func isNoSuchKey(err error) bool {
var apiErr smithy.APIError
if errors.As(err, &apiErr) {
code := apiErr.ErrorCode()
return strings.EqualFold(code, "NoSuchKey") || strings.EqualFold(code, "NotFound")
}
return false
}
func isInvalidRange(err error) bool {
var apiErr smithy.APIError
if errors.As(err, &apiErr) {
return strings.EqualFold(apiErr.ErrorCode(), "InvalidRange")
}
return false
}
func isNoSuchBucket(err error) bool {
+59
View File
@@ -0,0 +1,59 @@
// Package store builds the per-domain metadata repositories for a chosen
// backend. Domains depend on their Repository interfaces; this factory is the
// single place that knows the concrete implementations.
package store
import (
"database/sql"
"github.com/jackc/pgx/v5/pgxpool"
"shared-album/internal/albums"
"shared-album/internal/auth"
"shared-album/internal/members"
"shared-album/internal/notifications"
"shared-album/internal/photos"
"shared-album/internal/settings"
"shared-album/internal/sharelinks"
"shared-album/internal/social"
)
// Repos bundles one repository per domain.
type Repos struct {
Auth auth.Repository
Albums albums.Repository
Members members.Repository
Photos photos.Repository
Social social.Repository
Notifications notifications.Repository
ShareLinks sharelinks.Repository
Settings settings.Repository
}
// NewPostgres builds the repositories backed by PostgreSQL.
func NewPostgres(pool *pgxpool.Pool) Repos {
return Repos{
Auth: auth.NewStore(pool),
Albums: albums.NewStore(pool),
Members: members.NewStore(pool),
Photos: photos.NewStore(pool),
Social: social.NewStore(pool),
Notifications: notifications.NewStore(pool),
ShareLinks: sharelinks.NewStore(pool),
Settings: settings.NewStore(pool),
}
}
// NewSQLite builds the repositories backed by SQLite.
func NewSQLite(db *sql.DB) Repos {
return Repos{
Auth: auth.NewSQLiteStore(db),
Albums: albums.NewSQLiteStore(db),
Members: members.NewSQLiteStore(db),
Photos: photos.NewSQLiteStore(db),
Social: social.NewSQLiteStore(db),
Notifications: notifications.NewSQLiteStore(db),
ShareLinks: sharelinks.NewSQLiteStore(db),
Settings: settings.NewSQLiteStore(db),
}
}
+55
View File
@@ -0,0 +1,55 @@
package store
import (
"context"
"io/fs"
"os"
"path/filepath"
"testing"
"github.com/jackc/pgx/v5/pgxpool"
assets "shared-album"
"shared-album/internal/db"
)
func assertComplete(t *testing.T, r Repos) {
t.Helper()
if r.Auth == nil || r.Albums == nil || r.Members == nil || r.Photos == nil ||
r.Social == nil || r.Notifications == nil || r.ShareLinks == nil {
t.Fatalf("incomplete repository bundle: %+v", r)
}
}
func TestNewSQLite(t *testing.T) {
path := filepath.Join(t.TempDir(), "meta.db")
migFS, err := fs.Sub(assets.Migrations, "migrations/sqlite")
if err != nil {
t.Fatalf("migrations fs: %v", err)
}
if err := db.MigrateSQLite(path, migFS); err != nil {
t.Fatalf("migrate: %v", err)
}
database, err := db.OpenSQLite(path)
if err != nil {
t.Fatalf("open: %v", err)
}
defer database.Close()
assertComplete(t, NewSQLite(database))
}
func TestNewPostgres(t *testing.T) {
dsn := os.Getenv("TEST_DATABASE_URL")
if dsn == "" {
t.Skip("TEST_DATABASE_URL not set")
}
pool, err := pgxpool.New(context.Background(), dsn)
if err != nil {
t.Skipf("cannot connect: %v", err)
}
if err := pool.Ping(context.Background()); err != nil {
t.Skipf("cannot ping: %v", err)
}
t.Cleanup(pool.Close)
assertComplete(t, NewPostgres(pool))
}
@@ -0,0 +1,3 @@
DROP TABLE IF EXISTS variant_jobs;
DROP TABLE IF EXISTS photo_objects;
DROP TABLE IF EXISTS objects;
@@ -0,0 +1,39 @@
-- Physical objects stored in S3. Keys are opaque and never encode album, photo
-- or any other metadata entity. The row is the ledger entry that governs the
-- lifecycle of the corresponding blob.
CREATE TABLE IF NOT EXISTS objects (
id text PRIMARY KEY,
storage_key text NOT NULL UNIQUE,
mime_type text NOT NULL DEFAULT '',
size bigint NOT NULL DEFAULT 0,
width integer NOT NULL DEFAULT 0,
height integer NOT NULL DEFAULT 0,
checksum text NOT NULL DEFAULT '',
state text NOT NULL DEFAULT 'pending'
CHECK (state IN ('pending', 'ready', 'deleting')),
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS objects_state_created_idx ON objects (state, created_at);
-- Binding of a photo to its objects by role. The role is a property of the
-- binding, not of the object itself.
CREATE TABLE IF NOT EXISTS photo_objects (
photo_id text NOT NULL REFERENCES photos(id) ON DELETE CASCADE,
object_id text NOT NULL REFERENCES objects(id) ON DELETE RESTRICT,
role text NOT NULL CHECK (role IN ('original', 'thumb', 'preview')),
PRIMARY KEY (photo_id, role)
);
CREATE INDEX IF NOT EXISTS photo_objects_object_idx ON photo_objects (object_id);
-- Durable queue of pending variant generation work, one job per photo.
CREATE TABLE IF NOT EXISTS variant_jobs (
photo_id text PRIMARY KEY REFERENCES photos(id) ON DELETE CASCADE,
attempts integer NOT NULL DEFAULT 0,
next_attempt_at timestamptz NOT NULL DEFAULT now(),
last_error text NOT NULL DEFAULT '',
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS variant_jobs_due_idx ON variant_jobs (next_attempt_at);
@@ -0,0 +1,8 @@
ALTER TABLE photos
ADD COLUMN IF NOT EXISTS storage_key text NOT NULL DEFAULT '',
ADD COLUMN IF NOT EXISTS thumb_key text,
ADD COLUMN IF NOT EXISTS preview_key text,
ADD COLUMN IF NOT EXISTS size bigint NOT NULL DEFAULT 0,
ADD COLUMN IF NOT EXISTS width integer NOT NULL DEFAULT 0,
ADD COLUMN IF NOT EXISTS height integer NOT NULL DEFAULT 0,
ADD COLUMN IF NOT EXISTS mime_type text NOT NULL DEFAULT '';
@@ -0,0 +1,10 @@
-- Physical facts move to the objects table; photos keeps only membership and
-- presentation data. size/width/height/mime_type are derived from the original
-- object via photo_objects + objects.
ALTER TABLE photos DROP COLUMN IF EXISTS storage_key;
ALTER TABLE photos DROP COLUMN IF EXISTS thumb_key;
ALTER TABLE photos DROP COLUMN IF EXISTS preview_key;
ALTER TABLE photos DROP COLUMN IF EXISTS size;
ALTER TABLE photos DROP COLUMN IF EXISTS width;
ALTER TABLE photos DROP COLUMN IF EXISTS height;
ALTER TABLE photos DROP COLUMN IF EXISTS mime_type;
@@ -0,0 +1 @@
DROP TABLE IF EXISTS app_settings;
@@ -0,0 +1,6 @@
CREATE TABLE IF NOT EXISTS app_settings (
key text PRIMARY KEY,
value text NOT NULL,
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now()
);
@@ -0,0 +1,15 @@
DROP TABLE IF EXISTS variant_jobs;
DROP TABLE IF EXISTS photo_objects;
DROP TABLE IF EXISTS objects;
DROP TABLE IF EXISTS photo_tags;
DROP TABLE IF EXISTS tags;
DROP TABLE IF EXISTS refresh_tokens;
DROP TABLE IF EXISTS share_links;
DROP TABLE IF EXISTS notifications;
DROP TABLE IF EXISTS likes;
DROP TABLE IF EXISTS comments;
DROP TABLE IF EXISTS invites;
DROP TABLE IF EXISTS album_members;
DROP TABLE IF EXISTS photos;
DROP TABLE IF EXISTS albums;
DROP TABLE IF EXISTS users;
@@ -0,0 +1,159 @@
-- Consolidated SQLite schema equivalent to the PostgreSQL migration history.
CREATE TABLE users (
id TEXT PRIMARY KEY,
email TEXT NOT NULL UNIQUE,
password_hash TEXT NOT NULL,
name TEXT NOT NULL,
avatar_url TEXT,
preferences TEXT NOT NULL DEFAULT '{}',
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
);
CREATE TABLE albums (
id TEXT PRIMARY KEY,
owner_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
title TEXT NOT NULL,
description TEXT NOT NULL DEFAULT '',
cover_photo_id TEXT REFERENCES photos(id) ON DELETE SET NULL,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
);
CREATE TABLE photos (
id TEXT PRIMARY KEY,
album_id TEXT REFERENCES albums(id) ON DELETE SET NULL,
uploader_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
original_filename TEXT NOT NULL DEFAULT '',
taken_at TEXT,
latitude REAL,
longitude REAL,
caption TEXT NOT NULL DEFAULT '',
exif TEXT NOT NULL DEFAULT '{}',
deleted_at TEXT,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
);
CREATE INDEX photos_album_created_idx ON photos (album_id, created_at DESC, id DESC);
CREATE INDEX photos_deleted_idx ON photos (deleted_at) WHERE deleted_at IS NOT NULL;
CREATE TABLE album_members (
album_id TEXT NOT NULL REFERENCES albums(id) ON DELETE CASCADE,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
role TEXT NOT NULL,
invited_by TEXT REFERENCES users(id) ON DELETE SET NULL,
joined_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
PRIMARY KEY (album_id, user_id)
);
CREATE TABLE invites (
id TEXT PRIMARY KEY,
album_id TEXT NOT NULL REFERENCES albums(id) ON DELETE CASCADE,
token TEXT NOT NULL UNIQUE,
role TEXT NOT NULL,
created_by TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
expires_at TEXT NOT NULL,
max_uses INTEGER NOT NULL DEFAULT 1,
uses INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
);
CREATE TABLE comments (
id TEXT PRIMARY KEY,
photo_id TEXT NOT NULL REFERENCES photos(id) ON DELETE CASCADE,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
text TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
);
CREATE INDEX comments_photo_idx ON comments (photo_id, created_at);
CREATE TABLE likes (
photo_id TEXT NOT NULL REFERENCES photos(id) ON DELETE CASCADE,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
PRIMARY KEY (photo_id, user_id)
);
CREATE TABLE notifications (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
type TEXT NOT NULL,
payload_json TEXT NOT NULL DEFAULT '{}',
read_at TEXT,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
);
CREATE INDEX notifications_user_idx ON notifications (user_id, created_at DESC);
CREATE TABLE share_links (
id TEXT PRIMARY KEY,
album_id TEXT NOT NULL REFERENCES albums(id) ON DELETE CASCADE,
token TEXT NOT NULL UNIQUE,
created_by TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
expires_at TEXT NOT NULL,
revoked_at TEXT
);
CREATE TABLE refresh_tokens (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
token_hash TEXT NOT NULL UNIQUE,
expires_at TEXT NOT NULL,
revoked_at TEXT,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
);
CREATE INDEX refresh_tokens_user_idx ON refresh_tokens (user_id);
CREATE TABLE tags (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
name TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
);
CREATE UNIQUE INDEX tags_user_name_idx ON tags (user_id, lower(name));
CREATE TABLE photo_tags (
photo_id TEXT NOT NULL REFERENCES photos(id) ON DELETE CASCADE,
tag_id TEXT NOT NULL REFERENCES tags(id) ON DELETE CASCADE,
PRIMARY KEY (photo_id, tag_id)
);
CREATE INDEX photo_tags_tag_idx ON photo_tags (tag_id);
CREATE TABLE objects (
id TEXT PRIMARY KEY,
storage_key TEXT NOT NULL UNIQUE,
mime_type TEXT NOT NULL DEFAULT '',
size INTEGER NOT NULL DEFAULT 0,
width INTEGER NOT NULL DEFAULT 0,
height INTEGER NOT NULL DEFAULT 0,
checksum TEXT NOT NULL DEFAULT '',
state TEXT NOT NULL DEFAULT 'pending'
CHECK (state IN ('pending', 'ready', 'deleting')),
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
);
CREATE INDEX objects_state_created_idx ON objects (state, created_at);
CREATE TABLE photo_objects (
photo_id TEXT NOT NULL REFERENCES photos(id) ON DELETE CASCADE,
object_id TEXT NOT NULL REFERENCES objects(id) ON DELETE RESTRICT,
role TEXT NOT NULL CHECK (role IN ('original', 'thumb', 'preview')),
PRIMARY KEY (photo_id, role)
);
CREATE INDEX photo_objects_object_idx ON photo_objects (object_id);
CREATE TABLE variant_jobs (
photo_id TEXT PRIMARY KEY REFERENCES photos(id) ON DELETE CASCADE,
attempts INTEGER NOT NULL DEFAULT 0,
next_attempt_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
last_error TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
);
CREATE INDEX variant_jobs_due_idx ON variant_jobs (next_attempt_at);
@@ -0,0 +1 @@
DROP TABLE IF EXISTS app_settings;
@@ -0,0 +1,6 @@
CREATE TABLE app_settings (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
);
+8
View File
@@ -21,6 +21,14 @@ server {
chunked_transfer_encoding on;
}
location /api/objects/ {
proxy_pass http://shared_album;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_buffering off; # стриминг объектов без буферизации
proxy_cache off;
}
location / {
proxy_pass http://shared_album;
proxy_http_version 1.1;
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-10-10
@@ -0,0 +1,134 @@
# Design
## Context
См. `proposal.md` — Why. Текущее состояние, которое формирует подход:
- `photos` хранит `storage_key`, `thumb_key`, `preview_key` (`migrations/000001_init.up.sql:45-47`),
а ключи строятся как `albums/{album_id}/...` (`internal/photos/service.go:72,114,115`).
- `Clone` копирует строку фото вместе с ключами, `MoveBatch` меняет только `album_id`,
`PermanentDelete` удаляет объекты по счётчику ссылок `StorageKeyRefCount`
(`internal/photos/store.go:201`).
- `DeletePrefix` (`internal/storage/storage.go:107`) существует, но не вызывается.
- Очередь вариантов отсутствует: генерация запускается goroutine из HTTP-запроса
(`internal/photos/service.go:98`), результат не переживает рестарт.
- Внешний API уже отдаёт `thumb_url/preview_url/original_url` и
`size/width/height/mime_type`; это контракт, который надо сохранить.
- В БД только дев-данные, миграция может быть разрушительной.
## Goals / Non-Goals
**Goals:**
- Убрать сущность «альбом» (и любую модель метаданных) из ключей и операций S3.
- Сделать БД журналом истины: физический объект существует, пока на него есть ссылка.
- Дать явный, устойчивый к сбоям сборщик мусора и самовосстанавливающуюся генерацию вариантов.
- Сохранить внешний JSON API без изменений.
**Non-Goals:**
- Content-addressing и дедупликация одинаковых байтов между разными загрузками
(задел — колонка `checksum`, политика позже).
- Мультивариантность сверх фиксированных original/thumb/preview (join-таблица это выдержит).
- Перенос существующих данных: миграция разрушительная.
## Decisions
### D1. Join-таблица `photo_objects` вместо колонок-FK
Фото ссылается на объекты через `photo_objects(photo_id, object_id, role)` с
`PK(photo_id, role)`. Роль — свойство **связки**, не объекта: сам объект не знает,
кто он. Альтернатива (три FK-колонки на `photos`) проще, но фиксирует ровно три
варианта и снова смешивает представление с физикой. Выбран вариант, который
расширяется до нескольких размеров/видео без переделки схемы.
### D2. Разделение фактов: физические vs логические
`objects` — только физические факты: `storage_key`, `mime_type`, `size`,
`width`, `height`, `checksum`, `state`, `created_at`.
`photos` — членство и представление: `album_id`, `uploader_id`,
`original_filename`, `caption`, `exif`, `taken_at`, `latitude`, `longitude`,
`deleted_at`, `created_at`.
`size/width/height/mime_type` в API выводятся джойном к объекту-оригиналу.
```sql
objects(
id text pk, storage_key text unique not null, mime_type text,
size bigint, width int, height int, checksum text,
state text, created_at timestamptz)
photo_objects(
photo_id -> photos(id) on delete cascade,
object_id -> objects(id) on delete restrict,
role text check (role in ('original','thumb','preview')),
primary key (photo_id, role))
create index on photo_objects(object_id);
variant_jobs(
photo_id -> photos(id) on delete cascade,
attempts int, next_attempt_at timestamptz, last_error text,
primary key (photo_id))
```
У `photos` удаляются `storage_key`, `thumb_key`, `preview_key`.
### D3. Album-free ключи, случайный id объекта
Ключ: `objects/{object_id}/{role}.{ext}`, где `object_id` — случайный
(`auth.NewID()`). Роль в ключе — косметика для отладки, истина о роли — в
`photo_objects`. Бакет, region, path-style не меняются. `DeletePrefix` удаляется.
### D4. Ledger состояний и явный GC
Объект: `pending -> ready -> deleting`. Порядок операций — **сначала строка, потом
Put**, поэтому неубиваемый мусор «файл есть, строки нет» не возникает.
```
Upload : INSERT objects(pending) -> S3 Put -> ready -> bind original + INSERT variant_jobs
Copy : INSERT photos + bind те же objects -- 0 S3
Move : UPDATE photos.album_id -- 0 S3
Delete : tx DELETE photos -> cascade unbind -> mark orphaned 'deleting'
Worker : deleting -> re-check 0 ссылок -> S3 Delete -> DELETE row
pending старше T (Put не завершился) -> deleting -> S3 Delete -> DELETE row
```
Ссылочные гонки закрываются блокировками:
- Пометить `deleting` можно только объект с нулём связок, под `SELECT ... FOR UPDATE`.
- Перед S3 Delete воркер **перепроверяет** ноль связок под `FOR UPDATE`.
- Привязка объекта (bind) берёт `FOR UPDATE` и отвергает `state != 'ready'`.
### D5. Генерация вариантов через `variant_jobs`
Вместо goroutine из запроса — строка задания в той же транзакции, что и загрузка.
Воркер забирает задания через `FOR UPDATE SKIP LOCKED`, регенерирует thumb/preview,
создаёт объекты (pending->Put->ready), вставляет связки, публикует SSE
`photo.created`, при ошибке увеличивает `attempts` и переносит `next_attempt_at`.
Это даёт однократное выполнение, ретраи и восстановление после рестарта.
### D6. Один фоновый воркер в том же бинаре
Петля-тикер внутри процесса (соответствует модели «один бинарь + systemd»):
(1) доделать варианты, (2) удалить `deleting`, (3) дососить `pending`. Интервал и
батч — конфигурируемые. Пометка объектов при удалении — синхронно в транзакции.
## Risks / Trade-offs
- [Двойная генерация вариантов после рестарта] → задание живёт в `variant_jobs`,
забирается под `SKIP LOCKED`; повторное создание объекта безопасно, лишний объект
соберёт GC.
- [Удаление объекта, который вот-вот свяжут] → `FOR UPDATE` на объекте в bind и
повторная проверка ссылок в GC.
- [Строка `photos` есть, объекта нет при сбое Put] → состояние `pending` + sweeper.
- [Джойн добавляет стоимость в список фото] → один join к объекту-оригиналу; индексы
по `photo_objects(photo_id)` (PK) и `photo_objects(object_id)`.
- [Ошибка в GC удалит живые байты необратимо] → удаляем только при доказанном нуле
ссылок; удаление идемпотентно, состояние фиксируется до S3-операции.
- [API-совместимость] → JSON-поля сохраняются, меняется только источник значений.
## Migration Plan
Разрушительная (только дев-данные):
1. Миграция создаёт `objects`, `photo_objects`, `variant_jobs`.
2. Миграция удаляет `photos.storage_key`, `photos.thumb_key`, `photos.preview_key`.
3. Существующие строки фото и объекты S3 не переносятся.
4. Откат: down-миграция возвращает колонки и удаляет новые таблицы.
5. `project.md` §10 приводится к новой структуре ключей.
## Open Questions
- Точные значения интервала воркера, размера батча и порога `T` для `pending` —
подбираются при реализации, на схему и контракты не влияют.
- Формат будущего дедупа по `checksum` (политика, не схема) — вне этого change.
@@ -0,0 +1,49 @@
# Proposal
## Why
Слой хранения S3 сейчас знает о сущности «альбом»: ключи объектов имеют вид
`albums/{album_id}/original/{photo_id}.{ext}` (`photos/service.go:72,114,115`).
Это протекание модели метаданных в физический слой: перенос фото оставляет ключ в
старом namespace, копия делит объекты под префиксом исходного альбома, а
неиспользуемый `DeletePrefix` (`storage/storage.go:107`) создаёт латентный риск
удаления живых объектов. Нужно развести слои: S3 хранит непрозрачные объекты, а
сущности и их жизненный цикл живут в БД.
## What Changes
- Ввести таблицу `objects` — физические блобы (ключ, mime, размер, габариты,
checksum, состояние) и `photo_objects` — связка photo↔object с ролью
(`original|thumb|preview`).
- **BREAKING**: удалить `storage_key`, `thumb_key`, `preview_key` из `photos`;
ключи фото выводятся через джойн к объекту-оригиналу. Миграция разрушительная
(в базе только дев-данные).
- Сделать ключи S3 album-free: `objects/{id}/{role}.{ext}`, где `id` — случайный
идентификатор объекта.
- Ввести явный GC через ledger-состояния объектов `pending | ready | deleting`:
физический объект удаляется только когда на него не осталось ни одной связки.
- Перенести генерацию вариантов в фоновый воркер: таблица `variant_jobs`
(ровно-один-раз, ретраи с backoff), воркер также удаляет осиротевшие объекты и
досасывает зависшие `pending`. SSE `photo.created` публикует воркер.
- Удалить `DeletePrefix` из клиента хранения — entity-namespace больше нет.
## Capabilities
### New Capabilities
- `media-objects`: физическое объектное хранилище с непрозрачными ключами, журнал
объектов и их жизненный цикл, сборка мусора по отсутствию ссылок, привязка
объектов к фото по ролям.
### Modified Capabilities
- `photos`: уточняется семантика удаления (корзина не удаляет объекты хранилища) и
разделение объектов между фото при переносе и копировании.
## Impact
- Backend: `internal/photos/{service,store}.go`, `internal/storage/storage.go`,
`internal/albums/*` (косвенно, через ключи обложки), wiring фонового воркера,
`migrations/` (новая схема + удаление колонок).
- Данные: разрушительная миграция `photos` (dev-only), новые таблицы.
- Внешний JSON API не меняется: `thumb_url/preview_url/original_url` и
`size/width/height/mime_type` по-прежнему отдаются, но выводятся из объектов.
- Документация: `project.md` §10 (структура в S3).
@@ -0,0 +1,90 @@
# Spec Delta
## Purpose
Физическое объектное хранилище с непрозрачными ключами и журналом объектов:
привязка объектов к фото по ролям, совместное использование объектов несколькими
фото и сборка мусора по отсутствию ссылок.
## ADDED Requirements
### Requirement: Непрозрачные ключи хранилища
Слой хранения SHALL адресоваться непрозрачными ключами объектов, не содержащими
идентификаторов сущностей метаданных (таких как альбом или фото); сопоставление
ключей и сущностей SHALL храниться только в слое метаданных.
#### Scenario: Ключ без альбома
- **WHEN** система сохраняет объект в хранилище
- **THEN** ключ объекта не содержит идентификатора альбома
#### Scenario: Перенос не трогает ключи
- **WHEN** фото переносится в другой альбом
- **THEN** ключи его объектов остаются неизменными
### Requirement: Журнал объектов
Каждый физический объект SHALL иметь ровно одну запись в слое метаданных с ключом,
MIME-типом, размером, габаритами и состоянием; фото SHALL ссылаться на объекты по
ролям original, thumb и preview.
#### Scenario: Привязка ролей
- **WHEN** фото загружено и обработано
- **THEN** фото ссылается на объект-оригинал и, когда они готовы, на объекты thumb и preview
#### Scenario: Оригинал без вариантов
- **WHEN** варианты ещё не сгенерированы
- **THEN** фото доступно с оригиналом, а thumb и preview отсутствуют
### Requirement: Совместные объекты
Несколько фото SHALL иметь возможность ссылаться на один и тот же физический
объект; перенос и копирование SHALL изменять только ссылки и SHALL не изменять
объекты хранилища.
#### Scenario: Копирование
- **WHEN** пользователь копирует фото в другой альбом
- **THEN** создаётся новая запись, ссылающаяся на те же объекты, без операций с хранилищем
#### Scenario: Перенос
- **WHEN** пользователь переносит фото в другой альбом
- **THEN** изменяется только привязка к альбому, объекты хранилища не меняются
### Requirement: Сборка мусора по ссылкам
Объект хранилища SHALL удаляться только когда на него не ссылается ни одно фото;
удаление альбома и попадание фото в корзину SHALL сохранять объекты.
#### Scenario: Последняя ссылка удалена
- **WHEN** фото удаляется навсегда и больше ни одно фото не ссылается на его объекты
- **THEN** объекты удаляются из хранилища
#### Scenario: Остались ссылки
- **WHEN** на объект ссылается другое фото
- **THEN** объект сохраняется в хранилище
#### Scenario: Удаление альбома
- **WHEN** владелец удаляет альбом
- **THEN** объекты хранилища не удаляются
### Requirement: Устойчивость жизненного цикла объекта
Объект SHALL иметь состояние жизненного цикла, и незавершённые операции SHALL
самовосстанавливаться: неудачная загрузка не оставляет осиротевших объектов, а
удаление объекта SHALL доводиться до конца после сбоя.
#### Scenario: Сбой загрузки
- **WHEN** запись объекта создана, но загрузка в хранилище не завершилась
- **THEN** объект очищается без ручного вмешательства
#### Scenario: Сбой удаления
- **WHEN** удаление объекта из хранилища не удалось
- **THEN** попытка повторяется до успешного удаления
### Requirement: Генерация вариантов
Генерация thumb и preview SHALL выполняться через надёжную очередь заданий с
однократным выполнением и повтором при сбое; готовый результат SHALL
транслироваться подписчикам альбома.
#### Scenario: Повтор после рестарта
- **WHEN** процесс перезапускается с незавершёнными заданиями
- **THEN** генерация вариантов продолжается
#### Scenario: Событие после генерации
- **WHEN** варианты фото готовы
- **THEN** подписчикам альбома публикуется событие с этим фото
@@ -0,0 +1,14 @@
# Spec Delta
## MODIFIED Requirements
### Requirement: Удаление фото
Автор SHALL иметь возможность удалить своё фото; владелец и админ SHALL иметь возможность удалить любое фото. Удаление SHALL помещать фото в корзину и SHALL не удалять объекты хранилища.
#### Scenario: Удаление своего фото
- **WHEN** автор удаляет своё фото
- **THEN** фото помещается в корзину, а объекты хранилища сохраняются
#### Scenario: Удаление чужого фото участником
- **WHEN** участник с ролью member пытается удалить чужое фото
- **THEN** система отклоняет запрос
@@ -0,0 +1,37 @@
# Tasks
## 1. Схема и миграции
- [x] 1.1 Добавить миграцию, создающую `objects`, `photo_objects` (PK `photo_id,role`, индекс `object_id`) и `variant_jobs`; проверить, что `make migrate` применяется и таблицы видны в БД
- [x] 1.2 Добавить миграцию, удаляющую `photos.storage_key`, `photos.thumb_key`, `photos.preview_key`, `photos.size`, `photos.width`, `photos.height`, `photos.mime_type`; проверить, что up и down миграции проходят в обе стороны без ошибок
## 2. Слой хранения
- [x] 2.1 Удалить `DeletePrefix` из `internal/storage/storage.go`; проверить `go build ./...` и отсутствие вызовов метода в коде
- [x] 2.2 Ввести хелпер ключа объекта вида `objects/{id}/{role}.{ext}` без идентификатора альбома; покрыть юнит-тестом, что ключ не содержит album_id
## 3. Метаданные объектов
- [x] 3.1 Реализовать стор объектов: создание `pending`, активация `ready`, пометка `deleting`, удаление строки, выборка `pending` старше порога; покрыть тестами переходы состояний
- [x] 3.2 Реализовать привязку `photo_objects` с блокировкой объекта `FOR UPDATE` и отказом при `state != 'ready'`; тест на повторную привязку одного объекта к двум фото
- [x] 3.3 Реализовать запрос GC (объекты `ready` без связок) и тесты сценариев «последняя ссылка удалена» и «остались ссылки»
## 4. Сервис фото
- [x] 4.1 Переписать `Upload`: строка объекта `pending` → S3 Put → `ready` → привязка `original` + задание в `variant_jobs`; тест, что загрузка даёт фото с URL оригинала и ключом без альбома
- [x] 4.2 Переписать `List`/`Get`/`attachURLs` на вывод ключей и `size/width/height/mime_type` джойном к объекту-оригиналу; тест, что JSON-поля и URL присутствуют как раньше
- [x] 4.3 Переписать `CopyBatch`/`MoveBatch` на работу только через связки; тест, что объекты хранилища не меняются и делятся между фото
- [x] 4.4 Переписать `PermanentDelete`: удалить связки и пометить осиротевшие объекты `deleting`; тест, что объект с нулём ссылок становится кандидатом на GC
- [x] 4.5 Убрать обращения к удалённым колонкам во всех запросах и моделях; проверить `go build ./...` и прохождение `internal/repo` тестов
## 5. Фоновый воркер
- [x] 5.1 Реализовать цикл забора `variant_jobs` через `FOR UPDATE SKIP LOCKED`, генерацию thumb/preview, привязку и публикацию SSE; тест обработки задания и повтора при ошибке
- [x] 5.2 Реализовать в воркере удаление `deleting` (с повторной проверкой нуля ссылок) и досасывание зависших `pending`; тест повторной попытки S3 Delete и очистки `pending`
- [x] 5.3 Встроить старт/останов воркера в жизненный цикл приложения; проверить, что сервер запускается и воркер выполняет петлю (по логам/интеграционному тесту)
## 6. Интеграция и документация
- [x] 6.1 Перевести обложку альбома (`CoverThumbKey`) на вывод через `photo_objects`+`objects`; тест, что список альбомов отдаёт URL обложки
- [x] 6.2 Обновить `project.md` §10 (структура в S3 и слои); проверить, что описание совпадает с реализованными ключами
- [x] 6.3 Сквозная проверка: загрузка → генерация вариантов → копирование между альбомами (0 операций S3) → перенос → окончательное удаление → сборка объектов; проверить, что объекты удаляются только при нуле ссылок
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-10-11
@@ -0,0 +1,30 @@
# Proposal
## Why
Слои данных были по умолчанию внешними (`postgres` и `s3`), поэтому самодостаточный
локальный запуск без внешних сервисов требовал явной настройки. Нужно сделать локальные
бэкенды значением по умолчанию.
## What Changes
- Сменить значения по умолчанию: `DB_BACKEND=sqlite` (`SQLITE_PATH=./galo.db`) и
`STORAGE_BACKEND=fs` (`FS_ROOT=./data`).
- Обновить `.env.example`, `project.md`, `README.md`.
- Поведение при явном выборе `postgres`/`s3` не меняется.
## Capabilities
### New Capabilities
<!-- нет -->
### Modified Capabilities
- `metadata-backends`: значением по умолчанию становится встроенная SQLite (ранее PostgreSQL).
- `storage-backends`: значением по умолчанию становится локальная ФС (ранее S3-совместимая).
## Impact
- `backend/internal/config` (значения по умолчанию) и тесты.
- `.env.example`, `project.md`, `README.md`.
- **BREAKING (поведение):** развёртывания без явных `DB_BACKEND`/`STORAGE_BACKEND`
переключатся на локальные бэкенды; для прежнего режима задайте `postgres`/`s3` явно.
@@ -0,0 +1,23 @@
# Spec Delta
## MODIFIED Requirements
### Requirement: Выбор бэкенда метаданных
Система SHALL поддерживать выбор бэкенда слоя метаданных параметром конфигурации,
при этом встроенная SQLite SHALL быть значением по умолчанию.
#### Scenario: Выбор по умолчанию
- **WHEN** бэкенд метаданных не задан в конфигурации
- **THEN** система использует встроенную SQLite с путём по умолчанию
#### Scenario: Выбор PostgreSQL
- **WHEN** в конфигурации задан бэкенд PostgreSQL
- **THEN** метаданные читаются и пишутся во внешнюю PostgreSQL
#### Scenario: Выбор встроенной SQLite
- **WHEN** в конфигурации задан бэкенд SQLite
- **THEN** метаданные читаются и пишутся в локальный файл SQLite без внешней службы
#### Scenario: Неполная конфигурация
- **WHEN** конфигурация выбранного бэкенда неполна
- **THEN** система отказывается стартовать с явной ошибкой конфигурации
@@ -0,0 +1,23 @@
# Spec Delta
## MODIFIED Requirements
### Requirement: Выбор бэкенда хранилища
Система SHALL поддерживать выбор бэкенда объектного хранилища параметром конфигурации,
при этом локальная файловая система SHALL быть значением по умолчанию.
#### Scenario: Выбор по умолчанию
- **WHEN** бэкенд хранилища не задан в конфигурации
- **THEN** система хранит и отдаёт блобы из локальной файловой системы с корнем по умолчанию
#### Scenario: Выбор S3-совместимого бэкенда
- **WHEN** в конфигурации задан S3-совместимый бэкенд
- **THEN** система хранит и отдаёт блобы через S3-совместимое хранилище
#### Scenario: Выбор локальной ФС
- **WHEN** в конфигурации задан бэкенд локальной ФС
- **THEN** система хранит и отдаёт блобы из каталога локальной файловой системы
#### Scenario: Неполная конфигурация
- **WHEN** конфигурация выбранного бэкенда неполна
- **THEN** система отказывается стартовать с явной ошибкой конфигурации
@@ -0,0 +1,6 @@
# Tasks
## 1. Дефолты и документация
- [x] 1.1 Сменить значения по умолчанию в `config` (`DB_BACKEND=sqlite`, `SQLITE_PATH=./galo.db`, `STORAGE_BACKEND=fs`, `FS_ROOT=./data`) и обновить тесты конфига; проверка — `go test ./internal/config` зелёный.
- [x] 1.2 Обновить `.env.example`, `project.md`, `README.md`; проверка — запуск без переменных окружения стартует на SQLite+ФС и health показывает `db=ok, fs=ok`.
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-10-11
@@ -0,0 +1,107 @@
# Design
## Context
См. `proposal.md` — Why. Текущее состояние, определяющее подход:
- `storage.Backend` уже введён; `storage.S3` возвращает presigned URL на Garage, а
`storage.FS` отдаёт блобы через приложение. Presign уводит браузер напрямую в S3.
- В проде Garage (`s3.localnet`) недоступен извне, поэтому presigned URL не работают:
отдача обязана идти через бэкенд для обоих бэкендов.
- Подпись URL сейчас живёт внутри `FS` (`fs.go: sign/verify/URL`).
- Маршрут `/api/objects/*key` и `app.serveObject` уже есть и диспетчеризуются по
необязательному интерфейсу `storage.ObjectServer`.
- Ключи объектов непрозрачны (`objects/{id}/{role}{ext}`) и одинаковы для всех бэкендов.
- Сборка `CGO_ENABLED=0`, один бинарник + systemd, перед Go стоит nginx.
## Goals / Non-Goals
**Goals:**
- Единая модель доставки: любой блоб клиент получает только через бэкенд.
- Убрать presigned URL полностью; S3 отдаёт байты через прокси бэкенда.
- Сохранить выбор бэкенда хранилища (`s3|fs`) и непрозрачные ключи.
- Сохранить работу анонимных share-ссылок без пользовательской сессии.
**Non-Goals:**
- Миграция существующих блобов между бэкендами.
- Переключение бэкенда в рантайме.
- Кэширующий CDN и раздача через внешний веб-сервер.
- Изменение слоя метаданных (см. `sqlite-metadata-backend`).
## Decisions
### D1. `ServeObject` — обязательная часть `storage.Backend`
Отдача через бэкенд нужна всем бэкендам, поэтому `ServeObject(w, r, key)` входит в
интерфейс `Backend`, а не остаётся опциональным `ObjectServer`. Компилятор не даст
собрать бэкенд без отдачи, а `app.serveObject` диспетчеризуется без type-assert.
- Альтернатива: оставить `ObjectServer` опциональным — теперь это всегда реализуется,
опциональность только маскирует ошибку.
### D2. `URL` всегда возвращает подписанный URL приложения
`Backend.URL` больше не зависит от бэкенда в способе доставки: и `s3`, и `fs` возвращают
`${APP_BASE_URL}/api/objects/<key>?e=<exp>&s=<sig>`. Presign-клиент AWS удаляется.
- Альтернатива (dev): оставить presign для S3 — отвергнута по требованию «только через
бэкенд» и чтобы не иметь двух режимов доставки.
### D3. Общий `Signer`
HMAC-SHA256 от `key + exp` выносится в общий `Signer` (секрет `URL_SECRET` с fallback на
`JWT_SECRET`, срок `OBJECT_URL_TTL`). Его используют и FS, и S3, и маршрут проверяет
подпись одинаково. Внутри `FS` дублирующая подпись удаляется.
- Альтернатива: дублировать HMAC в каждом бэкенде — риск рассинхрона проверки.
### D4. Отдача S3: ranged `GetObject` и стриминг
S3-бэкенд в `ServeObject` определяет `Range` из запроса, вызывает `GetObject` с тем же
диапазоном (Garage отдаёт `Content-Range`) и стримит тело в ответ через `io.Copy`
(200/206/416). HEAD обслуживается `HeadObject`. Память не растёт, размер берётся из
`Content-Length` ответа S3.
- Альтернатива: `http.ServeContent` требует `io.ReadSeeker`; для S3 это ranged-GET-адаптер
и лишний size-probe — отвергнуто ради простоты нативного passthrough.
- `S3.Get` (в память) сохраняется только для воркера/backfill, отдачу не использует.
### D5. Отдача FS: без изменений
`FS.ServeObject` остаётся на `http.ServeContent` (Range и Content-Type бесплатно).
Меняется лишь источник подписи — общий `Signer`.
### D6. Публичный маршрут и доступ
`/api/objects/*key` вне группы авторизации; подпись и срок — единственный контроль
доступа, поэтому он обслуживает и авторизованных, и анонимные share-ссылки. Наружу не
уходят ни S3-endpoint, ни AWS-подпись, ни ключ с секретом.
### D7. Конфигурация
`STORAGE_BACKEND=s3|fs` без изменений (дефолт `s3`). `FS_ROOT` — для `fs`.
`S3_PRESIGN_TTL` переименовывается в `OBJECT_URL_TTL` (семантика: срок подписи URL).
`S3_PUBLIC_URL` удаляется как неиспользуемая. `APP_BASE_URL` становится определяющим для
корректности ссылок; пустое значение даёт относительный `/api/objects/...` (полагается на
nginx того же origin).
### D8. nginx для прокси-отдачи
`location /api/objects/` обслуживается как `location /`, но с `proxy_buffering off` и
`X-Accel-Buffering: no`, чтобы стриминг и Range не буферизовались. SSE-правило остаётся.
## Risks / Trade-offs
- [Все байты медиа идут через Go] → стриминг без чтения в память; `proxy_buffering off`;
для крупных инсталляций это принятая плата за требование «только через бэкенд».
- [Подписанный URL действует как bearer до истечения] → срок `OBJECT_URL_TTL` (1 час),
отдельный `URL_SECRET` для отзыва.
- [Range/HEAD через прокси] → passthrough `Range` в S3 и `HeadObject` для HEAD; покрыто
httptest-тестами (200/206/416/404).
- [Ломающее переименование `S3_PRESIGN_TTL`] → задокументировать в `.env.example` и
`project.md`; при откате вернуть имя.
- [Обход каталога через ключ FS] → нормализация пути и проверка префикса `FS_ROOT`.
- [Смена бэкенда на существующих данных не переносит блобы] → документируем; миграция вне
рамок.
## Migration Plan
1. Вынести `Signer`, сделать `ServeObject` обязательным, `URL` — всегда app-signed.
2. Реализовать S3 `ServeObject` (ranged `GetObject` + стриминг, HEAD).
3. Удалить presign-путь и `S3_PUBLIC_URL`; переименовать TTL; обновить доки и nginx.
4. Существующие развёртывания: дефолт `s3` сохраняется, но доставка теперь проксируется —
внешний доступ к S3 больше не нужен.
5. Перенос блобов между бэкендами не автоматизируется.
## Open Questions
- Нужно ли выставлять `Cache-Control` на ответы `/api/objects/` помимо браузерного кэша
по уникальному URL.
@@ -0,0 +1,40 @@
# Proposal
## Why
S3-совместимое хранилище (Garage на `s3.localnet`) в проде недоступно извне, поэтому
отдача медиа не может идти напрямую из хранилища в браузер. Нужен слой хранения,
который всегда отдаёт блобы через бэкенд, и локальный бэкенд для инсталляций без S3.
## What Changes
- Ввести интерфейс `storage.Backend`: Put/Delete/Get/Ping/EnsureBucket/Bucket, `URL`
(всегда подписанный URL приложения) и обязательный `ServeObject`.
- Реализовать бэкенд локальной ФС: блобы под корневым каталогом по непрозрачным ключам
`objects/{id}/{role}{ext}`, защита от обхода пути, отдача через `ServeContent`.
- Реализовать отдачу S3 через бэкенд: `ServeObject` тянет объект ranged `GetObject` из
Garage (внутренняя сеть) и стримит браузеру; presigned URL больше не используются.
- Публичный маршрут `/api/objects/<key>` с HMAC-подписью, сроком, Range и Content-Type
обслуживает и авторизованных, и анонимные share-ссылки.
- Ввести `STORAGE_BACKEND=s3|fs` и `FS_ROOT`; убрать `S3_PUBLIC_URL`; переименовать
`S3_PRESIGN_TTL` → `OBJECT_URL_TTL`; дефолт — `s3`.
- Сделать метку хранилища в health динамической.
- Сделать формулировки отдачи файлов нейтральными к бэкенду в спеках `photos` и `share-links`.
## Capabilities
### New Capabilities
- `storage-backends`: выбор и работа объектного хранилища — S3-совместимое (Garage) и локальная ФС; блобы всегда отдаются через бэкенд по подписанному URL, хранилище клиенту не раскрывается.
### Modified Capabilities
- `photos`: получение фото и скачивание не требуют presigned URL и обслуживаются бэкендом (подписанный URL).
- `share-links`: просмотр по ссылке отдаёт подписанный URL, обслуживаемый бэкендом.
## Impact
- `backend/internal/storage` (интерфейс, общий `Signer`, FS-бэкенд, S3-proxy-отдача),
сервисы `photos` и `albums`, `app` (маршрут отдачи, health), `cmd/*`, `config`.
- Удаление presign-пути AWS SDK из `storage`.
- `.env.example`, `project.md`, `deploy/nginx-sse.conf`.
- Спеки `photos`, `share-links`, новая `storage-backends`.
- Не затрагивает слой метаданных.
@@ -0,0 +1,31 @@
# Spec Delta
## MODIFIED Requirements
### Requirement: Получение фото по требованию
Клиент SHALL иметь возможность получить актуальные данные фото с подписанным URL,
обслуживаемым бэкендом, независимо от бэкенда хранения.
#### Scenario: Выдача свежего URL
- **WHEN** клиент запрашивает фото по id
- **THEN** система возвращает свежий подписанный URL бэкенда (срок до 1 часа) для просмотра
### Requirement: Скачивание одиночного фото
Любой участник, включая viewer, SHALL иметь возможность скачать одиночное фото.
#### Scenario: Скачивание фото
- **WHEN** участник запрашивает скачивание фото
- **THEN** система возвращает подписанный URL оригинала, обслуживаемый бэкендом
### Requirement: Перенос и копирование фото между альбомами
Пользователь с правом удаления в исходном альбоме и правом загрузки в целевом SHALL иметь
возможность перенести или скопировать фото в другой альбом. Операции SHALL изменять только
метаданные и SHALL не изменять объекты хранилища.
#### Scenario: Перенос
- **WHEN** пользователь переносит фото в другой альбом
- **THEN** фото исчезает из исходного альбома и появляется в целевом, объекты хранилища не меняются
#### Scenario: Копирование
- **WHEN** пользователь копирует фото в другой альбом
- **THEN** в целевом альбоме появляется новая запись, ссылающаяся на те же объекты хранилища

Some files were not shown because too many files have changed in this diff Show More