Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e0b4db926f | ||
|
|
8638d1bbfc | ||
|
|
cb272398ce | ||
|
|
176a8d8ced | ||
|
|
ec8d907565 | ||
|
|
7ae9bcf305 | ||
|
|
96268ee16c | ||
|
|
9effc0fc82 |
+23
-5
@@ -6,18 +6,31 @@ PG_PASSWORD=change_me
|
||||
PG_DATABASE=shared_album
|
||||
PG_SSLMODE=disable
|
||||
|
||||
# S3 (Garage on s3.localnet)
|
||||
# Metadata backend: sqlite (default, embedded single instance) or postgres
|
||||
DB_BACKEND=sqlite
|
||||
# Path to the SQLite database file (used when DB_BACKEND=sqlite)
|
||||
SQLITE_PATH=./galo.db
|
||||
|
||||
# S3 (Garage on s3.localnet) - internal storage, never exposed to clients
|
||||
S3_ENDPOINT=http://s3.localnet:3900
|
||||
S3_REGION=garage
|
||||
S3_ACCESS_KEY=GKchange_me
|
||||
S3_SECRET_KEY=change_me
|
||||
S3_BUCKET=galo-data
|
||||
S3_USE_PATH_STYLE=true
|
||||
S3_PUBLIC_URL=http://s3.localnet:3900/galo-data
|
||||
S3_PRESIGN_TTL=1h
|
||||
|
||||
# Auth
|
||||
JWT_SECRET=change_me_long_random
|
||||
# Storage backend: fs (default, local filesystem) or s3 (S3-compatible/Garage).
|
||||
# Blobs are always delivered through the backend via /api/objects/<key>.
|
||||
STORAGE_BACKEND=fs
|
||||
# Root directory for object blobs (STORAGE_BACKEND=fs)
|
||||
FS_ROOT=./data
|
||||
# Signs object URLs; falls back to the metadata-layer secret when empty
|
||||
URL_SECRET=
|
||||
# Lifetime of signed object URLs
|
||||
OBJECT_URL_TTL=1h
|
||||
|
||||
# Auth. The token signing secret is generated on first start and stored in the
|
||||
# metadata layer (app_settings); it is not configured here.
|
||||
JWT_ACCESS_TTL=15m
|
||||
JWT_REFRESH_TTL=720h
|
||||
|
||||
@@ -28,3 +41,8 @@ APP_BASE_URL=http://localhost:8080
|
||||
# SSE
|
||||
SSE_HEARTBEAT_INTERVAL=25s
|
||||
SSE_CLIENT_BUFFER=64
|
||||
|
||||
# Background worker (variant generation + object GC)
|
||||
WORKER_INTERVAL=5s
|
||||
WORKER_BATCH=10
|
||||
WORKER_PENDING_TTL=10m
|
||||
|
||||
@@ -7,3 +7,8 @@ frontend/build/
|
||||
frontend/.svelte-kit/
|
||||
node_modules/
|
||||
*.log
|
||||
|
||||
# Local data backends (defaults)
|
||||
galo.db
|
||||
galo.db-*
|
||||
data/
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
.PHONY: build frontend backend run migrate tidy test clean
|
||||
.PHONY: build frontend backend run migrate reconcile reconcile-delete tidy test clean
|
||||
|
||||
frontend:
|
||||
cd frontend && npm ci && npm run build
|
||||
@@ -20,6 +20,15 @@ run: backend
|
||||
migrate:
|
||||
cd backend && go run ./cmd/migrate
|
||||
|
||||
# Reconcile object storage against the metadata journal (report only).
|
||||
reconcile:
|
||||
cd backend && go run ./cmd/reconcile
|
||||
|
||||
# Reconcile and delete orphaned blobs older than --min-age (default 1h).
|
||||
# Override the grace period when needed: make reconcile-delete MIN_AGE=1m
|
||||
reconcile-delete:
|
||||
cd backend && go run ./cmd/reconcile --delete $(if $(MIN_AGE),--min-age $(MIN_AGE),)
|
||||
|
||||
tidy:
|
||||
cd backend && go mod tidy
|
||||
|
||||
|
||||
@@ -30,8 +30,19 @@ cp .env.example .env
|
||||
```
|
||||
|
||||
Ключевые переменные: `PG_*`, `S3_*` (обязательны `UsePathStyle=true` и регион
|
||||
`garage`), `JWT_SECRET`, `JWT_ACCESS_TTL`, `JWT_REFRESH_TTL`, `HTTP_ADDR`,
|
||||
`SSE_HEARTBEAT_INTERVAL`, `SSE_CLIENT_BUFFER`.
|
||||
`garage`), `JWT_ACCESS_TTL`, `JWT_REFRESH_TTL`, `HTTP_ADDR`,
|
||||
`SSE_HEARTBEAT_INTERVAL`, `SSE_CLIENT_BUFFER`. Секрет подписи токенов не задаётся
|
||||
в окружении: он генерируется при первом старте и хранится в слое метаданных
|
||||
(`app_settings`).
|
||||
|
||||
Слои данных выбираются параметрами (по умолчанию — локальный режим):
|
||||
|
||||
- `STORAGE_BACKEND=fs|s3` — где лежат блобы: `fs` (по умолчанию, `FS_ROOT=./data`)
|
||||
или `s3` (S3/Garage); наружу всегда отдаётся бэкенд (`/api/objects/<key>`).
|
||||
- `DB_BACKEND=sqlite|postgres` — где лежат метаданные: `sqlite` (по умолчанию,
|
||||
`SQLITE_PATH=./galo.db`) или `postgres`. SQLite — встроенный режим без внешней БД,
|
||||
**только для одного экземпляра приложения на файл** (без горизонтального
|
||||
масштабирования).
|
||||
|
||||
## Сборка
|
||||
|
||||
@@ -94,6 +105,21 @@ TEST_DATABASE_URL='postgres://user:pass@pg.localnet:5432/galo?sslmode=disable' \
|
||||
cd backend && go test ./internal/repo/ -v
|
||||
```
|
||||
|
||||
## Реконсиляция хранилища
|
||||
|
||||
Сборка мусора воркера управляется журналом `objects` и не сканирует хранилище, поэтому
|
||||
блобы без строки в журнале (после сброса БД в обход журнала и т.п.) накапливаются.
|
||||
Ручная сверка:
|
||||
|
||||
```bash
|
||||
go run ./cmd/reconcile # отчёт: сироты (есть в хранилище, нет в журнале) и пропавшие
|
||||
go run ./cmd/reconcile --delete # удалить сироты старше --min-age (по умолчанию 1h)
|
||||
```
|
||||
|
||||
- По умолчанию ничего не удаляется; удаление только по явному `--delete`.
|
||||
- Область — префикс `objects/`; блобы старого формата `albums/` не затрагиваются.
|
||||
- Пропавшие ключи (в журнале есть, в хранилище нет) только сообщаются.
|
||||
|
||||
## Деплой через systemd
|
||||
|
||||
1. Соберите бинарник: `make build`.
|
||||
@@ -129,11 +155,12 @@ backend/
|
||||
cmd/migrate/ # применение миграций
|
||||
internal/config # чтение env
|
||||
internal/db # pgx + golang-migrate
|
||||
internal/storage # S3 (Garage)
|
||||
internal/storage # S3 (Garage) / локальная ФС, отдача через бэкенд
|
||||
internal/store # фабрика репозиториев (postgres / sqlite)
|
||||
internal/auth # JWT, bcrypt, refresh
|
||||
internal/albums # альбомы
|
||||
internal/members # участники, роли, инвайты
|
||||
internal/photos # загрузка, варианты, presigned URL
|
||||
internal/photos # загрузка, варианты, отдача через бэкенд
|
||||
internal/social # лайки, комментарии
|
||||
internal/notifications # персональные уведомления
|
||||
internal/sharelinks # ссылки для просмотра
|
||||
|
||||
@@ -9,8 +9,9 @@ import (
|
||||
|
||||
"shared-album/internal/config"
|
||||
"shared-album/internal/db"
|
||||
"shared-album/internal/photos"
|
||||
"shared-album/internal/settings"
|
||||
"shared-album/internal/storage"
|
||||
"shared-album/internal/store"
|
||||
)
|
||||
|
||||
func main() {
|
||||
@@ -35,14 +36,20 @@ func main() {
|
||||
}
|
||||
defer pool.Close()
|
||||
|
||||
store, err := storage.New(ctx, cfg.S3)
|
||||
repos := store.NewPostgres(pool)
|
||||
secret, err := repos.Settings.Ensure(ctx, settings.JWTSecretKey, settings.GenerateSecret)
|
||||
if err != nil {
|
||||
slog.Error("secret init failed", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
blob, err := storage.Open(ctx, cfg, secret)
|
||||
if err != nil {
|
||||
slog.Error("storage init failed", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
photoStore := photos.NewStore(pool)
|
||||
updated, err := photoStore.BackfillEXIF(ctx, store.Get)
|
||||
updated, err := repos.Photos.BackfillEXIF(ctx, blob.Get)
|
||||
if err != nil {
|
||||
slog.Error("backfill failed", "error", err)
|
||||
os.Exit(1)
|
||||
|
||||
@@ -0,0 +1,169 @@
|
||||
// Command reconcile compares the object storage with the metadata journal and
|
||||
// reports orphaned blobs (present in storage, absent from the journal). With
|
||||
// --delete it removes orphans older than --min-age. It never touches missing
|
||||
// keys (in the journal but not in storage) and never runs automatically.
|
||||
package main
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"flag"
|
||||
"io/fs"
|
||||
"log/slog"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
assets "shared-album"
|
||||
"shared-album/internal/config"
|
||||
"shared-album/internal/db"
|
||||
"shared-album/internal/reconcile"
|
||||
"shared-album/internal/settings"
|
||||
"shared-album/internal/storage"
|
||||
"shared-album/internal/store"
|
||||
)
|
||||
|
||||
const objectPrefix = "objects/"
|
||||
|
||||
func main() {
|
||||
loadDotEnv(".env")
|
||||
loadDotEnv("../.env")
|
||||
|
||||
slog.SetDefault(slog.New(slog.NewTextHandler(os.Stdout, &slog.HandlerOptions{Level: slog.LevelInfo})))
|
||||
|
||||
deleteFlag := flag.Bool("delete", false, "delete orphaned blobs older than --min-age")
|
||||
minAge := flag.Duration("min-age", time.Hour, "minimum blob age before deletion is allowed")
|
||||
sample := flag.Int("sample", 20, "number of example keys to print")
|
||||
flag.Parse()
|
||||
|
||||
cfg, err := config.Load()
|
||||
if err != nil {
|
||||
slog.Error("config error", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
ctx := context.Background()
|
||||
|
||||
repos, closer, err := openRepos(ctx, cfg)
|
||||
if err != nil {
|
||||
slog.Error("metadata open failed", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
defer closer()
|
||||
|
||||
secret, err := repos.Settings.Ensure(ctx, settings.JWTSecretKey, settings.GenerateSecret)
|
||||
if err != nil {
|
||||
slog.Error("secret init failed", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
blob, err := storage.Open(ctx, cfg, secret)
|
||||
if err != nil {
|
||||
slog.Error("storage init failed", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
objects, err := blob.List(ctx, objectPrefix)
|
||||
if err != nil {
|
||||
slog.Error("list storage", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
journaled, err := repos.Photos.AllStorageKeys(ctx)
|
||||
if err != nil {
|
||||
slog.Error("list journal", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
res := reconcile.Classify(objects, journaled)
|
||||
slog.Info("reconciliation report",
|
||||
"storage", cfg.Storage.Backend,
|
||||
"storage_objects", len(objects),
|
||||
"journal_keys", len(journaled),
|
||||
"orphans", len(res.Orphans),
|
||||
"missing", len(res.Missing))
|
||||
printSamples("orphan", res.Orphans, *sample)
|
||||
printSamples("missing", res.Missing, *sample)
|
||||
|
||||
if !*deleteFlag {
|
||||
slog.Info("dry run: nothing deleted (pass --delete to remove orphans)")
|
||||
return
|
||||
}
|
||||
|
||||
targets := reconcile.Deletable(objects, res, objectPrefix, *minAge, time.Now())
|
||||
if len(targets) == 0 {
|
||||
slog.Info("no orphaned blobs old enough to delete", "min_age", minAge.String())
|
||||
return
|
||||
}
|
||||
if err := blob.Delete(ctx, targets...); err != nil {
|
||||
slog.Error("delete orphans", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
slog.Info("deleted orphaned blobs", "count", len(targets), "min_age", minAge.String())
|
||||
}
|
||||
|
||||
// openRepos builds the metadata repositories for the configured backend.
|
||||
func openRepos(ctx context.Context, cfg *config.Config) (store.Repos, func(), error) {
|
||||
if cfg.Database.Backend == "sqlite" {
|
||||
migFS, err := fs.Sub(assets.Migrations, "migrations/sqlite")
|
||||
if err != nil {
|
||||
return store.Repos{}, nil, err
|
||||
}
|
||||
if err := db.MigrateSQLite(cfg.Database.SQLitePath, migFS); err != nil {
|
||||
return store.Repos{}, nil, err
|
||||
}
|
||||
database, err := db.OpenSQLite(cfg.Database.SQLitePath)
|
||||
if err != nil {
|
||||
return store.Repos{}, nil, err
|
||||
}
|
||||
return store.NewSQLite(database), func() { _ = database.Close() }, nil
|
||||
}
|
||||
|
||||
migFS, err := fs.Sub(assets.Migrations, "migrations")
|
||||
if err != nil {
|
||||
return store.Repos{}, nil, err
|
||||
}
|
||||
if err := db.Migrate(cfg.Postgres.DSN(), migFS); err != nil {
|
||||
return store.Repos{}, nil, err
|
||||
}
|
||||
pool, err := db.Connect(ctx, cfg.Postgres.DSN())
|
||||
if err != nil {
|
||||
return store.Repos{}, nil, err
|
||||
}
|
||||
return store.NewPostgres(pool), pool.Close, nil
|
||||
}
|
||||
|
||||
func printSamples(label string, keys []string, n int) {
|
||||
if len(keys) == 0 {
|
||||
return
|
||||
}
|
||||
if n > len(keys) {
|
||||
n = len(keys)
|
||||
}
|
||||
for _, k := range keys[:n] {
|
||||
slog.Info("sample", "kind", label, "key", k)
|
||||
}
|
||||
}
|
||||
|
||||
func loadDotEnv(path string) {
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
defer f.Close()
|
||||
scanner := bufio.NewScanner(f)
|
||||
for scanner.Scan() {
|
||||
line := strings.TrimSpace(scanner.Text())
|
||||
if line == "" || strings.HasPrefix(line, "#") {
|
||||
continue
|
||||
}
|
||||
key, value, ok := strings.Cut(line, "=")
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
key = strings.TrimSpace(key)
|
||||
value = strings.TrimSpace(value)
|
||||
if _, exists := os.LookupEnv(key); !exists {
|
||||
_ = os.Setenv(key, value)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -15,10 +15,13 @@ import (
|
||||
|
||||
assets "shared-album"
|
||||
"shared-album/internal/app"
|
||||
"shared-album/internal/auth"
|
||||
"shared-album/internal/config"
|
||||
"shared-album/internal/db"
|
||||
"shared-album/internal/settings"
|
||||
"shared-album/internal/sse"
|
||||
"shared-album/internal/storage"
|
||||
"shared-album/internal/store"
|
||||
)
|
||||
|
||||
func main() {
|
||||
@@ -34,9 +37,49 @@ func main() {
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
switch cfg.Database.Backend {
|
||||
case "sqlite":
|
||||
slog.Info("metadata backend", "backend", "sqlite", "path", cfg.Database.SQLitePath)
|
||||
default:
|
||||
slog.Info("metadata backend", "backend", "postgres",
|
||||
"host", cfg.Postgres.Host, "database", cfg.Postgres.Database)
|
||||
}
|
||||
switch cfg.Storage.Backend {
|
||||
case "fs":
|
||||
slog.Info("storage backend", "backend", "fs", "root", cfg.Storage.FSRoot)
|
||||
default:
|
||||
slog.Info("storage backend", "backend", "s3",
|
||||
"endpoint", cfg.S3.Endpoint, "bucket", cfg.S3.Bucket)
|
||||
}
|
||||
|
||||
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
|
||||
defer stop()
|
||||
|
||||
var (
|
||||
repos store.Repos
|
||||
ping func(ctx context.Context) error
|
||||
closer func()
|
||||
)
|
||||
switch cfg.Database.Backend {
|
||||
case "sqlite":
|
||||
migFS, err := fs.Sub(assets.Migrations, "migrations/sqlite")
|
||||
if err != nil {
|
||||
slog.Error("migrations fs", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
if err := db.MigrateSQLite(cfg.Database.SQLitePath, migFS); err != nil {
|
||||
slog.Error("migration failed", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
database, err := db.OpenSQLite(cfg.Database.SQLitePath)
|
||||
if err != nil {
|
||||
slog.Error("database open failed", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
repos = store.NewSQLite(database)
|
||||
ping = database.PingContext
|
||||
closer = func() { _ = database.Close() }
|
||||
default:
|
||||
migFS, err := fs.Sub(assets.Migrations, "migrations")
|
||||
if err != nil {
|
||||
slog.Error("migrations fs", "error", err)
|
||||
@@ -46,28 +89,40 @@ func main() {
|
||||
slog.Error("migration failed", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
slog.Info("migrations applied")
|
||||
|
||||
pool, err := db.Connect(ctx, cfg.Postgres.DSN())
|
||||
if err != nil {
|
||||
slog.Error("database connect failed", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
defer pool.Close()
|
||||
repos = store.NewPostgres(pool)
|
||||
ping = pool.Ping
|
||||
closer = pool.Close
|
||||
}
|
||||
defer closer()
|
||||
slog.Info("migrations applied", "db_backend", cfg.Database.Backend)
|
||||
|
||||
store, err := storage.New(ctx, cfg.S3)
|
||||
secret, err := repos.Settings.Ensure(ctx, settings.JWTSecretKey, settings.GenerateSecret)
|
||||
if err != nil {
|
||||
slog.Error("secret init failed", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
tm := auth.NewTokenManager(secret, cfg.Auth.AccessTTL, cfg.Auth.RefreshTTL)
|
||||
|
||||
blob, err := storage.Open(ctx, cfg, secret)
|
||||
if err != nil {
|
||||
slog.Error("storage init failed", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
if err := store.EnsureBucket(ctx); err != nil {
|
||||
if err := blob.EnsureBucket(ctx); err != nil {
|
||||
slog.Warn("could not ensure bucket", "error", err)
|
||||
} else {
|
||||
slog.Info("storage bucket ready", "bucket", store.Bucket())
|
||||
slog.Info("storage ready", "location", blob.Bucket())
|
||||
}
|
||||
|
||||
hub := sse.NewHub(cfg.SSE.ClientBuffer)
|
||||
server := app.New(cfg, pool, store, hub)
|
||||
server := app.New(cfg, repos, ping, tm, blob, hub)
|
||||
|
||||
go server.StartWorker(ctx)
|
||||
|
||||
httpServer := &http.Server{
|
||||
Addr: cfg.Server.Addr,
|
||||
|
||||
+2
-2
@@ -7,7 +7,7 @@ import "embed"
|
||||
//go:embed all:static
|
||||
var Static embed.FS
|
||||
|
||||
// Migrations holds the SQL migration files.
|
||||
// Migrations holds the SQL migration files for every supported backend.
|
||||
//
|
||||
//go:embed migrations/*.sql
|
||||
//go:embed migrations/*.sql migrations/sqlite/*.sql
|
||||
var Migrations embed.FS
|
||||
|
||||
+8
-1
@@ -17,6 +17,7 @@ require (
|
||||
github.com/google/uuid v1.6.0
|
||||
github.com/jackc/pgx/v5 v5.11.0
|
||||
golang.org/x/crypto v0.58.0
|
||||
modernc.org/sqlite v1.60.1
|
||||
)
|
||||
|
||||
require (
|
||||
@@ -39,6 +40,7 @@ require (
|
||||
github.com/cloudwego/base64x v0.1.6 // indirect
|
||||
github.com/dsoprea/go-logging v0.0.0-20200710184922-b02d349568dd // indirect
|
||||
github.com/dsoprea/go-utility/v2 v2.0.0-20221003172846-a3e1774ef349 // indirect
|
||||
github.com/dustin/go-humanize v1.0.1 // indirect
|
||||
github.com/ebitengine/purego v0.10.1 // indirect
|
||||
github.com/gabriel-vasile/mimetype v1.4.12 // indirect
|
||||
github.com/gin-contrib/sse v1.1.0 // indirect
|
||||
@@ -56,12 +58,14 @@ require (
|
||||
github.com/json-iterator/go v1.1.12 // indirect
|
||||
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
|
||||
github.com/leodido/go-urn v1.4.0 // indirect
|
||||
github.com/mattn/go-isatty v0.0.20 // indirect
|
||||
github.com/mattn/go-isatty v0.0.24 // indirect
|
||||
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
|
||||
github.com/modern-go/reflect2 v1.0.2 // indirect
|
||||
github.com/ncruces/go-strftime v1.0.0 // indirect
|
||||
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
|
||||
github.com/quic-go/qpack v0.6.0 // indirect
|
||||
github.com/quic-go/quic-go v0.59.0 // indirect
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
|
||||
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
|
||||
github.com/ugorji/go/codec v1.3.1 // indirect
|
||||
go.mongodb.org/mongo-driver/v2 v2.5.0 // indirect
|
||||
@@ -73,4 +77,7 @@ require (
|
||||
golang.org/x/text v0.43.0 // indirect
|
||||
google.golang.org/protobuf v1.36.11 // indirect
|
||||
gopkg.in/yaml.v2 v2.4.0 // indirect
|
||||
modernc.org/libc v1.77.1 // indirect
|
||||
modernc.org/mathutil v1.7.1 // indirect
|
||||
modernc.org/memory v1.12.1 // indirect
|
||||
)
|
||||
|
||||
+44
-3
@@ -85,6 +85,8 @@ github.com/dsoprea/go-utility/v2 v2.0.0-20221003142440-7a1927d49d9d/go.mod h1:LV
|
||||
github.com/dsoprea/go-utility/v2 v2.0.0-20221003160719-7bc88537c05e/go.mod h1:VZ7cB0pTjm1ADBWhJUOHESu4ZYy9JN+ZPqjfiW09EPU=
|
||||
github.com/dsoprea/go-utility/v2 v2.0.0-20221003172846-a3e1774ef349 h1:DilThiXje0z+3UQ5YjYiSRRzVdtamFpvBQXKwMglWqw=
|
||||
github.com/dsoprea/go-utility/v2 v2.0.0-20221003172846-a3e1774ef349/go.mod h1:4GC5sXji84i/p+irqghpPFZBF8tRN/Q7+700G0/DLe8=
|
||||
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
|
||||
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
|
||||
github.com/ebitengine/purego v0.10.1 h1:dewVBCBT2GaMu1SrNTYxQhgQBethzfhiwvZiLGP/qyY=
|
||||
github.com/ebitengine/purego v0.10.1/go.mod h1:iIjxzd6CiRiOG0UyXP+V1+jWqUXVjPKLAI0mRfJZTmQ=
|
||||
github.com/felixge/httpsnoop v1.1.0 h1:3YtUj32ZZkqZtt3sZZsClsymw/QDuVfpNhoA31zeORc=
|
||||
@@ -129,8 +131,12 @@ github.com/golang/geo v0.0.0-20210211234256-740aa86cb551/go.mod h1:QZ0nwyI2jOfgR
|
||||
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
|
||||
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
|
||||
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
|
||||
github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3 h1:LMLX+LgTNWpfvCBdFebv6EsYotImrt/Ppc5cXIriCSo=
|
||||
github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3/go.mod h1:jl5iWTm0/hd5PjEYEOuwAJ57L/CibdZfrqZ5XA5GrCk=
|
||||
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
|
||||
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
|
||||
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
|
||||
github.com/jackc/pgerrcode v0.0.0-20220416144525-469b46aa5efa h1:s+4MhCQ6YrzisK6hFJUX53drDT4UsSW3DEhKn0ifuHw=
|
||||
github.com/jackc/pgerrcode v0.0.0-20220416144525-469b46aa5efa/go.mod h1:a/s9Lp5W7n/DD0VrVoyJ00FbP2ytTPDVOivvn2bMlds=
|
||||
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
|
||||
@@ -155,8 +161,8 @@ github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
|
||||
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
|
||||
github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
|
||||
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
|
||||
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
|
||||
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
|
||||
github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=
|
||||
github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
|
||||
github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0=
|
||||
github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
|
||||
github.com/moby/moby/api v1.54.2 h1:wiat9QAhnDQjA7wk1kh/TqHz2I1uUA7M7t9SAl/JNXg=
|
||||
@@ -172,6 +178,8 @@ github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9G
|
||||
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
|
||||
github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
|
||||
github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
|
||||
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
|
||||
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
|
||||
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
|
||||
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
|
||||
github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040=
|
||||
@@ -187,6 +195,8 @@ github.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8=
|
||||
github.com/quic-go/qpack v0.6.0/go.mod h1:lUpLKChi8njB4ty2bFLX2x4gzDqXwUpaO1DP9qMDZII=
|
||||
github.com/quic-go/quic-go v0.59.0 h1:OLJkp1Mlm/aS7dpKgTc6cnpynnD2Xg7C1pwL6vy/SAw=
|
||||
github.com/quic-go/quic-go v0.59.0/go.mod h1:upnsH4Ju1YkqpLXC305eW3yDZ4NfnNbmQRCMWS58IKU=
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
|
||||
github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ=
|
||||
github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog=
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
@@ -226,6 +236,8 @@ golang.org/x/crypto v0.58.0 h1:COkYLr4k7nDI4r6QtMML2AfzeIswWHAJthLS6K/M/54=
|
||||
golang.org/x/crypto v0.58.0/go.mod h1:Xh+kl5A+M0gMiWDytH2Hhr5Z7WbNCpv/eB/EFgfSk+w=
|
||||
golang.org/x/image v0.0.0-20191009234506-e7c1f5e7dbb8 h1:hVwzHzIUGRjiF7EcUjqNxk3NCfkPxbDKRdnNE1Rpg0U=
|
||||
golang.org/x/image v0.0.0-20191009234506-e7c1f5e7dbb8/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
|
||||
golang.org/x/mod v0.41.0 h1:qJmnOUb4YB+FsEuM3HcWucdZASCPGhsX6uljO6pog0c=
|
||||
golang.org/x/mod v0.41.0/go.mod h1:Ek9pY8RKWXwsWvd3rQiHYtMqkjSUV+s1Rj7j4H5Ur6o=
|
||||
golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
||||
golang.org/x/net v0.0.0-20200320220750-118fecf932d8/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
||||
golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
|
||||
@@ -241,7 +253,6 @@ golang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4/go.mod h1:h1NjWce9XRLGQEsW7w
|
||||
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20220728004956-3c1f35247d10/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20220928140112-f11e5e49a4ec/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.49.0 h1:XbzkgYJHdqh/8m2Uu0W/dQv8nktxx4BFHp1M0gROTXA=
|
||||
golang.org/x/sys v0.49.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
|
||||
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
|
||||
@@ -250,6 +261,8 @@ golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
|
||||
golang.org/x/text v0.43.0 h1:1QivrlhwAsnMOcqgOdl9my7s9OyZl8O5/UGUTJVXoWc=
|
||||
golang.org/x/text v0.43.0/go.mod h1:hbSIYA/amXcRXTFSZqh8tWHEoqRvvbKPJ2uz/KU+a+s=
|
||||
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
|
||||
golang.org/x/tools v0.51.0 h1:k4Xc/1Om9jwkBJBo4NVLMSARBoWtK10mx+W5BnXCeAI=
|
||||
golang.org/x/tools v0.51.0/go.mod h1:9eEncMayCV6zRMGhR5eZEC2iBx98qWcF1HZ9Z7wJOoA=
|
||||
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
|
||||
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
@@ -262,3 +275,31 @@ gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
|
||||
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
modernc.org/cc/v4 v4.29.7 h1:q+NXGJ0bK3b4TXFYQQVr9pYETGnmwFWkrUzJnMya/Tg=
|
||||
modernc.org/cc/v4 v4.29.7/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI=
|
||||
modernc.org/ccgo/v4 v4.36.1 h1:ZNIUZAryN0UgnJwtyxrdEzcFc3yD4Cu4AzjfPXsLsIE=
|
||||
modernc.org/ccgo/v4 v4.36.1/go.mod h1:rrtGc2QkS239nYb/mQNuBMyjq3/y3ZXWbBjPoV3wqzA=
|
||||
modernc.org/fileutil v1.4.0 h1:j6ZzNTftVS054gi281TyLjHPp6CPHr2KCxEXjEbD6SM=
|
||||
modernc.org/fileutil v1.4.0/go.mod h1:EqdKFDxiByqxLk8ozOxObDSfcVOv/54xDs/DUHdvCUU=
|
||||
modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
|
||||
modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
|
||||
modernc.org/gc/v3 v3.1.5 h1:21ldfPfRYE31Tb7B3mwAK8gy1AxP4+dKjrOQPfqakoc=
|
||||
modernc.org/gc/v3 v3.1.5/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
|
||||
modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
|
||||
modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
|
||||
modernc.org/libc v1.77.1 h1:Ct8j47QtiZ1Enj2DtFXQtUqrPCAjdCmPjtCuvrYQ0Hs=
|
||||
modernc.org/libc v1.77.1/go.mod h1:87/pZ4L6nD1zqW4nItuS12YO7hN1igAah34xjnQo/W0=
|
||||
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
|
||||
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
|
||||
modernc.org/memory v1.12.1 h1:nFMiWrpStgZczNl6XI9GnIk/rWhYIyHGUaR04pGbp9g=
|
||||
modernc.org/memory v1.12.1/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
|
||||
modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg=
|
||||
modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
|
||||
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
|
||||
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
|
||||
modernc.org/sqlite v1.60.1 h1:/blz53O951KWFOso4QQvEs/Fq6cDBKLtMVrYNSeJVKw=
|
||||
modernc.org/sqlite v1.60.1/go.mod h1:1dIoEagfDE72QytD5scH1lxARtaUgKgHC/NuApA27r0=
|
||||
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
|
||||
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
|
||||
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
|
||||
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
package albums
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"shared-album/internal/models"
|
||||
)
|
||||
|
||||
// Repository is the persistence contract for the albums domain.
|
||||
type Repository interface {
|
||||
Create(ctx context.Context, a *models.Album) error
|
||||
OwnerID(ctx context.Context, id string) (string, error)
|
||||
GetByID(ctx context.Context, id string) (*models.Album, error)
|
||||
ListForUser(ctx context.Context, userID string) ([]models.Album, error)
|
||||
Update(ctx context.Context, a *models.Album) error
|
||||
Delete(ctx context.Context, id string) error
|
||||
TrashPhotosByAlbum(ctx context.Context, albumID string) error
|
||||
}
|
||||
@@ -18,13 +18,13 @@ type MemberLookup interface {
|
||||
}
|
||||
|
||||
type Service struct {
|
||||
store *Store
|
||||
store Repository
|
||||
members MemberLookup
|
||||
hub *sse.Hub
|
||||
storage *storage.Client
|
||||
storage storage.Backend
|
||||
}
|
||||
|
||||
func NewService(store *Store, members MemberLookup, hub *sse.Hub, storage *storage.Client) *Service {
|
||||
func NewService(store Repository, members MemberLookup, hub *sse.Hub, storage storage.Backend) *Service {
|
||||
return &Service{store: store, members: members, hub: hub, storage: storage}
|
||||
}
|
||||
|
||||
@@ -61,12 +61,12 @@ func (s *Service) List(ctx context.Context, userID string) ([]models.Album, erro
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// attachCover fills CoverThumbURL from the presigned thumbnail key, when present.
|
||||
// attachCover fills CoverThumbURL from the signed thumbnail URL, when present.
|
||||
func (s *Service) attachCover(ctx context.Context, a *models.Album) {
|
||||
if s.storage == nil || a == nil || a.CoverThumbKey == "" {
|
||||
return
|
||||
}
|
||||
if url, err := s.storage.PresignGet(ctx, a.CoverThumbKey); err == nil {
|
||||
if url, err := s.storage.URL(ctx, a.CoverThumbKey); err == nil {
|
||||
a.CoverThumbURL = url
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,142 @@
|
||||
package albums
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
"time"
|
||||
|
||||
"shared-album/internal/models"
|
||||
"shared-album/internal/sqliteutil"
|
||||
)
|
||||
|
||||
// SQLiteStore is the SQLite implementation of Repository.
|
||||
type SQLiteStore struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func NewSQLiteStore(db *sql.DB) *SQLiteStore { return &SQLiteStore{db: db} }
|
||||
|
||||
func (s *SQLiteStore) Create(ctx context.Context, a *models.Album) error {
|
||||
var createdAt, updatedAt string
|
||||
err := s.db.QueryRowContext(ctx,
|
||||
`INSERT INTO albums (id, owner_id, title, description, cover_photo_id)
|
||||
VALUES ($1, $2, $3, $4, $5)
|
||||
RETURNING created_at, updated_at`,
|
||||
a.ID, a.OwnerID, a.Title, a.Description, a.CoverPhotoID,
|
||||
).Scan(&createdAt, &updatedAt)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
a.CreatedAt = sqliteutil.ParseTime(createdAt)
|
||||
a.UpdatedAt = sqliteutil.ParseTime(updatedAt)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) OwnerID(ctx context.Context, id string) (string, error) {
|
||||
var ownerID string
|
||||
err := s.db.QueryRowContext(ctx, `SELECT owner_id FROM albums WHERE id = $1`, id).Scan(&ownerID)
|
||||
if err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return "", ErrNotFound
|
||||
}
|
||||
return "", err
|
||||
}
|
||||
return ownerID, nil
|
||||
}
|
||||
|
||||
const sqliteAlbumColumns = `a.id, a.owner_id, a.title, a.description, a.cover_photo_id,
|
||||
a.created_at, a.updated_at, COALESCE(co.storage_key, ''),
|
||||
(SELECT count(*) FROM photos WHERE album_id = a.id),
|
||||
(SELECT count(*) FROM album_members WHERE album_id = a.id)`
|
||||
|
||||
const sqliteAlbumJoins = `
|
||||
LEFT JOIN photo_objects cpo ON cpo.photo_id = a.cover_photo_id AND cpo.role = 'thumb'
|
||||
LEFT JOIN objects co ON co.id = cpo.object_id`
|
||||
|
||||
func scanSQLiteAlbum(scan func(...any) error) (*models.Album, error) {
|
||||
var a models.Album
|
||||
var cover sql.NullString
|
||||
var createdAt, updatedAt string
|
||||
if err := scan(&a.ID, &a.OwnerID, &a.Title, &a.Description, &cover, &createdAt, &updatedAt,
|
||||
&a.CoverThumbKey, &a.PhotosCount, &a.MembersCount); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if cover.Valid {
|
||||
a.CoverPhotoID = &cover.String
|
||||
}
|
||||
a.CreatedAt = sqliteutil.ParseTime(createdAt)
|
||||
a.UpdatedAt = sqliteutil.ParseTime(updatedAt)
|
||||
return &a, nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) GetByID(ctx context.Context, id string) (*models.Album, error) {
|
||||
row := s.db.QueryRowContext(ctx,
|
||||
`SELECT `+sqliteAlbumColumns+` FROM albums a`+sqliteAlbumJoins+` WHERE a.id = $1`, id)
|
||||
a, err := scanSQLiteAlbum(row.Scan)
|
||||
if err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return nil, ErrNotFound
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
return a, nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) ListForUser(ctx context.Context, userID string) ([]models.Album, error) {
|
||||
rows, err := s.db.QueryContext(ctx,
|
||||
`SELECT `+sqliteAlbumColumns+` FROM albums a
|
||||
LEFT JOIN album_members m ON m.album_id = a.id AND m.user_id = $1`+sqliteAlbumJoins+`
|
||||
WHERE a.owner_id = $1 OR m.user_id IS NOT NULL
|
||||
ORDER BY a.created_at DESC`, userID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
out := []models.Album{}
|
||||
for rows.Next() {
|
||||
a, err := scanSQLiteAlbum(rows.Scan)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, *a)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) Update(ctx context.Context, a *models.Album) error {
|
||||
var updatedAt string
|
||||
err := s.db.QueryRowContext(ctx,
|
||||
`UPDATE albums
|
||||
SET title = $2, description = $3, cover_photo_id = $4, updated_at = $5
|
||||
WHERE id = $1
|
||||
RETURNING updated_at`,
|
||||
a.ID, a.Title, a.Description, a.CoverPhotoID, sqliteutil.FormatTime(time.Now().UTC()),
|
||||
).Scan(&updatedAt)
|
||||
if err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return ErrNotFound
|
||||
}
|
||||
return err
|
||||
}
|
||||
a.UpdatedAt = sqliteutil.ParseTime(updatedAt)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) Delete(ctx context.Context, id string) error {
|
||||
res, err := s.db.ExecContext(ctx, `DELETE FROM albums WHERE id = $1`, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if n, _ := res.RowsAffected(); n == 0 {
|
||||
return ErrNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) TrashPhotosByAlbum(ctx context.Context, albumID string) error {
|
||||
_, err := s.db.ExecContext(ctx,
|
||||
`UPDATE photos SET deleted_at = $2 WHERE album_id = $1 AND deleted_at IS NULL`,
|
||||
albumID, sqliteutil.FormatTime(time.Now().UTC()))
|
||||
return err
|
||||
}
|
||||
@@ -44,11 +44,12 @@ func (s *Store) GetByID(ctx context.Context, id string) (*models.Album, error) {
|
||||
var a models.Album
|
||||
err := s.pool.QueryRow(ctx,
|
||||
`SELECT a.id, a.owner_id, a.title, a.description, a.cover_photo_id, a.created_at, a.updated_at,
|
||||
COALESCE(p.thumb_key, ''),
|
||||
COALESCE(co.storage_key, ''),
|
||||
(SELECT count(*) FROM photos WHERE album_id = a.id),
|
||||
(SELECT count(*) FROM album_members WHERE album_id = a.id)
|
||||
FROM albums a
|
||||
LEFT JOIN photos p ON p.id = a.cover_photo_id
|
||||
LEFT JOIN photo_objects cpo ON cpo.photo_id = a.cover_photo_id AND cpo.role = 'thumb'
|
||||
LEFT JOIN objects co ON co.id = cpo.object_id
|
||||
WHERE a.id = $1`, id).
|
||||
Scan(&a.ID, &a.OwnerID, &a.Title, &a.Description, &a.CoverPhotoID, &a.CreatedAt, &a.UpdatedAt, &a.CoverThumbKey, &a.PhotosCount, &a.MembersCount)
|
||||
if err != nil {
|
||||
@@ -63,12 +64,13 @@ func (s *Store) GetByID(ctx context.Context, id string) (*models.Album, error) {
|
||||
func (s *Store) ListForUser(ctx context.Context, userID string) ([]models.Album, error) {
|
||||
rows, err := s.pool.Query(ctx,
|
||||
`SELECT a.id, a.owner_id, a.title, a.description, a.cover_photo_id, a.created_at, a.updated_at,
|
||||
COALESCE(p.thumb_key, ''),
|
||||
COALESCE(co.storage_key, ''),
|
||||
(SELECT count(*) FROM photos WHERE album_id = a.id),
|
||||
(SELECT count(*) FROM album_members WHERE album_id = a.id)
|
||||
FROM albums a
|
||||
LEFT JOIN album_members m ON m.album_id = a.id AND m.user_id = $1
|
||||
LEFT JOIN photos p ON p.id = a.cover_photo_id
|
||||
LEFT JOIN photo_objects cpo ON cpo.photo_id = a.cover_photo_id AND cpo.role = 'thumb'
|
||||
LEFT JOIN objects co ON co.id = cpo.object_id
|
||||
WHERE a.owner_id = $1 OR m.user_id IS NOT NULL
|
||||
ORDER BY a.created_at DESC`, userID)
|
||||
if err != nil {
|
||||
|
||||
+56
-29
@@ -1,13 +1,14 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"io/fs"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
|
||||
assets "shared-album"
|
||||
"shared-album/internal/albums"
|
||||
@@ -21,15 +22,16 @@ import (
|
||||
"shared-album/internal/social"
|
||||
"shared-album/internal/sse"
|
||||
"shared-album/internal/storage"
|
||||
"shared-album/internal/store"
|
||||
)
|
||||
|
||||
type Server struct {
|
||||
cfg *config.Config
|
||||
pool *pgxpool.Pool
|
||||
storage *storage.Client
|
||||
repos store.Repos
|
||||
ping func(ctx context.Context) error
|
||||
storage storage.Backend
|
||||
hub *sse.Hub
|
||||
|
||||
albumsStore *albums.Store
|
||||
albums *albums.Service
|
||||
|
||||
auth *auth.Service
|
||||
@@ -37,6 +39,7 @@ type Server struct {
|
||||
membersSvc *members.Service
|
||||
|
||||
photos *photos.Service
|
||||
worker *photos.Worker
|
||||
|
||||
social *social.Service
|
||||
|
||||
@@ -45,45 +48,47 @@ type Server struct {
|
||||
share *sharelinks.Service
|
||||
}
|
||||
|
||||
func New(cfg *config.Config, pool *pgxpool.Pool, st *storage.Client, hub *sse.Hub) *Server {
|
||||
authStore := auth.NewStore(pool)
|
||||
tm := auth.NewTokenManager(cfg.Auth.JWTSecret, cfg.Auth.AccessTTL, cfg.Auth.RefreshTTL)
|
||||
authSvc := auth.NewService(authStore, tm)
|
||||
// New wires the domain services over the selected metadata repositories. ping
|
||||
// reports the health of the metadata backend so health stays backend-agnostic.
|
||||
func New(cfg *config.Config, repos store.Repos, ping func(ctx context.Context) error, tm *auth.TokenManager, st storage.Backend, hub *sse.Hub) *Server {
|
||||
authSvc := auth.NewService(repos.Auth, tm)
|
||||
|
||||
albumsStore := albums.NewStore(pool)
|
||||
membersStore := members.NewStore(pool)
|
||||
notifSvc := notifications.NewService(repos.Notifications, hub)
|
||||
|
||||
notifStore := notifications.NewStore(pool)
|
||||
notifSvc := notifications.NewService(notifStore, hub)
|
||||
albumsSvc := albums.NewService(repos.Albums, repos.Members, hub, st)
|
||||
membersSvc := members.NewService(repos.Members, repos.Albums, hub, notifSvc)
|
||||
|
||||
albumsSvc := albums.NewService(albumsStore, membersStore, hub, st)
|
||||
membersSvc := members.NewService(membersStore, albumsStore, hub, notifSvc)
|
||||
photosSvc := photos.NewService(repos.Photos, albumsSvc, st, hub)
|
||||
photosWorker := photos.NewWorker(photosSvc, cfg.Worker.Interval, cfg.Worker.Batch, cfg.Worker.PendingTTL)
|
||||
|
||||
photosStore := photos.NewStore(pool)
|
||||
photosSvc := photos.NewService(photosStore, albumsSvc, st, hub)
|
||||
socialSvc := social.NewService(repos.Social, albumsSvc, repos.Photos, hub, notifSvc)
|
||||
|
||||
socialStore := social.NewStore(pool)
|
||||
socialSvc := social.NewService(socialStore, albumsSvc, photosStore, hub, notifSvc)
|
||||
|
||||
shareStore := sharelinks.NewStore(pool)
|
||||
shareSvc := sharelinks.NewService(shareStore, albumsSvc)
|
||||
shareSvc := sharelinks.NewService(repos.ShareLinks, albumsSvc)
|
||||
|
||||
return &Server{
|
||||
cfg: cfg,
|
||||
pool: pool,
|
||||
repos: repos,
|
||||
ping: ping,
|
||||
storage: st,
|
||||
hub: hub,
|
||||
albumsStore: albumsStore,
|
||||
albums: albumsSvc,
|
||||
auth: authSvc,
|
||||
membersSvc: membersSvc,
|
||||
photos: photosSvc,
|
||||
worker: photosWorker,
|
||||
social: socialSvc,
|
||||
notifications: notifSvc,
|
||||
share: shareSvc,
|
||||
}
|
||||
}
|
||||
|
||||
// StartWorker runs the background reconciler until ctx is cancelled.
|
||||
func (s *Server) StartWorker(ctx context.Context) {
|
||||
if s.worker != nil {
|
||||
s.worker.Run(ctx)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Server) Router() *gin.Engine {
|
||||
r := gin.New()
|
||||
r.Use(gin.Recovery())
|
||||
@@ -98,6 +103,8 @@ func (s *Server) Router() *gin.Engine {
|
||||
|
||||
api := r.Group("/api")
|
||||
api.GET("/health", s.health)
|
||||
api.GET("/objects/*key", s.serveObject)
|
||||
api.HEAD("/objects/*key", s.serveObject)
|
||||
|
||||
api.POST("/auth/register", authHandler.Register)
|
||||
api.POST("/auth/login", authHandler.Login)
|
||||
@@ -169,20 +176,40 @@ func (s *Server) Router() *gin.Engine {
|
||||
func (s *Server) health(c *gin.Context) {
|
||||
ctx := c.Request.Context()
|
||||
dbStatus := "ok"
|
||||
if err := s.pool.Ping(ctx); err != nil {
|
||||
if err := s.ping(ctx); err != nil {
|
||||
dbStatus = "error"
|
||||
}
|
||||
s3Status := "ok"
|
||||
storageLabel := s.cfg.Storage.Backend
|
||||
if storageLabel == "" {
|
||||
storageLabel = "s3"
|
||||
}
|
||||
storeStatus := "ok"
|
||||
if err := s.storage.Ping(ctx); err != nil {
|
||||
s3Status = "error"
|
||||
storeStatus = "error"
|
||||
}
|
||||
status := http.StatusOK
|
||||
overall := "ok"
|
||||
if dbStatus != "ok" || s3Status != "ok" {
|
||||
if dbStatus != "ok" || storeStatus != "ok" {
|
||||
status = http.StatusServiceUnavailable
|
||||
overall = "degraded"
|
||||
}
|
||||
c.JSON(status, gin.H{"status": overall, "db": dbStatus, "s3": s3Status})
|
||||
c.JSON(status, gin.H{"status": overall, "db": dbStatus, storageLabel: storeStatus})
|
||||
}
|
||||
|
||||
func (s *Server) serveObject(c *gin.Context) {
|
||||
key := strings.TrimPrefix(c.Param("key"), "/")
|
||||
switch err := s.storage.ServeObject(c.Writer, c.Request, key); {
|
||||
case err == nil:
|
||||
return
|
||||
case errors.Is(err, storage.ErrSignedURLInvalid):
|
||||
c.Status(http.StatusForbidden)
|
||||
case errors.Is(err, storage.ErrObjectNotFound):
|
||||
httpx.RespondError(c, httpx.NotFound("object_not_found", "object not found"))
|
||||
case errors.Is(err, storage.ErrRangeNotSatisfiable):
|
||||
c.Status(http.StatusRequestedRangeNotSatisfiable)
|
||||
default:
|
||||
c.Status(http.StatusInternalServerError)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Server) albumSSE(c *gin.Context) {
|
||||
@@ -209,7 +236,7 @@ func (s *Server) shareAlbum(c *gin.Context) {
|
||||
httpx.RespondError(c, err)
|
||||
return
|
||||
}
|
||||
album, err := s.albumsStore.GetByID(c.Request.Context(), link.AlbumID)
|
||||
album, err := s.repos.Albums.GetByID(c.Request.Context(), link.AlbumID)
|
||||
if err != nil {
|
||||
httpx.RespondError(c, httpx.NotFound("album_not_found", "album not found"))
|
||||
return
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
|
||||
"shared-album/internal/config"
|
||||
"shared-album/internal/storage"
|
||||
)
|
||||
|
||||
func mustSigner(t *testing.T) *storage.Signer {
|
||||
t.Helper()
|
||||
signer, err := storage.NewSigner("http://app.test", "secret", time.Hour)
|
||||
if err != nil {
|
||||
t.Fatalf("signer: %v", err)
|
||||
}
|
||||
return signer
|
||||
}
|
||||
|
||||
func TestHealthStorageLabel(t *testing.T) {
|
||||
gin.SetMode(gin.TestMode)
|
||||
fs, err := storage.NewFS(t.TempDir(), mustSigner(t))
|
||||
if err != nil {
|
||||
t.Fatalf("fs: %v", err)
|
||||
}
|
||||
ok := func(context.Context) error { return nil }
|
||||
srv := &Server{cfg: &config.Config{}, ping: ok, storage: fs}
|
||||
|
||||
call := func() map[string]any {
|
||||
rec := httptest.NewRecorder()
|
||||
c, _ := gin.CreateTestContext(rec)
|
||||
c.Request = httptest.NewRequest(http.MethodGet, "/api/health", nil)
|
||||
srv.health(c)
|
||||
var body map[string]any
|
||||
if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil {
|
||||
t.Fatalf("decode: %v", err)
|
||||
}
|
||||
return body
|
||||
}
|
||||
|
||||
srv.cfg.Storage.Backend = "fs"
|
||||
if body := call(); body["fs"] != "ok" {
|
||||
t.Fatalf("expected fs label ok, got %v", body)
|
||||
}
|
||||
|
||||
srv.cfg.Storage.Backend = "s3"
|
||||
if body := call(); body["s3"] != "ok" {
|
||||
t.Fatalf("expected s3 label ok, got %v", body)
|
||||
}
|
||||
|
||||
srv.cfg.Storage.Backend = ""
|
||||
if body := call(); body["s3"] != "ok" {
|
||||
t.Fatalf("expected default s3 label, got %v", body)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,114 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
|
||||
"shared-album/internal/auth"
|
||||
"shared-album/internal/config"
|
||||
"shared-album/internal/sse"
|
||||
"shared-album/internal/storage"
|
||||
"shared-album/internal/store"
|
||||
)
|
||||
|
||||
func newTestServerWithFS(t *testing.T) (*storage.FS, *gin.Engine) {
|
||||
t.Helper()
|
||||
gin.SetMode(gin.TestMode)
|
||||
signer, err := storage.NewSigner("http://app.test", "secret", time.Hour)
|
||||
if err != nil {
|
||||
t.Fatalf("signer: %v", err)
|
||||
}
|
||||
fs, err := storage.NewFS(t.TempDir(), signer)
|
||||
if err != nil {
|
||||
t.Fatalf("new fs: %v", err)
|
||||
}
|
||||
noop := func(context.Context) error { return nil }
|
||||
tm := auth.NewTokenManager("test-secret", 15*time.Minute, time.Hour)
|
||||
srv := New(&config.Config{}, store.Repos{}, noop, tm, fs, sse.NewHub(1))
|
||||
return fs, srv.Router()
|
||||
}
|
||||
|
||||
func TestServeObjectRoute(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
fs, r := newTestServerWithFS(t)
|
||||
key := "objects/abc/original.jpg"
|
||||
if err := fs.Put(ctx, key, strings.NewReader("0123456789"), 10, "image/jpeg"); err != nil {
|
||||
t.Fatalf("put: %v", err)
|
||||
}
|
||||
raw, err := fs.URL(ctx, key)
|
||||
if err != nil {
|
||||
t.Fatalf("url: %v", err)
|
||||
}
|
||||
u, err := url.Parse(raw)
|
||||
if err != nil {
|
||||
t.Fatalf("parse: %v", err)
|
||||
}
|
||||
|
||||
do := func(target string, rangeHeader string) *httptest.ResponseRecorder {
|
||||
req := httptest.NewRequest(http.MethodGet, target, nil)
|
||||
if rangeHeader != "" {
|
||||
req.Header.Set("Range", rangeHeader)
|
||||
}
|
||||
rec := httptest.NewRecorder()
|
||||
r.ServeHTTP(rec, req)
|
||||
return rec
|
||||
}
|
||||
|
||||
t.Run("valid", func(t *testing.T) {
|
||||
rec := do(u.Path+"?"+u.RawQuery, "")
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200", rec.Code)
|
||||
}
|
||||
if rec.Body.String() != "0123456789" {
|
||||
t.Fatalf("body = %q", rec.Body.String())
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("range", func(t *testing.T) {
|
||||
rec := do(u.Path+"?"+u.RawQuery, "bytes=2-5")
|
||||
if rec.Code != http.StatusPartialContent {
|
||||
t.Fatalf("status = %d, want 206", rec.Code)
|
||||
}
|
||||
if rec.Body.String() != "2345" {
|
||||
t.Fatalf("body = %q, want 2345", rec.Body.String())
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("expired", func(t *testing.T) {
|
||||
q := u.Query()
|
||||
q.Set("e", strconv.FormatInt(time.Now().Add(-time.Minute).Unix(), 10))
|
||||
rec := do(u.Path+"?"+q.Encode(), "")
|
||||
if rec.Code != http.StatusForbidden {
|
||||
t.Fatalf("status = %d, want 403", rec.Code)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("forged", func(t *testing.T) {
|
||||
q := u.Query()
|
||||
q.Set("s", "deadbeef")
|
||||
rec := do(u.Path+"?"+q.Encode(), "")
|
||||
if rec.Code != http.StatusForbidden {
|
||||
t.Fatalf("status = %d, want 403", rec.Code)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("missing", func(t *testing.T) {
|
||||
raw, err := fs.URL(ctx, "objects/def/original.jpg")
|
||||
if err != nil {
|
||||
t.Fatalf("url: %v", err)
|
||||
}
|
||||
mu, _ := url.Parse(raw)
|
||||
rec := do(mu.Path+"?"+mu.RawQuery, "")
|
||||
if rec.Code != http.StatusNotFound {
|
||||
t.Fatalf("status = %d, want 404", rec.Code)
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,213 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"image"
|
||||
"image/color"
|
||||
"image/png"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
|
||||
"shared-album/internal/albums"
|
||||
"shared-album/internal/auth"
|
||||
"shared-album/internal/config"
|
||||
"shared-album/internal/members"
|
||||
"shared-album/internal/models"
|
||||
"shared-album/internal/photos"
|
||||
"shared-album/internal/sharelinks"
|
||||
"shared-album/internal/sse"
|
||||
"shared-album/internal/storage"
|
||||
"shared-album/internal/store"
|
||||
)
|
||||
|
||||
func integrationPool(t *testing.T) *pgxpool.Pool {
|
||||
t.Helper()
|
||||
dsn := os.Getenv("TEST_DATABASE_URL")
|
||||
if dsn == "" {
|
||||
t.Skip("TEST_DATABASE_URL not set")
|
||||
}
|
||||
pool, err := pgxpool.New(context.Background(), dsn)
|
||||
if err != nil {
|
||||
t.Skipf("cannot connect: %v", err)
|
||||
}
|
||||
if err := pool.Ping(context.Background()); err != nil {
|
||||
t.Skipf("cannot ping: %v", err)
|
||||
}
|
||||
t.Cleanup(pool.Close)
|
||||
return pool
|
||||
}
|
||||
|
||||
func makeTestPNG(t *testing.T) []byte {
|
||||
t.Helper()
|
||||
img := image.NewRGBA(image.Rect(0, 0, 4, 4))
|
||||
for x := 0; x < 4; x++ {
|
||||
for y := 0; y < 4; y++ {
|
||||
img.Set(x, y, color.RGBA{R: uint8(x * 40), G: uint8(y * 40), B: 128, A: 255})
|
||||
}
|
||||
}
|
||||
var buf bytes.Buffer
|
||||
if err := png.Encode(&buf, img); err != nil {
|
||||
t.Fatalf("encode png: %v", err)
|
||||
}
|
||||
return buf.Bytes()
|
||||
}
|
||||
|
||||
// runShareFlow uploads a photo and validates the read-only share flow returns a
|
||||
// usable photo URL. verify inspects/serves the URL according to the backend.
|
||||
func runShareFlow(t *testing.T, backend storage.Backend, verify func(t *testing.T, router *gin.Engine, raw string)) {
|
||||
t.Helper()
|
||||
ctx := context.Background()
|
||||
pool := integrationPool(t)
|
||||
gin.SetMode(gin.TestMode)
|
||||
|
||||
hub := sse.NewHub(1)
|
||||
tm := auth.NewTokenManager("test-secret", 15*time.Minute, time.Hour)
|
||||
srv := New(&config.Config{}, store.NewPostgres(pool), pool.Ping, tm, backend, hub)
|
||||
router := srv.Router()
|
||||
|
||||
authStore := auth.NewStore(pool)
|
||||
owner := &models.User{ID: auth.NewID(), Email: auth.NewID()[:8] + "@test.local", PasswordHash: "x", Name: "Owner"}
|
||||
if err := authStore.CreateUser(ctx, owner); err != nil {
|
||||
t.Fatalf("create user: %v", err)
|
||||
}
|
||||
|
||||
albumsStore := albums.NewStore(pool)
|
||||
albumsSvc := albums.NewService(albumsStore, members.NewStore(pool), hub, backend)
|
||||
album, err := albumsSvc.Create(ctx, owner.ID, albums.CreateInput{Title: "Share Album"})
|
||||
if err != nil {
|
||||
t.Fatalf("create album: %v", err)
|
||||
}
|
||||
|
||||
photosSvc := photos.NewService(photos.NewStore(pool), albumsSvc, backend, hub)
|
||||
pngData := makeTestPNG(t)
|
||||
if _, err := photosSvc.Upload(ctx, photos.UploadInput{
|
||||
AlbumID: album.ID, UserID: owner.ID, Filename: "pic.png", Data: pngData,
|
||||
}); err != nil {
|
||||
t.Fatalf("upload: %v", err)
|
||||
}
|
||||
|
||||
var keys []string
|
||||
rows, err := pool.Query(ctx,
|
||||
`SELECT o.storage_key FROM photo_objects po
|
||||
JOIN objects o ON o.id = po.object_id
|
||||
JOIN photos p ON p.id = po.photo_id
|
||||
WHERE p.album_id = $1`, album.ID)
|
||||
if err == nil {
|
||||
for rows.Next() {
|
||||
var k string
|
||||
if rows.Scan(&k) == nil {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
}
|
||||
rows.Close()
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM photos WHERE album_id = $1`, album.ID)
|
||||
for _, k := range keys {
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM objects WHERE storage_key = $1`, k)
|
||||
}
|
||||
_ = backend.Delete(context.Background(), keys...)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM albums WHERE id = $1`, album.ID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM users WHERE id = $1`, owner.ID)
|
||||
})
|
||||
|
||||
shareSvc := sharelinks.NewService(sharelinks.NewStore(pool), albumsSvc)
|
||||
link, err := shareSvc.Create(ctx, album.ID, owner.ID, time.Hour)
|
||||
if err != nil {
|
||||
t.Fatalf("share create: %v", err)
|
||||
}
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
router.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/api/share/"+link.Token+"/photos", nil))
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("share photos status = %d body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
|
||||
var body struct {
|
||||
Photos []models.Photo `json:"photos"`
|
||||
}
|
||||
if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil {
|
||||
t.Fatalf("decode: %v", err)
|
||||
}
|
||||
if len(body.Photos) != 1 {
|
||||
t.Fatalf("photos len = %d, want 1", len(body.Photos))
|
||||
}
|
||||
if body.Photos[0].OriginalURL == "" {
|
||||
t.Fatalf("empty original url")
|
||||
}
|
||||
verify(t, router, body.Photos[0].OriginalURL)
|
||||
}
|
||||
|
||||
func TestShareFlowWithFS(t *testing.T) {
|
||||
signer, err := storage.NewSigner("http://app.test", "secret", time.Hour)
|
||||
if err != nil {
|
||||
t.Fatalf("signer: %v", err)
|
||||
}
|
||||
fs, err := storage.NewFS(t.TempDir(), signer)
|
||||
if err != nil {
|
||||
t.Fatalf("new fs: %v", err)
|
||||
}
|
||||
runShareFlow(t, fs, func(t *testing.T, router *gin.Engine, raw string) {
|
||||
if !strings.Contains(raw, "/api/objects/") {
|
||||
t.Fatalf("expected fs signed url, got %q", raw)
|
||||
}
|
||||
u, err := url.Parse(raw)
|
||||
if err != nil {
|
||||
t.Fatalf("parse url: %v", err)
|
||||
}
|
||||
rec := httptest.NewRecorder()
|
||||
router.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, u.Path+"?"+u.RawQuery, nil))
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("fs object fetch status = %d", rec.Code)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestShareFlowWithS3(t *testing.T) {
|
||||
endpoint := os.Getenv("S3_ENDPOINT")
|
||||
if endpoint == "" {
|
||||
t.Skip("S3_ENDPOINT not set")
|
||||
}
|
||||
signer, err := storage.NewSigner("http://app.test", "secret", time.Hour)
|
||||
if err != nil {
|
||||
t.Skipf("signer init: %v", err)
|
||||
}
|
||||
cl, err := storage.NewS3(context.Background(), config.S3{
|
||||
Endpoint: endpoint,
|
||||
Region: os.Getenv("S3_REGION"),
|
||||
AccessKey: os.Getenv("S3_ACCESS_KEY"),
|
||||
SecretKey: os.Getenv("S3_SECRET_KEY"),
|
||||
Bucket: os.Getenv("S3_BUCKET"),
|
||||
UsePathStyle: true,
|
||||
}, signer)
|
||||
if err != nil {
|
||||
t.Skipf("storage init: %v", err)
|
||||
}
|
||||
runShareFlow(t, cl, func(t *testing.T, router *gin.Engine, raw string) {
|
||||
if !strings.Contains(raw, "/api/objects/") {
|
||||
t.Fatalf("expected backend url, got %q", raw)
|
||||
}
|
||||
if strings.Contains(raw, "X-Amz-Signature") {
|
||||
t.Fatalf("presigned url leaked to client: %q", raw)
|
||||
}
|
||||
u, err := url.Parse(raw)
|
||||
if err != nil {
|
||||
t.Fatalf("parse url: %v", err)
|
||||
}
|
||||
rec := httptest.NewRecorder()
|
||||
router.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, u.Path+"?"+u.RawQuery, nil))
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("s3 object fetch status = %d", rec.Code)
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"shared-album/internal/models"
|
||||
)
|
||||
|
||||
// Repository is the persistence contract for the auth domain.
|
||||
type Repository interface {
|
||||
CreateUser(ctx context.Context, u *models.User) error
|
||||
GetUserByEmail(ctx context.Context, email string) (*models.User, error)
|
||||
GetUserByID(ctx context.Context, id string) (*models.User, error)
|
||||
UpdatePreferences(ctx context.Context, userID string, raw []byte) error
|
||||
CreateRefreshToken(ctx context.Context, t *models.RefreshToken) error
|
||||
GetRefreshToken(ctx context.Context, hash string) (*models.RefreshToken, error)
|
||||
RevokeRefreshToken(ctx context.Context, id string) error
|
||||
RevokeAllUserTokens(ctx context.Context, userID string) error
|
||||
}
|
||||
@@ -18,11 +18,11 @@ type TokenPair struct {
|
||||
}
|
||||
|
||||
type Service struct {
|
||||
store *Store
|
||||
store Repository
|
||||
tm *TokenManager
|
||||
}
|
||||
|
||||
func NewService(store *Store, tm *TokenManager) *Service {
|
||||
func NewService(store Repository, tm *TokenManager) *Service {
|
||||
return &Service{store: store, tm: tm}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,111 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"time"
|
||||
|
||||
"shared-album/internal/models"
|
||||
"shared-album/internal/sqliteutil"
|
||||
)
|
||||
|
||||
// SQLiteStore is the SQLite implementation of Repository.
|
||||
type SQLiteStore struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func NewSQLiteStore(db *sql.DB) *SQLiteStore { return &SQLiteStore{db: db} }
|
||||
|
||||
func (s *SQLiteStore) CreateUser(ctx context.Context, u *models.User) error {
|
||||
_, err := s.db.ExecContext(ctx,
|
||||
`INSERT INTO users (id, email, password_hash, name, avatar_url)
|
||||
VALUES ($1, $2, $3, $4, $5)`,
|
||||
u.ID, u.Email, u.PasswordHash, u.Name, nullIfEmpty(u.AvatarURL))
|
||||
return err
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) GetUserByEmail(ctx context.Context, email string) (*models.User, error) {
|
||||
return s.scanUser(s.db.QueryRowContext(ctx,
|
||||
`SELECT id, email, password_hash, name, COALESCE(avatar_url, ''),
|
||||
COALESCE(preferences, '{}'), created_at
|
||||
FROM users WHERE lower(email) = lower($1)`, email))
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) GetUserByID(ctx context.Context, id string) (*models.User, error) {
|
||||
return s.scanUser(s.db.QueryRowContext(ctx,
|
||||
`SELECT id, email, password_hash, name, COALESCE(avatar_url, ''),
|
||||
COALESCE(preferences, '{}'), created_at
|
||||
FROM users WHERE id = $1`, id))
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) scanUser(row *sql.Row) (*models.User, error) {
|
||||
var u models.User
|
||||
var prefs, createdAt string
|
||||
if err := row.Scan(&u.ID, &u.Email, &u.PasswordHash, &u.Name, &u.AvatarURL, &prefs, &createdAt); err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return nil, ErrNotFound
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
u.CreatedAt = sqliteutil.ParseTime(createdAt)
|
||||
if prefs != "" && prefs != "null" {
|
||||
_ = json.Unmarshal([]byte(prefs), &u.Preferences)
|
||||
}
|
||||
return &u, nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) UpdatePreferences(ctx context.Context, userID string, raw []byte) error {
|
||||
res, err := s.db.ExecContext(ctx,
|
||||
`UPDATE users SET preferences = $2 WHERE id = $1`, userID, string(raw))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if n, _ := res.RowsAffected(); n == 0 {
|
||||
return ErrNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) CreateRefreshToken(ctx context.Context, t *models.RefreshToken) error {
|
||||
_, err := s.db.ExecContext(ctx,
|
||||
`INSERT INTO refresh_tokens (id, user_id, token_hash, expires_at)
|
||||
VALUES ($1, $2, $3, $4)`,
|
||||
t.ID, t.UserID, t.TokenHash, sqliteutil.FormatTime(t.ExpiresAt))
|
||||
return err
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) GetRefreshToken(ctx context.Context, hash string) (*models.RefreshToken, error) {
|
||||
var t models.RefreshToken
|
||||
var expiresAt, createdAt string
|
||||
var revokedAt sql.NullString
|
||||
err := s.db.QueryRowContext(ctx,
|
||||
`SELECT id, user_id, token_hash, expires_at, revoked_at, created_at
|
||||
FROM refresh_tokens WHERE token_hash = $1`, hash).
|
||||
Scan(&t.ID, &t.UserID, &t.TokenHash, &expiresAt, &revokedAt, &createdAt)
|
||||
if err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return nil, ErrNotFound
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
t.ExpiresAt = sqliteutil.ParseTime(expiresAt)
|
||||
t.CreatedAt = sqliteutil.ParseTime(createdAt)
|
||||
t.RevokedAt = sqliteutil.ParseNullTime(revokedAt)
|
||||
return &t, nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) RevokeRefreshToken(ctx context.Context, id string) error {
|
||||
_, err := s.db.ExecContext(ctx,
|
||||
`UPDATE refresh_tokens SET revoked_at = $2 WHERE id = $1 AND revoked_at IS NULL`,
|
||||
id, sqliteutil.FormatTime(time.Now()))
|
||||
return err
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) RevokeAllUserTokens(ctx context.Context, userID string) error {
|
||||
_, err := s.db.ExecContext(ctx,
|
||||
`UPDATE refresh_tokens SET revoked_at = $2 WHERE user_id = $1 AND revoked_at IS NULL`,
|
||||
userID, sqliteutil.FormatTime(time.Now()))
|
||||
return err
|
||||
}
|
||||
@@ -28,12 +28,9 @@ type S3 struct {
|
||||
SecretKey string
|
||||
Bucket string
|
||||
UsePathStyle bool
|
||||
PublicURL string
|
||||
PresignTTL time.Duration
|
||||
}
|
||||
|
||||
type Auth struct {
|
||||
JWTSecret string
|
||||
AccessTTL time.Duration
|
||||
RefreshTTL time.Duration
|
||||
}
|
||||
@@ -48,12 +45,33 @@ type SSE struct {
|
||||
ClientBuffer int
|
||||
}
|
||||
|
||||
type Worker struct {
|
||||
Interval time.Duration
|
||||
Batch int
|
||||
PendingTTL time.Duration
|
||||
}
|
||||
|
||||
type Storage struct {
|
||||
Backend string
|
||||
FSRoot string
|
||||
URLSecret string
|
||||
URLTTL time.Duration
|
||||
}
|
||||
|
||||
type Database struct {
|
||||
Backend string
|
||||
SQLitePath string
|
||||
}
|
||||
|
||||
type Config struct {
|
||||
Postgres Postgres
|
||||
S3 S3
|
||||
Storage Storage
|
||||
Database Database
|
||||
Auth Auth
|
||||
Server Server
|
||||
SSE SSE
|
||||
Worker Worker
|
||||
}
|
||||
|
||||
func Load() (*Config, error) {
|
||||
@@ -72,9 +90,9 @@ func Load() (*Config, error) {
|
||||
return nil, fmt.Errorf("invalid JWT_REFRESH_TTL: %w", err)
|
||||
}
|
||||
|
||||
presignTTL, err := time.ParseDuration(getEnv("S3_PRESIGN_TTL", "1h"))
|
||||
urlTTL, err := time.ParseDuration(getEnv("OBJECT_URL_TTL", "1h"))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid S3_PRESIGN_TTL: %w", err)
|
||||
return nil, fmt.Errorf("invalid OBJECT_URL_TTL: %w", err)
|
||||
}
|
||||
|
||||
heartbeat, err := time.ParseDuration(getEnv("SSE_HEARTBEAT_INTERVAL", "25s"))
|
||||
@@ -92,6 +110,21 @@ func Load() (*Config, error) {
|
||||
return nil, fmt.Errorf("invalid S3_USE_PATH_STYLE: %w", err)
|
||||
}
|
||||
|
||||
workerInterval, err := time.ParseDuration(getEnv("WORKER_INTERVAL", "5s"))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid WORKER_INTERVAL: %w", err)
|
||||
}
|
||||
|
||||
workerBatch, err := strconv.Atoi(getEnv("WORKER_BATCH", "10"))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid WORKER_BATCH: %w", err)
|
||||
}
|
||||
|
||||
workerPendingTTL, err := time.ParseDuration(getEnv("WORKER_PENDING_TTL", "10m"))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid WORKER_PENDING_TTL: %w", err)
|
||||
}
|
||||
|
||||
cfg := &Config{
|
||||
Postgres: Postgres{
|
||||
Host: getEnv("PG_HOST", "pg.localnet"),
|
||||
@@ -108,11 +141,18 @@ func Load() (*Config, error) {
|
||||
SecretKey: getEnv("S3_SECRET_KEY", ""),
|
||||
Bucket: getEnv("S3_BUCKET", "albums"),
|
||||
UsePathStyle: pathStyle,
|
||||
PublicURL: getEnv("S3_PUBLIC_URL", ""),
|
||||
PresignTTL: presignTTL,
|
||||
},
|
||||
Storage: Storage{
|
||||
Backend: getEnv("STORAGE_BACKEND", "fs"),
|
||||
FSRoot: getEnv("FS_ROOT", "./data"),
|
||||
URLSecret: getEnv("URL_SECRET", ""),
|
||||
URLTTL: urlTTL,
|
||||
},
|
||||
Database: Database{
|
||||
Backend: getEnv("DB_BACKEND", "sqlite"),
|
||||
SQLitePath: getEnv("SQLITE_PATH", "./galo.db"),
|
||||
},
|
||||
Auth: Auth{
|
||||
JWTSecret: getEnv("JWT_SECRET", ""),
|
||||
AccessTTL: accessTTL,
|
||||
RefreshTTL: refreshTTL,
|
||||
},
|
||||
@@ -124,14 +164,35 @@ func Load() (*Config, error) {
|
||||
HeartbeatInterval: heartbeat,
|
||||
ClientBuffer: bufSize,
|
||||
},
|
||||
Worker: Worker{
|
||||
Interval: workerInterval,
|
||||
Batch: workerBatch,
|
||||
PendingTTL: workerPendingTTL,
|
||||
},
|
||||
}
|
||||
|
||||
if cfg.Auth.JWTSecret == "" {
|
||||
return nil, fmt.Errorf("JWT_SECRET is required")
|
||||
}
|
||||
switch cfg.Database.Backend {
|
||||
case "postgres":
|
||||
if cfg.Postgres.Password == "" {
|
||||
return nil, fmt.Errorf("PG_PASSWORD is required")
|
||||
}
|
||||
case "sqlite":
|
||||
if cfg.Database.SQLitePath == "" {
|
||||
return nil, fmt.Errorf("SQLITE_PATH is required when DB_BACKEND=sqlite")
|
||||
}
|
||||
default:
|
||||
return nil, fmt.Errorf("invalid DB_BACKEND %q (want postgres or sqlite)", cfg.Database.Backend)
|
||||
}
|
||||
|
||||
switch cfg.Storage.Backend {
|
||||
case "s3":
|
||||
case "fs":
|
||||
if cfg.Storage.FSRoot == "" {
|
||||
return nil, fmt.Errorf("FS_ROOT is required when STORAGE_BACKEND=fs")
|
||||
}
|
||||
default:
|
||||
return nil, fmt.Errorf("invalid STORAGE_BACKEND %q (want s3 or fs)", cfg.Storage.Backend)
|
||||
}
|
||||
|
||||
return cfg, nil
|
||||
}
|
||||
|
||||
@@ -0,0 +1,134 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func setRequiredEnv(t *testing.T) {
|
||||
t.Helper()
|
||||
t.Setenv("JWT_SECRET", "test-secret")
|
||||
t.Setenv("PG_PASSWORD", "test-pass")
|
||||
}
|
||||
|
||||
func TestLoadStorageBackend(t *testing.T) {
|
||||
t.Run("defaults to fs", func(t *testing.T) {
|
||||
setRequiredEnv(t)
|
||||
t.Setenv("STORAGE_BACKEND", "")
|
||||
cfg, err := Load()
|
||||
if err != nil {
|
||||
t.Fatalf("load: %v", err)
|
||||
}
|
||||
if cfg.Storage.Backend != "fs" || cfg.Storage.FSRoot != "./data" {
|
||||
t.Fatalf("storage = %+v", cfg.Storage)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("fs with root", func(t *testing.T) {
|
||||
setRequiredEnv(t)
|
||||
t.Setenv("STORAGE_BACKEND", "fs")
|
||||
t.Setenv("FS_ROOT", "/var/lib/galo/objects")
|
||||
cfg, err := Load()
|
||||
if err != nil {
|
||||
t.Fatalf("load: %v", err)
|
||||
}
|
||||
if cfg.Storage.Backend != "fs" || cfg.Storage.FSRoot != "/var/lib/galo/objects" {
|
||||
t.Fatalf("storage = %+v", cfg.Storage)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("fs empty root falls back to default", func(t *testing.T) {
|
||||
setRequiredEnv(t)
|
||||
t.Setenv("STORAGE_BACKEND", "fs")
|
||||
t.Setenv("FS_ROOT", "")
|
||||
t.Setenv("URL_SECRET", "")
|
||||
cfg, err := Load()
|
||||
if err != nil {
|
||||
t.Fatalf("load: %v", err)
|
||||
}
|
||||
if cfg.Storage.FSRoot != "./data" {
|
||||
t.Fatalf("FSRoot = %q, want ./data", cfg.Storage.FSRoot)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("invalid backend", func(t *testing.T) {
|
||||
setRequiredEnv(t)
|
||||
t.Setenv("STORAGE_BACKEND", "gcs")
|
||||
if _, err := Load(); err == nil {
|
||||
t.Fatal("expected error for invalid backend")
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("object url ttl", func(t *testing.T) {
|
||||
setRequiredEnv(t)
|
||||
t.Setenv("STORAGE_BACKEND", "")
|
||||
t.Setenv("OBJECT_URL_TTL", "30m")
|
||||
cfg, err := Load()
|
||||
if err != nil {
|
||||
t.Fatalf("load: %v", err)
|
||||
}
|
||||
if cfg.Storage.URLTTL != 30*time.Minute {
|
||||
t.Fatalf("URLTTL = %v, want 30m", cfg.Storage.URLTTL)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestLoadDatabaseBackend(t *testing.T) {
|
||||
t.Run("defaults to sqlite", func(t *testing.T) {
|
||||
setRequiredEnv(t)
|
||||
t.Setenv("DB_BACKEND", "")
|
||||
cfg, err := Load()
|
||||
if err != nil {
|
||||
t.Fatalf("load: %v", err)
|
||||
}
|
||||
if cfg.Database.Backend != "sqlite" || cfg.Database.SQLitePath != "./galo.db" {
|
||||
t.Fatalf("database = %+v", cfg.Database)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("postgres requires password", func(t *testing.T) {
|
||||
t.Setenv("JWT_SECRET", "s")
|
||||
t.Setenv("PG_PASSWORD", "")
|
||||
t.Setenv("DB_BACKEND", "postgres")
|
||||
if _, err := Load(); err == nil {
|
||||
t.Fatal("expected error for postgres without PG_PASSWORD")
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("sqlite with path", func(t *testing.T) {
|
||||
t.Setenv("JWT_SECRET", "s")
|
||||
t.Setenv("PG_PASSWORD", "")
|
||||
t.Setenv("DB_BACKEND", "sqlite")
|
||||
t.Setenv("SQLITE_PATH", "/data/galo.db")
|
||||
cfg, err := Load()
|
||||
if err != nil {
|
||||
t.Fatalf("load: %v", err)
|
||||
}
|
||||
if cfg.Database.Backend != "sqlite" || cfg.Database.SQLitePath != "/data/galo.db" {
|
||||
t.Fatalf("database = %+v", cfg.Database)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("sqlite empty path falls back to default", func(t *testing.T) {
|
||||
t.Setenv("JWT_SECRET", "s")
|
||||
t.Setenv("PG_PASSWORD", "")
|
||||
t.Setenv("DB_BACKEND", "sqlite")
|
||||
t.Setenv("SQLITE_PATH", "")
|
||||
cfg, err := Load()
|
||||
if err != nil {
|
||||
t.Fatalf("load: %v", err)
|
||||
}
|
||||
if cfg.Database.SQLitePath != "./galo.db" {
|
||||
t.Fatalf("SQLitePath = %q, want ./galo.db", cfg.Database.SQLitePath)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("invalid backend", func(t *testing.T) {
|
||||
t.Setenv("JWT_SECRET", "s")
|
||||
t.Setenv("PG_PASSWORD", "p")
|
||||
t.Setenv("DB_BACKEND", "mysql")
|
||||
if _, err := Load(); err == nil {
|
||||
t.Fatal("expected error for invalid backend")
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -8,8 +8,10 @@ import (
|
||||
"database/sql"
|
||||
"github.com/golang-migrate/migrate/v4"
|
||||
"github.com/golang-migrate/migrate/v4/database/pgx/v5"
|
||||
sqlitemig "github.com/golang-migrate/migrate/v4/database/sqlite"
|
||||
"github.com/golang-migrate/migrate/v4/source/iofs"
|
||||
_ "github.com/jackc/pgx/v5/stdlib"
|
||||
_ "modernc.org/sqlite"
|
||||
)
|
||||
|
||||
// Migrate applies all up migrations from the given fs (root contains *.sql files).
|
||||
@@ -39,3 +41,32 @@ func Migrate(dsn string, migrations fs.FS) error {
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// MigrateSQLite applies all up migrations from the given fs to a SQLite database
|
||||
// at path (creating it if needed).
|
||||
func MigrateSQLite(path string, migrations fs.FS) error {
|
||||
source, err := iofs.New(migrations, ".")
|
||||
if err != nil {
|
||||
return fmt.Errorf("migrations source: %w", err)
|
||||
}
|
||||
|
||||
sqlDB, err := sql.Open("sqlite", path)
|
||||
if err != nil {
|
||||
return fmt.Errorf("open sql db: %w", err)
|
||||
}
|
||||
defer sqlDB.Close()
|
||||
|
||||
driver, err := sqlitemig.WithInstance(sqlDB, &sqlitemig.Config{})
|
||||
if err != nil {
|
||||
return fmt.Errorf("migrate driver: %w", err)
|
||||
}
|
||||
|
||||
m, err := migrate.NewWithInstance("iofs", source, "sqlite", driver)
|
||||
if err != nil {
|
||||
return fmt.Errorf("migrate instance: %w", err)
|
||||
}
|
||||
if err := m.Up(); err != nil && !errors.Is(err, migrate.ErrNoChange) {
|
||||
return fmt.Errorf("migrate up: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
package db
|
||||
|
||||
import (
|
||||
"io/fs"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
assets "shared-album"
|
||||
)
|
||||
|
||||
func TestMigrateSQLite(t *testing.T) {
|
||||
path := filepath.Join(t.TempDir(), "meta.db")
|
||||
migFS, err := fs.Sub(assets.Migrations, "migrations/sqlite")
|
||||
if err != nil {
|
||||
t.Fatalf("migrations fs: %v", err)
|
||||
}
|
||||
if err := MigrateSQLite(path, migFS); err != nil {
|
||||
t.Fatalf("migrate: %v", err)
|
||||
}
|
||||
if err := MigrateSQLite(path, migFS); err != nil {
|
||||
t.Fatalf("migrate again: %v", err)
|
||||
}
|
||||
|
||||
db, err := OpenSQLite(path)
|
||||
if err != nil {
|
||||
t.Fatalf("open: %v", err)
|
||||
}
|
||||
defer db.Close()
|
||||
|
||||
var n int
|
||||
if err := db.QueryRow(`SELECT count(*) FROM sqlite_master WHERE type='table' AND name='objects'`).Scan(&n); err != nil {
|
||||
t.Fatalf("query: %v", err)
|
||||
}
|
||||
if n != 1 {
|
||||
t.Fatalf("objects table missing after migration")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
package db
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
_ "modernc.org/sqlite"
|
||||
)
|
||||
|
||||
// OpenSQLite opens (creating if needed) a SQLite database tuned for a single
|
||||
// writer with concurrent readers: WAL journalling, a busy timeout so writers
|
||||
// wait instead of failing, and enforced foreign keys.
|
||||
func OpenSQLite(path string) (*sql.DB, error) {
|
||||
if strings.TrimSpace(path) == "" {
|
||||
return nil, fmt.Errorf("sqlite path is required")
|
||||
}
|
||||
dsn := fmt.Sprintf(
|
||||
"file:%s?_pragma=journal_mode(WAL)&_pragma=busy_timeout(5000)&_pragma=foreign_keys(1)&_pragma=synchronous(NORMAL)&_txlock=immediate",
|
||||
path,
|
||||
)
|
||||
db, err := sql.Open("sqlite", dsn)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open sqlite: %w", err)
|
||||
}
|
||||
if err := db.Ping(); err != nil {
|
||||
_ = db.Close()
|
||||
return nil, fmt.Errorf("ping sqlite: %w", err)
|
||||
}
|
||||
return db, nil
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
package db
|
||||
|
||||
import (
|
||||
"context"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestOpenSQLite(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
db, err := OpenSQLite(filepath.Join(t.TempDir(), "test.db"))
|
||||
if err != nil {
|
||||
t.Fatalf("open: %v", err)
|
||||
}
|
||||
defer db.Close()
|
||||
|
||||
if err := db.PingContext(ctx); err != nil {
|
||||
t.Fatalf("ping: %v", err)
|
||||
}
|
||||
|
||||
var mode string
|
||||
if err := db.QueryRowContext(ctx, "PRAGMA journal_mode").Scan(&mode); err != nil {
|
||||
t.Fatalf("journal_mode: %v", err)
|
||||
}
|
||||
if !strings.EqualFold(mode, "wal") {
|
||||
t.Fatalf("journal_mode = %q, want wal", mode)
|
||||
}
|
||||
|
||||
var fk int
|
||||
if err := db.QueryRowContext(ctx, "PRAGMA foreign_keys").Scan(&fk); err != nil {
|
||||
t.Fatalf("foreign_keys: %v", err)
|
||||
}
|
||||
if fk != 1 {
|
||||
t.Fatalf("foreign_keys = %d, want 1", fk)
|
||||
}
|
||||
|
||||
var busy int
|
||||
if err := db.QueryRowContext(ctx, "PRAGMA busy_timeout").Scan(&busy); err != nil {
|
||||
t.Fatalf("busy_timeout: %v", err)
|
||||
}
|
||||
if busy != 5000 {
|
||||
t.Fatalf("busy_timeout = %d, want 5000", busy)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,198 @@
|
||||
package e2e
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"database/sql"
|
||||
"image"
|
||||
"image/color"
|
||||
"image/png"
|
||||
"io/fs"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
assets "shared-album"
|
||||
"shared-album/internal/albums"
|
||||
"shared-album/internal/auth"
|
||||
"shared-album/internal/db"
|
||||
"shared-album/internal/photos"
|
||||
"shared-album/internal/settings"
|
||||
"shared-album/internal/sse"
|
||||
"shared-album/internal/storage"
|
||||
"shared-album/internal/store"
|
||||
)
|
||||
|
||||
func makePNG(t *testing.T) []byte {
|
||||
t.Helper()
|
||||
img := image.NewRGBA(image.Rect(0, 0, 4, 4))
|
||||
for x := 0; x < 4; x++ {
|
||||
for y := 0; y < 4; y++ {
|
||||
img.Set(x, y, color.RGBA{R: uint8(x * 40), G: uint8(y * 40), B: 128, A: 255})
|
||||
}
|
||||
}
|
||||
var buf bytes.Buffer
|
||||
if err := png.Encode(&buf, img); err != nil {
|
||||
t.Fatalf("encode png: %v", err)
|
||||
}
|
||||
return buf.Bytes()
|
||||
}
|
||||
|
||||
type stack struct {
|
||||
db *sql.DB
|
||||
photos *photos.Service
|
||||
albums *albums.Service
|
||||
auth *auth.Service
|
||||
}
|
||||
|
||||
// openStack opens a migrated SQLite database at path and builds the services.
|
||||
func openStack(t *testing.T, path, blobRoot string) (*stack, func()) {
|
||||
t.Helper()
|
||||
migFS, err := fs.Sub(assets.Migrations, "migrations/sqlite")
|
||||
if err != nil {
|
||||
t.Fatalf("migrations fs: %v", err)
|
||||
}
|
||||
if err := db.MigrateSQLite(path, migFS); err != nil {
|
||||
t.Fatalf("migrate: %v", err)
|
||||
}
|
||||
database, err := db.OpenSQLite(path)
|
||||
if err != nil {
|
||||
t.Fatalf("open: %v", err)
|
||||
}
|
||||
repos := store.NewSQLite(database)
|
||||
hub := sse.NewHub(1)
|
||||
signer, err := storage.NewSigner("http://app.test", "secret", time.Hour)
|
||||
if err != nil {
|
||||
t.Fatalf("signer: %v", err)
|
||||
}
|
||||
blob, err := storage.NewFS(blobRoot, signer)
|
||||
if err != nil {
|
||||
t.Fatalf("fs: %v", err)
|
||||
}
|
||||
authSvc := auth.NewService(repos.Auth, auth.NewTokenManager("secret", 15*time.Minute, time.Hour))
|
||||
albumsSvc := albums.NewService(repos.Albums, repos.Members, hub, blob)
|
||||
photosSvc := photos.NewService(repos.Photos, albumsSvc, blob, hub)
|
||||
return &stack{db: database, photos: photosSvc, albums: albumsSvc, auth: authSvc}, func() { database.Close() }
|
||||
}
|
||||
|
||||
func TestSQLiteEndToEndWithRestart(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
path := filepath.Join(t.TempDir(), "galo.db")
|
||||
blobs := t.TempDir()
|
||||
|
||||
st, closeDB := openStack(t, path, blobs)
|
||||
worker := photos.NewWorker(st.photos, 20*time.Millisecond, 10, time.Minute)
|
||||
wctx, cancel := context.WithCancel(ctx)
|
||||
go worker.Run(wctx)
|
||||
|
||||
user, _, err := st.auth.Register(ctx, "e2e@test.local", "password123", "E2E")
|
||||
if err != nil {
|
||||
t.Fatalf("register: %v", err)
|
||||
}
|
||||
album, err := st.albums.Create(ctx, user.ID, albums.CreateInput{Title: "Trip"})
|
||||
if err != nil {
|
||||
t.Fatalf("create album: %v", err)
|
||||
}
|
||||
p, err := st.photos.Upload(ctx, photos.UploadInput{AlbumID: album.ID, UserID: user.ID, Filename: "a.png", Data: makePNG(t)})
|
||||
if err != nil {
|
||||
t.Fatalf("upload: %v", err)
|
||||
}
|
||||
waitFor(t, "variants of first photo", func() bool {
|
||||
got, err := st.photos.Get(ctx, p.ID, user.ID)
|
||||
return err == nil && got.ThumbURL != "" && got.PreviewURL != ""
|
||||
})
|
||||
|
||||
if err := st.photos.PermanentDelete(ctx, p.ID, user.ID); err != nil {
|
||||
t.Fatalf("permanent delete: %v", err)
|
||||
}
|
||||
waitFor(t, "garbage collection", func() bool {
|
||||
var n int
|
||||
return st.db.QueryRowContext(ctx, `SELECT count(*) FROM objects`).Scan(&n) == nil && n == 0
|
||||
})
|
||||
|
||||
// Stop the worker, then queue another upload whose variant job must survive
|
||||
// the restart and be processed by a fresh worker over the same file.
|
||||
cancel()
|
||||
time.Sleep(50 * time.Millisecond)
|
||||
|
||||
p2, err := st.photos.Upload(ctx, photos.UploadInput{AlbumID: album.ID, UserID: user.ID, Filename: "b.png", Data: makePNG(t)})
|
||||
if err != nil {
|
||||
t.Fatalf("upload p2: %v", err)
|
||||
}
|
||||
got2, _ := st.photos.Get(ctx, p2.ID, user.ID)
|
||||
if got2.PreviewURL != "" {
|
||||
t.Fatalf("variants generated before restart")
|
||||
}
|
||||
closeDB()
|
||||
|
||||
st2, close2 := openStack(t, path, blobs)
|
||||
defer close2()
|
||||
worker2 := photos.NewWorker(st2.photos, 20*time.Millisecond, 10, time.Minute)
|
||||
wctx2, cancel2 := context.WithCancel(ctx)
|
||||
defer cancel2()
|
||||
go worker2.Run(wctx2)
|
||||
|
||||
waitFor(t, "variants of second photo after restart", func() bool {
|
||||
got, err := st2.photos.Get(ctx, p2.ID, user.ID)
|
||||
return err == nil && got.ThumbURL != "" && got.PreviewURL != ""
|
||||
})
|
||||
}
|
||||
|
||||
func TestJWTSecretPersistsAcrossRestart(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
path := filepath.Join(t.TempDir(), "galo.db")
|
||||
migFS, err := fs.Sub(assets.Migrations, "migrations/sqlite")
|
||||
if err != nil {
|
||||
t.Fatalf("migrations fs: %v", err)
|
||||
}
|
||||
|
||||
open := func() (*sql.DB, store.Repos) {
|
||||
t.Helper()
|
||||
if err := db.MigrateSQLite(path, migFS); err != nil {
|
||||
t.Fatalf("migrate: %v", err)
|
||||
}
|
||||
database, err := db.OpenSQLite(path)
|
||||
if err != nil {
|
||||
t.Fatalf("open: %v", err)
|
||||
}
|
||||
return database, store.NewSQLite(database)
|
||||
}
|
||||
|
||||
db1, repos1 := open()
|
||||
secret1, err := repos1.Settings.Ensure(ctx, settings.JWTSecretKey, settings.GenerateSecret)
|
||||
if err != nil || secret1 == "" {
|
||||
t.Fatalf("ensure secret: %v %q", err, secret1)
|
||||
}
|
||||
tm1 := auth.NewTokenManager(secret1, 15*time.Minute, time.Hour)
|
||||
token, err := tm1.GenerateAccess("user-1", "u@test.local")
|
||||
if err != nil {
|
||||
t.Fatalf("generate: %v", err)
|
||||
}
|
||||
_ = db1.Close()
|
||||
|
||||
db2, repos2 := open()
|
||||
defer db2.Close()
|
||||
secret2, err := repos2.Settings.Ensure(ctx, settings.JWTSecretKey, settings.GenerateSecret)
|
||||
if err != nil {
|
||||
t.Fatalf("ensure after restart: %v", err)
|
||||
}
|
||||
if secret2 != secret1 {
|
||||
t.Fatalf("secret changed across restart")
|
||||
}
|
||||
tm2 := auth.NewTokenManager(secret2, 15*time.Minute, time.Hour)
|
||||
if _, err := tm2.ParseAccess(token); err != nil {
|
||||
t.Fatalf("token issued before restart not valid after: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func waitFor(t *testing.T, what string, cond func() bool) {
|
||||
t.Helper()
|
||||
deadline := time.Now().Add(10 * time.Second)
|
||||
for time.Now().Before(deadline) {
|
||||
if cond() {
|
||||
return
|
||||
}
|
||||
time.Sleep(20 * time.Millisecond)
|
||||
}
|
||||
t.Fatalf("timed out waiting for %s", what)
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
package members
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"shared-album/internal/models"
|
||||
)
|
||||
|
||||
// Repository is the persistence contract for the members domain.
|
||||
type Repository interface {
|
||||
Role(ctx context.Context, albumID, userID string) (string, error)
|
||||
List(ctx context.Context, albumID string) ([]models.AlbumMember, error)
|
||||
Add(ctx context.Context, albumID, userID, role string, invitedBy *string) error
|
||||
UpdateRole(ctx context.Context, albumID, userID, role string) error
|
||||
Remove(ctx context.Context, albumID, userID string) error
|
||||
CreateInvite(ctx context.Context, inv *models.Invite) error
|
||||
GetInviteByToken(ctx context.Context, token string) (*models.Invite, error)
|
||||
IncrementUses(ctx context.Context, id string) error
|
||||
}
|
||||
@@ -21,13 +21,13 @@ type Notifier interface {
|
||||
}
|
||||
|
||||
type Service struct {
|
||||
store *Store
|
||||
store Repository
|
||||
albums AlbumReader
|
||||
hub *sse.Hub
|
||||
notify Notifier
|
||||
}
|
||||
|
||||
func NewService(store *Store, albums AlbumReader, hub *sse.Hub, notify Notifier) *Service {
|
||||
func NewService(store Repository, albums AlbumReader, hub *sse.Hub, notify Notifier) *Service {
|
||||
return &Service{store: store, albums: albums, hub: hub, notify: notify}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
package members
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
|
||||
"shared-album/internal/models"
|
||||
"shared-album/internal/sqliteutil"
|
||||
)
|
||||
|
||||
// SQLiteStore is the SQLite implementation of Repository.
|
||||
type SQLiteStore struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func NewSQLiteStore(db *sql.DB) *SQLiteStore { return &SQLiteStore{db: db} }
|
||||
|
||||
func (s *SQLiteStore) Role(ctx context.Context, albumID, userID string) (string, error) {
|
||||
var role string
|
||||
err := s.db.QueryRowContext(ctx,
|
||||
`SELECT role FROM album_members WHERE album_id = $1 AND user_id = $2`,
|
||||
albumID, userID).Scan(&role)
|
||||
if err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return "", nil
|
||||
}
|
||||
return "", err
|
||||
}
|
||||
return role, nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) List(ctx context.Context, albumID string) ([]models.AlbumMember, error) {
|
||||
rows, err := s.db.QueryContext(ctx,
|
||||
`SELECT m.album_id, m.user_id, m.role, m.invited_by, m.joined_at,
|
||||
u.id, u.email, u.name, COALESCE(u.avatar_url, '')
|
||||
FROM album_members m
|
||||
JOIN users u ON u.id = m.user_id
|
||||
WHERE m.album_id = $1
|
||||
ORDER BY m.joined_at`, albumID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
out := []models.AlbumMember{}
|
||||
for rows.Next() {
|
||||
var m models.AlbumMember
|
||||
var invitedBy sql.NullString
|
||||
var joinedAt string
|
||||
if err := rows.Scan(&m.AlbumID, &m.UserID, &m.Role, &invitedBy, &joinedAt,
|
||||
&m.User.ID, &m.User.Email, &m.User.Name, &m.User.AvatarURL); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if invitedBy.Valid {
|
||||
m.InvitedBy = &invitedBy.String
|
||||
}
|
||||
m.JoinedAt = sqliteutil.ParseTime(joinedAt)
|
||||
out = append(out, m)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) Add(ctx context.Context, albumID, userID, role string, invitedBy *string) error {
|
||||
_, err := s.db.ExecContext(ctx,
|
||||
`INSERT INTO album_members (album_id, user_id, role, invited_by)
|
||||
VALUES ($1, $2, $3, $4)
|
||||
ON CONFLICT (album_id, user_id) DO UPDATE SET role = excluded.role`,
|
||||
albumID, userID, role, invitedBy)
|
||||
return err
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) UpdateRole(ctx context.Context, albumID, userID, role string) error {
|
||||
res, err := s.db.ExecContext(ctx,
|
||||
`UPDATE album_members SET role = $3 WHERE album_id = $1 AND user_id = $2`,
|
||||
albumID, userID, role)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if n, _ := res.RowsAffected(); n == 0 {
|
||||
return ErrNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) Remove(ctx context.Context, albumID, userID string) error {
|
||||
res, err := s.db.ExecContext(ctx,
|
||||
`DELETE FROM album_members WHERE album_id = $1 AND user_id = $2`, albumID, userID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if n, _ := res.RowsAffected(); n == 0 {
|
||||
return ErrNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) CreateInvite(ctx context.Context, inv *models.Invite) error {
|
||||
var createdAt string
|
||||
err := s.db.QueryRowContext(ctx,
|
||||
`INSERT INTO invites (id, album_id, token, role, created_by, expires_at, max_uses)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7)
|
||||
RETURNING created_at`,
|
||||
inv.ID, inv.AlbumID, inv.Token, inv.Role, inv.CreatedBy, sqliteutil.FormatTime(inv.ExpiresAt), inv.MaxUses,
|
||||
).Scan(&createdAt)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
inv.CreatedAt = sqliteutil.ParseTime(createdAt)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) GetInviteByToken(ctx context.Context, token string) (*models.Invite, error) {
|
||||
var inv models.Invite
|
||||
var expiresAt, createdAt string
|
||||
err := s.db.QueryRowContext(ctx,
|
||||
`SELECT id, album_id, token, role, created_by, expires_at, max_uses, uses, created_at
|
||||
FROM invites WHERE token = $1`, token).
|
||||
Scan(&inv.ID, &inv.AlbumID, &inv.Token, &inv.Role, &inv.CreatedBy, &expiresAt, &inv.MaxUses, &inv.Uses, &createdAt)
|
||||
if err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return nil, ErrNotFound
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
inv.ExpiresAt = sqliteutil.ParseTime(expiresAt)
|
||||
inv.CreatedAt = sqliteutil.ParseTime(createdAt)
|
||||
return &inv, nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) IncrementUses(ctx context.Context, id string) error {
|
||||
_, err := s.db.ExecContext(ctx, `UPDATE invites SET uses = uses + 1 WHERE id = $1`, id)
|
||||
return err
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
package notifications
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"shared-album/internal/models"
|
||||
)
|
||||
|
||||
// Repository is the persistence contract for the notifications domain.
|
||||
type Repository interface {
|
||||
Create(ctx context.Context, n *models.Notification) error
|
||||
List(ctx context.Context, userID string, limit int) ([]models.Notification, error)
|
||||
MarkRead(ctx context.Context, id, userID string) error
|
||||
}
|
||||
@@ -10,11 +10,11 @@ import (
|
||||
)
|
||||
|
||||
type Service struct {
|
||||
store *Store
|
||||
store Repository
|
||||
hub *sse.Hub
|
||||
}
|
||||
|
||||
func NewService(store *Store, hub *sse.Hub) *Service {
|
||||
func NewService(store Repository, hub *sse.Hub) *Service {
|
||||
return &Service{store: store, hub: hub}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
package notifications
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"time"
|
||||
|
||||
"shared-album/internal/models"
|
||||
"shared-album/internal/sqliteutil"
|
||||
)
|
||||
|
||||
// SQLiteStore is the SQLite implementation of Repository.
|
||||
type SQLiteStore struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func NewSQLiteStore(db *sql.DB) *SQLiteStore { return &SQLiteStore{db: db} }
|
||||
|
||||
func (s *SQLiteStore) Create(ctx context.Context, n *models.Notification) error {
|
||||
payload, err := json.Marshal(n.PayloadJSON)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var createdAt string
|
||||
err = s.db.QueryRowContext(ctx,
|
||||
`INSERT INTO notifications (id, user_id, type, payload_json)
|
||||
VALUES ($1, $2, $3, $4)
|
||||
RETURNING created_at`,
|
||||
n.ID, n.UserID, n.Type, string(payload)).Scan(&createdAt)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
n.CreatedAt = sqliteutil.ParseTime(createdAt)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) List(ctx context.Context, userID string, limit int) ([]models.Notification, error) {
|
||||
if limit <= 0 || limit > 200 {
|
||||
limit = 100
|
||||
}
|
||||
rows, err := s.db.QueryContext(ctx,
|
||||
`SELECT id, user_id, type, payload_json, read_at, created_at
|
||||
FROM notifications
|
||||
WHERE user_id = $1
|
||||
ORDER BY created_at DESC
|
||||
LIMIT $2`, userID, limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
out := []models.Notification{}
|
||||
for rows.Next() {
|
||||
var n models.Notification
|
||||
var payload, createdAt string
|
||||
var readAt sql.NullString
|
||||
if err := rows.Scan(&n.ID, &n.UserID, &n.Type, &payload, &readAt, &createdAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
_ = json.Unmarshal([]byte(payload), &n.PayloadJSON)
|
||||
n.ReadAt = sqliteutil.ParseNullTime(readAt)
|
||||
n.CreatedAt = sqliteutil.ParseTime(createdAt)
|
||||
out = append(out, n)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) MarkRead(ctx context.Context, id, userID string) error {
|
||||
res, err := s.db.ExecContext(ctx,
|
||||
`UPDATE notifications SET read_at = $3 WHERE id = $1 AND user_id = $2 AND read_at IS NULL`,
|
||||
id, userID, sqliteutil.FormatTime(time.Now()))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if n, _ := res.RowsAffected(); n == 0 {
|
||||
var exists int
|
||||
err := s.db.QueryRowContext(ctx,
|
||||
`SELECT 1 FROM notifications WHERE id = $1 AND user_id = $2`, id, userID).Scan(&exists)
|
||||
if err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return ErrNotFound
|
||||
}
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,92 @@
|
||||
package photos
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestCopySharesStorageObjects(t *testing.T) {
|
||||
pool := testPool(t)
|
||||
ctx := context.Background()
|
||||
svc, cl := newTestService(t, pool)
|
||||
user := createUser(t, ctx, pool)
|
||||
albumA := createAlbum(t, ctx, pool, user)
|
||||
albumB := createAlbum(t, ctx, pool, user)
|
||||
|
||||
p, err := svc.Upload(ctx, UploadInput{
|
||||
AlbumID: albumA, UserID: user, Filename: "p.png", MimeType: "image/png", Data: makePNG(t),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("upload: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { cleanupPhoto(t, ctx, pool, cl, p.ID) })
|
||||
|
||||
w := NewWorker(svc, time.Second, 10, time.Minute)
|
||||
w.processVariantJobs(ctx)
|
||||
|
||||
srcIDs, err := svc.store.PhotoObjectIDs(ctx, p.ID)
|
||||
if err != nil || len(srcIDs) != 3 {
|
||||
t.Fatalf("source objects = %v err=%v", srcIDs, err)
|
||||
}
|
||||
|
||||
clones, err := svc.CopyBatch(ctx, []string{p.ID}, user, albumB)
|
||||
if err != nil || len(clones) != 1 {
|
||||
t.Fatalf("copy: %v len=%d", err, len(clones))
|
||||
}
|
||||
clone := clones[0]
|
||||
t.Cleanup(func() { cleanupPhoto(t, ctx, pool, cl, clone.ID) })
|
||||
|
||||
cloneIDs, err := svc.store.PhotoObjectIDs(ctx, clone.ID)
|
||||
if err != nil || len(cloneIDs) != 3 {
|
||||
t.Fatalf("clone objects = %v err=%v", cloneIDs, err)
|
||||
}
|
||||
if !sameSet(srcIDs, cloneIDs) {
|
||||
t.Fatalf("clone does not share objects: src=%v clone=%v", srcIDs, cloneIDs)
|
||||
}
|
||||
for _, id := range cloneIDs {
|
||||
if n, err := svc.store.ObjectRefCount(ctx, id); err != nil || n != 2 {
|
||||
t.Fatalf("object %s ref count = %d err=%v, want 2", id, n, err)
|
||||
}
|
||||
}
|
||||
|
||||
// Move the source into the same album; the storage key and shared objects
|
||||
// must be untouched.
|
||||
beforeKey := p.StorageKey
|
||||
moved, err := svc.MoveBatch(ctx, []string{p.ID}, user, albumB)
|
||||
if err != nil || moved != 1 {
|
||||
t.Fatalf("move: %v moved=%d", err, moved)
|
||||
}
|
||||
after, err := svc.store.GetByID(ctx, p.ID, "")
|
||||
if err != nil {
|
||||
t.Fatalf("get after move: %v", err)
|
||||
}
|
||||
if after.AlbumID != albumB {
|
||||
t.Fatalf("photo not moved: album=%q", after.AlbumID)
|
||||
}
|
||||
if after.StorageKey != beforeKey {
|
||||
t.Fatalf("move changed storage key: %q -> %q", beforeKey, after.StorageKey)
|
||||
}
|
||||
for _, id := range cloneIDs {
|
||||
if n, err := svc.store.ObjectRefCount(ctx, id); err != nil || n != 2 {
|
||||
t.Fatalf("object %s ref count after move = %d err=%v, want 2", id, n, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func sameSet(a, b []string) bool {
|
||||
if len(a) != len(b) {
|
||||
return false
|
||||
}
|
||||
m := make(map[string]int, len(a))
|
||||
for _, v := range a {
|
||||
m[v]++
|
||||
}
|
||||
for _, v := range b {
|
||||
m[v]--
|
||||
if m[v] < 0 {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
@@ -0,0 +1,167 @@
|
||||
package photos
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"image"
|
||||
"image/color"
|
||||
"image/png"
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
|
||||
"shared-album/internal/auth"
|
||||
"shared-album/internal/config"
|
||||
"shared-album/internal/models"
|
||||
"shared-album/internal/roles"
|
||||
"shared-album/internal/sse"
|
||||
"shared-album/internal/storage"
|
||||
)
|
||||
|
||||
func testPool(t *testing.T) *pgxpool.Pool {
|
||||
t.Helper()
|
||||
dsn := os.Getenv("TEST_DATABASE_URL")
|
||||
if dsn == "" {
|
||||
t.Skip("TEST_DATABASE_URL not set")
|
||||
}
|
||||
pool, err := pgxpool.New(context.Background(), dsn)
|
||||
if err != nil {
|
||||
t.Skipf("cannot connect: %v", err)
|
||||
}
|
||||
if err := pool.Ping(context.Background()); err != nil {
|
||||
t.Skipf("cannot ping: %v", err)
|
||||
}
|
||||
t.Cleanup(pool.Close)
|
||||
return pool
|
||||
}
|
||||
|
||||
func testEnvOr(key, fallback string) string {
|
||||
if v := os.Getenv(key); v != "" {
|
||||
return v
|
||||
}
|
||||
return fallback
|
||||
}
|
||||
|
||||
type stubAlbums struct{}
|
||||
|
||||
func (stubAlbums) Access(ctx context.Context, albumID, userID string) (*models.Album, string, error) {
|
||||
return &models.Album{ID: albumID}, roles.Owner, nil
|
||||
}
|
||||
|
||||
func createUser(t *testing.T, ctx context.Context, pool *pgxpool.Pool) string {
|
||||
t.Helper()
|
||||
id := auth.NewID()
|
||||
if _, err := pool.Exec(ctx,
|
||||
`INSERT INTO users (id, email, password_hash, name) VALUES ($1, $2, 'x', 't')`,
|
||||
id, id+"@test.local"); err != nil {
|
||||
t.Fatalf("create user: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { _, _ = pool.Exec(ctx, `DELETE FROM users WHERE id = $1`, id) })
|
||||
return id
|
||||
}
|
||||
|
||||
func createAlbum(t *testing.T, ctx context.Context, pool *pgxpool.Pool, ownerID string) string {
|
||||
t.Helper()
|
||||
id := auth.NewID()
|
||||
if _, err := pool.Exec(ctx,
|
||||
`INSERT INTO albums (id, owner_id, title) VALUES ($1, $2, 't')`, id, ownerID); err != nil {
|
||||
t.Fatalf("create album: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { _, _ = pool.Exec(ctx, `DELETE FROM albums WHERE id = $1`, id) })
|
||||
return id
|
||||
}
|
||||
|
||||
func createPhoto(t *testing.T, ctx context.Context, pool *pgxpool.Pool, uploaderID string) string {
|
||||
t.Helper()
|
||||
id := auth.NewID()
|
||||
if _, err := pool.Exec(ctx,
|
||||
`INSERT INTO photos (id, album_id, uploader_id) VALUES ($1, NULL, $2)`, id, uploaderID); err != nil {
|
||||
t.Fatalf("create photo: %v", err)
|
||||
}
|
||||
return id
|
||||
}
|
||||
|
||||
func makePNG(t *testing.T) []byte {
|
||||
t.Helper()
|
||||
img := image.NewRGBA(image.Rect(0, 0, 4, 4))
|
||||
for x := 0; x < 4; x++ {
|
||||
for y := 0; y < 4; y++ {
|
||||
img.Set(x, y, color.RGBA{R: uint8(x * 40), G: uint8(y * 40), B: 128, A: 255})
|
||||
}
|
||||
}
|
||||
var buf bytes.Buffer
|
||||
if err := png.Encode(&buf, img); err != nil {
|
||||
t.Fatalf("encode png: %v", err)
|
||||
}
|
||||
return buf.Bytes()
|
||||
}
|
||||
|
||||
// newTestService builds a service backed by the real dev storage. It skips when
|
||||
// S3 env is absent so unit runs stay hermetic.
|
||||
func newTestService(t *testing.T, pool *pgxpool.Pool) (*Service, storage.Backend) {
|
||||
t.Helper()
|
||||
endpoint := os.Getenv("S3_ENDPOINT")
|
||||
if endpoint == "" {
|
||||
t.Skip("S3_ENDPOINT not set")
|
||||
}
|
||||
signer, err := storage.NewSigner("http://app.test", "test-secret", time.Hour)
|
||||
if err != nil {
|
||||
t.Skipf("signer init: %v", err)
|
||||
}
|
||||
cl, err := storage.NewS3(context.Background(), config.S3{
|
||||
Endpoint: endpoint,
|
||||
Region: testEnvOr("S3_REGION", "garage"),
|
||||
AccessKey: os.Getenv("S3_ACCESS_KEY"),
|
||||
SecretKey: os.Getenv("S3_SECRET_KEY"),
|
||||
Bucket: testEnvOr("S3_BUCKET", "galo-data"),
|
||||
UsePathStyle: testEnvOr("S3_USE_PATH_STYLE", "true") != "false",
|
||||
}, signer)
|
||||
if err != nil {
|
||||
t.Skipf("storage init: %v", err)
|
||||
}
|
||||
return NewService(NewStore(pool), stubAlbums{}, cl, sse.NewHub(1)), cl
|
||||
}
|
||||
|
||||
// newTestServiceFS builds a service backed by a temporary local file-system
|
||||
// storage backend so the full flow can run without external infrastructure.
|
||||
func newTestServiceFS(t *testing.T, pool *pgxpool.Pool) (*Service, storage.Backend) {
|
||||
t.Helper()
|
||||
signer, err := storage.NewSigner("http://app.test", "test-secret", time.Hour)
|
||||
if err != nil {
|
||||
t.Fatalf("storage init: %v", err)
|
||||
}
|
||||
fs, err := storage.NewFS(t.TempDir(), signer)
|
||||
if err != nil {
|
||||
t.Fatalf("storage init: %v", err)
|
||||
}
|
||||
return NewService(NewStore(pool), stubAlbums{}, fs, sse.NewHub(1)), fs
|
||||
}
|
||||
|
||||
// cleanupPhoto removes a photo, its objects (rows and blobs) and bindings.
|
||||
func cleanupPhoto(t *testing.T, ctx context.Context, pool *pgxpool.Pool, cl storage.Backend, photoID string) {
|
||||
t.Helper()
|
||||
rows, err := pool.Query(ctx,
|
||||
`SELECT o.id, o.storage_key FROM photo_objects po JOIN objects o ON o.id = po.object_id
|
||||
WHERE po.photo_id = $1`, photoID)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
var ids, keys []string
|
||||
for rows.Next() {
|
||||
var id, key string
|
||||
if rows.Scan(&id, &key) == nil {
|
||||
ids = append(ids, id)
|
||||
keys = append(keys, key)
|
||||
}
|
||||
}
|
||||
rows.Close()
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM photos WHERE id = $1`, photoID)
|
||||
for _, id := range ids {
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM objects WHERE id = $1`, id)
|
||||
}
|
||||
if cl != nil && len(keys) > 0 {
|
||||
_ = cl.Delete(ctx, keys...)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
package photos
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// objectKey builds an opaque, album-free storage key for a physical object.
|
||||
// The key never encodes album, photo or any other metadata entity; the mapping
|
||||
// between a key and its photo binding lives only in the metadata layer. The
|
||||
// role segment is cosmetic (for debugging) and is not authoritative.
|
||||
func objectKey(objectID, role, ext string) string {
|
||||
if ext != "" && !strings.HasPrefix(ext, ".") {
|
||||
ext = "." + ext
|
||||
}
|
||||
return fmt.Sprintf("objects/%s/%s%s", objectID, role, ext)
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
package photos
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestObjectKeyIsAlbumFree(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
id string
|
||||
role string
|
||||
ext string
|
||||
want string
|
||||
}{
|
||||
{"original with dot", "obj1", RoleOriginal, ".jpg", "objects/obj1/original.jpg"},
|
||||
{"thumb without dot", "obj1", RoleThumb, "webp", "objects/obj1/thumb.webp"},
|
||||
{"preview", "obj2", RolePreview, ".webp", "objects/obj2/preview.webp"},
|
||||
{"empty ext", "obj3", RoleOriginal, "", "objects/obj3/original"},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
got := objectKey(tc.id, tc.role, tc.ext)
|
||||
if got != tc.want {
|
||||
t.Fatalf("objectKey = %q, want %q", got, tc.want)
|
||||
}
|
||||
if strings.Contains(got, "album") {
|
||||
t.Fatalf("key must not encode an album entity: %q", got)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,388 @@
|
||||
package photos
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"time"
|
||||
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
// Object lifecycle states. The DB row is the ledger entry that governs the
|
||||
// lifecycle of the corresponding blob in the storage layer.
|
||||
const (
|
||||
ObjectPending = "pending"
|
||||
ObjectReady = "ready"
|
||||
ObjectDeleting = "deleting"
|
||||
)
|
||||
|
||||
// Object roles as bound to a photo. The role is a property of the binding.
|
||||
const (
|
||||
RoleOriginal = "original"
|
||||
RoleThumb = "thumb"
|
||||
RolePreview = "preview"
|
||||
)
|
||||
|
||||
var (
|
||||
// ErrObjectNotFound is returned when an object row does not exist.
|
||||
ErrObjectNotFound = errors.New("object not found")
|
||||
// ErrObjectNotReady is returned when binding an object that is not ready.
|
||||
ErrObjectNotReady = errors.New("object is not ready")
|
||||
)
|
||||
|
||||
// Object is a physical file in the storage layer, described by metadata.
|
||||
type Object struct {
|
||||
ID string
|
||||
StorageKey string
|
||||
MimeType string
|
||||
Size int64
|
||||
Width int
|
||||
Height int
|
||||
Checksum string
|
||||
State string
|
||||
CreatedAt time.Time
|
||||
}
|
||||
|
||||
// CreateObject inserts a new object row in the pending state.
|
||||
func (s *Store) CreateObject(ctx context.Context, o *Object) error {
|
||||
if o.State == "" {
|
||||
o.State = ObjectPending
|
||||
}
|
||||
return s.pool.QueryRow(ctx,
|
||||
`INSERT INTO objects (id, storage_key, mime_type, size, width, height, checksum, state)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
|
||||
RETURNING created_at`,
|
||||
o.ID, o.StorageKey, o.MimeType, o.Size, o.Width, o.Height, o.Checksum, o.State).Scan(&o.CreatedAt)
|
||||
}
|
||||
|
||||
// SetObjectState updates an object's lifecycle state.
|
||||
func (s *Store) SetObjectState(ctx context.Context, id, state string) error {
|
||||
tag, err := s.pool.Exec(ctx, `UPDATE objects SET state = $2 WHERE id = $1`, id, state)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if tag.RowsAffected() == 0 {
|
||||
return ErrObjectNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// ActivateObject marks an object ready after its bytes have been stored.
|
||||
func (s *Store) ActivateObject(ctx context.Context, id string) error {
|
||||
return s.SetObjectState(ctx, id, ObjectReady)
|
||||
}
|
||||
|
||||
// MarkObjectDeleting marks an object as pending removal from storage.
|
||||
func (s *Store) MarkObjectDeleting(ctx context.Context, id string) error {
|
||||
return s.SetObjectState(ctx, id, ObjectDeleting)
|
||||
}
|
||||
|
||||
// DeleteObjectRow permanently removes an object row (after its blob is gone).
|
||||
func (s *Store) DeleteObjectRow(ctx context.Context, id string) error {
|
||||
tag, err := s.pool.Exec(ctx, `DELETE FROM objects WHERE id = $1`, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if tag.RowsAffected() == 0 {
|
||||
return ErrObjectNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// BindObject attaches a ready object to a photo under the given role. It locks
|
||||
// the object row and refuses objects that are not ready, so an object being
|
||||
// garbage-collected can never be bound.
|
||||
func (s *Store) BindObject(ctx context.Context, photoID, objectID, role string) error {
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer func() { _ = tx.Rollback(ctx) }()
|
||||
|
||||
var state string
|
||||
if err := tx.QueryRow(ctx, `SELECT state FROM objects WHERE id = $1 FOR UPDATE`, objectID).Scan(&state); err != nil {
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return ErrObjectNotFound
|
||||
}
|
||||
return err
|
||||
}
|
||||
if state != ObjectReady {
|
||||
return ErrObjectNotReady
|
||||
}
|
||||
if _, err := tx.Exec(ctx,
|
||||
`INSERT INTO photo_objects (photo_id, object_id, role) VALUES ($1, $2, $3)
|
||||
ON CONFLICT (photo_id, role) DO UPDATE SET object_id = EXCLUDED.object_id`,
|
||||
photoID, objectID, role); err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Commit(ctx)
|
||||
}
|
||||
|
||||
// PhotoObjectIDs returns the object ids currently bound to a photo.
|
||||
func (s *Store) PhotoObjectIDs(ctx context.Context, photoID string) ([]string, error) {
|
||||
rows, err := s.pool.Query(ctx, `SELECT object_id FROM photo_objects WHERE photo_id = $1`, photoID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var ids []string
|
||||
for rows.Next() {
|
||||
var id string
|
||||
if err := rows.Scan(&id); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
ids = append(ids, id)
|
||||
}
|
||||
return ids, rows.Err()
|
||||
}
|
||||
|
||||
// ObjectRefCount counts how many photo bindings reference an object.
|
||||
func (s *Store) ObjectRefCount(ctx context.Context, objectID string) (int, error) {
|
||||
var n int
|
||||
err := s.pool.QueryRow(ctx, `SELECT count(*) FROM photo_objects WHERE object_id = $1`, objectID).Scan(&n)
|
||||
return n, err
|
||||
}
|
||||
|
||||
// OriginalObject returns the original object bound to a photo.
|
||||
func (s *Store) OriginalObject(ctx context.Context, photoID string) (*Object, error) {
|
||||
var o Object
|
||||
err := s.pool.QueryRow(ctx,
|
||||
`SELECT o.id, o.storage_key, o.mime_type, o.size, o.width, o.height, o.checksum, o.state, o.created_at
|
||||
FROM photo_objects po JOIN objects o ON o.id = po.object_id
|
||||
WHERE po.photo_id = $1 AND po.role = $2`, photoID, RoleOriginal).
|
||||
Scan(&o.ID, &o.StorageKey, &o.MimeType, &o.Size, &o.Width, &o.Height, &o.Checksum, &o.State, &o.CreatedAt)
|
||||
if err != nil {
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return nil, ErrObjectNotFound
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
return &o, nil
|
||||
}
|
||||
|
||||
// PhotosByOriginalObject returns the ids of all photos bound to the given
|
||||
// original object, so generated variants can be shared between copies.
|
||||
func (s *Store) PhotosByOriginalObject(ctx context.Context, originalObjectID string) ([]string, error) {
|
||||
rows, err := s.pool.Query(ctx,
|
||||
`SELECT photo_id FROM photo_objects WHERE object_id = $1 AND role = $2`, originalObjectID, RoleOriginal)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var ids []string
|
||||
for rows.Next() {
|
||||
var id string
|
||||
if err := rows.Scan(&id); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
ids = append(ids, id)
|
||||
}
|
||||
return ids, rows.Err()
|
||||
}
|
||||
|
||||
// ClaimGarbage claims collectible objects: ready objects with no bindings and
|
||||
// objects already marked deleting. Rows are locked with SKIP LOCKED so
|
||||
// concurrent workers do not contend and a concurrent bind cannot slip in.
|
||||
func (s *Store) ClaimGarbage(ctx context.Context, limit int) ([]Object, error) {
|
||||
if limit <= 0 {
|
||||
limit = 10
|
||||
}
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer func() { _ = tx.Rollback(ctx) }()
|
||||
|
||||
rows, err := tx.Query(ctx,
|
||||
`SELECT o.id, o.storage_key FROM objects o
|
||||
WHERE o.state = ANY($1)
|
||||
AND NOT EXISTS (SELECT 1 FROM photo_objects po WHERE po.object_id = o.id)
|
||||
ORDER BY o.created_at
|
||||
LIMIT $2
|
||||
FOR UPDATE SKIP LOCKED`, []string{ObjectReady, ObjectDeleting}, limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var objs []Object
|
||||
for rows.Next() {
|
||||
var o Object
|
||||
if err := rows.Scan(&o.ID, &o.StorageKey); err != nil {
|
||||
rows.Close()
|
||||
return nil, err
|
||||
}
|
||||
objs = append(objs, o)
|
||||
}
|
||||
rows.Close()
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(objs) == 0 {
|
||||
return nil, tx.Commit(ctx)
|
||||
}
|
||||
ids := make([]string, len(objs))
|
||||
for i, o := range objs {
|
||||
ids[i] = o.ID
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `UPDATE objects SET state = $2 WHERE id = ANY($1)`, ids, ObjectDeleting); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return objs, nil
|
||||
}
|
||||
|
||||
// ClaimStalePending marks abandoned pending objects (whose upload never
|
||||
// completed) as deleting and returns them for cleanup.
|
||||
func (s *Store) ClaimStalePending(ctx context.Context, olderThan time.Duration, limit int) ([]Object, error) {
|
||||
if limit <= 0 {
|
||||
limit = 10
|
||||
}
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer func() { _ = tx.Rollback(ctx) }()
|
||||
|
||||
rows, err := tx.Query(ctx,
|
||||
`SELECT id, storage_key FROM objects
|
||||
WHERE state = $1 AND created_at < $2
|
||||
ORDER BY created_at
|
||||
LIMIT $3
|
||||
FOR UPDATE SKIP LOCKED`, ObjectPending, time.Now().Add(-olderThan), limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var objs []Object
|
||||
for rows.Next() {
|
||||
var o Object
|
||||
if err := rows.Scan(&o.ID, &o.StorageKey); err != nil {
|
||||
rows.Close()
|
||||
return nil, err
|
||||
}
|
||||
objs = append(objs, o)
|
||||
}
|
||||
rows.Close()
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(objs) == 0 {
|
||||
return nil, tx.Commit(ctx)
|
||||
}
|
||||
ids := make([]string, len(objs))
|
||||
for i, o := range objs {
|
||||
ids[i] = o.ID
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `UPDATE objects SET state = $2 WHERE id = ANY($1)`, ids, ObjectDeleting); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return objs, nil
|
||||
}
|
||||
|
||||
// ClaimVariantJobs leases due variant jobs and returns their photo ids. The
|
||||
// lease pushes next_attempt_at forward so the same job is not picked up twice
|
||||
// while it is being processed.
|
||||
func (s *Store) ClaimVariantJobs(ctx context.Context, limit int, lease time.Duration) ([]string, error) {
|
||||
if limit <= 0 {
|
||||
limit = 10
|
||||
}
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer func() { _ = tx.Rollback(ctx) }()
|
||||
|
||||
rows, err := tx.Query(ctx,
|
||||
`SELECT photo_id FROM variant_jobs
|
||||
WHERE next_attempt_at <= now()
|
||||
ORDER BY next_attempt_at
|
||||
LIMIT $1
|
||||
FOR UPDATE SKIP LOCKED`, limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var ids []string
|
||||
for rows.Next() {
|
||||
var id string
|
||||
if err := rows.Scan(&id); err != nil {
|
||||
rows.Close()
|
||||
return nil, err
|
||||
}
|
||||
ids = append(ids, id)
|
||||
}
|
||||
rows.Close()
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(ids) == 0 {
|
||||
return nil, tx.Commit(ctx)
|
||||
}
|
||||
if _, err := tx.Exec(ctx,
|
||||
`UPDATE variant_jobs SET next_attempt_at = $2 WHERE photo_id = ANY($1)`,
|
||||
ids, time.Now().Add(lease)); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return ids, nil
|
||||
}
|
||||
|
||||
// CompleteVariantJob removes a finished job.
|
||||
func (s *Store) CompleteVariantJob(ctx context.Context, photoID string) error {
|
||||
_, err := s.pool.Exec(ctx, `DELETE FROM variant_jobs WHERE photo_id = $1`, photoID)
|
||||
return err
|
||||
}
|
||||
|
||||
// FailVariantJob records a failure and schedules a retry with exponential
|
||||
// backoff. After maxAttempts the job is abandoned so a permanently broken photo
|
||||
// cannot loop forever.
|
||||
func (s *Store) FailVariantJob(ctx context.Context, photoID, errMsg string, maxAttempts int) error {
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer func() { _ = tx.Rollback(ctx) }()
|
||||
|
||||
var attempts int
|
||||
err = tx.QueryRow(ctx,
|
||||
`UPDATE variant_jobs
|
||||
SET attempts = attempts + 1,
|
||||
next_attempt_at = now() + make_interval(secs => least(300, 10 * power(2, attempts))),
|
||||
last_error = $2
|
||||
WHERE photo_id = $1
|
||||
RETURNING attempts`, photoID, errMsg).Scan(&attempts)
|
||||
if err != nil {
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return tx.Commit(ctx)
|
||||
}
|
||||
return err
|
||||
}
|
||||
if maxAttempts > 0 && attempts >= maxAttempts {
|
||||
if _, err := tx.Exec(ctx, `DELETE FROM variant_jobs WHERE photo_id = $1`, photoID); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return tx.Commit(ctx)
|
||||
}
|
||||
|
||||
// AllStorageKeys returns every key registered in the object journal.
|
||||
func (s *Store) AllStorageKeys(ctx context.Context) ([]string, error) {
|
||||
rows, err := s.pool.Query(ctx, `SELECT storage_key FROM objects`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var keys []string
|
||||
for rows.Next() {
|
||||
var k string
|
||||
if err := rows.Scan(&k); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
keys = append(keys, k)
|
||||
}
|
||||
return keys, rows.Err()
|
||||
}
|
||||
@@ -0,0 +1,187 @@
|
||||
package photos
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
|
||||
"shared-album/internal/auth"
|
||||
)
|
||||
|
||||
func TestObjectStateTransitions(t *testing.T) {
|
||||
pool := testPool(t)
|
||||
ctx := context.Background()
|
||||
st := NewStore(pool)
|
||||
|
||||
objID := auth.NewID()
|
||||
obj := &Object{ID: objID, StorageKey: "objects/" + objID + "/original.jpg", MimeType: "image/jpeg", Size: 42}
|
||||
t.Cleanup(func() { _, _ = pool.Exec(ctx, `DELETE FROM objects WHERE id = $1`, objID) })
|
||||
|
||||
if err := st.CreateObject(ctx, obj); err != nil {
|
||||
t.Fatalf("create object: %v", err)
|
||||
}
|
||||
if obj.State != ObjectPending {
|
||||
t.Fatalf("new object state = %q, want %q", obj.State, ObjectPending)
|
||||
}
|
||||
if obj.CreatedAt.IsZero() {
|
||||
t.Fatalf("created_at not populated")
|
||||
}
|
||||
|
||||
if err := st.ActivateObject(ctx, objID); err != nil {
|
||||
t.Fatalf("activate: %v", err)
|
||||
}
|
||||
if got := objectState(t, ctx, pool, objID); got != ObjectReady {
|
||||
t.Fatalf("state after activate = %q", got)
|
||||
}
|
||||
if err := st.MarkObjectDeleting(ctx, objID); err != nil {
|
||||
t.Fatalf("mark deleting: %v", err)
|
||||
}
|
||||
if got := objectState(t, ctx, pool, objID); got != ObjectDeleting {
|
||||
t.Fatalf("state after mark = %q", got)
|
||||
}
|
||||
if err := st.DeleteObjectRow(ctx, objID); err != nil {
|
||||
t.Fatalf("delete row: %v", err)
|
||||
}
|
||||
if err := st.DeleteObjectRow(ctx, objID); !errors.Is(err, ErrObjectNotFound) {
|
||||
t.Fatalf("second delete = %v, want ErrObjectNotFound", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClaimStalePending(t *testing.T) {
|
||||
pool := testPool(t)
|
||||
ctx := context.Background()
|
||||
st := NewStore(pool)
|
||||
|
||||
objID := auth.NewID()
|
||||
obj := &Object{ID: objID, StorageKey: "objects/" + objID + "/original.jpg"}
|
||||
t.Cleanup(func() { _, _ = pool.Exec(ctx, `DELETE FROM objects WHERE id = $1`, objID) })
|
||||
if err := st.CreateObject(ctx, obj); err != nil {
|
||||
t.Fatalf("create object: %v", err)
|
||||
}
|
||||
if _, err := pool.Exec(ctx, `UPDATE objects SET created_at = now() - interval '1 hour' WHERE id = $1`, objID); err != nil {
|
||||
t.Fatalf("age object: %v", err)
|
||||
}
|
||||
|
||||
objs, err := st.ClaimStalePending(ctx, 10*time.Minute, 100)
|
||||
if err != nil {
|
||||
t.Fatalf("claim stale: %v", err)
|
||||
}
|
||||
if !containsObject(objs, objID) {
|
||||
t.Fatalf("stale pending object not claimed: %+v", objs)
|
||||
}
|
||||
if got := objectState(t, ctx, pool, objID); got != ObjectDeleting {
|
||||
t.Fatalf("claimed stale object state = %q, want deleting", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBindObjectRequiresReadyAndShares(t *testing.T) {
|
||||
pool := testPool(t)
|
||||
ctx := context.Background()
|
||||
st := NewStore(pool)
|
||||
|
||||
user := createUser(t, ctx, pool)
|
||||
p1 := createPhoto(t, ctx, pool, user)
|
||||
p2 := createPhoto(t, ctx, pool, user)
|
||||
|
||||
objID := auth.NewID()
|
||||
obj := &Object{ID: objID, StorageKey: "objects/" + objID + "/original.jpg"}
|
||||
t.Cleanup(func() {
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM photo_objects WHERE object_id = $1`, objID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM objects WHERE id = $1`, objID)
|
||||
})
|
||||
if err := st.CreateObject(ctx, obj); err != nil {
|
||||
t.Fatalf("create object: %v", err)
|
||||
}
|
||||
|
||||
if err := st.BindObject(ctx, p1, objID, RoleOriginal); !errors.Is(err, ErrObjectNotReady) {
|
||||
t.Fatalf("bind pending object = %v, want ErrObjectNotReady", err)
|
||||
}
|
||||
if err := st.ActivateObject(ctx, objID); err != nil {
|
||||
t.Fatalf("activate: %v", err)
|
||||
}
|
||||
if err := st.BindObject(ctx, p1, objID, RoleOriginal); err != nil {
|
||||
t.Fatalf("bind p1: %v", err)
|
||||
}
|
||||
if err := st.BindObject(ctx, p2, objID, RoleOriginal); err != nil {
|
||||
t.Fatalf("bind p2: %v", err)
|
||||
}
|
||||
|
||||
if n, err := st.ObjectRefCount(ctx, objID); err != nil || n != 2 {
|
||||
t.Fatalf("ref count = %d err=%v, want 2", n, err)
|
||||
}
|
||||
siblings, err := st.PhotosByOriginalObject(ctx, objID)
|
||||
if err != nil || len(siblings) != 2 {
|
||||
t.Fatalf("siblings = %v err=%v, want 2", siblings, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClaimGarbageRespectsReferences(t *testing.T) {
|
||||
pool := testPool(t)
|
||||
ctx := context.Background()
|
||||
st := NewStore(pool)
|
||||
|
||||
user := createUser(t, ctx, pool)
|
||||
photo := createPhoto(t, ctx, pool, user)
|
||||
|
||||
boundID := auth.NewID()
|
||||
freeID := auth.NewID()
|
||||
t.Cleanup(func() {
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM photo_objects WHERE object_id IN ($1, $2)`, boundID, freeID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM objects WHERE id IN ($1, $2)`, boundID, freeID)
|
||||
})
|
||||
for _, id := range []string{boundID, freeID} {
|
||||
if err := st.CreateObject(ctx, &Object{ID: id, StorageKey: "objects/" + id + "/original.jpg"}); err != nil {
|
||||
t.Fatalf("create object: %v", err)
|
||||
}
|
||||
if err := st.ActivateObject(ctx, id); err != nil {
|
||||
t.Fatalf("activate: %v", err)
|
||||
}
|
||||
}
|
||||
if err := st.BindObject(ctx, photo, boundID, RoleOriginal); err != nil {
|
||||
t.Fatalf("bind: %v", err)
|
||||
}
|
||||
|
||||
claimed, err := st.ClaimGarbage(ctx, 100)
|
||||
if err != nil {
|
||||
t.Fatalf("claim garbage: %v", err)
|
||||
}
|
||||
if !containsObject(claimed, freeID) {
|
||||
t.Fatalf("unreferenced object not claimed: %+v", claimed)
|
||||
}
|
||||
if containsObject(claimed, boundID) {
|
||||
t.Fatalf("referenced object was claimed for deletion")
|
||||
}
|
||||
|
||||
// Removing the last reference makes both collectible.
|
||||
if _, err := pool.Exec(ctx, `DELETE FROM photos WHERE id = $1`, photo); err != nil {
|
||||
t.Fatalf("delete photo: %v", err)
|
||||
}
|
||||
claimed, err = st.ClaimGarbage(ctx, 100)
|
||||
if err != nil {
|
||||
t.Fatalf("claim garbage 2: %v", err)
|
||||
}
|
||||
if !containsObject(claimed, boundID) {
|
||||
t.Fatalf("object with no remaining references not claimed: %+v", claimed)
|
||||
}
|
||||
}
|
||||
|
||||
func objectState(t *testing.T, ctx context.Context, pool *pgxpool.Pool, id string) string {
|
||||
t.Helper()
|
||||
var state string
|
||||
if err := pool.QueryRow(ctx, `SELECT state FROM objects WHERE id = $1`, id).Scan(&state); err != nil {
|
||||
t.Fatalf("read state: %v", err)
|
||||
}
|
||||
return state
|
||||
}
|
||||
|
||||
func containsObject(objs []Object, id string) bool {
|
||||
for _, o := range objs {
|
||||
if o.ID == id {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -2,6 +2,8 @@ package photos
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"image"
|
||||
"strconv"
|
||||
@@ -138,3 +140,9 @@ func buildVariants(data []byte) (thumb, preview []byte, err error) {
|
||||
}
|
||||
return thumbBuf.Bytes(), previewBuf.Bytes(), nil
|
||||
}
|
||||
|
||||
// checksum returns the hex-encoded SHA-256 of the given bytes.
|
||||
func checksum(data []byte) string {
|
||||
sum := sha256.Sum256(data)
|
||||
return hex.EncodeToString(sum[:])
|
||||
}
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
package photos
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
|
||||
"shared-album/internal/models"
|
||||
)
|
||||
|
||||
// Repository is the persistence contract for the photos domain, including the
|
||||
// object ledger, variant queue and garbage collection.
|
||||
type Repository interface {
|
||||
CreateWithOriginal(ctx context.Context, p *models.Photo, obj *Object) error
|
||||
DeleteUpload(ctx context.Context, photoID, objectID string) error
|
||||
GetByID(ctx context.Context, id, viewerID string) (*models.Photo, error)
|
||||
List(ctx context.Context, in ListParams) ([]models.Photo, error)
|
||||
SetDeleted(ctx context.Context, id string, deleted bool) error
|
||||
UpdateAlbum(ctx context.Context, id, albumID string) error
|
||||
Clone(ctx context.Context, src *models.Photo, albumID, uploaderID, id string) error
|
||||
HardDelete(ctx context.Context, id string) error
|
||||
ListTrash(ctx context.Context, userID string) ([]models.Photo, error)
|
||||
TrashCount(ctx context.Context, userID string) (int, error)
|
||||
RestoreToAlbum(ctx context.Context, id, albumID string) error
|
||||
AlbumExists(ctx context.Context, albumID string) (bool, error)
|
||||
ListTags(ctx context.Context, userID, q string) ([]models.Tag, error)
|
||||
EnsureTag(ctx context.Context, userID, name string) (*models.Tag, error)
|
||||
AttachTag(ctx context.Context, photoID, tagID string) error
|
||||
DetachTag(ctx context.Context, photoID, tagID string) error
|
||||
TagOwnedBy(ctx context.Context, tagID, userID string) (bool, error)
|
||||
SetCoverIfEmpty(ctx context.Context, albumID, photoID string)
|
||||
UpdateFilename(ctx context.Context, id, filename string) error
|
||||
BackfillEXIF(ctx context.Context, fetch func(context.Context, string) ([]byte, error)) (int, error)
|
||||
|
||||
CreateObject(ctx context.Context, o *Object) error
|
||||
SetObjectState(ctx context.Context, id, state string) error
|
||||
ActivateObject(ctx context.Context, id string) error
|
||||
MarkObjectDeleting(ctx context.Context, id string) error
|
||||
DeleteObjectRow(ctx context.Context, id string) error
|
||||
BindObject(ctx context.Context, photoID, objectID, role string) error
|
||||
PhotoObjectIDs(ctx context.Context, photoID string) ([]string, error)
|
||||
ObjectRefCount(ctx context.Context, objectID string) (int, error)
|
||||
OriginalObject(ctx context.Context, photoID string) (*Object, error)
|
||||
PhotosByOriginalObject(ctx context.Context, originalObjectID string) ([]string, error)
|
||||
ClaimGarbage(ctx context.Context, limit int) ([]Object, error)
|
||||
ClaimStalePending(ctx context.Context, olderThan time.Duration, limit int) ([]Object, error)
|
||||
ClaimVariantJobs(ctx context.Context, limit int, lease time.Duration) ([]string, error)
|
||||
CompleteVariantJob(ctx context.Context, photoID string) error
|
||||
FailVariantJob(ctx context.Context, photoID, errMsg string, maxAttempts int) error
|
||||
|
||||
// AllStorageKeys returns every key registered in the object journal.
|
||||
AllStorageKeys(ctx context.Context) ([]string, error)
|
||||
}
|
||||
@@ -4,7 +4,6 @@ import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
@@ -23,13 +22,13 @@ type AlbumAccess interface {
|
||||
}
|
||||
|
||||
type Service struct {
|
||||
store *Store
|
||||
store Repository
|
||||
albums AlbumAccess
|
||||
storage *storage.Client
|
||||
storage storage.Backend
|
||||
hub *sse.Hub
|
||||
}
|
||||
|
||||
func NewService(store *Store, albums AlbumAccess, storage *storage.Client, hub *sse.Hub) *Service {
|
||||
func NewService(store Repository, albums AlbumAccess, storage storage.Backend, hub *sse.Hub) *Service {
|
||||
return &Service{store: store, albums: albums, storage: storage, hub: hub}
|
||||
}
|
||||
|
||||
@@ -64,22 +63,20 @@ func (s *Service) Upload(ctx context.Context, in UploadInput) (*models.Photo, er
|
||||
}
|
||||
meta := extractEXIF(in.Data)
|
||||
|
||||
id := auth.NewID()
|
||||
ext := strings.ToLower(filepath.Ext(in.Filename))
|
||||
if ext == "" {
|
||||
ext = ".bin"
|
||||
}
|
||||
storageKey := fmt.Sprintf("albums/%s/original/%s%s", in.AlbumID, id, ext)
|
||||
|
||||
if err := s.storage.Put(ctx, storageKey, bytes.NewReader(in.Data), int64(len(in.Data)), mime); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
photoID := auth.NewID()
|
||||
objectID := auth.NewID()
|
||||
key := objectKey(objectID, RoleOriginal, ext)
|
||||
|
||||
p := &models.Photo{
|
||||
ID: id,
|
||||
ID: photoID,
|
||||
AlbumID: in.AlbumID,
|
||||
UploaderID: in.UserID,
|
||||
StorageKey: storageKey,
|
||||
StorageKey: key,
|
||||
OriginalFilename: in.Filename,
|
||||
Size: int64(len(in.Data)),
|
||||
Width: width,
|
||||
@@ -90,12 +87,28 @@ func (s *Service) Upload(ctx context.Context, in UploadInput) (*models.Photo, er
|
||||
Longitude: meta.Longitude,
|
||||
EXIF: meta.Attributes,
|
||||
}
|
||||
if err := s.store.Create(ctx, p); err != nil {
|
||||
return nil, err
|
||||
obj := &Object{
|
||||
ID: objectID,
|
||||
StorageKey: key,
|
||||
MimeType: mime,
|
||||
Size: int64(len(in.Data)),
|
||||
Width: width,
|
||||
Height: height,
|
||||
Checksum: checksum(in.Data),
|
||||
}
|
||||
|
||||
data := in.Data
|
||||
go s.processVariants(p.ID, in.AlbumID, data)
|
||||
// The metadata layer is the ledger of record: the row is written first,
|
||||
// then the bytes are stored, then the object is activated.
|
||||
if err := s.store.CreateWithOriginal(ctx, p, obj); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := s.storage.Put(ctx, key, bytes.NewReader(in.Data), int64(len(in.Data)), mime); err != nil {
|
||||
_ = s.store.DeleteUpload(ctx, photoID, objectID)
|
||||
return nil, err
|
||||
}
|
||||
if err := s.store.ActivateObject(ctx, objectID); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if err := s.attachURLs(ctx, p); err != nil {
|
||||
return nil, err
|
||||
@@ -103,36 +116,9 @@ func (s *Service) Upload(ctx context.Context, in UploadInput) (*models.Photo, er
|
||||
return p, nil
|
||||
}
|
||||
|
||||
func (s *Service) processVariants(photoID, albumID string, data []byte) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
|
||||
defer cancel()
|
||||
|
||||
thumb, preview, err := buildVariants(data)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
thumbKey := fmt.Sprintf("albums/%s/thumb/%s.webp", albumID, photoID)
|
||||
previewKey := fmt.Sprintf("albums/%s/preview/%s.webp", albumID, photoID)
|
||||
if err := s.storage.Put(ctx, thumbKey, bytes.NewReader(thumb), int64(len(thumb)), "image/webp"); err != nil {
|
||||
return
|
||||
}
|
||||
if err := s.storage.Put(ctx, previewKey, bytes.NewReader(preview), int64(len(preview)), "image/webp"); err != nil {
|
||||
return
|
||||
}
|
||||
if err := s.store.SetVariants(ctx, photoID, thumbKey, previewKey); err != nil {
|
||||
return
|
||||
}
|
||||
s.store.SetCoverIfEmpty(ctx, albumID, photoID)
|
||||
if p, err := s.store.GetByID(ctx, photoID, ""); err == nil {
|
||||
if err := s.attachURLs(ctx, p); err == nil {
|
||||
s.hub.Publish(albumID, "photo.created", p)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Service) attachURLs(ctx context.Context, p *models.Photo) error {
|
||||
if p.StorageKey != "" {
|
||||
url, err := s.storage.PresignGet(ctx, p.StorageKey)
|
||||
url, err := s.storage.URL(ctx, p.StorageKey)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -140,12 +126,12 @@ func (s *Service) attachURLs(ctx context.Context, p *models.Photo) error {
|
||||
p.DownloadURL = url
|
||||
}
|
||||
if p.ThumbKey != "" {
|
||||
if url, err := s.storage.PresignGet(ctx, p.ThumbKey); err == nil {
|
||||
if url, err := s.storage.URL(ctx, p.ThumbKey); err == nil {
|
||||
p.ThumbURL = url
|
||||
}
|
||||
}
|
||||
if p.PreviewKey != "" {
|
||||
if url, err := s.storage.PresignGet(ctx, p.PreviewKey); err == nil {
|
||||
if url, err := s.storage.URL(ctx, p.PreviewKey); err == nil {
|
||||
p.PreviewURL = url
|
||||
}
|
||||
}
|
||||
@@ -216,13 +202,13 @@ func (s *Service) GetPublic(ctx context.Context, photoID string) (*models.Photo,
|
||||
return p, nil
|
||||
}
|
||||
|
||||
// DownloadPublic returns a presigned original URL for a share link.
|
||||
// DownloadPublic returns a signed original URL for a share link.
|
||||
func (s *Service) DownloadPublic(ctx context.Context, photoID string) (string, error) {
|
||||
p, err := s.GetPublic(ctx, photoID)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return s.storage.PresignGet(ctx, p.StorageKey)
|
||||
return s.storage.URL(ctx, p.StorageKey)
|
||||
}
|
||||
|
||||
func (s *Service) Get(ctx context.Context, photoID, userID string) (*models.Photo, error) {
|
||||
@@ -250,7 +236,7 @@ func (s *Service) Download(ctx context.Context, photoID, userID string) (string,
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
url, err := s.storage.PresignGet(ctx, p.StorageKey)
|
||||
url, err := s.storage.URL(ctx, p.StorageKey)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
@@ -416,21 +402,22 @@ func (s *Service) PermanentDelete(ctx context.Context, photoID, userID string) e
|
||||
if !canDeletePhoto(role, p.UploaderID == userID) {
|
||||
return httpx.Forbidden("forbidden", "you cannot delete this photo")
|
||||
}
|
||||
objectIDs, err := s.store.PhotoObjectIDs(ctx, photoID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := s.store.HardDelete(ctx, photoID); err != nil {
|
||||
return err
|
||||
}
|
||||
if p.AlbumID != "" {
|
||||
s.publishDeleted(p)
|
||||
}
|
||||
if n, err := s.store.StorageKeyRefCount(ctx, p.StorageKey); err == nil && n == 0 {
|
||||
keys := []string{p.StorageKey}
|
||||
if p.ThumbKey != "" {
|
||||
keys = append(keys, p.ThumbKey)
|
||||
// Mark objects that no longer have any binding for garbage collection.
|
||||
// The background worker removes the blobs from storage.
|
||||
for _, oid := range objectIDs {
|
||||
if n, err := s.store.ObjectRefCount(ctx, oid); err == nil && n == 0 {
|
||||
_ = s.store.MarkObjectDeleting(ctx, oid)
|
||||
}
|
||||
if p.PreviewKey != "" {
|
||||
keys = append(keys, p.PreviewKey)
|
||||
}
|
||||
_ = s.storage.Delete(ctx, keys...)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
package photos
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"shared-album/internal/sqliteutil"
|
||||
)
|
||||
|
||||
func nullIfEmpty(s string) any {
|
||||
if s == "" {
|
||||
return nil
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
func nullTime(t *time.Time) any {
|
||||
if t == nil {
|
||||
return nil
|
||||
}
|
||||
return sqliteutil.FormatTime(*t)
|
||||
}
|
||||
|
||||
func nullFloat(f *float64) any {
|
||||
if f == nil {
|
||||
return nil
|
||||
}
|
||||
return *f
|
||||
}
|
||||
|
||||
func nowUTC() time.Time { return time.Now().UTC() }
|
||||
@@ -0,0 +1,365 @@
|
||||
package photos
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
"time"
|
||||
|
||||
"shared-album/internal/sqliteutil"
|
||||
)
|
||||
|
||||
// CreateObject inserts a new object row in the pending state.
|
||||
func (s *SQLiteStore) CreateObject(ctx context.Context, o *Object) error {
|
||||
if o.State == "" {
|
||||
o.State = ObjectPending
|
||||
}
|
||||
var createdAt string
|
||||
err := s.db.QueryRowContext(ctx,
|
||||
`INSERT INTO objects (id, storage_key, mime_type, size, width, height, checksum, state)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
|
||||
RETURNING created_at`,
|
||||
o.ID, o.StorageKey, o.MimeType, o.Size, o.Width, o.Height, o.Checksum, o.State).Scan(&createdAt)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
o.CreatedAt = sqliteutil.ParseTime(createdAt)
|
||||
return nil
|
||||
}
|
||||
|
||||
// SetObjectState updates an object's lifecycle state.
|
||||
func (s *SQLiteStore) SetObjectState(ctx context.Context, id, state string) error {
|
||||
res, err := s.db.ExecContext(ctx, `UPDATE objects SET state = $2 WHERE id = $1`, id, state)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if n, _ := res.RowsAffected(); n == 0 {
|
||||
return ErrObjectNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// ActivateObject marks an object ready after its bytes have been stored.
|
||||
func (s *SQLiteStore) ActivateObject(ctx context.Context, id string) error {
|
||||
return s.SetObjectState(ctx, id, ObjectReady)
|
||||
}
|
||||
|
||||
// MarkObjectDeleting marks an object as pending removal from storage.
|
||||
func (s *SQLiteStore) MarkObjectDeleting(ctx context.Context, id string) error {
|
||||
return s.SetObjectState(ctx, id, ObjectDeleting)
|
||||
}
|
||||
|
||||
// DeleteObjectRow permanently removes an object row (after its blob is gone).
|
||||
func (s *SQLiteStore) DeleteObjectRow(ctx context.Context, id string) error {
|
||||
res, err := s.db.ExecContext(ctx, `DELETE FROM objects WHERE id = $1`, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if n, _ := res.RowsAffected(); n == 0 {
|
||||
return ErrObjectNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// BindObject attaches a ready object to a photo under the given role. The write
|
||||
// lock of the immediate transaction serializes it against garbage collection.
|
||||
func (s *SQLiteStore) BindObject(ctx context.Context, photoID, objectID, role string) error {
|
||||
tx, err := s.db.BeginTx(ctx, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer func() { _ = tx.Rollback() }()
|
||||
|
||||
var state string
|
||||
if err := tx.QueryRowContext(ctx, `SELECT state FROM objects WHERE id = $1`, objectID).Scan(&state); err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return ErrObjectNotFound
|
||||
}
|
||||
return err
|
||||
}
|
||||
if state != ObjectReady {
|
||||
return ErrObjectNotReady
|
||||
}
|
||||
if _, err := tx.ExecContext(ctx,
|
||||
`INSERT INTO photo_objects (photo_id, object_id, role) VALUES ($1, $2, $3)
|
||||
ON CONFLICT (photo_id, role) DO UPDATE SET object_id = excluded.object_id`,
|
||||
photoID, objectID, role); err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Commit()
|
||||
}
|
||||
|
||||
// PhotoObjectIDs returns the object ids currently bound to a photo.
|
||||
func (s *SQLiteStore) PhotoObjectIDs(ctx context.Context, photoID string) ([]string, error) {
|
||||
rows, err := s.db.QueryContext(ctx, `SELECT object_id FROM photo_objects WHERE photo_id = $1`, photoID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var ids []string
|
||||
for rows.Next() {
|
||||
var id string
|
||||
if err := rows.Scan(&id); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
ids = append(ids, id)
|
||||
}
|
||||
return ids, rows.Err()
|
||||
}
|
||||
|
||||
// ObjectRefCount counts how many photo bindings reference an object.
|
||||
func (s *SQLiteStore) ObjectRefCount(ctx context.Context, objectID string) (int, error) {
|
||||
var n int
|
||||
err := s.db.QueryRowContext(ctx, `SELECT count(*) FROM photo_objects WHERE object_id = $1`, objectID).Scan(&n)
|
||||
return n, err
|
||||
}
|
||||
|
||||
// OriginalObject returns the original object bound to a photo.
|
||||
func (s *SQLiteStore) OriginalObject(ctx context.Context, photoID string) (*Object, error) {
|
||||
var o Object
|
||||
var createdAt string
|
||||
err := s.db.QueryRowContext(ctx,
|
||||
`SELECT o.id, o.storage_key, o.mime_type, o.size, o.width, o.height, o.checksum, o.state, o.created_at
|
||||
FROM photo_objects po JOIN objects o ON o.id = po.object_id
|
||||
WHERE po.photo_id = $1 AND po.role = $2`, photoID, RoleOriginal).
|
||||
Scan(&o.ID, &o.StorageKey, &o.MimeType, &o.Size, &o.Width, &o.Height, &o.Checksum, &o.State, &createdAt)
|
||||
if err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return nil, ErrObjectNotFound
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
o.CreatedAt = sqliteutil.ParseTime(createdAt)
|
||||
return &o, nil
|
||||
}
|
||||
|
||||
// PhotosByOriginalObject returns the ids of all photos bound to the given
|
||||
// original object, so generated variants can be shared between copies.
|
||||
func (s *SQLiteStore) PhotosByOriginalObject(ctx context.Context, originalObjectID string) ([]string, error) {
|
||||
rows, err := s.db.QueryContext(ctx,
|
||||
`SELECT photo_id FROM photo_objects WHERE object_id = $1 AND role = $2`, originalObjectID, RoleOriginal)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var ids []string
|
||||
for rows.Next() {
|
||||
var id string
|
||||
if err := rows.Scan(&id); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
ids = append(ids, id)
|
||||
}
|
||||
return ids, rows.Err()
|
||||
}
|
||||
|
||||
// ClaimGarbage claims collectible objects: ready objects with no bindings and
|
||||
// objects already marked deleting.
|
||||
func (s *SQLiteStore) ClaimGarbage(ctx context.Context, limit int) ([]Object, error) {
|
||||
if limit <= 0 {
|
||||
limit = 10
|
||||
}
|
||||
tx, err := s.db.BeginTx(ctx, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer func() { _ = tx.Rollback() }()
|
||||
|
||||
rows, err := tx.QueryContext(ctx,
|
||||
`SELECT o.id, o.storage_key FROM objects o
|
||||
WHERE o.state IN ('ready', 'deleting')
|
||||
AND NOT EXISTS (SELECT 1 FROM photo_objects po WHERE po.object_id = o.id)
|
||||
ORDER BY o.created_at
|
||||
LIMIT $1`, limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
objs, err := scanObjectKeys(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(objs) == 0 {
|
||||
return nil, tx.Commit()
|
||||
}
|
||||
if err := markDeleting(ctx, tx, objs); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := tx.Commit(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return objs, nil
|
||||
}
|
||||
|
||||
// ClaimStalePending marks abandoned pending objects as deleting and returns them.
|
||||
func (s *SQLiteStore) ClaimStalePending(ctx context.Context, olderThan time.Duration, limit int) ([]Object, error) {
|
||||
if limit <= 0 {
|
||||
limit = 10
|
||||
}
|
||||
threshold := sqliteutil.FormatTime(time.Now().UTC().Add(-olderThan))
|
||||
tx, err := s.db.BeginTx(ctx, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer func() { _ = tx.Rollback() }()
|
||||
|
||||
rows, err := tx.QueryContext(ctx,
|
||||
`SELECT id, storage_key FROM objects
|
||||
WHERE state = 'pending' AND created_at < $1
|
||||
ORDER BY created_at
|
||||
LIMIT $2`, threshold, limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
objs, err := scanObjectKeys(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(objs) == 0 {
|
||||
return nil, tx.Commit()
|
||||
}
|
||||
if err := markDeleting(ctx, tx, objs); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := tx.Commit(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return objs, nil
|
||||
}
|
||||
|
||||
// ClaimVariantJobs leases due variant jobs and returns their photo ids.
|
||||
func (s *SQLiteStore) ClaimVariantJobs(ctx context.Context, limit int, lease time.Duration) ([]string, error) {
|
||||
if limit <= 0 {
|
||||
limit = 10
|
||||
}
|
||||
now := sqliteutil.FormatTime(time.Now().UTC())
|
||||
next := sqliteutil.FormatTime(time.Now().UTC().Add(lease))
|
||||
tx, err := s.db.BeginTx(ctx, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer func() { _ = tx.Rollback() }()
|
||||
|
||||
rows, err := tx.QueryContext(ctx,
|
||||
`SELECT photo_id FROM variant_jobs
|
||||
WHERE next_attempt_at <= $1
|
||||
ORDER BY next_attempt_at
|
||||
LIMIT $2`, now, limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var ids []string
|
||||
for rows.Next() {
|
||||
var id string
|
||||
if err := rows.Scan(&id); err != nil {
|
||||
rows.Close()
|
||||
return nil, err
|
||||
}
|
||||
ids = append(ids, id)
|
||||
}
|
||||
rows.Close()
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(ids) == 0 {
|
||||
return nil, tx.Commit()
|
||||
}
|
||||
args := make([]any, 0, len(ids)+1)
|
||||
args = append(args, next)
|
||||
for _, id := range ids {
|
||||
args = append(args, id)
|
||||
}
|
||||
if _, err := tx.ExecContext(ctx,
|
||||
`UPDATE variant_jobs SET next_attempt_at = $1 WHERE photo_id IN (`+sqliteutil.Placeholders(2, len(ids))+`)`,
|
||||
args...); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := tx.Commit(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return ids, nil
|
||||
}
|
||||
|
||||
// CompleteVariantJob removes a finished job.
|
||||
func (s *SQLiteStore) CompleteVariantJob(ctx context.Context, photoID string) error {
|
||||
_, err := s.db.ExecContext(ctx, `DELETE FROM variant_jobs WHERE photo_id = $1`, photoID)
|
||||
return err
|
||||
}
|
||||
|
||||
// FailVariantJob records a failure and schedules a retry with exponential
|
||||
// backoff, abandoning the job after maxAttempts.
|
||||
func (s *SQLiteStore) FailVariantJob(ctx context.Context, photoID, errMsg string, maxAttempts int) error {
|
||||
tx, err := s.db.BeginTx(ctx, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer func() { _ = tx.Rollback() }()
|
||||
|
||||
var attempts int
|
||||
err = tx.QueryRowContext(ctx, `SELECT attempts FROM variant_jobs WHERE photo_id = $1`, photoID).Scan(&attempts)
|
||||
if err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return tx.Commit()
|
||||
}
|
||||
return err
|
||||
}
|
||||
newAttempts := attempts + 1
|
||||
if maxAttempts > 0 && newAttempts >= maxAttempts {
|
||||
if _, err := tx.ExecContext(ctx, `DELETE FROM variant_jobs WHERE photo_id = $1`, photoID); err != nil {
|
||||
return err
|
||||
}
|
||||
} else {
|
||||
secs := 10 * (1 << attempts)
|
||||
if secs > 300 {
|
||||
secs = 300
|
||||
}
|
||||
next := sqliteutil.FormatTime(time.Now().UTC().Add(time.Duration(secs) * time.Second))
|
||||
if _, err := tx.ExecContext(ctx,
|
||||
`UPDATE variant_jobs SET attempts = $2, next_attempt_at = $3, last_error = $4 WHERE photo_id = $1`,
|
||||
photoID, newAttempts, next, errMsg); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return tx.Commit()
|
||||
}
|
||||
|
||||
func scanObjectKeys(rows *sql.Rows) ([]Object, error) {
|
||||
var objs []Object
|
||||
for rows.Next() {
|
||||
var o Object
|
||||
if err := rows.Scan(&o.ID, &o.StorageKey); err != nil {
|
||||
rows.Close()
|
||||
return nil, err
|
||||
}
|
||||
objs = append(objs, o)
|
||||
}
|
||||
rows.Close()
|
||||
return objs, rows.Err()
|
||||
}
|
||||
|
||||
func markDeleting(ctx context.Context, tx *sql.Tx, objs []Object) error {
|
||||
args := make([]any, len(objs))
|
||||
for i, o := range objs {
|
||||
args[i] = o.ID
|
||||
}
|
||||
_, err := tx.ExecContext(ctx,
|
||||
`UPDATE objects SET state = 'deleting' WHERE id IN (`+sqliteutil.Placeholders(1, len(objs))+`)`, args...)
|
||||
return err
|
||||
}
|
||||
|
||||
// AllStorageKeys returns every key registered in the object journal.
|
||||
func (s *SQLiteStore) AllStorageKeys(ctx context.Context) ([]string, error) {
|
||||
rows, err := s.db.QueryContext(ctx, `SELECT storage_key FROM objects`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var keys []string
|
||||
for rows.Next() {
|
||||
var k string
|
||||
if err := rows.Scan(&k); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
keys = append(keys, k)
|
||||
}
|
||||
return keys, rows.Err()
|
||||
}
|
||||
@@ -0,0 +1,470 @@
|
||||
package photos
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"strings"
|
||||
|
||||
"shared-album/internal/auth"
|
||||
"shared-album/internal/models"
|
||||
"shared-album/internal/sqliteutil"
|
||||
)
|
||||
|
||||
// SQLiteStore is the SQLite implementation of Repository.
|
||||
type SQLiteStore struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func NewSQLiteStore(db *sql.DB) *SQLiteStore { return &SQLiteStore{db: db} }
|
||||
|
||||
const sqlitePhotoColumns = `p.id, COALESCE(p.album_id, ''), p.uploader_id,
|
||||
COALESCE(o_o.storage_key, ''),
|
||||
COALESCE(o_t.storage_key, ''),
|
||||
COALESCE(o_p.storage_key, ''),
|
||||
p.original_filename, COALESCE(o_o.size, 0), COALESCE(o_o.width, 0), COALESCE(o_o.height, 0), COALESCE(o_o.mime_type, ''),
|
||||
p.taken_at, p.latitude, p.longitude, COALESCE(p.caption, ''), p.created_at,
|
||||
u.id, u.name, COALESCE(u.avatar_url, ''),
|
||||
(SELECT count(*) FROM likes l WHERE l.photo_id = p.id),
|
||||
(SELECT count(*) FROM comments c WHERE c.photo_id = p.id),
|
||||
EXISTS(SELECT 1 FROM likes l WHERE l.photo_id = p.id AND l.user_id = $2),
|
||||
COALESCE(p.exif, '{}'), p.deleted_at,
|
||||
COALESCE((SELECT json_group_array(json_object('id', t.id, 'name', t.name))
|
||||
FROM (SELECT t.id, t.name FROM photo_tags pt JOIN tags t ON t.id = pt.tag_id
|
||||
WHERE pt.photo_id = p.id AND t.user_id = $2
|
||||
ORDER BY lower(t.name)) t), '[]')`
|
||||
|
||||
const sqlitePhotoJoins = `
|
||||
LEFT JOIN photo_objects po_o ON po_o.photo_id = p.id AND po_o.role = 'original'
|
||||
LEFT JOIN objects o_o ON o_o.id = po_o.object_id
|
||||
LEFT JOIN photo_objects po_t ON po_t.photo_id = p.id AND po_t.role = 'thumb'
|
||||
LEFT JOIN objects o_t ON o_t.id = po_t.object_id
|
||||
LEFT JOIN photo_objects po_p ON po_p.photo_id = p.id AND po_p.role = 'preview'
|
||||
LEFT JOIN objects o_p ON o_p.id = po_p.object_id`
|
||||
|
||||
func scanSQLitePhoto(scan func(...any) error) (*models.Photo, error) {
|
||||
var p models.Photo
|
||||
var exifText, tagsText, createdAt string
|
||||
var takenAt, deletedAt sql.NullString
|
||||
var lat, long sql.NullFloat64
|
||||
var liked int
|
||||
if err := scan(&p.ID, &p.AlbumID, &p.UploaderID, &p.StorageKey,
|
||||
&p.ThumbKey, &p.PreviewKey,
|
||||
&p.OriginalFilename, &p.Size, &p.Width, &p.Height, &p.MimeType,
|
||||
&takenAt, &lat, &long, &p.Caption, &createdAt,
|
||||
&p.Uploader.ID, &p.Uploader.Name, &p.Uploader.AvatarURL,
|
||||
&p.LikesCount, &p.CommentsCount, &liked, &exifText, &deletedAt, &tagsText); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
p.TakenAt = sqliteutil.ParseNullTime(takenAt)
|
||||
if lat.Valid {
|
||||
p.Latitude = &lat.Float64
|
||||
}
|
||||
if long.Valid {
|
||||
p.Longitude = &long.Float64
|
||||
}
|
||||
p.CreatedAt = sqliteutil.ParseTime(createdAt)
|
||||
p.DeletedAt = sqliteutil.ParseNullTime(deletedAt)
|
||||
p.LikedByMe = liked != 0
|
||||
setExif(&p, exifText)
|
||||
setTags(&p, tagsText)
|
||||
return &p, nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) CreateWithOriginal(ctx context.Context, p *models.Photo, obj *Object) error {
|
||||
exifJSON, err := json.Marshal(p.EXIF)
|
||||
if err != nil || len(exifJSON) == 0 {
|
||||
exifJSON = []byte("{}")
|
||||
}
|
||||
tx, err := s.db.BeginTx(ctx, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer func() { _ = tx.Rollback() }()
|
||||
|
||||
if _, err := tx.ExecContext(ctx,
|
||||
`INSERT INTO objects (id, storage_key, mime_type, size, width, height, checksum, state)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)`,
|
||||
obj.ID, obj.StorageKey, obj.MimeType, obj.Size, obj.Width, obj.Height, obj.Checksum, ObjectPending); err != nil {
|
||||
return err
|
||||
}
|
||||
var createdAt string
|
||||
if err := tx.QueryRowContext(ctx,
|
||||
`INSERT INTO photos (id, album_id, uploader_id, original_filename, taken_at, latitude, longitude, caption, exif)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)
|
||||
RETURNING created_at`,
|
||||
p.ID, nullIfEmpty(p.AlbumID), p.UploaderID, p.OriginalFilename,
|
||||
nullTime(p.TakenAt), nullFloat(p.Latitude), nullFloat(p.Longitude), p.Caption, string(exifJSON),
|
||||
).Scan(&createdAt); err != nil {
|
||||
return err
|
||||
}
|
||||
p.CreatedAt = sqliteutil.ParseTime(createdAt)
|
||||
if _, err := tx.ExecContext(ctx,
|
||||
`INSERT INTO photo_objects (photo_id, object_id, role) VALUES ($1, $2, $3)`,
|
||||
p.ID, obj.ID, RoleOriginal); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := tx.ExecContext(ctx,
|
||||
`INSERT INTO variant_jobs (photo_id) VALUES ($1) ON CONFLICT (photo_id) DO NOTHING`, p.ID); err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Commit()
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) DeleteUpload(ctx context.Context, photoID, objectID string) error {
|
||||
tx, err := s.db.BeginTx(ctx, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer func() { _ = tx.Rollback() }()
|
||||
if _, err := tx.ExecContext(ctx, `DELETE FROM photos WHERE id = $1`, photoID); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := tx.ExecContext(ctx, `DELETE FROM objects WHERE id = $1`, objectID); err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Commit()
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) GetByID(ctx context.Context, id, viewerID string) (*models.Photo, error) {
|
||||
row := s.db.QueryRowContext(ctx,
|
||||
`SELECT `+sqlitePhotoColumns+`
|
||||
FROM photos p JOIN users u ON u.id = p.uploader_id`+sqlitePhotoJoins+`
|
||||
WHERE p.id = $1`, id, viewerID)
|
||||
p, err := scanSQLitePhoto(row.Scan)
|
||||
if err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return nil, ErrNotFound
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
return p, nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) List(ctx context.Context, in ListParams) ([]models.Photo, error) {
|
||||
if in.Limit <= 0 || in.Limit > 100 {
|
||||
in.Limit = 50
|
||||
}
|
||||
var since any
|
||||
if in.Since != nil {
|
||||
since = sqliteutil.FormatTime(*in.Since)
|
||||
}
|
||||
rows, err := s.db.QueryContext(ctx,
|
||||
`SELECT `+sqlitePhotoColumns+`
|
||||
FROM photos p JOIN users u ON u.id = p.uploader_id`+sqlitePhotoJoins+`
|
||||
WHERE p.album_id = $1
|
||||
AND p.deleted_at IS NULL
|
||||
AND ($3 = '' OR (p.created_at, p.id) < (SELECT created_at, id FROM photos WHERE id = $3))
|
||||
AND ($4 IS NULL OR p.created_at > $4)
|
||||
ORDER BY p.created_at DESC, p.id DESC
|
||||
LIMIT $5`,
|
||||
in.AlbumID, in.Viewer, in.Cursor, since, in.Limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
out := []models.Photo{}
|
||||
for rows.Next() {
|
||||
p, err := scanSQLitePhoto(rows.Scan)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, *p)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) SetDeleted(ctx context.Context, id string, deleted bool) error {
|
||||
var res sql.Result
|
||||
var err error
|
||||
if deleted {
|
||||
res, err = s.db.ExecContext(ctx,
|
||||
`UPDATE photos SET deleted_at = $2 WHERE id = $1 AND deleted_at IS NULL`,
|
||||
id, sqliteutil.FormatTime(nowUTC()))
|
||||
} else {
|
||||
res, err = s.db.ExecContext(ctx, `UPDATE photos SET deleted_at = NULL WHERE id = $1`, id)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if n, _ := res.RowsAffected(); n == 0 {
|
||||
return ErrNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) UpdateAlbum(ctx context.Context, id, albumID string) error {
|
||||
res, err := s.db.ExecContext(ctx,
|
||||
`UPDATE photos SET album_id = $2 WHERE id = $1 AND deleted_at IS NULL`, id, albumID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if n, _ := res.RowsAffected(); n == 0 {
|
||||
return ErrNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) Clone(ctx context.Context, src *models.Photo, albumID, uploaderID, id string) error {
|
||||
exifJSON, err := json.Marshal(src.EXIF)
|
||||
if err != nil || len(exifJSON) == 0 {
|
||||
exifJSON = []byte("{}")
|
||||
}
|
||||
tx, err := s.db.BeginTx(ctx, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer func() { _ = tx.Rollback() }()
|
||||
|
||||
if _, err := tx.ExecContext(ctx,
|
||||
`INSERT INTO photos (id, album_id, uploader_id, original_filename, taken_at, latitude, longitude, caption, exif)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)`,
|
||||
id, nullIfEmpty(albumID), uploaderID, src.OriginalFilename,
|
||||
nullTime(src.TakenAt), nullFloat(src.Latitude), nullFloat(src.Longitude), src.Caption, string(exifJSON)); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := tx.ExecContext(ctx,
|
||||
`INSERT INTO photo_objects (photo_id, object_id, role)
|
||||
SELECT $1, object_id, role FROM photo_objects WHERE photo_id = $2`,
|
||||
id, src.ID); err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Commit()
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) HardDelete(ctx context.Context, id string) error {
|
||||
res, err := s.db.ExecContext(ctx, `DELETE FROM photos WHERE id = $1`, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if n, _ := res.RowsAffected(); n == 0 {
|
||||
return ErrNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) ListTrash(ctx context.Context, userID string) ([]models.Photo, error) {
|
||||
rows, err := s.db.QueryContext(ctx,
|
||||
`SELECT `+sqlitePhotoColumns+`, COALESCE(a.title, '')
|
||||
FROM photos p
|
||||
JOIN users u ON u.id = p.uploader_id
|
||||
LEFT JOIN albums a ON a.id = p.album_id`+sqlitePhotoJoins+`
|
||||
WHERE p.deleted_at IS NOT NULL
|
||||
AND (
|
||||
(a.id IS NOT NULL AND (a.owner_id = $1 OR EXISTS(
|
||||
SELECT 1 FROM album_members m WHERE m.album_id = a.id AND m.user_id = $1)))
|
||||
OR (p.album_id IS NULL AND p.uploader_id = $1)
|
||||
)
|
||||
ORDER BY p.deleted_at DESC
|
||||
LIMIT 200`,
|
||||
userID, userID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
out := []models.Photo{}
|
||||
for rows.Next() {
|
||||
var p models.Photo
|
||||
var exifText, tagsText, createdAt, albumTitle string
|
||||
var takenAt, deletedAt sql.NullString
|
||||
var lat, long sql.NullFloat64
|
||||
var liked int
|
||||
if err := rows.Scan(&p.ID, &p.AlbumID, &p.UploaderID, &p.StorageKey,
|
||||
&p.ThumbKey, &p.PreviewKey,
|
||||
&p.OriginalFilename, &p.Size, &p.Width, &p.Height, &p.MimeType,
|
||||
&takenAt, &lat, &long, &p.Caption, &createdAt,
|
||||
&p.Uploader.ID, &p.Uploader.Name, &p.Uploader.AvatarURL,
|
||||
&p.LikesCount, &p.CommentsCount, &liked, &exifText, &deletedAt, &albumTitle, &tagsText); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
p.TakenAt = sqliteutil.ParseNullTime(takenAt)
|
||||
if lat.Valid {
|
||||
p.Latitude = &lat.Float64
|
||||
}
|
||||
if long.Valid {
|
||||
p.Longitude = &long.Float64
|
||||
}
|
||||
p.CreatedAt = sqliteutil.ParseTime(createdAt)
|
||||
p.DeletedAt = sqliteutil.ParseNullTime(deletedAt)
|
||||
p.AlbumTitle = albumTitle
|
||||
p.LikedByMe = liked != 0
|
||||
setExif(&p, exifText)
|
||||
setTags(&p, tagsText)
|
||||
out = append(out, p)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) TrashCount(ctx context.Context, userID string) (int, error) {
|
||||
var n int
|
||||
err := s.db.QueryRowContext(ctx,
|
||||
`SELECT count(*) FROM photos p
|
||||
LEFT JOIN albums a ON a.id = p.album_id
|
||||
WHERE p.deleted_at IS NOT NULL
|
||||
AND (
|
||||
(a.id IS NOT NULL AND (a.owner_id = $1 OR EXISTS(
|
||||
SELECT 1 FROM album_members m WHERE m.album_id = a.id AND m.user_id = $1)))
|
||||
OR (p.album_id IS NULL AND p.uploader_id = $1)
|
||||
)`, userID).Scan(&n)
|
||||
return n, err
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) RestoreToAlbum(ctx context.Context, id, albumID string) error {
|
||||
res, err := s.db.ExecContext(ctx,
|
||||
`UPDATE photos SET album_id = $2, deleted_at = NULL WHERE id = $1`, id, albumID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if n, _ := res.RowsAffected(); n == 0 {
|
||||
return ErrNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) AlbumExists(ctx context.Context, albumID string) (bool, error) {
|
||||
var ok int
|
||||
err := s.db.QueryRowContext(ctx, `SELECT EXISTS(SELECT 1 FROM albums WHERE id = $1)`, albumID).Scan(&ok)
|
||||
return ok != 0, err
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) ListTags(ctx context.Context, userID, q string) ([]models.Tag, error) {
|
||||
rows, err := s.db.QueryContext(ctx,
|
||||
`SELECT id, name FROM tags
|
||||
WHERE user_id = $1 AND ($2 = '' OR name LIKE '%' || $2 || '%')
|
||||
ORDER BY lower(name) LIMIT 50`, userID, q)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
out := []models.Tag{}
|
||||
for rows.Next() {
|
||||
var t models.Tag
|
||||
if err := rows.Scan(&t.ID, &t.Name); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, t)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) EnsureTag(ctx context.Context, userID, name string) (*models.Tag, error) {
|
||||
name = strings.TrimSpace(name)
|
||||
if name == "" || len(name) > 64 {
|
||||
return nil, ErrNotFound
|
||||
}
|
||||
var t models.Tag
|
||||
err := s.db.QueryRowContext(ctx,
|
||||
`SELECT id, name FROM tags WHERE user_id = $1 AND lower(name) = lower($2)`, userID, name).
|
||||
Scan(&t.ID, &t.Name)
|
||||
if err == nil {
|
||||
return &t, nil
|
||||
}
|
||||
if !errors.Is(err, sql.ErrNoRows) {
|
||||
return nil, err
|
||||
}
|
||||
id := auth.NewID()
|
||||
err = s.db.QueryRowContext(ctx,
|
||||
`INSERT INTO tags (id, user_id, name) VALUES ($1, $2, $3) RETURNING id, name`, id, userID, name).
|
||||
Scan(&t.ID, &t.Name)
|
||||
if err != nil {
|
||||
if e2 := s.db.QueryRowContext(ctx,
|
||||
`SELECT id, name FROM tags WHERE user_id = $1 AND lower(name) = lower($2)`, userID, name).
|
||||
Scan(&t.ID, &t.Name); e2 == nil {
|
||||
return &t, nil
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
return &t, nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) AttachTag(ctx context.Context, photoID, tagID string) error {
|
||||
_, err := s.db.ExecContext(ctx,
|
||||
`INSERT INTO photo_tags (photo_id, tag_id) VALUES ($1, $2) ON CONFLICT DO NOTHING`, photoID, tagID)
|
||||
return err
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) DetachTag(ctx context.Context, photoID, tagID string) error {
|
||||
_, err := s.db.ExecContext(ctx,
|
||||
`DELETE FROM photo_tags WHERE photo_id = $1 AND tag_id = $2`, photoID, tagID)
|
||||
return err
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) TagOwnedBy(ctx context.Context, tagID, userID string) (bool, error) {
|
||||
var ok int
|
||||
err := s.db.QueryRowContext(ctx,
|
||||
`SELECT EXISTS(SELECT 1 FROM tags WHERE id = $1 AND user_id = $2)`, tagID, userID).Scan(&ok)
|
||||
return ok != 0, err
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) SetCoverIfEmpty(ctx context.Context, albumID, photoID string) {
|
||||
_, _ = s.db.ExecContext(ctx,
|
||||
`UPDATE albums SET cover_photo_id = $2 WHERE id = $1 AND cover_photo_id IS NULL`,
|
||||
albumID, photoID)
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) UpdateFilename(ctx context.Context, id, filename string) error {
|
||||
res, err := s.db.ExecContext(ctx, `UPDATE photos SET original_filename = $2 WHERE id = $1`, id, filename)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if n, _ := res.RowsAffected(); n == 0 {
|
||||
return ErrNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// BackfillEXIF mirrors the PostgreSQL implementation, reading the original blob
|
||||
// of every photo with empty exif.
|
||||
func (s *SQLiteStore) BackfillEXIF(ctx context.Context, fetch func(context.Context, string) ([]byte, error)) (int, error) {
|
||||
rows, err := s.db.QueryContext(ctx,
|
||||
`SELECT p.id, o.storage_key
|
||||
FROM photos p
|
||||
JOIN photo_objects po ON po.photo_id = p.id AND po.role = 'original'
|
||||
JOIN objects o ON o.id = po.object_id
|
||||
WHERE p.exif IS NULL OR p.exif = '{}'
|
||||
ORDER BY p.created_at`)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
var pending []backfillRow
|
||||
for rows.Next() {
|
||||
var r backfillRow
|
||||
if err := rows.Scan(&r.ID, &r.StorageKey); err != nil {
|
||||
rows.Close()
|
||||
return 0, err
|
||||
}
|
||||
pending = append(pending, r)
|
||||
}
|
||||
rows.Close()
|
||||
if err := rows.Err(); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
updated := 0
|
||||
for _, r := range pending {
|
||||
data, err := fetch(ctx, r.StorageKey)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
meta := extractEXIF(data)
|
||||
if len(meta.Attributes) == 0 && meta.TakenAt == nil && meta.Latitude == nil && meta.Longitude == nil {
|
||||
continue
|
||||
}
|
||||
attrs, err := json.Marshal(meta.Attributes)
|
||||
if err != nil || len(attrs) == 0 {
|
||||
attrs = []byte("{}")
|
||||
}
|
||||
if _, err := s.db.ExecContext(ctx,
|
||||
`UPDATE photos SET exif = $2,
|
||||
taken_at = COALESCE(taken_at, $3),
|
||||
latitude = COALESCE(latitude, $4),
|
||||
longitude = COALESCE(longitude, $5)
|
||||
WHERE id = $1`,
|
||||
r.ID, string(attrs), nullTime(meta.TakenAt), nullFloat(meta.Latitude), nullFloat(meta.Longitude)); err != nil {
|
||||
continue
|
||||
}
|
||||
updated++
|
||||
}
|
||||
return updated, nil
|
||||
}
|
||||
@@ -23,9 +23,14 @@ type Store struct {
|
||||
|
||||
func NewStore(pool *pgxpool.Pool) *Store { return &Store{pool: pool} }
|
||||
|
||||
const photoColumns = `p.id, COALESCE(p.album_id, ''), p.uploader_id, p.storage_key,
|
||||
COALESCE(p.thumb_key, ''), COALESCE(p.preview_key, ''),
|
||||
p.original_filename, p.size, p.width, p.height, p.mime_type,
|
||||
// photoColumns selects photo rows together with the keys and physical facts of
|
||||
// their bound objects (original, thumb, preview) and the original's size,
|
||||
// dimensions and mime type.
|
||||
const photoColumns = `p.id, COALESCE(p.album_id, ''), p.uploader_id,
|
||||
COALESCE(o_o.storage_key, ''),
|
||||
COALESCE(o_t.storage_key, ''),
|
||||
COALESCE(o_p.storage_key, ''),
|
||||
p.original_filename, COALESCE(o_o.size, 0), COALESCE(o_o.width, 0), COALESCE(o_o.height, 0), COALESCE(o_o.mime_type, ''),
|
||||
p.taken_at, p.latitude, p.longitude, COALESCE(p.caption, ''), p.created_at,
|
||||
u.id, u.name, COALESCE(u.avatar_url, ''),
|
||||
(SELECT count(*) FROM likes l WHERE l.photo_id = p.id),
|
||||
@@ -36,6 +41,14 @@ const photoColumns = `p.id, COALESCE(p.album_id, ''), p.uploader_id, p.storage_k
|
||||
FROM photo_tags pt JOIN tags t ON t.id = pt.tag_id
|
||||
WHERE pt.photo_id = p.id AND t.user_id = $2), '[]')::text`
|
||||
|
||||
const photoJoins = `
|
||||
LEFT JOIN photo_objects po_o ON po_o.photo_id = p.id AND po_o.role = 'original'
|
||||
LEFT JOIN objects o_o ON o_o.id = po_o.object_id
|
||||
LEFT JOIN photo_objects po_t ON po_t.photo_id = p.id AND po_t.role = 'thumb'
|
||||
LEFT JOIN objects o_t ON o_t.id = po_t.object_id
|
||||
LEFT JOIN photo_objects po_p ON po_p.photo_id = p.id AND po_p.role = 'preview'
|
||||
LEFT JOIN objects o_p ON o_p.id = po_p.object_id`
|
||||
|
||||
func scanPhoto(row pgx.Row) (*models.Photo, error) {
|
||||
var p models.Photo
|
||||
var exifText, tagsText string
|
||||
@@ -76,31 +89,68 @@ func setExif(p *models.Photo, raw string) {
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Store) Create(ctx context.Context, p *models.Photo) error {
|
||||
// CreateWithOriginal records a new photo together with its pending original
|
||||
// object, the original binding and a variant job — all in one transaction. The
|
||||
// caller stores the bytes and then activates the object. The metadata layer is
|
||||
// the ledger of record; the storage layer follows.
|
||||
func (s *Store) CreateWithOriginal(ctx context.Context, p *models.Photo, obj *Object) error {
|
||||
exifJSON, err := json.Marshal(p.EXIF)
|
||||
if err != nil || len(exifJSON) == 0 {
|
||||
exifJSON = []byte("{}")
|
||||
}
|
||||
return s.pool.QueryRow(ctx,
|
||||
`INSERT INTO photos (id, album_id, uploader_id, storage_key, original_filename,
|
||||
size, width, height, mime_type, taken_at, latitude, longitude, caption, exif)
|
||||
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14::jsonb)
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer func() { _ = tx.Rollback(ctx) }()
|
||||
|
||||
if _, err := tx.Exec(ctx,
|
||||
`INSERT INTO objects (id, storage_key, mime_type, size, width, height, checksum, state)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)`,
|
||||
obj.ID, obj.StorageKey, obj.MimeType, obj.Size, obj.Width, obj.Height, obj.Checksum, ObjectPending); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := tx.QueryRow(ctx,
|
||||
`INSERT INTO photos (id, album_id, uploader_id, original_filename, taken_at, latitude, longitude, caption, exif)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9::jsonb)
|
||||
RETURNING created_at`,
|
||||
p.ID, p.AlbumID, p.UploaderID, p.StorageKey, p.OriginalFilename,
|
||||
p.Size, p.Width, p.Height, p.MimeType, p.TakenAt, p.Latitude, p.Longitude, p.Caption, string(exifJSON),
|
||||
).Scan(&p.CreatedAt)
|
||||
p.ID, p.AlbumID, p.UploaderID, p.OriginalFilename, p.TakenAt, p.Latitude, p.Longitude, p.Caption, string(exifJSON),
|
||||
).Scan(&p.CreatedAt); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := tx.Exec(ctx,
|
||||
`INSERT INTO photo_objects (photo_id, object_id, role) VALUES ($1, $2, $3)`,
|
||||
p.ID, obj.ID, RoleOriginal); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := tx.Exec(ctx,
|
||||
`INSERT INTO variant_jobs (photo_id) VALUES ($1) ON CONFLICT (photo_id) DO NOTHING`, p.ID); err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Commit(ctx)
|
||||
}
|
||||
|
||||
func (s *Store) SetVariants(ctx context.Context, id, thumbKey, previewKey string) error {
|
||||
_, err := s.pool.Exec(ctx,
|
||||
`UPDATE photos SET thumb_key = $2, preview_key = $3 WHERE id = $1`, id, thumbKey, previewKey)
|
||||
// DeleteUpload rolls back a failed upload: the photo (and its bindings and job)
|
||||
// plus the pending object row are removed.
|
||||
func (s *Store) DeleteUpload(ctx context.Context, photoID, objectID string) error {
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer func() { _ = tx.Rollback(ctx) }()
|
||||
if _, err := tx.Exec(ctx, `DELETE FROM photos WHERE id = $1`, photoID); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `DELETE FROM objects WHERE id = $1`, objectID); err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Commit(ctx)
|
||||
}
|
||||
|
||||
func (s *Store) GetByID(ctx context.Context, id, viewerID string) (*models.Photo, error) {
|
||||
return scanPhoto(s.pool.QueryRow(ctx,
|
||||
`SELECT `+photoColumns+`
|
||||
FROM photos p JOIN users u ON u.id = p.uploader_id
|
||||
FROM photos p JOIN users u ON u.id = p.uploader_id`+photoJoins+`
|
||||
WHERE p.id = $1`, id, viewerID))
|
||||
}
|
||||
|
||||
@@ -118,7 +168,7 @@ func (s *Store) List(ctx context.Context, in ListParams) ([]models.Photo, error)
|
||||
}
|
||||
rows, err := s.pool.Query(ctx,
|
||||
`SELECT `+photoColumns+`
|
||||
FROM photos p JOIN users u ON u.id = p.uploader_id
|
||||
FROM photos p JOIN users u ON u.id = p.uploader_id`+photoJoins+`
|
||||
WHERE p.album_id = $1
|
||||
AND p.deleted_at IS NULL
|
||||
AND ($3 = '' OR (p.created_at, p.id) < (SELECT created_at, id FROM photos WHERE id = $3))
|
||||
@@ -181,30 +231,36 @@ func (s *Store) UpdateAlbum(ctx context.Context, id, albumID string) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// Clone creates a new photo row that references the same stored objects as src.
|
||||
// Clone creates a new photo row in albumID that references the same objects as
|
||||
// src. No storage operation is performed: the clone shares the physical files.
|
||||
func (s *Store) Clone(ctx context.Context, src *models.Photo, albumID, uploaderID, id string) error {
|
||||
exifJSON, err := json.Marshal(src.EXIF)
|
||||
if err != nil || len(exifJSON) == 0 {
|
||||
exifJSON = []byte("{}")
|
||||
}
|
||||
_, err = s.pool.Exec(ctx,
|
||||
`INSERT INTO photos (id, album_id, uploader_id, storage_key, thumb_key, preview_key,
|
||||
original_filename, size, width, height, mime_type, taken_at, latitude, longitude, caption, exif)
|
||||
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16::jsonb)`,
|
||||
id, albumID, uploaderID, src.StorageKey, nullIfEmpty(src.ThumbKey), nullIfEmpty(src.PreviewKey),
|
||||
src.OriginalFilename, src.Size, src.Width, src.Height, src.MimeType,
|
||||
src.TakenAt, src.Latitude, src.Longitude, src.Caption, string(exifJSON))
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer func() { _ = tx.Rollback(ctx) }()
|
||||
|
||||
// StorageKeyRefCount counts photo rows (any state) referencing the stored object.
|
||||
func (s *Store) StorageKeyRefCount(ctx context.Context, storageKey string) (int, error) {
|
||||
var n int
|
||||
err := s.pool.QueryRow(ctx, `SELECT count(*) FROM photos WHERE storage_key = $1`, storageKey).Scan(&n)
|
||||
return n, err
|
||||
if _, err := tx.Exec(ctx,
|
||||
`INSERT INTO photos (id, album_id, uploader_id, original_filename, taken_at, latitude, longitude, caption, exif)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9::jsonb)`,
|
||||
id, albumID, uploaderID, src.OriginalFilename, src.TakenAt, src.Latitude, src.Longitude, src.Caption, string(exifJSON)); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := tx.Exec(ctx,
|
||||
`INSERT INTO photo_objects (photo_id, object_id, role)
|
||||
SELECT $1, object_id, role FROM photo_objects WHERE photo_id = $2`,
|
||||
id, src.ID); err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Commit(ctx)
|
||||
}
|
||||
|
||||
// HardDelete permanently removes the photo row.
|
||||
// HardDelete permanently removes the photo row. Bound objects are not touched
|
||||
// here; the caller marks orphaned objects for collection.
|
||||
func (s *Store) HardDelete(ctx context.Context, id string) error {
|
||||
tag, err := s.pool.Exec(ctx, `DELETE FROM photos WHERE id = $1`, id)
|
||||
if err != nil {
|
||||
@@ -223,7 +279,7 @@ func (s *Store) ListTrash(ctx context.Context, userID string) ([]models.Photo, e
|
||||
`SELECT `+photoColumns+`, COALESCE(a.title, '')
|
||||
FROM photos p
|
||||
JOIN users u ON u.id = p.uploader_id
|
||||
LEFT JOIN albums a ON a.id = p.album_id
|
||||
LEFT JOIN albums a ON a.id = p.album_id`+photoJoins+`
|
||||
WHERE p.deleted_at IS NOT NULL
|
||||
AND (
|
||||
(a.id IS NOT NULL AND (a.owner_id = $1 OR EXISTS(
|
||||
@@ -362,13 +418,6 @@ func (s *Store) TagOwnedBy(ctx context.Context, tagID, userID string) (bool, err
|
||||
return ok, err
|
||||
}
|
||||
|
||||
func nullIfEmpty(s string) *string {
|
||||
if s == "" {
|
||||
return nil
|
||||
}
|
||||
return &s
|
||||
}
|
||||
|
||||
func (s *Store) SetCoverIfEmpty(ctx context.Context, albumID, photoID string) {
|
||||
_, _ = s.pool.Exec(ctx,
|
||||
`UPDATE albums SET cover_photo_id = $2 WHERE id = $1 AND cover_photo_id IS NULL`,
|
||||
@@ -391,11 +440,16 @@ type backfillRow struct {
|
||||
StorageKey string
|
||||
}
|
||||
|
||||
// BackfillEXIF re-reads the original object of every photo with empty exif and
|
||||
// fills the exif/geo fields on the photo.
|
||||
func (s *Store) BackfillEXIF(ctx context.Context, fetch func(context.Context, string) ([]byte, error)) (int, error) {
|
||||
rows, err := s.pool.Query(ctx,
|
||||
`SELECT id, storage_key FROM photos
|
||||
WHERE exif IS NULL OR exif = '{}'::jsonb
|
||||
ORDER BY created_at`)
|
||||
`SELECT p.id, o.storage_key
|
||||
FROM photos p
|
||||
JOIN photo_objects po ON po.photo_id = p.id AND po.role = 'original'
|
||||
JOIN objects o ON o.id = po.object_id
|
||||
WHERE p.exif IS NULL OR p.exif = '{}'::jsonb
|
||||
ORDER BY p.created_at`)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
package photos
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestUploadCreatesLedgerAndAlbumFreeKey(t *testing.T) {
|
||||
pool := testPool(t)
|
||||
ctx := context.Background()
|
||||
svc, cl := newTestService(t, pool)
|
||||
user := createUser(t, ctx, pool)
|
||||
album := createAlbum(t, ctx, pool, user)
|
||||
|
||||
p, err := svc.Upload(ctx, UploadInput{
|
||||
AlbumID: album,
|
||||
UserID: user,
|
||||
Filename: "photo.png",
|
||||
MimeType: "image/png",
|
||||
Data: makePNG(t),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("upload: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { cleanupPhoto(t, ctx, pool, cl, p.ID) })
|
||||
|
||||
if !strings.HasPrefix(p.StorageKey, "objects/") {
|
||||
t.Fatalf("storage key %q is not an object key", p.StorageKey)
|
||||
}
|
||||
if strings.Contains(p.StorageKey, album) {
|
||||
t.Fatalf("storage key leaks album id: %q", p.StorageKey)
|
||||
}
|
||||
if p.OriginalURL == "" {
|
||||
t.Fatalf("original url not set")
|
||||
}
|
||||
if p.ThumbURL != "" || p.PreviewURL != "" {
|
||||
t.Fatalf("variants should be pending, got thumb=%q preview=%q", p.ThumbURL, p.PreviewURL)
|
||||
}
|
||||
|
||||
orig, err := svc.store.OriginalObject(ctx, p.ID)
|
||||
if err != nil {
|
||||
t.Fatalf("original object: %v", err)
|
||||
}
|
||||
if orig.State != ObjectReady {
|
||||
t.Fatalf("original object state = %q, want ready", orig.State)
|
||||
}
|
||||
if n, err := svc.store.ObjectRefCount(ctx, orig.ID); err != nil || n != 1 {
|
||||
t.Fatalf("original ref count = %d err=%v, want 1", n, err)
|
||||
}
|
||||
|
||||
var jobs int
|
||||
if err := pool.QueryRow(ctx, `SELECT count(*) FROM variant_jobs WHERE photo_id = $1`, p.ID).Scan(&jobs); err != nil {
|
||||
t.Fatalf("count jobs: %v", err)
|
||||
}
|
||||
if jobs != 1 {
|
||||
t.Fatalf("variant jobs = %d, want 1", jobs)
|
||||
}
|
||||
|
||||
list, _, err := svc.List(ctx, album, user, "", nil, 50)
|
||||
if err != nil || len(list) != 1 {
|
||||
t.Fatalf("list: %v len=%d", err, len(list))
|
||||
}
|
||||
if list[0].MimeType != "image/png" || list[0].Width != 4 || list[0].Height != 4 {
|
||||
t.Fatalf("derived fields wrong: mime=%q %dx%d", list[0].MimeType, list[0].Width, list[0].Height)
|
||||
}
|
||||
if list[0].OriginalURL == "" {
|
||||
t.Fatalf("list original url not set")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,210 @@
|
||||
package photos
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"log/slog"
|
||||
"time"
|
||||
|
||||
"shared-album/internal/auth"
|
||||
)
|
||||
|
||||
const maxVariantAttempts = 5
|
||||
|
||||
// Worker is the background reconciler. It drives the variant generation queue
|
||||
// and the object garbage collector entirely from the state recorded in the
|
||||
// metadata layer, so it resumes correctly after a restart.
|
||||
type Worker struct {
|
||||
svc *Service
|
||||
interval time.Duration
|
||||
batch int
|
||||
pendingTTL time.Duration
|
||||
lease time.Duration
|
||||
}
|
||||
|
||||
// NewWorker builds a worker. Parameters are clamped to sane defaults.
|
||||
func NewWorker(svc *Service, interval time.Duration, batch int, pendingTTL time.Duration) *Worker {
|
||||
if interval <= 0 {
|
||||
interval = 5 * time.Second
|
||||
}
|
||||
if batch <= 0 {
|
||||
batch = 10
|
||||
}
|
||||
if pendingTTL <= 0 {
|
||||
pendingTTL = 10 * time.Minute
|
||||
}
|
||||
return &Worker{
|
||||
svc: svc,
|
||||
interval: interval,
|
||||
batch: batch,
|
||||
pendingTTL: pendingTTL,
|
||||
lease: interval * 4,
|
||||
}
|
||||
}
|
||||
|
||||
// Run loops until the context is cancelled.
|
||||
func (w *Worker) Run(ctx context.Context) {
|
||||
slog.Info("background worker started", "interval", w.interval)
|
||||
defer slog.Info("background worker stopped")
|
||||
ticker := time.NewTicker(w.interval)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
w.tick(ctx)
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-ticker.C:
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (w *Worker) tick(ctx context.Context) {
|
||||
w.processVariantJobs(ctx)
|
||||
w.collectGarbage(ctx)
|
||||
w.reapStalePending(ctx)
|
||||
}
|
||||
|
||||
func (w *Worker) processVariantJobs(ctx context.Context) {
|
||||
ids, err := w.svc.store.ClaimVariantJobs(ctx, w.batch, w.lease)
|
||||
if err != nil {
|
||||
slog.Error("claim variant jobs", "error", err)
|
||||
return
|
||||
}
|
||||
for _, id := range ids {
|
||||
w.generateVariants(ctx, id)
|
||||
}
|
||||
}
|
||||
|
||||
func (w *Worker) generateVariants(ctx context.Context, photoID string) {
|
||||
orig, err := w.svc.store.OriginalObject(ctx, photoID)
|
||||
if err != nil {
|
||||
// The photo or its original binding is gone; nothing to do.
|
||||
_ = w.svc.store.CompleteVariantJob(ctx, photoID)
|
||||
return
|
||||
}
|
||||
if orig.State != ObjectReady {
|
||||
w.fail(ctx, photoID, "original object is not ready")
|
||||
return
|
||||
}
|
||||
data, err := w.svc.storage.Get(ctx, orig.StorageKey)
|
||||
if err != nil {
|
||||
w.fail(ctx, photoID, "fetch original: "+err.Error())
|
||||
return
|
||||
}
|
||||
thumb, preview, err := buildVariants(data)
|
||||
if err != nil {
|
||||
w.fail(ctx, photoID, "build variants: "+err.Error())
|
||||
return
|
||||
}
|
||||
siblings, err := w.svc.store.PhotosByOriginalObject(ctx, orig.ID)
|
||||
if err != nil || len(siblings) == 0 {
|
||||
siblings = []string{photoID}
|
||||
}
|
||||
if err := w.createVariant(ctx, siblings, RoleThumb, thumb); err != nil {
|
||||
w.fail(ctx, photoID, "store thumb: "+err.Error())
|
||||
return
|
||||
}
|
||||
if err := w.createVariant(ctx, siblings, RolePreview, preview); err != nil {
|
||||
w.fail(ctx, photoID, "store preview: "+err.Error())
|
||||
return
|
||||
}
|
||||
_ = w.svc.store.CompleteVariantJob(ctx, photoID)
|
||||
for _, pid := range siblings {
|
||||
w.notify(ctx, pid)
|
||||
}
|
||||
}
|
||||
|
||||
// createVariant stores one derived object and binds it to every photo sharing
|
||||
// the original, so copies reuse the same physical variant.
|
||||
func (w *Worker) createVariant(ctx context.Context, photoIDs []string, role string, data []byte) error {
|
||||
objID := auth.NewID()
|
||||
key := objectKey(objID, role, ".webp")
|
||||
obj := &Object{
|
||||
ID: objID,
|
||||
StorageKey: key,
|
||||
MimeType: "image/webp",
|
||||
Size: int64(len(data)),
|
||||
Checksum: checksum(data),
|
||||
}
|
||||
if err := w.svc.store.CreateObject(ctx, obj); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := w.svc.storage.Put(ctx, key, bytes.NewReader(data), int64(len(data)), "image/webp"); err != nil {
|
||||
_ = w.svc.store.DeleteObjectRow(ctx, objID)
|
||||
return err
|
||||
}
|
||||
if err := w.svc.store.ActivateObject(ctx, objID); err != nil {
|
||||
return err
|
||||
}
|
||||
bound := false
|
||||
for _, pid := range photoIDs {
|
||||
if err := w.svc.store.BindObject(ctx, pid, objID, role); err != nil {
|
||||
if errors.Is(err, ErrObjectNotReady) || errors.Is(err, ErrObjectNotFound) {
|
||||
continue
|
||||
}
|
||||
return err
|
||||
}
|
||||
bound = true
|
||||
}
|
||||
if !bound {
|
||||
// No photo could take the variant; let the collector reclaim it.
|
||||
_ = w.svc.store.MarkObjectDeleting(ctx, objID)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (w *Worker) notify(ctx context.Context, photoID string) {
|
||||
p, err := w.svc.store.GetByID(ctx, photoID, "")
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
if err := w.svc.attachURLs(ctx, p); err != nil {
|
||||
return
|
||||
}
|
||||
w.svc.hub.Publish(p.AlbumID, "photo.created", p)
|
||||
}
|
||||
|
||||
func (w *Worker) fail(ctx context.Context, photoID, msg string) {
|
||||
slog.Warn("variant job failed", "photo", photoID, "error", msg)
|
||||
_ = w.svc.store.FailVariantJob(ctx, photoID, msg, maxVariantAttempts)
|
||||
}
|
||||
|
||||
func (w *Worker) collectGarbage(ctx context.Context) {
|
||||
objs, err := w.svc.store.ClaimGarbage(ctx, w.batch)
|
||||
if err != nil {
|
||||
slog.Error("claim garbage", "error", err)
|
||||
return
|
||||
}
|
||||
for _, o := range objs {
|
||||
w.deleteObject(ctx, o)
|
||||
}
|
||||
}
|
||||
|
||||
func (w *Worker) reapStalePending(ctx context.Context) {
|
||||
objs, err := w.svc.store.ClaimStalePending(ctx, w.pendingTTL, w.batch)
|
||||
if err != nil {
|
||||
slog.Error("claim stale pending", "error", err)
|
||||
return
|
||||
}
|
||||
for _, o := range objs {
|
||||
w.deleteObject(ctx, o)
|
||||
}
|
||||
}
|
||||
|
||||
// deleteObject removes an object's blob then its row. If a binding appeared
|
||||
// meanwhile the object is returned to ready; if the storage delete fails the
|
||||
// row stays deleting and is retried on the next pass.
|
||||
func (w *Worker) deleteObject(ctx context.Context, o Object) {
|
||||
if n, err := w.svc.store.ObjectRefCount(ctx, o.ID); err == nil && n > 0 {
|
||||
_ = w.svc.store.SetObjectState(ctx, o.ID, ObjectReady)
|
||||
return
|
||||
}
|
||||
if err := w.svc.storage.Delete(ctx, o.StorageKey); err != nil {
|
||||
slog.Warn("delete object blob", "object", o.ID, "error", err)
|
||||
return
|
||||
}
|
||||
if err := w.svc.store.DeleteObjectRow(ctx, o.ID); err != nil && !errors.Is(err, ErrObjectNotFound) {
|
||||
slog.Warn("delete object row", "object", o.ID, "error", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,92 @@
|
||||
package photos
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
|
||||
"shared-album/internal/storage"
|
||||
)
|
||||
|
||||
func TestWorkerGeneratesVariantsAndCollects(t *testing.T) {
|
||||
pool := testPool(t)
|
||||
svc, cl := newTestService(t, pool)
|
||||
runWorkerFlow(t, pool, svc, cl)
|
||||
}
|
||||
|
||||
func TestWorkerGeneratesVariantsWithFS(t *testing.T) {
|
||||
pool := testPool(t)
|
||||
svc, cl := newTestServiceFS(t, pool)
|
||||
runWorkerFlow(t, pool, svc, cl)
|
||||
}
|
||||
|
||||
func runWorkerFlow(t *testing.T, pool *pgxpool.Pool, svc *Service, cl storage.Backend) {
|
||||
t.Helper()
|
||||
ctx := context.Background()
|
||||
user := createUser(t, ctx, pool)
|
||||
album := createAlbum(t, ctx, pool, user)
|
||||
|
||||
p, err := svc.Upload(ctx, UploadInput{
|
||||
AlbumID: album,
|
||||
UserID: user,
|
||||
Filename: "photo.png",
|
||||
MimeType: "image/png",
|
||||
Data: makePNG(t),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("upload: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { cleanupPhoto(t, ctx, pool, cl, p.ID) })
|
||||
|
||||
w := NewWorker(svc, time.Second, 10, time.Minute)
|
||||
w.processVariantJobs(ctx)
|
||||
|
||||
refreshed, err := svc.store.GetByID(ctx, p.ID, "")
|
||||
if err != nil {
|
||||
t.Fatalf("get refreshed: %v", err)
|
||||
}
|
||||
if refreshed.ThumbKey == "" || refreshed.PreviewKey == "" {
|
||||
t.Fatalf("variants not bound: thumb=%q preview=%q", refreshed.ThumbKey, refreshed.PreviewKey)
|
||||
}
|
||||
if err := svc.attachURLs(ctx, refreshed); err != nil {
|
||||
t.Fatalf("attach urls: %v", err)
|
||||
}
|
||||
if refreshed.ThumbURL == "" || refreshed.PreviewURL == "" {
|
||||
t.Fatalf("variant urls not set")
|
||||
}
|
||||
|
||||
var jobs int
|
||||
if err := pool.QueryRow(ctx, `SELECT count(*) FROM variant_jobs WHERE photo_id = $1`, p.ID).Scan(&jobs); err != nil {
|
||||
t.Fatalf("count jobs: %v", err)
|
||||
}
|
||||
if jobs != 0 {
|
||||
t.Fatalf("variant job not completed: %d left", jobs)
|
||||
}
|
||||
|
||||
objectIDs, err := svc.store.PhotoObjectIDs(ctx, p.ID)
|
||||
if err != nil || len(objectIDs) != 3 {
|
||||
t.Fatalf("expected 3 bound objects, got %v err=%v", objectIDs, err)
|
||||
}
|
||||
|
||||
if err := svc.PermanentDelete(ctx, p.ID, user); err != nil {
|
||||
t.Fatalf("permanent delete: %v", err)
|
||||
}
|
||||
// Run the collector until the objects are gone (bounded to avoid hangs).
|
||||
for attempt := 0; attempt < 100 && countObjects(t, ctx, pool, objectIDs) > 0; attempt++ {
|
||||
w.collectGarbage(ctx)
|
||||
}
|
||||
if n := countObjects(t, ctx, pool, objectIDs); n != 0 {
|
||||
t.Fatalf("%d objects were not collected", n)
|
||||
}
|
||||
}
|
||||
|
||||
func countObjects(t *testing.T, ctx context.Context, pool *pgxpool.Pool, ids []string) int {
|
||||
t.Helper()
|
||||
var n int
|
||||
if err := pool.QueryRow(ctx, `SELECT count(*) FROM objects WHERE id = ANY($1)`, ids).Scan(&n); err != nil {
|
||||
t.Fatalf("count objects: %v", err)
|
||||
}
|
||||
return n
|
||||
}
|
||||
@@ -0,0 +1,74 @@
|
||||
// Package reconcile computes the difference between the object storage and the
|
||||
// metadata journal so orphaned blobs can be reported and, on request, removed.
|
||||
package reconcile
|
||||
|
||||
import (
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"shared-album/internal/storage"
|
||||
)
|
||||
|
||||
// Result holds the two kinds of divergence between storage and the journal.
|
||||
type Result struct {
|
||||
// Orphans are keys present in storage but absent from the journal; they are
|
||||
// candidates for deletion.
|
||||
Orphans []string
|
||||
// Missing are keys present in the journal but absent from storage; they are
|
||||
// reported only and never modified.
|
||||
Missing []string
|
||||
}
|
||||
|
||||
// Classify compares the listed storage objects with the journaled keys.
|
||||
func Classify(objects []storage.ObjectInfo, journaled []string) Result {
|
||||
have := make(map[string]struct{}, len(journaled))
|
||||
for _, k := range journaled {
|
||||
have[k] = struct{}{}
|
||||
}
|
||||
seen := make(map[string]struct{}, len(objects))
|
||||
var orphans []string
|
||||
for _, o := range objects {
|
||||
if _, dup := seen[o.Key]; dup {
|
||||
continue
|
||||
}
|
||||
seen[o.Key] = struct{}{}
|
||||
if _, ok := have[o.Key]; !ok {
|
||||
orphans = append(orphans, o.Key)
|
||||
}
|
||||
}
|
||||
var missing []string
|
||||
for _, k := range journaled {
|
||||
if _, ok := seen[k]; !ok {
|
||||
missing = append(missing, k)
|
||||
}
|
||||
}
|
||||
sort.Strings(orphans)
|
||||
sort.Strings(missing)
|
||||
return Result{Orphans: orphans, Missing: missing}
|
||||
}
|
||||
|
||||
// Deletable selects orphaned keys that are safe to remove: within prefix and
|
||||
// older than minAge (a guard against racing an in-flight upload whose row is
|
||||
// not written yet).
|
||||
func Deletable(objects []storage.ObjectInfo, res Result, prefix string, minAge time.Duration, now time.Time) []string {
|
||||
orphans := make(map[string]struct{}, len(res.Orphans))
|
||||
for _, k := range res.Orphans {
|
||||
orphans[k] = struct{}{}
|
||||
}
|
||||
var out []string
|
||||
for _, o := range objects {
|
||||
if _, ok := orphans[o.Key]; !ok {
|
||||
continue
|
||||
}
|
||||
if prefix != "" && !strings.HasPrefix(o.Key, prefix) {
|
||||
continue
|
||||
}
|
||||
if now.Sub(o.ModTime) < minAge {
|
||||
continue
|
||||
}
|
||||
out = append(out, o.Key)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
package reconcile
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"shared-album/internal/storage"
|
||||
)
|
||||
|
||||
func TestClassify(t *testing.T) {
|
||||
now := time.Now()
|
||||
objects := []storage.ObjectInfo{
|
||||
{Key: "objects/a/original.jpg", ModTime: now},
|
||||
{Key: "objects/b/original.jpg", ModTime: now},
|
||||
}
|
||||
journaled := []string{"objects/b/original.jpg", "objects/c/original.jpg"}
|
||||
|
||||
got := Classify(objects, journaled)
|
||||
if !reflect.DeepEqual(got.Orphans, []string{"objects/a/original.jpg"}) {
|
||||
t.Fatalf("orphans = %v", got.Orphans)
|
||||
}
|
||||
if !reflect.DeepEqual(got.Missing, []string{"objects/c/original.jpg"}) {
|
||||
t.Fatalf("missing = %v", got.Missing)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeletableRespectsAgeAndPrefix(t *testing.T) {
|
||||
now := time.Now()
|
||||
objects := []storage.ObjectInfo{
|
||||
{Key: "objects/old/original.jpg", ModTime: now.Add(-2 * time.Hour)},
|
||||
{Key: "objects/new/original.jpg", ModTime: now.Add(-1 * time.Minute)},
|
||||
{Key: "misc/other.bin", ModTime: now.Add(-2 * time.Hour)},
|
||||
}
|
||||
res := Result{Orphans: []string{"objects/old/original.jpg", "objects/new/original.jpg", "misc/other.bin"}}
|
||||
|
||||
got := Deletable(objects, res, "objects/", time.Hour, now)
|
||||
if !reflect.DeepEqual(got, []string{"objects/old/original.jpg"}) {
|
||||
t.Fatalf("deletable = %v, want only the old in-prefix orphan", got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,560 @@
|
||||
// Package contract holds a backend-agnostic test suite for the metadata
|
||||
// repositories. The same suite runs against PostgreSQL and SQLite so both
|
||||
// implementations are held to identical observable behaviour.
|
||||
package contract
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"shared-album/internal/albums"
|
||||
"shared-album/internal/auth"
|
||||
"shared-album/internal/members"
|
||||
"shared-album/internal/models"
|
||||
"shared-album/internal/notifications"
|
||||
"shared-album/internal/photos"
|
||||
"shared-album/internal/settings"
|
||||
"shared-album/internal/sharelinks"
|
||||
"shared-album/internal/social"
|
||||
)
|
||||
|
||||
// Repos bundles one repository per domain plus a Reset hook that clears all
|
||||
// metadata so each subtest starts from an empty state.
|
||||
type Repos struct {
|
||||
Auth auth.Repository
|
||||
Albums albums.Repository
|
||||
Members members.Repository
|
||||
Photos photos.Repository
|
||||
Social social.Repository
|
||||
Notifications notifications.Repository
|
||||
ShareLinks sharelinks.Repository
|
||||
Settings settings.Repository
|
||||
Reset func(ctx context.Context) error
|
||||
}
|
||||
|
||||
// Run executes the whole contract suite against the given repositories.
|
||||
func Run(t *testing.T, r Repos) {
|
||||
t.Helper()
|
||||
cases := []struct {
|
||||
name string
|
||||
fn func(*testing.T, context.Context, Repos)
|
||||
}{
|
||||
{"Auth", testAuth},
|
||||
{"Albums", testAlbums},
|
||||
{"Members", testMembers},
|
||||
{"Notifications", testNotifications},
|
||||
{"ShareLinks", testShareLinks},
|
||||
{"Social", testSocial},
|
||||
{"Photos", testPhotos},
|
||||
{"Objects", testObjects},
|
||||
{"Settings", testSettings},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
if err := r.Reset(context.Background()); err != nil {
|
||||
t.Fatalf("reset: %v", err)
|
||||
}
|
||||
tc.fn(t, context.Background(), r)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func newID() string { return auth.NewID() }
|
||||
|
||||
func seedUser(t *testing.T, ctx context.Context, r Repos, name string) *models.User {
|
||||
t.Helper()
|
||||
u := &models.User{ID: newID(), Email: newID()[:10] + "@test.local", PasswordHash: "x", Name: name}
|
||||
if err := r.Auth.CreateUser(ctx, u); err != nil {
|
||||
t.Fatalf("create user: %v", err)
|
||||
}
|
||||
return u
|
||||
}
|
||||
|
||||
func seedAlbum(t *testing.T, ctx context.Context, r Repos, ownerID string) *models.Album {
|
||||
t.Helper()
|
||||
a := &models.Album{ID: newID(), OwnerID: ownerID, Title: "Album"}
|
||||
if err := r.Albums.Create(ctx, a); err != nil {
|
||||
t.Fatalf("create album: %v", err)
|
||||
}
|
||||
return a
|
||||
}
|
||||
|
||||
func seedPhoto(t *testing.T, ctx context.Context, r Repos, uploaderID, albumID string) *models.Photo {
|
||||
t.Helper()
|
||||
obj := &photos.Object{ID: newID(), StorageKey: "objects/" + newID(), State: photos.ObjectPending}
|
||||
p := &models.Photo{ID: newID(), AlbumID: albumID, UploaderID: uploaderID, OriginalFilename: "p.jpg"}
|
||||
if err := r.Photos.CreateWithOriginal(ctx, p, obj); err != nil {
|
||||
t.Fatalf("create photo: %v", err)
|
||||
}
|
||||
if err := r.Photos.ActivateObject(ctx, obj.ID); err != nil {
|
||||
t.Fatalf("activate object: %v", err)
|
||||
}
|
||||
return p
|
||||
}
|
||||
|
||||
func testAuth(t *testing.T, ctx context.Context, r Repos) {
|
||||
u := seedUser(t, ctx, r, "Tester")
|
||||
|
||||
byEmail, err := r.Auth.GetUserByEmail(ctx, u.Email)
|
||||
if err != nil || byEmail.ID != u.ID {
|
||||
t.Fatalf("get by email: %v id=%v", err, byEmail)
|
||||
}
|
||||
if _, err := r.Auth.GetUserByID(ctx, u.ID); err != nil {
|
||||
t.Fatalf("get by id: %v", err)
|
||||
}
|
||||
if _, err := r.Auth.GetUserByEmail(ctx, "missing@test.local"); !errors.Is(err, auth.ErrNotFound) {
|
||||
t.Fatalf("missing email err = %v, want ErrNotFound", err)
|
||||
}
|
||||
|
||||
if err := r.Auth.UpdatePreferences(ctx, u.ID, []byte(`{"theme":"dark"}`)); err != nil {
|
||||
t.Fatalf("update preferences: %v", err)
|
||||
}
|
||||
got, _ := r.Auth.GetUserByID(ctx, u.ID)
|
||||
if got.Preferences["theme"] != "dark" {
|
||||
t.Fatalf("preferences not applied: %+v", got.Preferences)
|
||||
}
|
||||
if err := r.Auth.UpdatePreferences(ctx, "missing", []byte(`{}`)); !errors.Is(err, auth.ErrNotFound) {
|
||||
t.Fatalf("update missing prefs err = %v, want ErrNotFound", err)
|
||||
}
|
||||
|
||||
hash := newID()
|
||||
rt := &models.RefreshToken{ID: newID(), UserID: u.ID, TokenHash: hash, ExpiresAt: time.Now().Add(time.Hour)}
|
||||
if err := r.Auth.CreateRefreshToken(ctx, rt); err != nil {
|
||||
t.Fatalf("create refresh: %v", err)
|
||||
}
|
||||
fetched, err := r.Auth.GetRefreshToken(ctx, hash)
|
||||
if err != nil || fetched.ID != rt.ID {
|
||||
t.Fatalf("get refresh: %v id=%v", err, fetched)
|
||||
}
|
||||
if err := r.Auth.RevokeRefreshToken(ctx, rt.ID); err != nil {
|
||||
t.Fatalf("revoke refresh: %v", err)
|
||||
}
|
||||
fetched, _ = r.Auth.GetRefreshToken(ctx, hash)
|
||||
if fetched.RevokedAt == nil {
|
||||
t.Fatalf("refresh not revoked")
|
||||
}
|
||||
if err := r.Auth.RevokeAllUserTokens(ctx, u.ID); err != nil {
|
||||
t.Fatalf("revoke all: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func testAlbums(t *testing.T, ctx context.Context, r Repos) {
|
||||
owner := seedUser(t, ctx, r, "Owner")
|
||||
a := seedAlbum(t, ctx, r, owner.ID)
|
||||
if a.CreatedAt.IsZero() || a.UpdatedAt.IsZero() {
|
||||
t.Fatalf("album timestamps not returned: %+v", a)
|
||||
}
|
||||
|
||||
got, err := r.Albums.GetByID(ctx, a.ID)
|
||||
if err != nil || got.Title != "Album" {
|
||||
t.Fatalf("get: %v %+v", err, got)
|
||||
}
|
||||
if oid, err := r.Albums.OwnerID(ctx, a.ID); err != nil || oid != owner.ID {
|
||||
t.Fatalf("owner id: %v %v", oid, err)
|
||||
}
|
||||
list, err := r.Albums.ListForUser(ctx, owner.ID)
|
||||
if err != nil || len(list) != 1 {
|
||||
t.Fatalf("list: %v len=%d", err, len(list))
|
||||
}
|
||||
|
||||
a.Title = "Renamed"
|
||||
if err := r.Albums.Update(ctx, a); err != nil {
|
||||
t.Fatalf("update: %v", err)
|
||||
}
|
||||
got, _ = r.Albums.GetByID(ctx, a.ID)
|
||||
if got.Title != "Renamed" {
|
||||
t.Fatalf("update not applied: %q", got.Title)
|
||||
}
|
||||
|
||||
if err := r.Albums.Delete(ctx, a.ID); err != nil {
|
||||
t.Fatalf("delete: %v", err)
|
||||
}
|
||||
if _, err := r.Albums.GetByID(ctx, a.ID); !errors.Is(err, albums.ErrNotFound) {
|
||||
t.Fatalf("after delete err = %v, want ErrNotFound", err)
|
||||
}
|
||||
}
|
||||
|
||||
func testMembers(t *testing.T, ctx context.Context, r Repos) {
|
||||
owner := seedUser(t, ctx, r, "Owner")
|
||||
member := seedUser(t, ctx, r, "Member")
|
||||
a := seedAlbum(t, ctx, r, owner.ID)
|
||||
|
||||
if role, _ := r.Members.Role(ctx, a.ID, member.ID); role != "" {
|
||||
t.Fatalf("role before add = %q", role)
|
||||
}
|
||||
if err := r.Members.Add(ctx, a.ID, member.ID, "member", &owner.ID); err != nil {
|
||||
t.Fatalf("add: %v", err)
|
||||
}
|
||||
if role, err := r.Members.Role(ctx, a.ID, member.ID); err != nil || role != "member" {
|
||||
t.Fatalf("role = %q err=%v", role, err)
|
||||
}
|
||||
list, err := r.Members.List(ctx, a.ID)
|
||||
if err != nil || len(list) != 1 || list[0].User.Name != "Member" {
|
||||
t.Fatalf("list: %v %+v", err, list)
|
||||
}
|
||||
if err := r.Members.UpdateRole(ctx, a.ID, member.ID, "editor"); err != nil {
|
||||
t.Fatalf("update role: %v", err)
|
||||
}
|
||||
if role, _ := r.Members.Role(ctx, a.ID, member.ID); role != "editor" {
|
||||
t.Fatalf("role not updated: %q", role)
|
||||
}
|
||||
|
||||
inv := &models.Invite{ID: newID(), AlbumID: a.ID, Token: newID(), Role: "viewer", CreatedBy: owner.ID, ExpiresAt: time.Now().Add(time.Hour), MaxUses: 2}
|
||||
if err := r.Members.CreateInvite(ctx, inv); err != nil {
|
||||
t.Fatalf("create invite: %v", err)
|
||||
}
|
||||
if inv.CreatedAt.IsZero() {
|
||||
t.Fatalf("invite created_at not returned")
|
||||
}
|
||||
got, err := r.Members.GetInviteByToken(ctx, inv.Token)
|
||||
if err != nil || got.Role != "viewer" || got.MaxUses != 2 {
|
||||
t.Fatalf("get invite: %v %+v", err, got)
|
||||
}
|
||||
if err := r.Members.IncrementUses(ctx, inv.ID); err != nil {
|
||||
t.Fatalf("increment: %v", err)
|
||||
}
|
||||
got, _ = r.Members.GetInviteByToken(ctx, inv.Token)
|
||||
if got.Uses != 1 {
|
||||
t.Fatalf("uses = %d, want 1", got.Uses)
|
||||
}
|
||||
if _, err := r.Members.GetInviteByToken(ctx, "missing"); !errors.Is(err, members.ErrNotFound) {
|
||||
t.Fatalf("missing invite err = %v, want ErrNotFound", err)
|
||||
}
|
||||
|
||||
if err := r.Members.Remove(ctx, a.ID, member.ID); err != nil {
|
||||
t.Fatalf("remove: %v", err)
|
||||
}
|
||||
if role, _ := r.Members.Role(ctx, a.ID, member.ID); role != "" {
|
||||
t.Fatalf("member not removed: %q", role)
|
||||
}
|
||||
if err := r.Members.Remove(ctx, a.ID, member.ID); !errors.Is(err, members.ErrNotFound) {
|
||||
t.Fatalf("remove missing err = %v, want ErrNotFound", err)
|
||||
}
|
||||
}
|
||||
|
||||
func testNotifications(t *testing.T, ctx context.Context, r Repos) {
|
||||
u := seedUser(t, ctx, r, "Notify")
|
||||
n := &models.Notification{ID: newID(), UserID: u.ID, Type: "test", PayloadJSON: map[string]any{"k": "v"}}
|
||||
if err := r.Notifications.Create(ctx, n); err != nil {
|
||||
t.Fatalf("create: %v", err)
|
||||
}
|
||||
if n.CreatedAt.IsZero() {
|
||||
t.Fatalf("notification created_at not returned")
|
||||
}
|
||||
list, err := r.Notifications.List(ctx, u.ID, 10)
|
||||
if err != nil || len(list) != 1 || list[0].PayloadJSON["k"] != "v" {
|
||||
t.Fatalf("list: %v %+v", err, list)
|
||||
}
|
||||
if err := r.Notifications.MarkRead(ctx, n.ID, u.ID); err != nil {
|
||||
t.Fatalf("mark read: %v", err)
|
||||
}
|
||||
list, _ = r.Notifications.List(ctx, u.ID, 10)
|
||||
if list[0].ReadAt == nil {
|
||||
t.Fatalf("notification not marked read")
|
||||
}
|
||||
if err := r.Notifications.MarkRead(ctx, newID(), u.ID); !errors.Is(err, notifications.ErrNotFound) {
|
||||
t.Fatalf("mark missing err = %v, want ErrNotFound", err)
|
||||
}
|
||||
}
|
||||
|
||||
func testShareLinks(t *testing.T, ctx context.Context, r Repos) {
|
||||
owner := seedUser(t, ctx, r, "Owner")
|
||||
a := seedAlbum(t, ctx, r, owner.ID)
|
||||
|
||||
l := &models.ShareLink{ID: newID(), AlbumID: a.ID, Token: newID(), CreatedBy: owner.ID, ExpiresAt: time.Now().Add(time.Hour)}
|
||||
if err := r.ShareLinks.Create(ctx, l); err != nil {
|
||||
t.Fatalf("create: %v", err)
|
||||
}
|
||||
if l.CreatedAt.IsZero() {
|
||||
t.Fatalf("share link created_at not returned")
|
||||
}
|
||||
got, err := r.ShareLinks.GetByToken(ctx, l.Token)
|
||||
if err != nil || !got.Active(time.Now()) {
|
||||
t.Fatalf("get token: %v %+v", err, got)
|
||||
}
|
||||
list, err := r.ShareLinks.ListByAlbum(ctx, a.ID)
|
||||
if err != nil || len(list) != 1 {
|
||||
t.Fatalf("list: %v len=%d", err, len(list))
|
||||
}
|
||||
if err := r.ShareLinks.Revoke(ctx, l.ID, a.ID); err != nil {
|
||||
t.Fatalf("revoke: %v", err)
|
||||
}
|
||||
got, _ = r.ShareLinks.GetByToken(ctx, l.Token)
|
||||
if got.Active(time.Now()) {
|
||||
t.Fatalf("link still active after revoke")
|
||||
}
|
||||
if err := r.ShareLinks.Revoke(ctx, l.ID, a.ID); !errors.Is(err, sharelinks.ErrNotFound) {
|
||||
t.Fatalf("revoke again err = %v, want ErrNotFound", err)
|
||||
}
|
||||
if _, err := r.ShareLinks.GetByToken(ctx, "missing"); !errors.Is(err, sharelinks.ErrNotFound) {
|
||||
t.Fatalf("missing token err = %v, want ErrNotFound", err)
|
||||
}
|
||||
}
|
||||
|
||||
func testSocial(t *testing.T, ctx context.Context, r Repos) {
|
||||
owner := seedUser(t, ctx, r, "Owner")
|
||||
a := seedAlbum(t, ctx, r, owner.ID)
|
||||
p := seedPhoto(t, ctx, r, owner.ID, a.ID)
|
||||
|
||||
if err := r.Social.Like(ctx, p.ID, owner.ID); err != nil {
|
||||
t.Fatalf("like: %v", err)
|
||||
}
|
||||
if err := r.Social.Like(ctx, p.ID, owner.ID); err != nil {
|
||||
t.Fatalf("like twice: %v", err)
|
||||
}
|
||||
if err := r.Social.Unlike(ctx, p.ID, owner.ID); err != nil {
|
||||
t.Fatalf("unlike: %v", err)
|
||||
}
|
||||
|
||||
c := &models.Comment{ID: newID(), PhotoID: p.ID, UserID: owner.ID, Text: "hi"}
|
||||
if err := r.Social.CreateComment(ctx, c); err != nil {
|
||||
t.Fatalf("create comment: %v", err)
|
||||
}
|
||||
if c.CreatedAt.IsZero() || c.UpdatedAt.IsZero() {
|
||||
t.Fatalf("comment timestamps not returned: %+v", c)
|
||||
}
|
||||
list, err := r.Social.ListComments(ctx, p.ID)
|
||||
if err != nil || len(list) != 1 || list[0].Text != "hi" {
|
||||
t.Fatalf("list comments: %v %+v", err, list)
|
||||
}
|
||||
got, err := r.Social.GetComment(ctx, c.ID)
|
||||
if err != nil || got.Text != "hi" {
|
||||
t.Fatalf("get comment: %v %+v", err, got)
|
||||
}
|
||||
if err := r.Social.DeleteComment(ctx, c.ID); err != nil {
|
||||
t.Fatalf("delete comment: %v", err)
|
||||
}
|
||||
if _, err := r.Social.GetComment(ctx, c.ID); !errors.Is(err, social.ErrNotFound) {
|
||||
t.Fatalf("get deleted err = %v, want ErrNotFound", err)
|
||||
}
|
||||
if err := r.Social.DeleteComment(ctx, c.ID); !errors.Is(err, social.ErrNotFound) {
|
||||
t.Fatalf("delete again err = %v, want ErrNotFound", err)
|
||||
}
|
||||
}
|
||||
|
||||
func testPhotos(t *testing.T, ctx context.Context, r Repos) {
|
||||
owner := seedUser(t, ctx, r, "Owner")
|
||||
a := seedAlbum(t, ctx, r, owner.ID)
|
||||
p := seedPhoto(t, ctx, r, owner.ID, a.ID)
|
||||
if p.CreatedAt.IsZero() {
|
||||
t.Fatalf("photo created_at not returned")
|
||||
}
|
||||
got, err := r.Photos.GetByID(ctx, p.ID, "")
|
||||
if err != nil || got.StorageKey == "" {
|
||||
t.Fatalf("get photo: %v %+v", err, got)
|
||||
}
|
||||
list, err := r.Photos.List(ctx, photos.ListParams{AlbumID: a.ID, Limit: 10})
|
||||
if err != nil || len(list) != 1 {
|
||||
t.Fatalf("list: %v len=%d", err, len(list))
|
||||
}
|
||||
|
||||
tag, err := r.Photos.EnsureTag(ctx, owner.ID, "Vacation")
|
||||
if err != nil || tag.Name != "Vacation" {
|
||||
t.Fatalf("ensure tag: %v %+v", err, tag)
|
||||
}
|
||||
tag2, err := r.Photos.EnsureTag(ctx, owner.ID, "vacation")
|
||||
if err != nil || tag2.ID != tag.ID {
|
||||
t.Fatalf("ensure tag case-insensitive: %v %+v", err, tag2)
|
||||
}
|
||||
if err := r.Photos.AttachTag(ctx, p.ID, tag.ID); err != nil {
|
||||
t.Fatalf("attach tag: %v", err)
|
||||
}
|
||||
got, _ = r.Photos.GetByID(ctx, p.ID, owner.ID)
|
||||
if len(got.Tags) != 1 || got.Tags[0].Name != "Vacation" {
|
||||
t.Fatalf("photo tags = %+v", got.Tags)
|
||||
}
|
||||
if owned, _ := r.Photos.TagOwnedBy(ctx, tag.ID, owner.ID); !owned {
|
||||
t.Fatalf("tag not owned")
|
||||
}
|
||||
tags, err := r.Photos.ListTags(ctx, owner.ID, "vac")
|
||||
if err != nil || len(tags) != 1 {
|
||||
t.Fatalf("list tags: %v len=%d", err, len(tags))
|
||||
}
|
||||
if err := r.Photos.DetachTag(ctx, p.ID, tag.ID); err != nil {
|
||||
t.Fatalf("detach tag: %v", err)
|
||||
}
|
||||
|
||||
r.Photos.SetCoverIfEmpty(ctx, a.ID, p.ID)
|
||||
al, _ := r.Albums.GetByID(ctx, a.ID)
|
||||
if al.CoverPhotoID == nil || *al.CoverPhotoID != p.ID {
|
||||
t.Fatalf("cover not set: %+v", al.CoverPhotoID)
|
||||
}
|
||||
|
||||
if err := r.Photos.SetDeleted(ctx, p.ID, true); err != nil {
|
||||
t.Fatalf("soft delete: %v", err)
|
||||
}
|
||||
if n, _ := r.Photos.TrashCount(ctx, owner.ID); n != 1 {
|
||||
t.Fatalf("trash count = %d, want 1", n)
|
||||
}
|
||||
trash, err := r.Photos.ListTrash(ctx, owner.ID)
|
||||
if err != nil || len(trash) != 1 {
|
||||
t.Fatalf("trash: %v len=%d", err, len(trash))
|
||||
}
|
||||
if err := r.Photos.RestoreToAlbum(ctx, p.ID, a.ID); err != nil {
|
||||
t.Fatalf("restore: %v", err)
|
||||
}
|
||||
if n, _ := r.Photos.TrashCount(ctx, owner.ID); n != 0 {
|
||||
t.Fatalf("trash count after restore = %d", n)
|
||||
}
|
||||
|
||||
if err := r.Photos.UpdateFilename(ctx, p.ID, "renamed.jpg"); err != nil {
|
||||
t.Fatalf("update filename: %v", err)
|
||||
}
|
||||
got, _ = r.Photos.GetByID(ctx, p.ID, "")
|
||||
if got.OriginalFilename != "renamed.jpg" {
|
||||
t.Fatalf("filename = %q", got.OriginalFilename)
|
||||
}
|
||||
if ok, _ := r.Photos.AlbumExists(ctx, a.ID); !ok {
|
||||
t.Fatalf("album should exist")
|
||||
}
|
||||
|
||||
if err := r.Photos.HardDelete(ctx, p.ID); err != nil {
|
||||
t.Fatalf("hard delete: %v", err)
|
||||
}
|
||||
if _, err := r.Photos.GetByID(ctx, p.ID, ""); !errors.Is(err, photos.ErrNotFound) {
|
||||
t.Fatalf("get deleted err = %v, want ErrNotFound", err)
|
||||
}
|
||||
}
|
||||
|
||||
func testObjects(t *testing.T, ctx context.Context, r Repos) {
|
||||
owner := seedUser(t, ctx, r, "Owner")
|
||||
a := seedAlbum(t, ctx, r, owner.ID)
|
||||
p := seedPhoto(t, ctx, r, owner.ID, a.ID)
|
||||
|
||||
orig, err := r.Photos.OriginalObject(ctx, p.ID)
|
||||
if err != nil || orig.State != photos.ObjectReady {
|
||||
t.Fatalf("original object: %v %+v", err, orig)
|
||||
}
|
||||
keys, err := r.Photos.AllStorageKeys(ctx)
|
||||
if err != nil || !contains(keys, orig.StorageKey) {
|
||||
t.Fatalf("AllStorageKeys = %v err=%v, want %s", keys, err, orig.StorageKey)
|
||||
}
|
||||
if refs, _ := r.Photos.ObjectRefCount(ctx, orig.ID); refs != 1 {
|
||||
t.Fatalf("refs = %d, want 1", refs)
|
||||
}
|
||||
sibs, _ := r.Photos.PhotosByOriginalObject(ctx, orig.ID)
|
||||
if len(sibs) != 1 || sibs[0] != p.ID {
|
||||
t.Fatalf("siblings = %+v", sibs)
|
||||
}
|
||||
|
||||
thumb := &photos.Object{ID: newID(), StorageKey: "objects/" + newID(), MimeType: "image/webp", State: photos.ObjectPending}
|
||||
if err := r.Photos.CreateObject(ctx, thumb); err != nil {
|
||||
t.Fatalf("create object: %v", err)
|
||||
}
|
||||
if err := r.Photos.BindObject(ctx, p.ID, thumb.ID, photos.RoleThumb); !errors.Is(err, photos.ErrObjectNotReady) {
|
||||
t.Fatalf("bind pending err = %v, want ErrObjectNotReady", err)
|
||||
}
|
||||
if err := r.Photos.ActivateObject(ctx, thumb.ID); err != nil {
|
||||
t.Fatalf("activate: %v", err)
|
||||
}
|
||||
if err := r.Photos.BindObject(ctx, p.ID, thumb.ID, photos.RoleThumb); err != nil {
|
||||
t.Fatalf("bind thumb: %v", err)
|
||||
}
|
||||
if refs, _ := r.Photos.ObjectRefCount(ctx, thumb.ID); refs != 1 {
|
||||
t.Fatalf("thumb refs = %d, want 1", refs)
|
||||
}
|
||||
|
||||
// After the photo is gone, its objects have no bindings and are collectible.
|
||||
if err := r.Photos.HardDelete(ctx, p.ID); err != nil {
|
||||
t.Fatalf("hard delete: %v", err)
|
||||
}
|
||||
claimed, err := r.Photos.ClaimGarbage(ctx, 10)
|
||||
if err != nil {
|
||||
t.Fatalf("claim garbage: %v", err)
|
||||
}
|
||||
got := map[string]bool{}
|
||||
for _, o := range claimed {
|
||||
got[o.ID] = true
|
||||
}
|
||||
if !got[orig.ID] || !got[thumb.ID] {
|
||||
t.Fatalf("claimed = %+v, want original and thumb", got)
|
||||
}
|
||||
for _, o := range claimed {
|
||||
if err := r.Photos.DeleteObjectRow(ctx, o.ID); err != nil {
|
||||
t.Fatalf("delete object row: %v", err)
|
||||
}
|
||||
}
|
||||
if again, _ := r.Photos.ClaimGarbage(ctx, 10); len(again) != 0 {
|
||||
t.Fatalf("garbage reclaimed: %+v", again)
|
||||
}
|
||||
|
||||
// Stale pending objects are reaped. A negative age puts the threshold in
|
||||
// the future so the mechanism is exercised without depending on clock skew
|
||||
// between the test process and the database.
|
||||
stale := &photos.Object{ID: newID(), StorageKey: "objects/" + newID(), State: photos.ObjectPending}
|
||||
if err := r.Photos.CreateObject(ctx, stale); err != nil {
|
||||
t.Fatalf("create stale: %v", err)
|
||||
}
|
||||
staleClaimed, err := r.Photos.ClaimStalePending(ctx, -time.Hour, 10)
|
||||
if err != nil || len(staleClaimed) == 0 {
|
||||
t.Fatalf("claim stale: %v %+v", err, staleClaimed)
|
||||
}
|
||||
|
||||
// Variant queue: a fresh photo has one job.
|
||||
p2 := seedPhoto(t, ctx, r, owner.ID, a.ID)
|
||||
jobs, err := r.Photos.ClaimVariantJobs(ctx, 10, time.Minute)
|
||||
if err != nil {
|
||||
t.Fatalf("claim jobs: %v", err)
|
||||
}
|
||||
if !contains(jobs, p2.ID) {
|
||||
t.Fatalf("jobs = %+v, want %s", jobs, p2.ID)
|
||||
}
|
||||
if err := r.Photos.CompleteVariantJob(ctx, p2.ID); err != nil {
|
||||
t.Fatalf("complete job: %v", err)
|
||||
}
|
||||
if jobs, _ = r.Photos.ClaimVariantJobs(ctx, 10, time.Minute); contains(jobs, p2.ID) {
|
||||
t.Fatalf("completed job still claimed: %+v", jobs)
|
||||
}
|
||||
|
||||
// Failure schedules a retry and eventually abandons the job.
|
||||
p3 := seedPhoto(t, ctx, r, owner.ID, a.ID)
|
||||
if err := r.Photos.FailVariantJob(ctx, p3.ID, "boom", 3); err != nil {
|
||||
t.Fatalf("fail job: %v", err)
|
||||
}
|
||||
if jobs, _ = r.Photos.ClaimVariantJobs(ctx, 10, time.Minute); contains(jobs, p3.ID) {
|
||||
t.Fatalf("failed job retried immediately: %+v", jobs)
|
||||
}
|
||||
for i := 0; i < 3; i++ {
|
||||
if err := r.Photos.FailVariantJob(ctx, p3.ID, "boom", 3); err != nil {
|
||||
t.Fatalf("fail job again: %v", err)
|
||||
}
|
||||
}
|
||||
if jobs, _ = r.Photos.ClaimVariantJobs(ctx, 10, time.Minute); contains(jobs, p3.ID) {
|
||||
t.Fatalf("abandoned job still present: %+v", jobs)
|
||||
}
|
||||
}
|
||||
|
||||
func testSettings(t *testing.T, ctx context.Context, r Repos) {
|
||||
v, err := r.Settings.Ensure(ctx, settings.JWTSecretKey, func() (string, error) { return "generated-1", nil })
|
||||
if err != nil || v != "generated-1" {
|
||||
t.Fatalf("ensure first: %v %q", err, v)
|
||||
}
|
||||
v2, err := r.Settings.Ensure(ctx, settings.JWTSecretKey, func() (string, error) { return "generated-2", nil })
|
||||
if err != nil || v2 != "generated-1" {
|
||||
t.Fatalf("ensure not idempotent: %v %q", err, v2)
|
||||
}
|
||||
got, ok, err := r.Settings.Get(ctx, settings.JWTSecretKey)
|
||||
if err != nil || !ok || got != "generated-1" {
|
||||
t.Fatalf("get: %v ok=%v %q", err, ok, got)
|
||||
}
|
||||
if _, ok, _ := r.Settings.Get(ctx, "missing"); ok {
|
||||
t.Fatalf("missing key reported present")
|
||||
}
|
||||
if err := r.Settings.Set(ctx, settings.JWTSecretKey, "rotated"); err != nil {
|
||||
t.Fatalf("set: %v", err)
|
||||
}
|
||||
if got, _, _ := r.Settings.Get(ctx, settings.JWTSecretKey); got != "rotated" {
|
||||
t.Fatalf("set not applied: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func contains(list []string, v string) bool {
|
||||
for _, x := range list {
|
||||
if x == v {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,80 @@
|
||||
package contract
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
|
||||
assets "shared-album"
|
||||
"shared-album/internal/albums"
|
||||
"shared-album/internal/auth"
|
||||
"shared-album/internal/db"
|
||||
"shared-album/internal/members"
|
||||
"shared-album/internal/notifications"
|
||||
"shared-album/internal/photos"
|
||||
"shared-album/internal/settings"
|
||||
"shared-album/internal/sharelinks"
|
||||
"shared-album/internal/social"
|
||||
)
|
||||
|
||||
// TestPostgresContract runs the shared suite against PostgreSQL inside its own
|
||||
// schema, so it never clashes with other packages that use the same database.
|
||||
func TestPostgresContract(t *testing.T) {
|
||||
base := os.Getenv("TEST_DATABASE_URL")
|
||||
if base == "" {
|
||||
t.Skip("TEST_DATABASE_URL not set")
|
||||
}
|
||||
ctx := context.Background()
|
||||
|
||||
admin, err := pgxpool.New(ctx, base)
|
||||
if err != nil {
|
||||
t.Skipf("cannot connect: %v", err)
|
||||
}
|
||||
if err := admin.Ping(ctx); err != nil {
|
||||
t.Skipf("cannot ping: %v", err)
|
||||
}
|
||||
t.Cleanup(admin.Close)
|
||||
|
||||
schema := fmt.Sprintf("ct_%d", time.Now().UnixNano())
|
||||
if _, err := admin.Exec(ctx, "CREATE SCHEMA "+schema); err != nil {
|
||||
t.Fatalf("create schema: %v", err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
_, _ = admin.Exec(context.Background(), "DROP SCHEMA "+schema+" CASCADE")
|
||||
})
|
||||
|
||||
dsn := base + "&options=-c%20search_path%3D" + schema
|
||||
migFS, err := fs.Sub(assets.Migrations, "migrations")
|
||||
if err != nil {
|
||||
t.Fatalf("migrations fs: %v", err)
|
||||
}
|
||||
if err := db.Migrate(dsn, migFS); err != nil {
|
||||
t.Fatalf("migrate: %v", err)
|
||||
}
|
||||
|
||||
pool, err := pgxpool.New(ctx, dsn)
|
||||
if err != nil {
|
||||
t.Fatalf("pool: %v", err)
|
||||
}
|
||||
t.Cleanup(pool.Close)
|
||||
|
||||
Run(t, Repos{
|
||||
Auth: auth.NewStore(pool),
|
||||
Albums: albums.NewStore(pool),
|
||||
Members: members.NewStore(pool),
|
||||
Photos: photos.NewStore(pool),
|
||||
Social: social.NewStore(pool),
|
||||
Notifications: notifications.NewStore(pool),
|
||||
ShareLinks: sharelinks.NewStore(pool),
|
||||
Settings: settings.NewStore(pool),
|
||||
Reset: func(ctx context.Context) error {
|
||||
_, err := pool.Exec(ctx, `TRUNCATE users, objects, app_settings CASCADE`)
|
||||
return err
|
||||
},
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,59 @@
|
||||
package contract
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io/fs"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
assets "shared-album"
|
||||
"shared-album/internal/albums"
|
||||
"shared-album/internal/auth"
|
||||
"shared-album/internal/db"
|
||||
"shared-album/internal/members"
|
||||
"shared-album/internal/notifications"
|
||||
"shared-album/internal/photos"
|
||||
"shared-album/internal/settings"
|
||||
"shared-album/internal/sharelinks"
|
||||
"shared-album/internal/social"
|
||||
)
|
||||
|
||||
func TestSQLiteContract(t *testing.T) {
|
||||
path := filepath.Join(t.TempDir(), "meta.db")
|
||||
migFS, err := fs.Sub(assets.Migrations, "migrations/sqlite")
|
||||
if err != nil {
|
||||
t.Fatalf("migrations fs: %v", err)
|
||||
}
|
||||
if err := db.MigrateSQLite(path, migFS); err != nil {
|
||||
t.Fatalf("migrate: %v", err)
|
||||
}
|
||||
database, err := db.OpenSQLite(path)
|
||||
if err != nil {
|
||||
t.Fatalf("open: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { database.Close() })
|
||||
|
||||
Run(t, Repos{
|
||||
Auth: auth.NewSQLiteStore(database),
|
||||
Albums: albums.NewSQLiteStore(database),
|
||||
Members: members.NewSQLiteStore(database),
|
||||
Photos: photos.NewSQLiteStore(database),
|
||||
Social: social.NewSQLiteStore(database),
|
||||
Notifications: notifications.NewSQLiteStore(database),
|
||||
ShareLinks: sharelinks.NewSQLiteStore(database),
|
||||
Settings: settings.NewSQLiteStore(database),
|
||||
Reset: func(ctx context.Context) error {
|
||||
tables := []string{
|
||||
"variant_jobs", "photo_objects", "objects", "photo_tags", "tags",
|
||||
"refresh_tokens", "share_links", "notifications", "likes", "comments",
|
||||
"invites", "album_members", "photos", "albums", "users", "app_settings",
|
||||
}
|
||||
for _, tbl := range tables {
|
||||
if _, err := database.ExecContext(ctx, "DELETE FROM "+tbl); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
},
|
||||
})
|
||||
}
|
||||
@@ -29,6 +29,7 @@ func testPool(t *testing.T) *pgxpool.Pool {
|
||||
if err := pool.Ping(context.Background()); err != nil {
|
||||
t.Skipf("cannot ping: %v", err)
|
||||
}
|
||||
t.Cleanup(pool.Close)
|
||||
return pool
|
||||
}
|
||||
|
||||
@@ -48,7 +49,6 @@ func newUser(t *testing.T, ctx context.Context, pool *pgxpool.Pool, name string)
|
||||
|
||||
func TestUserRepository(t *testing.T) {
|
||||
pool := testPool(t)
|
||||
defer pool.Close()
|
||||
ctx := context.Background()
|
||||
store := auth.NewStore(pool)
|
||||
|
||||
@@ -82,7 +82,6 @@ func TestUserRepository(t *testing.T) {
|
||||
|
||||
func TestAlbumRepository(t *testing.T) {
|
||||
pool := testPool(t)
|
||||
defer pool.Close()
|
||||
ctx := context.Background()
|
||||
store := albums.NewStore(pool)
|
||||
|
||||
@@ -126,7 +125,6 @@ func TestAlbumRepository(t *testing.T) {
|
||||
|
||||
func TestAlbumCoverRepository(t *testing.T) {
|
||||
pool := testPool(t)
|
||||
defer pool.Close()
|
||||
ctx := context.Background()
|
||||
store := albums.NewStore(pool)
|
||||
|
||||
@@ -138,18 +136,33 @@ func TestAlbumCoverRepository(t *testing.T) {
|
||||
t.Fatalf("create album: %v", err)
|
||||
}
|
||||
|
||||
insertPhoto := func(id, thumbKey string) {
|
||||
insertPhoto := func(id string) string {
|
||||
t.Helper()
|
||||
origObj := auth.NewID()
|
||||
thumbObj := auth.NewID()
|
||||
thumbKey := "objects/" + thumbObj + "/thumb.webp"
|
||||
if _, err := pool.Exec(ctx,
|
||||
`INSERT INTO photos (id, album_id, uploader_id, storage_key, thumb_key)
|
||||
VALUES ($1, $2, $3, $4, $5)`,
|
||||
id, a.ID, owner.ID, "albums/"+a.ID+"/original/"+id+".jpg", thumbKey); err != nil {
|
||||
`INSERT INTO objects (id, storage_key, mime_type, state) VALUES
|
||||
($1, $2, 'image/jpeg', 'ready'),
|
||||
($3, $4, 'image/webp', 'ready')`,
|
||||
origObj, "objects/"+origObj+"/original.jpg", thumbObj, thumbKey); err != nil {
|
||||
t.Fatalf("insert objects: %v", err)
|
||||
}
|
||||
if _, err := pool.Exec(ctx,
|
||||
`INSERT INTO photos (id, album_id, uploader_id) VALUES ($1, $2, $3)`,
|
||||
id, a.ID, owner.ID); err != nil {
|
||||
t.Fatalf("insert photo: %v", err)
|
||||
}
|
||||
if _, err := pool.Exec(ctx,
|
||||
`INSERT INTO photo_objects (photo_id, object_id, role) VALUES ($1, $2, 'original'), ($1, $3, 'thumb')`,
|
||||
id, origObj, thumbObj); err != nil {
|
||||
t.Fatalf("insert bindings: %v", err)
|
||||
}
|
||||
return thumbKey
|
||||
}
|
||||
|
||||
photo1 := auth.NewID()
|
||||
insertPhoto(photo1, "albums/"+a.ID+"/thumb/"+photo1+".webp")
|
||||
thumb1 := insertPhoto(photo1)
|
||||
|
||||
got, err := store.GetByID(ctx, a.ID)
|
||||
if err != nil || got.CoverThumbKey != "" {
|
||||
@@ -162,12 +175,12 @@ func TestAlbumCoverRepository(t *testing.T) {
|
||||
if got.CoverPhotoID == nil || *got.CoverPhotoID != photo1 {
|
||||
t.Fatalf("cover not set: %+v", got.CoverPhotoID)
|
||||
}
|
||||
if want := "albums/" + a.ID + "/thumb/" + photo1 + ".webp"; got.CoverThumbKey != want {
|
||||
t.Fatalf("cover thumb key = %q want %q", got.CoverThumbKey, want)
|
||||
if got.CoverThumbKey != thumb1 {
|
||||
t.Fatalf("cover thumb key = %q want %q", got.CoverThumbKey, thumb1)
|
||||
}
|
||||
|
||||
photo2 := auth.NewID()
|
||||
insertPhoto(photo2, "thumb-2")
|
||||
thumb2 := insertPhoto(photo2)
|
||||
ps.SetCoverIfEmpty(ctx, a.ID, photo2)
|
||||
got, _ = store.GetByID(ctx, a.ID)
|
||||
if got.CoverPhotoID == nil || *got.CoverPhotoID != photo1 {
|
||||
@@ -184,14 +197,13 @@ func TestAlbumCoverRepository(t *testing.T) {
|
||||
|
||||
ps.SetCoverIfEmpty(ctx, a.ID, photo2)
|
||||
list, err := store.ListForUser(ctx, owner.ID)
|
||||
if err != nil || len(list) != 1 || list[0].CoverThumbKey != "thumb-2" {
|
||||
if err != nil || len(list) != 1 || list[0].CoverThumbKey != thumb2 {
|
||||
t.Fatalf("list cover key: %v %+v", err, list)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMemberRepository(t *testing.T) {
|
||||
pool := testPool(t)
|
||||
defer pool.Close()
|
||||
ctx := context.Background()
|
||||
|
||||
owner := newUser(t, ctx, pool, "Owner")
|
||||
@@ -255,7 +267,6 @@ func TestMemberRepository(t *testing.T) {
|
||||
|
||||
func TestShareLinkRepository(t *testing.T) {
|
||||
pool := testPool(t)
|
||||
defer pool.Close()
|
||||
ctx := context.Background()
|
||||
|
||||
owner := newUser(t, ctx, pool, "Owner")
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
package settings
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"encoding/base64"
|
||||
)
|
||||
|
||||
// GenerateSecret returns a new random secret suitable for HMAC signing.
|
||||
func GenerateSecret() (string, error) {
|
||||
buf := make([]byte, 32)
|
||||
if _, err := rand.Read(buf); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return base64.RawURLEncoding.EncodeToString(buf), nil
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
// Package settings persists application-level key/value settings in the
|
||||
// metadata layer. The main use is the token signing secret, generated on first
|
||||
// use and reused afterwards.
|
||||
package settings
|
||||
|
||||
import "context"
|
||||
|
||||
// JWTSecretKey is the settings key holding the token signing secret.
|
||||
const JWTSecretKey = "jwt_secret"
|
||||
|
||||
// Repository reads and writes application settings.
|
||||
type Repository interface {
|
||||
// Get returns the value for key and whether it exists.
|
||||
Get(ctx context.Context, key string) (string, bool, error)
|
||||
// Set stores or replaces the value for key.
|
||||
Set(ctx context.Context, key, value string) error
|
||||
// Ensure returns the existing value for key, or generates and stores one
|
||||
// using generate when absent. Concurrent first use yields one value.
|
||||
Ensure(ctx context.Context, key string, generate func() (string, error)) (string, error)
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
package settings
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"shared-album/internal/sqliteutil"
|
||||
)
|
||||
|
||||
// SQLiteStore is the SQLite implementation of Repository.
|
||||
type SQLiteStore struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func NewSQLiteStore(db *sql.DB) *SQLiteStore { return &SQLiteStore{db: db} }
|
||||
|
||||
func (s *SQLiteStore) Get(ctx context.Context, key string) (string, bool, error) {
|
||||
var v string
|
||||
err := s.db.QueryRowContext(ctx, `SELECT value FROM app_settings WHERE key = $1`, key).Scan(&v)
|
||||
if err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return "", false, nil
|
||||
}
|
||||
return "", false, err
|
||||
}
|
||||
return v, true, nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) Set(ctx context.Context, key, value string) error {
|
||||
_, err := s.db.ExecContext(ctx,
|
||||
`INSERT INTO app_settings (key, value) VALUES ($1, $2)
|
||||
ON CONFLICT (key) DO UPDATE SET value = excluded.value, updated_at = $3`,
|
||||
key, value, sqliteutil.FormatTime(time.Now()))
|
||||
return err
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) Ensure(ctx context.Context, key string, generate func() (string, error)) (string, error) {
|
||||
if v, ok, err := s.Get(ctx, key); err != nil {
|
||||
return "", err
|
||||
} else if ok {
|
||||
return v, nil
|
||||
}
|
||||
v, err := generate()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if _, err := s.db.ExecContext(ctx,
|
||||
`INSERT INTO app_settings (key, value) VALUES ($1, $2) ON CONFLICT (key) DO NOTHING`,
|
||||
key, v); err != nil {
|
||||
return "", err
|
||||
}
|
||||
got, ok, err := s.Get(ctx, key)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if !ok {
|
||||
return "", fmt.Errorf("settings: %q missing after insert", key)
|
||||
}
|
||||
return got, nil
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
package settings
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
|
||||
"github.com/jackc/pgx/v5"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
)
|
||||
|
||||
// Store is the PostgreSQL implementation of Repository.
|
||||
type Store struct {
|
||||
pool *pgxpool.Pool
|
||||
}
|
||||
|
||||
func NewStore(pool *pgxpool.Pool) *Store { return &Store{pool: pool} }
|
||||
|
||||
func (s *Store) Get(ctx context.Context, key string) (string, bool, error) {
|
||||
var v string
|
||||
err := s.pool.QueryRow(ctx, `SELECT value FROM app_settings WHERE key = $1`, key).Scan(&v)
|
||||
if err != nil {
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return "", false, nil
|
||||
}
|
||||
return "", false, err
|
||||
}
|
||||
return v, true, nil
|
||||
}
|
||||
|
||||
func (s *Store) Set(ctx context.Context, key, value string) error {
|
||||
_, err := s.pool.Exec(ctx,
|
||||
`INSERT INTO app_settings (key, value) VALUES ($1, $2)
|
||||
ON CONFLICT (key) DO UPDATE SET value = EXCLUDED.value, updated_at = now()`,
|
||||
key, value)
|
||||
return err
|
||||
}
|
||||
|
||||
func (s *Store) Ensure(ctx context.Context, key string, generate func() (string, error)) (string, error) {
|
||||
if v, ok, err := s.Get(ctx, key); err != nil {
|
||||
return "", err
|
||||
} else if ok {
|
||||
return v, nil
|
||||
}
|
||||
v, err := generate()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if _, err := s.pool.Exec(ctx,
|
||||
`INSERT INTO app_settings (key, value) VALUES ($1, $2) ON CONFLICT (key) DO NOTHING`,
|
||||
key, v); err != nil {
|
||||
return "", err
|
||||
}
|
||||
got, ok, err := s.Get(ctx, key)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if !ok {
|
||||
return "", fmt.Errorf("settings: %q missing after insert", key)
|
||||
}
|
||||
return got, nil
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
package sharelinks
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"shared-album/internal/models"
|
||||
)
|
||||
|
||||
// Repository is the persistence contract for the share-links domain.
|
||||
type Repository interface {
|
||||
Create(ctx context.Context, l *models.ShareLink) error
|
||||
ListByAlbum(ctx context.Context, albumID string) ([]models.ShareLink, error)
|
||||
GetByToken(ctx context.Context, token string) (*models.ShareLink, error)
|
||||
Revoke(ctx context.Context, id, albumID string) error
|
||||
}
|
||||
@@ -16,11 +16,11 @@ type AlbumAccess interface {
|
||||
}
|
||||
|
||||
type Service struct {
|
||||
store *Store
|
||||
store Repository
|
||||
albums AlbumAccess
|
||||
}
|
||||
|
||||
func NewService(store *Store, albums AlbumAccess) *Service {
|
||||
func NewService(store Repository, albums AlbumAccess) *Service {
|
||||
return &Service{store: store, albums: albums}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
package sharelinks
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
"time"
|
||||
|
||||
"shared-album/internal/models"
|
||||
"shared-album/internal/sqliteutil"
|
||||
)
|
||||
|
||||
// SQLiteStore is the SQLite implementation of Repository.
|
||||
type SQLiteStore struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func NewSQLiteStore(db *sql.DB) *SQLiteStore { return &SQLiteStore{db: db} }
|
||||
|
||||
func (s *SQLiteStore) Create(ctx context.Context, l *models.ShareLink) error {
|
||||
var createdAt string
|
||||
err := s.db.QueryRowContext(ctx,
|
||||
`INSERT INTO share_links (id, album_id, token, created_by, expires_at)
|
||||
VALUES ($1, $2, $3, $4, $5)
|
||||
RETURNING created_at`,
|
||||
l.ID, l.AlbumID, l.Token, l.CreatedBy, sqliteutil.FormatTime(l.ExpiresAt),
|
||||
).Scan(&createdAt)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
l.CreatedAt = sqliteutil.ParseTime(createdAt)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) ListByAlbum(ctx context.Context, albumID string) ([]models.ShareLink, error) {
|
||||
rows, err := s.db.QueryContext(ctx,
|
||||
`SELECT id, album_id, token, created_by, created_at, expires_at, revoked_at
|
||||
FROM share_links WHERE album_id = $1 ORDER BY created_at DESC`, albumID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
out := []models.ShareLink{}
|
||||
for rows.Next() {
|
||||
var l models.ShareLink
|
||||
var createdAt, expiresAt string
|
||||
var revokedAt sql.NullString
|
||||
if err := rows.Scan(&l.ID, &l.AlbumID, &l.Token, &l.CreatedBy, &createdAt, &expiresAt, &revokedAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
l.CreatedAt = sqliteutil.ParseTime(createdAt)
|
||||
l.ExpiresAt = sqliteutil.ParseTime(expiresAt)
|
||||
l.RevokedAt = sqliteutil.ParseNullTime(revokedAt)
|
||||
out = append(out, l)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) GetByToken(ctx context.Context, token string) (*models.ShareLink, error) {
|
||||
var l models.ShareLink
|
||||
var createdAt, expiresAt string
|
||||
var revokedAt sql.NullString
|
||||
err := s.db.QueryRowContext(ctx,
|
||||
`SELECT id, album_id, token, created_by, created_at, expires_at, revoked_at
|
||||
FROM share_links WHERE token = $1`, token).
|
||||
Scan(&l.ID, &l.AlbumID, &l.Token, &l.CreatedBy, &createdAt, &expiresAt, &revokedAt)
|
||||
if err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return nil, ErrNotFound
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
l.CreatedAt = sqliteutil.ParseTime(createdAt)
|
||||
l.ExpiresAt = sqliteutil.ParseTime(expiresAt)
|
||||
l.RevokedAt = sqliteutil.ParseNullTime(revokedAt)
|
||||
return &l, nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) Revoke(ctx context.Context, id, albumID string) error {
|
||||
res, err := s.db.ExecContext(ctx,
|
||||
`UPDATE share_links SET revoked_at = $3 WHERE id = $1 AND album_id = $2 AND revoked_at IS NULL`,
|
||||
id, albumID, sqliteutil.FormatTime(time.Now()))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if n, _ := res.RowsAffected(); n == 0 {
|
||||
return ErrNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
package social
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"shared-album/internal/models"
|
||||
)
|
||||
|
||||
// Repository is the persistence contract for the social domain.
|
||||
type Repository interface {
|
||||
Like(ctx context.Context, photoID, userID string) error
|
||||
Unlike(ctx context.Context, photoID, userID string) error
|
||||
CreateComment(ctx context.Context, c *models.Comment) error
|
||||
ListComments(ctx context.Context, photoID string) ([]models.Comment, error)
|
||||
GetComment(ctx context.Context, id string) (*models.Comment, error)
|
||||
DeleteComment(ctx context.Context, id string) error
|
||||
}
|
||||
@@ -25,14 +25,14 @@ type Notifier interface {
|
||||
}
|
||||
|
||||
type Service struct {
|
||||
store *Store
|
||||
store Repository
|
||||
albums AlbumAccess
|
||||
photos PhotoLookup
|
||||
hub *sse.Hub
|
||||
notify Notifier
|
||||
}
|
||||
|
||||
func NewService(store *Store, albums AlbumAccess, photos PhotoLookup, hub *sse.Hub, notify Notifier) *Service {
|
||||
func NewService(store Repository, albums AlbumAccess, photos PhotoLookup, hub *sse.Hub, notify Notifier) *Service {
|
||||
return &Service{store: store, albums: albums, photos: photos, hub: hub, notify: notify}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
package social
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
|
||||
"shared-album/internal/models"
|
||||
"shared-album/internal/sqliteutil"
|
||||
)
|
||||
|
||||
// SQLiteStore is the SQLite implementation of Repository.
|
||||
type SQLiteStore struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func NewSQLiteStore(db *sql.DB) *SQLiteStore { return &SQLiteStore{db: db} }
|
||||
|
||||
func (s *SQLiteStore) Like(ctx context.Context, photoID, userID string) error {
|
||||
_, err := s.db.ExecContext(ctx,
|
||||
`INSERT INTO likes (photo_id, user_id) VALUES ($1, $2)
|
||||
ON CONFLICT (photo_id, user_id) DO NOTHING`, photoID, userID)
|
||||
return err
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) Unlike(ctx context.Context, photoID, userID string) error {
|
||||
_, err := s.db.ExecContext(ctx,
|
||||
`DELETE FROM likes WHERE photo_id = $1 AND user_id = $2`, photoID, userID)
|
||||
return err
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) CreateComment(ctx context.Context, c *models.Comment) error {
|
||||
var createdAt, updatedAt string
|
||||
err := s.db.QueryRowContext(ctx,
|
||||
`INSERT INTO comments (id, photo_id, user_id, text)
|
||||
VALUES ($1, $2, $3, $4)
|
||||
RETURNING created_at, updated_at`,
|
||||
c.ID, c.PhotoID, c.UserID, c.Text,
|
||||
).Scan(&createdAt, &updatedAt)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
c.CreatedAt = sqliteutil.ParseTime(createdAt)
|
||||
c.UpdatedAt = sqliteutil.ParseTime(updatedAt)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) ListComments(ctx context.Context, photoID string) ([]models.Comment, error) {
|
||||
rows, err := s.db.QueryContext(ctx,
|
||||
`SELECT c.id, c.photo_id, c.user_id, c.text, c.created_at, c.updated_at,
|
||||
u.id, u.name, COALESCE(u.avatar_url, '')
|
||||
FROM comments c
|
||||
JOIN users u ON u.id = c.user_id
|
||||
WHERE c.photo_id = $1
|
||||
ORDER BY c.created_at`, photoID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
out := []models.Comment{}
|
||||
for rows.Next() {
|
||||
var c models.Comment
|
||||
var createdAt, updatedAt string
|
||||
if err := rows.Scan(&c.ID, &c.PhotoID, &c.UserID, &c.Text, &createdAt, &updatedAt,
|
||||
&c.User.ID, &c.User.Name, &c.User.AvatarURL); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c.CreatedAt = sqliteutil.ParseTime(createdAt)
|
||||
c.UpdatedAt = sqliteutil.ParseTime(updatedAt)
|
||||
out = append(out, c)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) GetComment(ctx context.Context, id string) (*models.Comment, error) {
|
||||
var c models.Comment
|
||||
var createdAt, updatedAt string
|
||||
err := s.db.QueryRowContext(ctx,
|
||||
`SELECT id, photo_id, user_id, text, created_at, updated_at FROM comments WHERE id = $1`, id).
|
||||
Scan(&c.ID, &c.PhotoID, &c.UserID, &c.Text, &createdAt, &updatedAt)
|
||||
if err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return nil, ErrNotFound
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
c.CreatedAt = sqliteutil.ParseTime(createdAt)
|
||||
c.UpdatedAt = sqliteutil.ParseTime(updatedAt)
|
||||
return &c, nil
|
||||
}
|
||||
|
||||
func (s *SQLiteStore) DeleteComment(ctx context.Context, id string) error {
|
||||
res, err := s.db.ExecContext(ctx, `DELETE FROM comments WHERE id = $1`, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if n, _ := res.RowsAffected(); n == 0 {
|
||||
return ErrNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
// Package sqliteutil holds small helpers shared by the SQLite repositories:
|
||||
// timestamp conversion and dynamic IN-lists.
|
||||
package sqliteutil
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// TimeLayout matches the SQLite column defaults produced by strftime with %f.
|
||||
const TimeLayout = "2006-01-02T15:04:05.000Z"
|
||||
|
||||
// FormatTime renders a time in the canonical TEXT format used by the schema.
|
||||
func FormatTime(t time.Time) string { return t.UTC().Format(TimeLayout) }
|
||||
|
||||
// ParseTime parses a canonical timestamp, tolerating the RFC3339 variants the
|
||||
// driver may return. An empty or unparseable value yields the zero time.
|
||||
func ParseTime(s string) time.Time {
|
||||
if s == "" {
|
||||
return time.Time{}
|
||||
}
|
||||
if t, err := time.Parse(TimeLayout, s); err == nil {
|
||||
return t
|
||||
}
|
||||
if t, err := time.Parse(time.RFC3339Nano, s); err == nil {
|
||||
return t
|
||||
}
|
||||
return time.Time{}
|
||||
}
|
||||
|
||||
// ParseNullTime converts a nullable TEXT timestamp into *time.Time.
|
||||
func ParseNullTime(ns sql.NullString) *time.Time {
|
||||
if !ns.Valid || ns.String == "" {
|
||||
return nil
|
||||
}
|
||||
t := ParseTime(ns.String)
|
||||
return &t
|
||||
}
|
||||
|
||||
// Placeholders returns an IN-list placeholder group of n items starting at
|
||||
// index start, e.g. Placeholders(3, 2) -> "$3,$4".
|
||||
func Placeholders(start, n int) string {
|
||||
parts := make([]string, n)
|
||||
for i := 0; i < n; i++ {
|
||||
parts[i] = "$" + strconv.Itoa(start+i)
|
||||
}
|
||||
return strings.Join(parts, ",")
|
||||
}
|
||||
@@ -0,0 +1,233 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"io/fs"
|
||||
"mime"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// errInvalidKey is returned for object keys that are empty, absolute or attempt
|
||||
// to escape the storage root.
|
||||
var errInvalidKey = errors.New("fs: invalid object key")
|
||||
|
||||
// FS is a Backend backed by a local directory tree. Blobs live under Root at the
|
||||
// path formed by the opaque key. Because a local directory has no external URL,
|
||||
// FS serves blobs through the app using the shared Signer.
|
||||
type FS struct {
|
||||
root string
|
||||
signer *Signer
|
||||
}
|
||||
|
||||
// NewFS builds a local file-system storage backend rooted at root. signer issues
|
||||
// and validates the app URLs used to serve blobs.
|
||||
func NewFS(root string, signer *Signer) (*FS, error) {
|
||||
if strings.TrimSpace(root) == "" {
|
||||
return nil, errors.New("fs: root is required")
|
||||
}
|
||||
if signer == nil {
|
||||
return nil, errors.New("fs: signer is required")
|
||||
}
|
||||
abs, err := filepath.Abs(root)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("fs: resolve root: %w", err)
|
||||
}
|
||||
if err := os.MkdirAll(abs, 0o755); err != nil {
|
||||
return nil, fmt.Errorf("fs: create root: %w", err)
|
||||
}
|
||||
return &FS{root: abs, signer: signer}, nil
|
||||
}
|
||||
|
||||
func (f *FS) Bucket() string { return f.root }
|
||||
|
||||
func (f *FS) EnsureBucket(context.Context) error {
|
||||
return os.MkdirAll(f.root, 0o755)
|
||||
}
|
||||
|
||||
func (f *FS) Ping(context.Context) error {
|
||||
info, err := os.Stat(f.root)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !info.IsDir() {
|
||||
return errors.New("fs: root is not a directory")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// objectPath maps an opaque key to an absolute path inside the root, rejecting
|
||||
// absolute keys and any key that resolves outside the root.
|
||||
func (f *FS) objectPath(key string) (string, error) {
|
||||
if key == "" || filepath.IsAbs(key) {
|
||||
return "", errInvalidKey
|
||||
}
|
||||
clean := filepath.Clean(key)
|
||||
if clean == ".." || strings.HasPrefix(clean, ".."+string(os.PathSeparator)) {
|
||||
return "", errInvalidKey
|
||||
}
|
||||
p := filepath.Join(f.root, clean)
|
||||
rel, err := filepath.Rel(f.root, p)
|
||||
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(os.PathSeparator)) {
|
||||
return "", errInvalidKey
|
||||
}
|
||||
return p, nil
|
||||
}
|
||||
|
||||
func (f *FS) Put(_ context.Context, key string, body io.Reader, _ int64, _ string) error {
|
||||
p, err := f.objectPath(key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
|
||||
return fmt.Errorf("fs: mkdir: %w", err)
|
||||
}
|
||||
tmp, err := os.CreateTemp(filepath.Dir(p), ".upload-*")
|
||||
if err != nil {
|
||||
return fmt.Errorf("fs: temp file: %w", err)
|
||||
}
|
||||
tmpName := tmp.Name()
|
||||
defer func() { _ = os.Remove(tmpName) }()
|
||||
if _, err := io.Copy(tmp, body); err != nil {
|
||||
_ = tmp.Close()
|
||||
return fmt.Errorf("fs: write: %w", err)
|
||||
}
|
||||
if err := tmp.Close(); err != nil {
|
||||
return fmt.Errorf("fs: close: %w", err)
|
||||
}
|
||||
if err := os.Rename(tmpName, p); err != nil {
|
||||
return fmt.Errorf("fs: rename: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (f *FS) Get(_ context.Context, key string) ([]byte, error) {
|
||||
p, err := f.objectPath(key)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
data, err := os.ReadFile(p)
|
||||
if err != nil {
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
return nil, ErrObjectNotFound
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
return data, nil
|
||||
}
|
||||
|
||||
func (f *FS) Delete(_ context.Context, keys ...string) error {
|
||||
var errs []error
|
||||
for _, key := range keys {
|
||||
p, err := f.objectPath(key)
|
||||
if err != nil {
|
||||
errs = append(errs, err)
|
||||
continue
|
||||
}
|
||||
if err := os.Remove(p); err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||
errs = append(errs, err)
|
||||
}
|
||||
}
|
||||
return errors.Join(errs...)
|
||||
}
|
||||
|
||||
// URL returns a signed app URL served by this process.
|
||||
func (f *FS) URL(_ context.Context, key string) (string, error) {
|
||||
if _, err := f.objectPath(key); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return f.signer.URL(key), nil
|
||||
}
|
||||
|
||||
// List returns every blob under prefix as a key relative to the storage root.
|
||||
func (f *FS) List(_ context.Context, prefix string) ([]ObjectInfo, error) {
|
||||
var out []ObjectInfo
|
||||
err := filepath.WalkDir(f.root, func(path string, d fs.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if d.IsDir() {
|
||||
return nil
|
||||
}
|
||||
rel, err := filepath.Rel(f.root, path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
rel = filepath.ToSlash(rel)
|
||||
if prefix != "" && !strings.HasPrefix(rel, prefix) {
|
||||
return nil
|
||||
}
|
||||
info, err := d.Info()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
out = append(out, ObjectInfo{Key: rel, Size: info.Size(), ModTime: info.ModTime()})
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// ServeObject validates the signed URL and serves the blob. It supports range
|
||||
// requests and sets the content type from the key extension. It writes the
|
||||
// response itself and returns nil on success.
|
||||
func (f *FS) ServeObject(w http.ResponseWriter, r *http.Request, key string) error {
|
||||
if unescaped, err := url.PathUnescape(key); err == nil {
|
||||
key = unescaped
|
||||
}
|
||||
if err := f.signer.VerifyRequest(key, r); err != nil {
|
||||
return err
|
||||
}
|
||||
p, err := f.objectPath(key)
|
||||
if err != nil {
|
||||
return ErrSignedURLInvalid
|
||||
}
|
||||
file, err := os.Open(p)
|
||||
if err != nil {
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
return ErrObjectNotFound
|
||||
}
|
||||
return err
|
||||
}
|
||||
defer file.Close()
|
||||
info, err := file.Stat()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
w.Header().Set("Content-Type", contentTypeForKey(key))
|
||||
http.ServeContent(w, r, info.Name(), info.ModTime(), file)
|
||||
return nil
|
||||
}
|
||||
|
||||
var imageContentTypes = map[string]string{
|
||||
".jpg": "image/jpeg",
|
||||
".jpeg": "image/jpeg",
|
||||
".png": "image/png",
|
||||
".gif": "image/gif",
|
||||
".webp": "image/webp",
|
||||
".heic": "image/heic",
|
||||
".heif": "image/heif",
|
||||
".tiff": "image/tiff",
|
||||
".tif": "image/tiff",
|
||||
".bmp": "image/bmp",
|
||||
".avif": "image/avif",
|
||||
}
|
||||
|
||||
func contentTypeForKey(key string) string {
|
||||
ext := strings.ToLower(filepath.Ext(key))
|
||||
if ct, ok := imageContentTypes[ext]; ok {
|
||||
return ct
|
||||
}
|
||||
if ct := mime.TypeByExtension(ext); ct != "" {
|
||||
return ct
|
||||
}
|
||||
return "application/octet-stream"
|
||||
}
|
||||
@@ -0,0 +1,204 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func newTestFS(t *testing.T) *FS {
|
||||
t.Helper()
|
||||
signer, err := NewSigner("http://app.test", "test-secret", time.Hour)
|
||||
if err != nil {
|
||||
t.Fatalf("signer: %v", err)
|
||||
}
|
||||
fs, err := NewFS(t.TempDir(), signer)
|
||||
if err != nil {
|
||||
t.Fatalf("new fs: %v", err)
|
||||
}
|
||||
return fs
|
||||
}
|
||||
|
||||
func TestFSRoundTrip(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
fs := newTestFS(t)
|
||||
key := "objects/abc/original.jpg"
|
||||
data := []byte("hello blob")
|
||||
|
||||
if err := fs.Put(ctx, key, bytes.NewReader(data), int64(len(data)), "image/jpeg"); err != nil {
|
||||
t.Fatalf("put: %v", err)
|
||||
}
|
||||
got, err := fs.Get(ctx, key)
|
||||
if err != nil {
|
||||
t.Fatalf("get: %v", err)
|
||||
}
|
||||
if !bytes.Equal(got, data) {
|
||||
t.Fatalf("get = %q, want %q", got, data)
|
||||
}
|
||||
if err := fs.Delete(ctx, key); err != nil {
|
||||
t.Fatalf("delete: %v", err)
|
||||
}
|
||||
if _, err := fs.Get(ctx, key); !errors.Is(err, ErrObjectNotFound) {
|
||||
t.Fatalf("get after delete = %v, want ErrObjectNotFound", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFSPathTraversalRejected(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
fs := newTestFS(t)
|
||||
for _, key := range []string{"../escape", "../../etc/passwd", "/etc/passwd", ""} {
|
||||
if err := fs.Put(ctx, key, strings.NewReader("x"), 1, "text/plain"); err == nil {
|
||||
t.Errorf("put %q: expected rejection", key)
|
||||
}
|
||||
if _, err := fs.Get(ctx, key); err == nil {
|
||||
t.Errorf("get %q: expected rejection", key)
|
||||
}
|
||||
if _, err := fs.URL(ctx, key); err == nil {
|
||||
t.Errorf("url %q: expected rejection", key)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestFSURLAndServe(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
fs := newTestFS(t)
|
||||
key := "objects/abc/thumb.webp"
|
||||
if err := fs.Put(ctx, key, strings.NewReader("webp-bytes"), 10, "image/webp"); err != nil {
|
||||
t.Fatalf("put: %v", err)
|
||||
}
|
||||
|
||||
raw, err := fs.URL(ctx, key)
|
||||
if err != nil {
|
||||
t.Fatalf("url: %v", err)
|
||||
}
|
||||
u, err := url.Parse(raw)
|
||||
if err != nil {
|
||||
t.Fatalf("parse url: %v", err)
|
||||
}
|
||||
if !strings.HasPrefix(u.Path, "/api/objects/objects/abc/thumb.webp") {
|
||||
t.Fatalf("unexpected path: %s", u.Path)
|
||||
}
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
req := httptest.NewRequest(http.MethodGet, raw, nil)
|
||||
if err := fs.ServeObject(rec, req, strings.TrimPrefix(u.Path, "/api/objects/")); err != nil {
|
||||
t.Fatalf("serve: %v", err)
|
||||
}
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200", rec.Code)
|
||||
}
|
||||
if ct := rec.Header().Get("Content-Type"); ct != "image/webp" {
|
||||
t.Fatalf("content-type = %q, want image/webp", ct)
|
||||
}
|
||||
if rec.Body.String() != "webp-bytes" {
|
||||
t.Fatalf("body = %q", rec.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestFSServeExpiredAndForged(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
fs := newTestFS(t)
|
||||
key := "objects/abc/original.jpg"
|
||||
if err := fs.Put(ctx, key, strings.NewReader("x"), 1, "image/jpeg"); err != nil {
|
||||
t.Fatalf("put: %v", err)
|
||||
}
|
||||
|
||||
past := time.Now().Add(-time.Minute).Unix()
|
||||
expired := "/api/objects/" + key + "?e=" + strconv.FormatInt(past, 10) + "&s=" + fs.signer.sign(key, past)
|
||||
rec := httptest.NewRecorder()
|
||||
err := fs.ServeObject(rec, httptest.NewRequest(http.MethodGet, expired, nil), key)
|
||||
if !errors.Is(err, ErrSignedURLInvalid) {
|
||||
t.Fatalf("expired = %v, want ErrSignedURLInvalid", err)
|
||||
}
|
||||
|
||||
future := time.Now().Add(time.Hour).Unix()
|
||||
forged := "/api/objects/" + key + "?e=" + strconv.FormatInt(future, 10) + "&s=deadbeef"
|
||||
rec = httptest.NewRecorder()
|
||||
err = fs.ServeObject(rec, httptest.NewRequest(http.MethodGet, forged, nil), key)
|
||||
if !errors.Is(err, ErrSignedURLInvalid) {
|
||||
t.Fatalf("forged = %v, want ErrSignedURLInvalid", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFSServeRangeAndMissing(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
fs := newTestFS(t)
|
||||
key := "objects/abc/original.jpg"
|
||||
if err := fs.Put(ctx, key, strings.NewReader("0123456789"), 10, "image/jpeg"); err != nil {
|
||||
t.Fatalf("put: %v", err)
|
||||
}
|
||||
exp := time.Now().Add(time.Hour).Unix()
|
||||
base := "/api/objects/" + key + "?e=" + strconv.FormatInt(exp, 10) + "&s=" + fs.signer.sign(key, exp)
|
||||
|
||||
req := httptest.NewRequest(http.MethodGet, base, nil)
|
||||
req.Header.Set("Range", "bytes=2-5")
|
||||
rec := httptest.NewRecorder()
|
||||
if err := fs.ServeObject(rec, req, key); err != nil {
|
||||
t.Fatalf("serve range: %v", err)
|
||||
}
|
||||
if rec.Code != http.StatusPartialContent {
|
||||
t.Fatalf("status = %d, want 206", rec.Code)
|
||||
}
|
||||
if body, _ := io.ReadAll(rec.Body); string(body) != "2345" {
|
||||
t.Fatalf("range body = %q, want 2345", body)
|
||||
}
|
||||
|
||||
missingKey := "objects/def/original.jpg"
|
||||
missing := "/api/objects/" + missingKey + "?e=" + strconv.FormatInt(exp, 10) + "&s=" + fs.signer.sign(missingKey, exp)
|
||||
rec = httptest.NewRecorder()
|
||||
err := fs.ServeObject(rec, httptest.NewRequest(http.MethodGet, missing, nil), missingKey)
|
||||
if !errors.Is(err, ErrObjectNotFound) {
|
||||
t.Fatalf("missing = %v, want ErrObjectNotFound", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestContentTypeForKey(t *testing.T) {
|
||||
cases := map[string]string{
|
||||
"objects/a/original.jpg": "image/jpeg",
|
||||
"objects/a/thumb.webp": "image/webp",
|
||||
"objects/a/odd.unknownext": "application/octet-stream",
|
||||
}
|
||||
for key, want := range cases {
|
||||
if got := contentTypeForKey(key); got != want {
|
||||
t.Errorf("contentTypeForKey(%q) = %q, want %q", key, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestFSList(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
fs := newTestFS(t)
|
||||
for _, key := range []string{"objects/a/original.jpg", "objects/b/original.jpg", "misc/readme.txt"} {
|
||||
if err := fs.Put(ctx, key, strings.NewReader("x"), 1, "text/plain"); err != nil {
|
||||
t.Fatalf("put %s: %v", key, err)
|
||||
}
|
||||
}
|
||||
|
||||
all, err := fs.List(ctx, "")
|
||||
if err != nil || len(all) != 3 {
|
||||
t.Fatalf("list all: %v len=%d", err, len(all))
|
||||
}
|
||||
objs, err := fs.List(ctx, "objects/")
|
||||
if err != nil {
|
||||
t.Fatalf("list objects: %v", err)
|
||||
}
|
||||
if len(objs) != 2 {
|
||||
t.Fatalf("list objects len=%d, want 2", len(objs))
|
||||
}
|
||||
for _, o := range objs {
|
||||
if !strings.HasPrefix(o.Key, "objects/") {
|
||||
t.Fatalf("unexpected key %q", o.Key)
|
||||
}
|
||||
if o.Size != 1 || o.ModTime.IsZero() {
|
||||
t.Fatalf("unexpected info %+v", o)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,169 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"shared-album/internal/config"
|
||||
)
|
||||
|
||||
func newTestS3(t *testing.T) (*S3, *Signer) {
|
||||
t.Helper()
|
||||
endpoint := os.Getenv("S3_ENDPOINT")
|
||||
if endpoint == "" {
|
||||
t.Skip("S3_ENDPOINT not set")
|
||||
}
|
||||
signer, err := NewSigner("http://app.test", "secret", time.Hour)
|
||||
if err != nil {
|
||||
t.Fatalf("signer: %v", err)
|
||||
}
|
||||
cl, err := NewS3(context.Background(), config.S3{
|
||||
Endpoint: endpoint,
|
||||
Region: os.Getenv("S3_REGION"),
|
||||
AccessKey: os.Getenv("S3_ACCESS_KEY"),
|
||||
SecretKey: os.Getenv("S3_SECRET_KEY"),
|
||||
Bucket: os.Getenv("S3_BUCKET"),
|
||||
UsePathStyle: true,
|
||||
}, signer)
|
||||
if err != nil {
|
||||
t.Skipf("storage init: %v", err)
|
||||
}
|
||||
return cl, signer
|
||||
}
|
||||
|
||||
func signedQuery(signer *Signer, key string, exp int64) string {
|
||||
return "?e=" + strconv.FormatInt(exp, 10) + "&s=" + signer.sign(key, exp)
|
||||
}
|
||||
|
||||
func TestS3ServeObject(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cl, signer := newTestS3(t)
|
||||
key := "objects/" + strconv.FormatInt(time.Now().UnixNano(), 10) + "/original.jpg"
|
||||
data := []byte("0123456789")
|
||||
if err := cl.Put(ctx, key, bytes.NewReader(data), int64(len(data)), "image/jpeg"); err != nil {
|
||||
t.Fatalf("put: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { _ = cl.Delete(context.Background(), key) })
|
||||
|
||||
exp := time.Now().Add(time.Hour).Unix()
|
||||
base := "/api/objects/" + key + signedQuery(signer, key, exp)
|
||||
|
||||
t.Run("full", func(t *testing.T) {
|
||||
rec := httptest.NewRecorder()
|
||||
req := httptest.NewRequest(http.MethodGet, base, nil)
|
||||
if err := cl.ServeObject(rec, req, key); err != nil {
|
||||
t.Fatalf("serve: %v", err)
|
||||
}
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200", rec.Code)
|
||||
}
|
||||
if !bytes.Equal(rec.Body.Bytes(), data) {
|
||||
t.Fatalf("body = %q", rec.Body.String())
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("range", func(t *testing.T) {
|
||||
rec := httptest.NewRecorder()
|
||||
req := httptest.NewRequest(http.MethodGet, base, nil)
|
||||
req.Header.Set("Range", "bytes=2-5")
|
||||
if err := cl.ServeObject(rec, req, key); err != nil {
|
||||
t.Fatalf("serve range: %v", err)
|
||||
}
|
||||
if rec.Code != http.StatusPartialContent {
|
||||
t.Fatalf("status = %d, want 206", rec.Code)
|
||||
}
|
||||
if body, _ := io.ReadAll(rec.Body); string(body) != "2345" {
|
||||
t.Fatalf("range body = %q, want 2345", body)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("invalid range", func(t *testing.T) {
|
||||
rec := httptest.NewRecorder()
|
||||
req := httptest.NewRequest(http.MethodGet, base, nil)
|
||||
req.Header.Set("Range", "bytes=1000-2000")
|
||||
err := cl.ServeObject(rec, req, key)
|
||||
if !errors.Is(err, ErrRangeNotSatisfiable) {
|
||||
t.Fatalf("err = %v, want ErrRangeNotSatisfiable", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("missing", func(t *testing.T) {
|
||||
missingKey := "objects/" + strconv.FormatInt(time.Now().UnixNano(), 10) + "/original.jpg"
|
||||
target := "/api/objects/" + missingKey + signedQuery(signer, missingKey, exp)
|
||||
rec := httptest.NewRecorder()
|
||||
err := cl.ServeObject(rec, httptest.NewRequest(http.MethodGet, target, nil), missingKey)
|
||||
if !errors.Is(err, ErrObjectNotFound) {
|
||||
t.Fatalf("err = %v, want ErrObjectNotFound", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("expired signature", func(t *testing.T) {
|
||||
past := time.Now().Add(-time.Minute).Unix()
|
||||
target := "/api/objects/" + key + signedQuery(signer, key, past)
|
||||
rec := httptest.NewRecorder()
|
||||
err := cl.ServeObject(rec, httptest.NewRequest(http.MethodGet, target, nil), key)
|
||||
if !errors.Is(err, ErrSignedURLInvalid) {
|
||||
t.Fatalf("err = %v, want ErrSignedURLInvalid", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestS3List(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cl, _ := newTestS3(t)
|
||||
prefix := "objects/" + strconv.FormatInt(time.Now().UnixNano(), 10) + "/"
|
||||
keys := []string{prefix + "original.jpg", prefix + "thumb.webp"}
|
||||
for _, k := range keys {
|
||||
if err := cl.Put(ctx, k, strings.NewReader("x"), 1, "image/jpeg"); err != nil {
|
||||
t.Fatalf("put %s: %v", k, err)
|
||||
}
|
||||
}
|
||||
t.Cleanup(func() { _ = cl.Delete(context.Background(), keys...) })
|
||||
|
||||
got, err := cl.List(ctx, prefix)
|
||||
if err != nil {
|
||||
t.Fatalf("list: %v", err)
|
||||
}
|
||||
found := map[string]bool{}
|
||||
for _, o := range got {
|
||||
found[o.Key] = true
|
||||
}
|
||||
for _, k := range keys {
|
||||
if !found[k] {
|
||||
t.Fatalf("key %s not listed: %+v", k, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestS3ServeObjectHead(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cl, signer := newTestS3(t)
|
||||
key := "objects/" + strconv.FormatInt(time.Now().UnixNano(), 10) + "/original.jpg"
|
||||
data := []byte("head-check")
|
||||
if err := cl.Put(ctx, key, strings.NewReader(string(data)), int64(len(data)), "image/jpeg"); err != nil {
|
||||
t.Fatalf("put: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { _ = cl.Delete(context.Background(), key) })
|
||||
|
||||
exp := time.Now().Add(time.Hour).Unix()
|
||||
target := "/api/objects/" + key + signedQuery(signer, key, exp)
|
||||
rec := httptest.NewRecorder()
|
||||
if err := cl.ServeObject(rec, httptest.NewRequest(http.MethodHead, target, nil), key); err != nil {
|
||||
t.Fatalf("head: %v", err)
|
||||
}
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200", rec.Code)
|
||||
}
|
||||
if rec.Body.Len() != 0 {
|
||||
t.Fatalf("HEAD body should be empty, got %d bytes", rec.Body.Len())
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,80 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"crypto/hmac"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Signer produces and verifies the signed app URLs used by the /api/objects
|
||||
// route. It is shared by every storage backend so delivery is identical: the
|
||||
// client always talks to the backend, never to object storage directly.
|
||||
type Signer struct {
|
||||
secret []byte
|
||||
baseURL string
|
||||
ttl time.Duration
|
||||
}
|
||||
|
||||
// NewSigner builds a signer. secret signs the URLs (required); baseURL is the
|
||||
// public base URL of the app; ttl limits URL lifetime (defaults to one hour).
|
||||
func NewSigner(baseURL, secret string, ttl time.Duration) (*Signer, error) {
|
||||
if strings.TrimSpace(secret) == "" {
|
||||
return nil, errors.New("url secret is required")
|
||||
}
|
||||
if ttl <= 0 {
|
||||
ttl = time.Hour
|
||||
}
|
||||
return &Signer{
|
||||
secret: []byte(secret),
|
||||
baseURL: strings.TrimRight(baseURL, "/"),
|
||||
ttl: ttl,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// URL returns a signed app URL for the given opaque key.
|
||||
func (s *Signer) URL(key string) string {
|
||||
exp := time.Now().Add(s.ttl).Unix()
|
||||
q := url.Values{}
|
||||
q.Set("e", strconv.FormatInt(exp, 10))
|
||||
q.Set("s", s.sign(key, exp))
|
||||
return fmt.Sprintf("%s/api/objects/%s?%s", s.baseURL, escapeKey(key), q.Encode())
|
||||
}
|
||||
|
||||
// VerifyRequest checks the signature and expiry carried by an object request.
|
||||
// It returns ErrSignedURLInvalid when the request is not authorized.
|
||||
func (s *Signer) VerifyRequest(key string, r *http.Request) error {
|
||||
q := r.URL.Query()
|
||||
exp, err := strconv.ParseInt(q.Get("e"), 10, 64)
|
||||
if err != nil || !s.verify(key, exp, q.Get("s")) {
|
||||
return ErrSignedURLInvalid
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Signer) sign(key string, exp int64) string {
|
||||
mac := hmac.New(sha256.New, s.secret)
|
||||
_, _ = fmt.Fprintf(mac, "%s\n%d", key, exp)
|
||||
return hex.EncodeToString(mac.Sum(nil))
|
||||
}
|
||||
|
||||
func (s *Signer) verify(key string, exp int64, sig string) bool {
|
||||
if sig == "" || time.Now().Unix() > exp {
|
||||
return false
|
||||
}
|
||||
return hmac.Equal([]byte(s.sign(key, exp)), []byte(sig))
|
||||
}
|
||||
|
||||
func escapeKey(key string) string {
|
||||
parts := strings.Split(key, "/")
|
||||
for i, p := range parts {
|
||||
parts[i] = url.PathEscape(p)
|
||||
}
|
||||
return strings.Join(parts, "/")
|
||||
}
|
||||
@@ -5,6 +5,8 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
@@ -18,14 +20,58 @@ import (
|
||||
"shared-album/internal/config"
|
||||
)
|
||||
|
||||
type Client struct {
|
||||
s3 *s3.Client
|
||||
presign *s3.PresignClient
|
||||
bucket string
|
||||
presignTTL time.Duration
|
||||
// ErrSignedURLInvalid is returned when an app-served object URL has a missing,
|
||||
// malformed, expired or forged signature.
|
||||
var ErrSignedURLInvalid = errors.New("invalid or expired object signature")
|
||||
|
||||
// ErrObjectNotFound is returned when the requested blob does not exist.
|
||||
var ErrObjectNotFound = errors.New("object not found")
|
||||
|
||||
// ErrRangeNotSatisfiable is returned when a requested byte range is invalid.
|
||||
var ErrRangeNotSatisfiable = errors.New("requested range not satisfiable")
|
||||
|
||||
// ObjectInfo describes one stored blob for inventory and reconciliation.
|
||||
type ObjectInfo struct {
|
||||
Key string
|
||||
Size int64
|
||||
ModTime time.Time
|
||||
}
|
||||
|
||||
func New(ctx context.Context, cfg config.S3) (*Client, error) {
|
||||
// Backend is the object storage abstraction. Keys are opaque and independent of
|
||||
// the backend: every implementation stores and serves the same logical blob.
|
||||
// Blobs are always delivered through the app.
|
||||
type Backend interface {
|
||||
Put(ctx context.Context, key string, body io.Reader, size int64, contentType string) error
|
||||
Delete(ctx context.Context, keys ...string) error
|
||||
Get(ctx context.Context, key string) ([]byte, error)
|
||||
// URL returns a time-limited signed app URL the client can use to fetch the blob.
|
||||
URL(ctx context.Context, key string) (string, error)
|
||||
// ServeObject validates the signed URL and writes the blob, including
|
||||
// Content-Type and range handling, to the response.
|
||||
ServeObject(w http.ResponseWriter, r *http.Request, key string) error
|
||||
// List returns the blobs under prefix for inventory and reconciliation.
|
||||
List(ctx context.Context, prefix string) ([]ObjectInfo, error)
|
||||
Ping(ctx context.Context) error
|
||||
EnsureBucket(ctx context.Context) error
|
||||
// Bucket returns a human-readable identifier for the storage location.
|
||||
Bucket() string
|
||||
}
|
||||
|
||||
// S3 is a Backend backed by an S3-compatible object store (Garage). Blobs are
|
||||
// delivered through the app: the client receives a signed app URL and the bytes
|
||||
// are streamed from S3 over the internal network.
|
||||
type S3 struct {
|
||||
s3 *s3.Client
|
||||
bucket string
|
||||
signer *Signer
|
||||
}
|
||||
|
||||
// NewS3 builds an S3-compatible storage backend. signer issues the app URLs used
|
||||
// to serve blobs.
|
||||
func NewS3(ctx context.Context, cfg config.S3, signer *Signer) (*S3, error) {
|
||||
if signer == nil {
|
||||
return nil, errors.New("s3: signer is required")
|
||||
}
|
||||
awsCfg, err := awsconfig.LoadDefaultConfig(ctx,
|
||||
awsconfig.WithRegion(cfg.Region),
|
||||
awsconfig.WithCredentialsProvider(credentials.NewStaticCredentialsProvider(
|
||||
@@ -41,17 +87,42 @@ func New(ctx context.Context, cfg config.S3) (*Client, error) {
|
||||
o.UsePathStyle = cfg.UsePathStyle
|
||||
})
|
||||
|
||||
return &Client{
|
||||
return &S3{
|
||||
s3: client,
|
||||
presign: s3.NewPresignClient(client),
|
||||
bucket: cfg.Bucket,
|
||||
presignTTL: cfg.PresignTTL,
|
||||
signer: signer,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (c *Client) Bucket() string { return c.bucket }
|
||||
func (c *S3) Bucket() string { return c.bucket }
|
||||
|
||||
func (c *Client) EnsureBucket(ctx context.Context) error {
|
||||
// Open builds the storage backend selected by configuration: S3-compatible by
|
||||
// default, or the local file system when STORAGE_BACKEND=fs. Every backend
|
||||
// delivers blobs through the app using a shared Signer. jwtSecret is the
|
||||
// application secret used to sign object URLs when URL_SECRET is not set.
|
||||
func Open(ctx context.Context, cfg *config.Config, jwtSecret string) (Backend, error) {
|
||||
secret := cfg.Storage.URLSecret
|
||||
if secret == "" {
|
||||
secret = jwtSecret
|
||||
}
|
||||
if secret == "" {
|
||||
return nil, errors.New("object URL secret is not available")
|
||||
}
|
||||
signer, err := NewSigner(cfg.Server.BaseURL, secret, cfg.Storage.URLTTL)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
switch cfg.Storage.Backend {
|
||||
case "", "s3":
|
||||
return NewS3(ctx, cfg.S3, signer)
|
||||
case "fs":
|
||||
return NewFS(cfg.Storage.FSRoot, signer)
|
||||
default:
|
||||
return nil, fmt.Errorf("unknown storage backend %q", cfg.Storage.Backend)
|
||||
}
|
||||
}
|
||||
|
||||
func (c *S3) EnsureBucket(ctx context.Context) error {
|
||||
_, err := c.s3.HeadBucket(ctx, &s3.HeadBucketInput{Bucket: aws.String(c.bucket)})
|
||||
if err == nil {
|
||||
return nil
|
||||
@@ -66,12 +137,12 @@ func (c *Client) EnsureBucket(ctx context.Context) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *Client) Ping(ctx context.Context) error {
|
||||
func (c *S3) Ping(ctx context.Context) error {
|
||||
_, err := c.s3.HeadBucket(ctx, &s3.HeadBucketInput{Bucket: aws.String(c.bucket)})
|
||||
return err
|
||||
}
|
||||
|
||||
func (c *Client) Put(ctx context.Context, key string, body io.Reader, size int64, contentType string) error {
|
||||
func (c *S3) Put(ctx context.Context, key string, body io.Reader, size int64, contentType string) error {
|
||||
in := &s3.PutObjectInput{
|
||||
Bucket: aws.String(c.bucket),
|
||||
Key: aws.String(key),
|
||||
@@ -87,7 +158,7 @@ func (c *Client) Put(ctx context.Context, key string, body io.Reader, size int64
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *Client) Delete(ctx context.Context, keys ...string) error {
|
||||
func (c *S3) Delete(ctx context.Context, keys ...string) error {
|
||||
if len(keys) == 0 {
|
||||
return nil
|
||||
}
|
||||
@@ -104,25 +175,7 @@ func (c *Client) Delete(ctx context.Context, keys ...string) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *Client) DeletePrefix(ctx context.Context, prefix string) error {
|
||||
var keys []string
|
||||
paginator := s3.NewListObjectsV2Paginator(c.s3, &s3.ListObjectsV2Input{
|
||||
Bucket: aws.String(c.bucket),
|
||||
Prefix: aws.String(prefix),
|
||||
})
|
||||
for paginator.HasMorePages() {
|
||||
page, err := paginator.NextPage(ctx)
|
||||
if err != nil {
|
||||
return fmt.Errorf("list objects: %w", err)
|
||||
}
|
||||
for _, o := range page.Contents {
|
||||
keys = append(keys, aws.ToString(o.Key))
|
||||
}
|
||||
}
|
||||
return c.Delete(ctx, keys...)
|
||||
}
|
||||
|
||||
func (c *Client) Get(ctx context.Context, key string) ([]byte, error) {
|
||||
func (c *S3) Get(ctx context.Context, key string) ([]byte, error) {
|
||||
out, err := c.s3.GetObject(ctx, &s3.GetObjectInput{
|
||||
Bucket: aws.String(c.bucket),
|
||||
Key: aws.String(key),
|
||||
@@ -138,14 +191,127 @@ func (c *Client) Get(ctx context.Context, key string) ([]byte, error) {
|
||||
return data, nil
|
||||
}
|
||||
|
||||
func (c *Client) PresignGet(ctx context.Context, key string) (string, error) { out, err := c.presign.PresignGetObject(ctx, &s3.GetObjectInput{
|
||||
// URL returns a signed app URL. The blob itself is streamed by ServeObject, so
|
||||
// the S3 endpoint is never exposed to the client.
|
||||
func (c *S3) URL(_ context.Context, key string) (string, error) {
|
||||
return c.signer.URL(key), nil
|
||||
}
|
||||
|
||||
// List returns every object under prefix using paginated ListObjectsV2.
|
||||
func (c *S3) List(ctx context.Context, prefix string) ([]ObjectInfo, error) {
|
||||
var out []ObjectInfo
|
||||
var token *string
|
||||
for {
|
||||
in := &s3.ListObjectsV2Input{Bucket: aws.String(c.bucket)}
|
||||
if prefix != "" {
|
||||
in.Prefix = aws.String(prefix)
|
||||
}
|
||||
if token != nil {
|
||||
in.ContinuationToken = token
|
||||
}
|
||||
res, err := c.s3.ListObjectsV2(ctx, in)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("list objects: %w", err)
|
||||
}
|
||||
for _, o := range res.Contents {
|
||||
out = append(out, ObjectInfo{
|
||||
Key: aws.ToString(o.Key),
|
||||
Size: aws.ToInt64(o.Size),
|
||||
ModTime: aws.ToTime(o.LastModified),
|
||||
})
|
||||
}
|
||||
if res.IsTruncated == nil || !*res.IsTruncated {
|
||||
return out, nil
|
||||
}
|
||||
token = res.NextContinuationToken
|
||||
}
|
||||
}
|
||||
|
||||
// ServeObject validates the signed URL and streams the blob from S3. It passes
|
||||
// the client's Range header through to S3, so byte ranges are handled natively,
|
||||
// and answers HEAD via HeadObject. It writes the response itself.
|
||||
func (c *S3) ServeObject(w http.ResponseWriter, r *http.Request, key string) error {
|
||||
if err := c.signer.VerifyRequest(key, r); err != nil {
|
||||
return err
|
||||
}
|
||||
if r.Method == http.MethodHead {
|
||||
return c.headObject(w, r, key)
|
||||
}
|
||||
in := &s3.GetObjectInput{Bucket: aws.String(c.bucket), Key: aws.String(key)}
|
||||
if rng := r.Header.Get("Range"); rng != "" {
|
||||
in.Range = aws.String(rng)
|
||||
}
|
||||
out, err := c.s3.GetObject(r.Context(), in)
|
||||
if err != nil {
|
||||
if isNoSuchKey(err) {
|
||||
return ErrObjectNotFound
|
||||
}
|
||||
if isInvalidRange(err) {
|
||||
return ErrRangeNotSatisfiable
|
||||
}
|
||||
return err
|
||||
}
|
||||
defer out.Body.Close()
|
||||
if ct := aws.ToString(out.ContentType); ct != "" {
|
||||
w.Header().Set("Content-Type", ct)
|
||||
} else {
|
||||
w.Header().Set("Content-Type", contentTypeForKey(key))
|
||||
}
|
||||
if out.ContentLength != nil {
|
||||
w.Header().Set("Content-Length", strconv.FormatInt(*out.ContentLength, 10))
|
||||
}
|
||||
if out.ContentRange != nil {
|
||||
w.Header().Set("Content-Range", *out.ContentRange)
|
||||
}
|
||||
status := http.StatusOK
|
||||
if out.ContentRange != nil {
|
||||
status = http.StatusPartialContent
|
||||
}
|
||||
w.WriteHeader(status)
|
||||
if _, err := io.Copy(w, out.Body); err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *S3) headObject(w http.ResponseWriter, r *http.Request, key string) error {
|
||||
out, err := c.s3.HeadObject(r.Context(), &s3.HeadObjectInput{
|
||||
Bucket: aws.String(c.bucket),
|
||||
Key: aws.String(key),
|
||||
}, s3.WithPresignExpires(c.presignTTL))
|
||||
})
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("presign %s: %w", key, err)
|
||||
if isNoSuchKey(err) {
|
||||
return ErrObjectNotFound
|
||||
}
|
||||
return out.URL, nil
|
||||
return err
|
||||
}
|
||||
if ct := aws.ToString(out.ContentType); ct != "" {
|
||||
w.Header().Set("Content-Type", ct)
|
||||
} else {
|
||||
w.Header().Set("Content-Type", contentTypeForKey(key))
|
||||
}
|
||||
if out.ContentLength != nil {
|
||||
w.Header().Set("Content-Length", strconv.FormatInt(*out.ContentLength, 10))
|
||||
}
|
||||
w.WriteHeader(http.StatusOK)
|
||||
return nil
|
||||
}
|
||||
|
||||
func isNoSuchKey(err error) bool {
|
||||
var apiErr smithy.APIError
|
||||
if errors.As(err, &apiErr) {
|
||||
code := apiErr.ErrorCode()
|
||||
return strings.EqualFold(code, "NoSuchKey") || strings.EqualFold(code, "NotFound")
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func isInvalidRange(err error) bool {
|
||||
var apiErr smithy.APIError
|
||||
if errors.As(err, &apiErr) {
|
||||
return strings.EqualFold(apiErr.ErrorCode(), "InvalidRange")
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func isNoSuchBucket(err error) bool {
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
// Package store builds the per-domain metadata repositories for a chosen
|
||||
// backend. Domains depend on their Repository interfaces; this factory is the
|
||||
// single place that knows the concrete implementations.
|
||||
package store
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
|
||||
"shared-album/internal/albums"
|
||||
"shared-album/internal/auth"
|
||||
"shared-album/internal/members"
|
||||
"shared-album/internal/notifications"
|
||||
"shared-album/internal/photos"
|
||||
"shared-album/internal/settings"
|
||||
"shared-album/internal/sharelinks"
|
||||
"shared-album/internal/social"
|
||||
)
|
||||
|
||||
// Repos bundles one repository per domain.
|
||||
type Repos struct {
|
||||
Auth auth.Repository
|
||||
Albums albums.Repository
|
||||
Members members.Repository
|
||||
Photos photos.Repository
|
||||
Social social.Repository
|
||||
Notifications notifications.Repository
|
||||
ShareLinks sharelinks.Repository
|
||||
Settings settings.Repository
|
||||
}
|
||||
|
||||
// NewPostgres builds the repositories backed by PostgreSQL.
|
||||
func NewPostgres(pool *pgxpool.Pool) Repos {
|
||||
return Repos{
|
||||
Auth: auth.NewStore(pool),
|
||||
Albums: albums.NewStore(pool),
|
||||
Members: members.NewStore(pool),
|
||||
Photos: photos.NewStore(pool),
|
||||
Social: social.NewStore(pool),
|
||||
Notifications: notifications.NewStore(pool),
|
||||
ShareLinks: sharelinks.NewStore(pool),
|
||||
Settings: settings.NewStore(pool),
|
||||
}
|
||||
}
|
||||
|
||||
// NewSQLite builds the repositories backed by SQLite.
|
||||
func NewSQLite(db *sql.DB) Repos {
|
||||
return Repos{
|
||||
Auth: auth.NewSQLiteStore(db),
|
||||
Albums: albums.NewSQLiteStore(db),
|
||||
Members: members.NewSQLiteStore(db),
|
||||
Photos: photos.NewSQLiteStore(db),
|
||||
Social: social.NewSQLiteStore(db),
|
||||
Notifications: notifications.NewSQLiteStore(db),
|
||||
ShareLinks: sharelinks.NewSQLiteStore(db),
|
||||
Settings: settings.NewSQLiteStore(db),
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
|
||||
assets "shared-album"
|
||||
"shared-album/internal/db"
|
||||
)
|
||||
|
||||
func assertComplete(t *testing.T, r Repos) {
|
||||
t.Helper()
|
||||
if r.Auth == nil || r.Albums == nil || r.Members == nil || r.Photos == nil ||
|
||||
r.Social == nil || r.Notifications == nil || r.ShareLinks == nil {
|
||||
t.Fatalf("incomplete repository bundle: %+v", r)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewSQLite(t *testing.T) {
|
||||
path := filepath.Join(t.TempDir(), "meta.db")
|
||||
migFS, err := fs.Sub(assets.Migrations, "migrations/sqlite")
|
||||
if err != nil {
|
||||
t.Fatalf("migrations fs: %v", err)
|
||||
}
|
||||
if err := db.MigrateSQLite(path, migFS); err != nil {
|
||||
t.Fatalf("migrate: %v", err)
|
||||
}
|
||||
database, err := db.OpenSQLite(path)
|
||||
if err != nil {
|
||||
t.Fatalf("open: %v", err)
|
||||
}
|
||||
defer database.Close()
|
||||
assertComplete(t, NewSQLite(database))
|
||||
}
|
||||
|
||||
func TestNewPostgres(t *testing.T) {
|
||||
dsn := os.Getenv("TEST_DATABASE_URL")
|
||||
if dsn == "" {
|
||||
t.Skip("TEST_DATABASE_URL not set")
|
||||
}
|
||||
pool, err := pgxpool.New(context.Background(), dsn)
|
||||
if err != nil {
|
||||
t.Skipf("cannot connect: %v", err)
|
||||
}
|
||||
if err := pool.Ping(context.Background()); err != nil {
|
||||
t.Skipf("cannot ping: %v", err)
|
||||
}
|
||||
t.Cleanup(pool.Close)
|
||||
assertComplete(t, NewPostgres(pool))
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
DROP TABLE IF EXISTS variant_jobs;
|
||||
DROP TABLE IF EXISTS photo_objects;
|
||||
DROP TABLE IF EXISTS objects;
|
||||
@@ -0,0 +1,39 @@
|
||||
-- Physical objects stored in S3. Keys are opaque and never encode album, photo
|
||||
-- or any other metadata entity. The row is the ledger entry that governs the
|
||||
-- lifecycle of the corresponding blob.
|
||||
CREATE TABLE IF NOT EXISTS objects (
|
||||
id text PRIMARY KEY,
|
||||
storage_key text NOT NULL UNIQUE,
|
||||
mime_type text NOT NULL DEFAULT '',
|
||||
size bigint NOT NULL DEFAULT 0,
|
||||
width integer NOT NULL DEFAULT 0,
|
||||
height integer NOT NULL DEFAULT 0,
|
||||
checksum text NOT NULL DEFAULT '',
|
||||
state text NOT NULL DEFAULT 'pending'
|
||||
CHECK (state IN ('pending', 'ready', 'deleting')),
|
||||
created_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS objects_state_created_idx ON objects (state, created_at);
|
||||
|
||||
-- Binding of a photo to its objects by role. The role is a property of the
|
||||
-- binding, not of the object itself.
|
||||
CREATE TABLE IF NOT EXISTS photo_objects (
|
||||
photo_id text NOT NULL REFERENCES photos(id) ON DELETE CASCADE,
|
||||
object_id text NOT NULL REFERENCES objects(id) ON DELETE RESTRICT,
|
||||
role text NOT NULL CHECK (role IN ('original', 'thumb', 'preview')),
|
||||
PRIMARY KEY (photo_id, role)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS photo_objects_object_idx ON photo_objects (object_id);
|
||||
|
||||
-- Durable queue of pending variant generation work, one job per photo.
|
||||
CREATE TABLE IF NOT EXISTS variant_jobs (
|
||||
photo_id text PRIMARY KEY REFERENCES photos(id) ON DELETE CASCADE,
|
||||
attempts integer NOT NULL DEFAULT 0,
|
||||
next_attempt_at timestamptz NOT NULL DEFAULT now(),
|
||||
last_error text NOT NULL DEFAULT '',
|
||||
created_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS variant_jobs_due_idx ON variant_jobs (next_attempt_at);
|
||||
@@ -0,0 +1,8 @@
|
||||
ALTER TABLE photos
|
||||
ADD COLUMN IF NOT EXISTS storage_key text NOT NULL DEFAULT '',
|
||||
ADD COLUMN IF NOT EXISTS thumb_key text,
|
||||
ADD COLUMN IF NOT EXISTS preview_key text,
|
||||
ADD COLUMN IF NOT EXISTS size bigint NOT NULL DEFAULT 0,
|
||||
ADD COLUMN IF NOT EXISTS width integer NOT NULL DEFAULT 0,
|
||||
ADD COLUMN IF NOT EXISTS height integer NOT NULL DEFAULT 0,
|
||||
ADD COLUMN IF NOT EXISTS mime_type text NOT NULL DEFAULT '';
|
||||
@@ -0,0 +1,10 @@
|
||||
-- Physical facts move to the objects table; photos keeps only membership and
|
||||
-- presentation data. size/width/height/mime_type are derived from the original
|
||||
-- object via photo_objects + objects.
|
||||
ALTER TABLE photos DROP COLUMN IF EXISTS storage_key;
|
||||
ALTER TABLE photos DROP COLUMN IF EXISTS thumb_key;
|
||||
ALTER TABLE photos DROP COLUMN IF EXISTS preview_key;
|
||||
ALTER TABLE photos DROP COLUMN IF EXISTS size;
|
||||
ALTER TABLE photos DROP COLUMN IF EXISTS width;
|
||||
ALTER TABLE photos DROP COLUMN IF EXISTS height;
|
||||
ALTER TABLE photos DROP COLUMN IF EXISTS mime_type;
|
||||
@@ -0,0 +1 @@
|
||||
DROP TABLE IF EXISTS app_settings;
|
||||
@@ -0,0 +1,6 @@
|
||||
CREATE TABLE IF NOT EXISTS app_settings (
|
||||
key text PRIMARY KEY,
|
||||
value text NOT NULL,
|
||||
created_at timestamptz NOT NULL DEFAULT now(),
|
||||
updated_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
@@ -0,0 +1,15 @@
|
||||
DROP TABLE IF EXISTS variant_jobs;
|
||||
DROP TABLE IF EXISTS photo_objects;
|
||||
DROP TABLE IF EXISTS objects;
|
||||
DROP TABLE IF EXISTS photo_tags;
|
||||
DROP TABLE IF EXISTS tags;
|
||||
DROP TABLE IF EXISTS refresh_tokens;
|
||||
DROP TABLE IF EXISTS share_links;
|
||||
DROP TABLE IF EXISTS notifications;
|
||||
DROP TABLE IF EXISTS likes;
|
||||
DROP TABLE IF EXISTS comments;
|
||||
DROP TABLE IF EXISTS invites;
|
||||
DROP TABLE IF EXISTS album_members;
|
||||
DROP TABLE IF EXISTS photos;
|
||||
DROP TABLE IF EXISTS albums;
|
||||
DROP TABLE IF EXISTS users;
|
||||
@@ -0,0 +1,159 @@
|
||||
-- Consolidated SQLite schema equivalent to the PostgreSQL migration history.
|
||||
CREATE TABLE users (
|
||||
id TEXT PRIMARY KEY,
|
||||
email TEXT NOT NULL UNIQUE,
|
||||
password_hash TEXT NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
avatar_url TEXT,
|
||||
preferences TEXT NOT NULL DEFAULT '{}',
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
|
||||
);
|
||||
|
||||
CREATE TABLE albums (
|
||||
id TEXT PRIMARY KEY,
|
||||
owner_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
title TEXT NOT NULL,
|
||||
description TEXT NOT NULL DEFAULT '',
|
||||
cover_photo_id TEXT REFERENCES photos(id) ON DELETE SET NULL,
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
|
||||
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
|
||||
);
|
||||
|
||||
CREATE TABLE photos (
|
||||
id TEXT PRIMARY KEY,
|
||||
album_id TEXT REFERENCES albums(id) ON DELETE SET NULL,
|
||||
uploader_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
original_filename TEXT NOT NULL DEFAULT '',
|
||||
taken_at TEXT,
|
||||
latitude REAL,
|
||||
longitude REAL,
|
||||
caption TEXT NOT NULL DEFAULT '',
|
||||
exif TEXT NOT NULL DEFAULT '{}',
|
||||
deleted_at TEXT,
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
|
||||
);
|
||||
|
||||
CREATE INDEX photos_album_created_idx ON photos (album_id, created_at DESC, id DESC);
|
||||
CREATE INDEX photos_deleted_idx ON photos (deleted_at) WHERE deleted_at IS NOT NULL;
|
||||
|
||||
CREATE TABLE album_members (
|
||||
album_id TEXT NOT NULL REFERENCES albums(id) ON DELETE CASCADE,
|
||||
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
role TEXT NOT NULL,
|
||||
invited_by TEXT REFERENCES users(id) ON DELETE SET NULL,
|
||||
joined_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
|
||||
PRIMARY KEY (album_id, user_id)
|
||||
);
|
||||
|
||||
CREATE TABLE invites (
|
||||
id TEXT PRIMARY KEY,
|
||||
album_id TEXT NOT NULL REFERENCES albums(id) ON DELETE CASCADE,
|
||||
token TEXT NOT NULL UNIQUE,
|
||||
role TEXT NOT NULL,
|
||||
created_by TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
expires_at TEXT NOT NULL,
|
||||
max_uses INTEGER NOT NULL DEFAULT 1,
|
||||
uses INTEGER NOT NULL DEFAULT 0,
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
|
||||
);
|
||||
|
||||
CREATE TABLE comments (
|
||||
id TEXT PRIMARY KEY,
|
||||
photo_id TEXT NOT NULL REFERENCES photos(id) ON DELETE CASCADE,
|
||||
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
text TEXT NOT NULL,
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
|
||||
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
|
||||
);
|
||||
|
||||
CREATE INDEX comments_photo_idx ON comments (photo_id, created_at);
|
||||
|
||||
CREATE TABLE likes (
|
||||
photo_id TEXT NOT NULL REFERENCES photos(id) ON DELETE CASCADE,
|
||||
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
|
||||
PRIMARY KEY (photo_id, user_id)
|
||||
);
|
||||
|
||||
CREATE TABLE notifications (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
type TEXT NOT NULL,
|
||||
payload_json TEXT NOT NULL DEFAULT '{}',
|
||||
read_at TEXT,
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
|
||||
);
|
||||
|
||||
CREATE INDEX notifications_user_idx ON notifications (user_id, created_at DESC);
|
||||
|
||||
CREATE TABLE share_links (
|
||||
id TEXT PRIMARY KEY,
|
||||
album_id TEXT NOT NULL REFERENCES albums(id) ON DELETE CASCADE,
|
||||
token TEXT NOT NULL UNIQUE,
|
||||
created_by TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
|
||||
expires_at TEXT NOT NULL,
|
||||
revoked_at TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE refresh_tokens (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
token_hash TEXT NOT NULL UNIQUE,
|
||||
expires_at TEXT NOT NULL,
|
||||
revoked_at TEXT,
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
|
||||
);
|
||||
|
||||
CREATE INDEX refresh_tokens_user_idx ON refresh_tokens (user_id);
|
||||
|
||||
CREATE TABLE tags (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
name TEXT NOT NULL,
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX tags_user_name_idx ON tags (user_id, lower(name));
|
||||
|
||||
CREATE TABLE photo_tags (
|
||||
photo_id TEXT NOT NULL REFERENCES photos(id) ON DELETE CASCADE,
|
||||
tag_id TEXT NOT NULL REFERENCES tags(id) ON DELETE CASCADE,
|
||||
PRIMARY KEY (photo_id, tag_id)
|
||||
);
|
||||
|
||||
CREATE INDEX photo_tags_tag_idx ON photo_tags (tag_id);
|
||||
|
||||
CREATE TABLE objects (
|
||||
id TEXT PRIMARY KEY,
|
||||
storage_key TEXT NOT NULL UNIQUE,
|
||||
mime_type TEXT NOT NULL DEFAULT '',
|
||||
size INTEGER NOT NULL DEFAULT 0,
|
||||
width INTEGER NOT NULL DEFAULT 0,
|
||||
height INTEGER NOT NULL DEFAULT 0,
|
||||
checksum TEXT NOT NULL DEFAULT '',
|
||||
state TEXT NOT NULL DEFAULT 'pending'
|
||||
CHECK (state IN ('pending', 'ready', 'deleting')),
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
|
||||
);
|
||||
|
||||
CREATE INDEX objects_state_created_idx ON objects (state, created_at);
|
||||
|
||||
CREATE TABLE photo_objects (
|
||||
photo_id TEXT NOT NULL REFERENCES photos(id) ON DELETE CASCADE,
|
||||
object_id TEXT NOT NULL REFERENCES objects(id) ON DELETE RESTRICT,
|
||||
role TEXT NOT NULL CHECK (role IN ('original', 'thumb', 'preview')),
|
||||
PRIMARY KEY (photo_id, role)
|
||||
);
|
||||
|
||||
CREATE INDEX photo_objects_object_idx ON photo_objects (object_id);
|
||||
|
||||
CREATE TABLE variant_jobs (
|
||||
photo_id TEXT PRIMARY KEY REFERENCES photos(id) ON DELETE CASCADE,
|
||||
attempts INTEGER NOT NULL DEFAULT 0,
|
||||
next_attempt_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
|
||||
last_error TEXT NOT NULL DEFAULT '',
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
|
||||
);
|
||||
|
||||
CREATE INDEX variant_jobs_due_idx ON variant_jobs (next_attempt_at);
|
||||
@@ -0,0 +1 @@
|
||||
DROP TABLE IF EXISTS app_settings;
|
||||
@@ -0,0 +1,6 @@
|
||||
CREATE TABLE app_settings (
|
||||
key TEXT PRIMARY KEY,
|
||||
value TEXT NOT NULL,
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
|
||||
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
|
||||
);
|
||||
@@ -21,6 +21,14 @@ server {
|
||||
chunked_transfer_encoding on;
|
||||
}
|
||||
|
||||
location /api/objects/ {
|
||||
proxy_pass http://shared_album;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_buffering off; # стриминг объектов без буферизации
|
||||
proxy_cache off;
|
||||
}
|
||||
|
||||
location / {
|
||||
proxy_pass http://shared_album;
|
||||
proxy_http_version 1.1;
|
||||
|
||||
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-10-10
|
||||
@@ -0,0 +1,134 @@
|
||||
# Design
|
||||
|
||||
## Context
|
||||
|
||||
См. `proposal.md` — Why. Текущее состояние, которое формирует подход:
|
||||
|
||||
- `photos` хранит `storage_key`, `thumb_key`, `preview_key` (`migrations/000001_init.up.sql:45-47`),
|
||||
а ключи строятся как `albums/{album_id}/...` (`internal/photos/service.go:72,114,115`).
|
||||
- `Clone` копирует строку фото вместе с ключами, `MoveBatch` меняет только `album_id`,
|
||||
`PermanentDelete` удаляет объекты по счётчику ссылок `StorageKeyRefCount`
|
||||
(`internal/photos/store.go:201`).
|
||||
- `DeletePrefix` (`internal/storage/storage.go:107`) существует, но не вызывается.
|
||||
- Очередь вариантов отсутствует: генерация запускается goroutine из HTTP-запроса
|
||||
(`internal/photos/service.go:98`), результат не переживает рестарт.
|
||||
- Внешний API уже отдаёт `thumb_url/preview_url/original_url` и
|
||||
`size/width/height/mime_type`; это контракт, который надо сохранить.
|
||||
- В БД только дев-данные, миграция может быть разрушительной.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Убрать сущность «альбом» (и любую модель метаданных) из ключей и операций S3.
|
||||
- Сделать БД журналом истины: физический объект существует, пока на него есть ссылка.
|
||||
- Дать явный, устойчивый к сбоям сборщик мусора и самовосстанавливающуюся генерацию вариантов.
|
||||
- Сохранить внешний JSON API без изменений.
|
||||
|
||||
**Non-Goals:**
|
||||
- Content-addressing и дедупликация одинаковых байтов между разными загрузками
|
||||
(задел — колонка `checksum`, политика позже).
|
||||
- Мультивариантность сверх фиксированных original/thumb/preview (join-таблица это выдержит).
|
||||
- Перенос существующих данных: миграция разрушительная.
|
||||
|
||||
## Decisions
|
||||
|
||||
### D1. Join-таблица `photo_objects` вместо колонок-FK
|
||||
Фото ссылается на объекты через `photo_objects(photo_id, object_id, role)` с
|
||||
`PK(photo_id, role)`. Роль — свойство **связки**, не объекта: сам объект не знает,
|
||||
кто он. Альтернатива (три FK-колонки на `photos`) проще, но фиксирует ровно три
|
||||
варианта и снова смешивает представление с физикой. Выбран вариант, который
|
||||
расширяется до нескольких размеров/видео без переделки схемы.
|
||||
|
||||
### D2. Разделение фактов: физические vs логические
|
||||
`objects` — только физические факты: `storage_key`, `mime_type`, `size`,
|
||||
`width`, `height`, `checksum`, `state`, `created_at`.
|
||||
`photos` — членство и представление: `album_id`, `uploader_id`,
|
||||
`original_filename`, `caption`, `exif`, `taken_at`, `latitude`, `longitude`,
|
||||
`deleted_at`, `created_at`.
|
||||
`size/width/height/mime_type` в API выводятся джойном к объекту-оригиналу.
|
||||
|
||||
```sql
|
||||
objects(
|
||||
id text pk, storage_key text unique not null, mime_type text,
|
||||
size bigint, width int, height int, checksum text,
|
||||
state text, created_at timestamptz)
|
||||
|
||||
photo_objects(
|
||||
photo_id -> photos(id) on delete cascade,
|
||||
object_id -> objects(id) on delete restrict,
|
||||
role text check (role in ('original','thumb','preview')),
|
||||
primary key (photo_id, role))
|
||||
create index on photo_objects(object_id);
|
||||
|
||||
variant_jobs(
|
||||
photo_id -> photos(id) on delete cascade,
|
||||
attempts int, next_attempt_at timestamptz, last_error text,
|
||||
primary key (photo_id))
|
||||
```
|
||||
|
||||
У `photos` удаляются `storage_key`, `thumb_key`, `preview_key`.
|
||||
|
||||
### D3. Album-free ключи, случайный id объекта
|
||||
Ключ: `objects/{object_id}/{role}.{ext}`, где `object_id` — случайный
|
||||
(`auth.NewID()`). Роль в ключе — косметика для отладки, истина о роли — в
|
||||
`photo_objects`. Бакет, region, path-style не меняются. `DeletePrefix` удаляется.
|
||||
|
||||
### D4. Ledger состояний и явный GC
|
||||
Объект: `pending -> ready -> deleting`. Порядок операций — **сначала строка, потом
|
||||
Put**, поэтому неубиваемый мусор «файл есть, строки нет» не возникает.
|
||||
|
||||
```
|
||||
Upload : INSERT objects(pending) -> S3 Put -> ready -> bind original + INSERT variant_jobs
|
||||
Copy : INSERT photos + bind те же objects -- 0 S3
|
||||
Move : UPDATE photos.album_id -- 0 S3
|
||||
Delete : tx DELETE photos -> cascade unbind -> mark orphaned 'deleting'
|
||||
Worker : deleting -> re-check 0 ссылок -> S3 Delete -> DELETE row
|
||||
pending старше T (Put не завершился) -> deleting -> S3 Delete -> DELETE row
|
||||
```
|
||||
|
||||
Ссылочные гонки закрываются блокировками:
|
||||
- Пометить `deleting` можно только объект с нулём связок, под `SELECT ... FOR UPDATE`.
|
||||
- Перед S3 Delete воркер **перепроверяет** ноль связок под `FOR UPDATE`.
|
||||
- Привязка объекта (bind) берёт `FOR UPDATE` и отвергает `state != 'ready'`.
|
||||
|
||||
### D5. Генерация вариантов через `variant_jobs`
|
||||
Вместо goroutine из запроса — строка задания в той же транзакции, что и загрузка.
|
||||
Воркер забирает задания через `FOR UPDATE SKIP LOCKED`, регенерирует thumb/preview,
|
||||
создаёт объекты (pending->Put->ready), вставляет связки, публикует SSE
|
||||
`photo.created`, при ошибке увеличивает `attempts` и переносит `next_attempt_at`.
|
||||
Это даёт однократное выполнение, ретраи и восстановление после рестарта.
|
||||
|
||||
### D6. Один фоновый воркер в том же бинаре
|
||||
Петля-тикер внутри процесса (соответствует модели «один бинарь + systemd»):
|
||||
(1) доделать варианты, (2) удалить `deleting`, (3) дососить `pending`. Интервал и
|
||||
батч — конфигурируемые. Пометка объектов при удалении — синхронно в транзакции.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- [Двойная генерация вариантов после рестарта] → задание живёт в `variant_jobs`,
|
||||
забирается под `SKIP LOCKED`; повторное создание объекта безопасно, лишний объект
|
||||
соберёт GC.
|
||||
- [Удаление объекта, который вот-вот свяжут] → `FOR UPDATE` на объекте в bind и
|
||||
повторная проверка ссылок в GC.
|
||||
- [Строка `photos` есть, объекта нет при сбое Put] → состояние `pending` + sweeper.
|
||||
- [Джойн добавляет стоимость в список фото] → один join к объекту-оригиналу; индексы
|
||||
по `photo_objects(photo_id)` (PK) и `photo_objects(object_id)`.
|
||||
- [Ошибка в GC удалит живые байты необратимо] → удаляем только при доказанном нуле
|
||||
ссылок; удаление идемпотентно, состояние фиксируется до S3-операции.
|
||||
- [API-совместимость] → JSON-поля сохраняются, меняется только источник значений.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
Разрушительная (только дев-данные):
|
||||
|
||||
1. Миграция создаёт `objects`, `photo_objects`, `variant_jobs`.
|
||||
2. Миграция удаляет `photos.storage_key`, `photos.thumb_key`, `photos.preview_key`.
|
||||
3. Существующие строки фото и объекты S3 не переносятся.
|
||||
4. Откат: down-миграция возвращает колонки и удаляет новые таблицы.
|
||||
5. `project.md` §10 приводится к новой структуре ключей.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Точные значения интервала воркера, размера батча и порога `T` для `pending` —
|
||||
подбираются при реализации, на схему и контракты не влияют.
|
||||
- Формат будущего дедупа по `checksum` (политика, не схема) — вне этого change.
|
||||
@@ -0,0 +1,49 @@
|
||||
# Proposal
|
||||
|
||||
## Why
|
||||
|
||||
Слой хранения S3 сейчас знает о сущности «альбом»: ключи объектов имеют вид
|
||||
`albums/{album_id}/original/{photo_id}.{ext}` (`photos/service.go:72,114,115`).
|
||||
Это протекание модели метаданных в физический слой: перенос фото оставляет ключ в
|
||||
старом namespace, копия делит объекты под префиксом исходного альбома, а
|
||||
неиспользуемый `DeletePrefix` (`storage/storage.go:107`) создаёт латентный риск
|
||||
удаления живых объектов. Нужно развести слои: S3 хранит непрозрачные объекты, а
|
||||
сущности и их жизненный цикл живут в БД.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Ввести таблицу `objects` — физические блобы (ключ, mime, размер, габариты,
|
||||
checksum, состояние) и `photo_objects` — связка photo↔object с ролью
|
||||
(`original|thumb|preview`).
|
||||
- **BREAKING**: удалить `storage_key`, `thumb_key`, `preview_key` из `photos`;
|
||||
ключи фото выводятся через джойн к объекту-оригиналу. Миграция разрушительная
|
||||
(в базе только дев-данные).
|
||||
- Сделать ключи S3 album-free: `objects/{id}/{role}.{ext}`, где `id` — случайный
|
||||
идентификатор объекта.
|
||||
- Ввести явный GC через ledger-состояния объектов `pending | ready | deleting`:
|
||||
физический объект удаляется только когда на него не осталось ни одной связки.
|
||||
- Перенести генерацию вариантов в фоновый воркер: таблица `variant_jobs`
|
||||
(ровно-один-раз, ретраи с backoff), воркер также удаляет осиротевшие объекты и
|
||||
досасывает зависшие `pending`. SSE `photo.created` публикует воркер.
|
||||
- Удалить `DeletePrefix` из клиента хранения — entity-namespace больше нет.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
- `media-objects`: физическое объектное хранилище с непрозрачными ключами, журнал
|
||||
объектов и их жизненный цикл, сборка мусора по отсутствию ссылок, привязка
|
||||
объектов к фото по ролям.
|
||||
|
||||
### Modified Capabilities
|
||||
- `photos`: уточняется семантика удаления (корзина не удаляет объекты хранилища) и
|
||||
разделение объектов между фото при переносе и копировании.
|
||||
|
||||
## Impact
|
||||
|
||||
- Backend: `internal/photos/{service,store}.go`, `internal/storage/storage.go`,
|
||||
`internal/albums/*` (косвенно, через ключи обложки), wiring фонового воркера,
|
||||
`migrations/` (новая схема + удаление колонок).
|
||||
- Данные: разрушительная миграция `photos` (dev-only), новые таблицы.
|
||||
- Внешний JSON API не меняется: `thumb_url/preview_url/original_url` и
|
||||
`size/width/height/mime_type` по-прежнему отдаются, но выводятся из объектов.
|
||||
- Документация: `project.md` §10 (структура в S3).
|
||||
@@ -0,0 +1,90 @@
|
||||
# Spec Delta
|
||||
|
||||
## Purpose
|
||||
|
||||
Физическое объектное хранилище с непрозрачными ключами и журналом объектов:
|
||||
привязка объектов к фото по ролям, совместное использование объектов несколькими
|
||||
фото и сборка мусора по отсутствию ссылок.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Непрозрачные ключи хранилища
|
||||
Слой хранения SHALL адресоваться непрозрачными ключами объектов, не содержащими
|
||||
идентификаторов сущностей метаданных (таких как альбом или фото); сопоставление
|
||||
ключей и сущностей SHALL храниться только в слое метаданных.
|
||||
|
||||
#### Scenario: Ключ без альбома
|
||||
- **WHEN** система сохраняет объект в хранилище
|
||||
- **THEN** ключ объекта не содержит идентификатора альбома
|
||||
|
||||
#### Scenario: Перенос не трогает ключи
|
||||
- **WHEN** фото переносится в другой альбом
|
||||
- **THEN** ключи его объектов остаются неизменными
|
||||
|
||||
### Requirement: Журнал объектов
|
||||
Каждый физический объект SHALL иметь ровно одну запись в слое метаданных с ключом,
|
||||
MIME-типом, размером, габаритами и состоянием; фото SHALL ссылаться на объекты по
|
||||
ролям original, thumb и preview.
|
||||
|
||||
#### Scenario: Привязка ролей
|
||||
- **WHEN** фото загружено и обработано
|
||||
- **THEN** фото ссылается на объект-оригинал и, когда они готовы, на объекты thumb и preview
|
||||
|
||||
#### Scenario: Оригинал без вариантов
|
||||
- **WHEN** варианты ещё не сгенерированы
|
||||
- **THEN** фото доступно с оригиналом, а thumb и preview отсутствуют
|
||||
|
||||
### Requirement: Совместные объекты
|
||||
Несколько фото SHALL иметь возможность ссылаться на один и тот же физический
|
||||
объект; перенос и копирование SHALL изменять только ссылки и SHALL не изменять
|
||||
объекты хранилища.
|
||||
|
||||
#### Scenario: Копирование
|
||||
- **WHEN** пользователь копирует фото в другой альбом
|
||||
- **THEN** создаётся новая запись, ссылающаяся на те же объекты, без операций с хранилищем
|
||||
|
||||
#### Scenario: Перенос
|
||||
- **WHEN** пользователь переносит фото в другой альбом
|
||||
- **THEN** изменяется только привязка к альбому, объекты хранилища не меняются
|
||||
|
||||
### Requirement: Сборка мусора по ссылкам
|
||||
Объект хранилища SHALL удаляться только когда на него не ссылается ни одно фото;
|
||||
удаление альбома и попадание фото в корзину SHALL сохранять объекты.
|
||||
|
||||
#### Scenario: Последняя ссылка удалена
|
||||
- **WHEN** фото удаляется навсегда и больше ни одно фото не ссылается на его объекты
|
||||
- **THEN** объекты удаляются из хранилища
|
||||
|
||||
#### Scenario: Остались ссылки
|
||||
- **WHEN** на объект ссылается другое фото
|
||||
- **THEN** объект сохраняется в хранилище
|
||||
|
||||
#### Scenario: Удаление альбома
|
||||
- **WHEN** владелец удаляет альбом
|
||||
- **THEN** объекты хранилища не удаляются
|
||||
|
||||
### Requirement: Устойчивость жизненного цикла объекта
|
||||
Объект SHALL иметь состояние жизненного цикла, и незавершённые операции SHALL
|
||||
самовосстанавливаться: неудачная загрузка не оставляет осиротевших объектов, а
|
||||
удаление объекта SHALL доводиться до конца после сбоя.
|
||||
|
||||
#### Scenario: Сбой загрузки
|
||||
- **WHEN** запись объекта создана, но загрузка в хранилище не завершилась
|
||||
- **THEN** объект очищается без ручного вмешательства
|
||||
|
||||
#### Scenario: Сбой удаления
|
||||
- **WHEN** удаление объекта из хранилища не удалось
|
||||
- **THEN** попытка повторяется до успешного удаления
|
||||
|
||||
### Requirement: Генерация вариантов
|
||||
Генерация thumb и preview SHALL выполняться через надёжную очередь заданий с
|
||||
однократным выполнением и повтором при сбое; готовый результат SHALL
|
||||
транслироваться подписчикам альбома.
|
||||
|
||||
#### Scenario: Повтор после рестарта
|
||||
- **WHEN** процесс перезапускается с незавершёнными заданиями
|
||||
- **THEN** генерация вариантов продолжается
|
||||
|
||||
#### Scenario: Событие после генерации
|
||||
- **WHEN** варианты фото готовы
|
||||
- **THEN** подписчикам альбома публикуется событие с этим фото
|
||||
@@ -0,0 +1,14 @@
|
||||
# Spec Delta
|
||||
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Удаление фото
|
||||
Автор SHALL иметь возможность удалить своё фото; владелец и админ SHALL иметь возможность удалить любое фото. Удаление SHALL помещать фото в корзину и SHALL не удалять объекты хранилища.
|
||||
|
||||
#### Scenario: Удаление своего фото
|
||||
- **WHEN** автор удаляет своё фото
|
||||
- **THEN** фото помещается в корзину, а объекты хранилища сохраняются
|
||||
|
||||
#### Scenario: Удаление чужого фото участником
|
||||
- **WHEN** участник с ролью member пытается удалить чужое фото
|
||||
- **THEN** система отклоняет запрос
|
||||
@@ -0,0 +1,37 @@
|
||||
# Tasks
|
||||
|
||||
## 1. Схема и миграции
|
||||
|
||||
- [x] 1.1 Добавить миграцию, создающую `objects`, `photo_objects` (PK `photo_id,role`, индекс `object_id`) и `variant_jobs`; проверить, что `make migrate` применяется и таблицы видны в БД
|
||||
- [x] 1.2 Добавить миграцию, удаляющую `photos.storage_key`, `photos.thumb_key`, `photos.preview_key`, `photos.size`, `photos.width`, `photos.height`, `photos.mime_type`; проверить, что up и down миграции проходят в обе стороны без ошибок
|
||||
|
||||
## 2. Слой хранения
|
||||
|
||||
- [x] 2.1 Удалить `DeletePrefix` из `internal/storage/storage.go`; проверить `go build ./...` и отсутствие вызовов метода в коде
|
||||
- [x] 2.2 Ввести хелпер ключа объекта вида `objects/{id}/{role}.{ext}` без идентификатора альбома; покрыть юнит-тестом, что ключ не содержит album_id
|
||||
|
||||
## 3. Метаданные объектов
|
||||
|
||||
- [x] 3.1 Реализовать стор объектов: создание `pending`, активация `ready`, пометка `deleting`, удаление строки, выборка `pending` старше порога; покрыть тестами переходы состояний
|
||||
- [x] 3.2 Реализовать привязку `photo_objects` с блокировкой объекта `FOR UPDATE` и отказом при `state != 'ready'`; тест на повторную привязку одного объекта к двум фото
|
||||
- [x] 3.3 Реализовать запрос GC (объекты `ready` без связок) и тесты сценариев «последняя ссылка удалена» и «остались ссылки»
|
||||
|
||||
## 4. Сервис фото
|
||||
|
||||
- [x] 4.1 Переписать `Upload`: строка объекта `pending` → S3 Put → `ready` → привязка `original` + задание в `variant_jobs`; тест, что загрузка даёт фото с URL оригинала и ключом без альбома
|
||||
- [x] 4.2 Переписать `List`/`Get`/`attachURLs` на вывод ключей и `size/width/height/mime_type` джойном к объекту-оригиналу; тест, что JSON-поля и URL присутствуют как раньше
|
||||
- [x] 4.3 Переписать `CopyBatch`/`MoveBatch` на работу только через связки; тест, что объекты хранилища не меняются и делятся между фото
|
||||
- [x] 4.4 Переписать `PermanentDelete`: удалить связки и пометить осиротевшие объекты `deleting`; тест, что объект с нулём ссылок становится кандидатом на GC
|
||||
- [x] 4.5 Убрать обращения к удалённым колонкам во всех запросах и моделях; проверить `go build ./...` и прохождение `internal/repo` тестов
|
||||
|
||||
## 5. Фоновый воркер
|
||||
|
||||
- [x] 5.1 Реализовать цикл забора `variant_jobs` через `FOR UPDATE SKIP LOCKED`, генерацию thumb/preview, привязку и публикацию SSE; тест обработки задания и повтора при ошибке
|
||||
- [x] 5.2 Реализовать в воркере удаление `deleting` (с повторной проверкой нуля ссылок) и досасывание зависших `pending`; тест повторной попытки S3 Delete и очистки `pending`
|
||||
- [x] 5.3 Встроить старт/останов воркера в жизненный цикл приложения; проверить, что сервер запускается и воркер выполняет петлю (по логам/интеграционному тесту)
|
||||
|
||||
## 6. Интеграция и документация
|
||||
|
||||
- [x] 6.1 Перевести обложку альбома (`CoverThumbKey`) на вывод через `photo_objects`+`objects`; тест, что список альбомов отдаёт URL обложки
|
||||
- [x] 6.2 Обновить `project.md` §10 (структура в S3 и слои); проверить, что описание совпадает с реализованными ключами
|
||||
- [x] 6.3 Сквозная проверка: загрузка → генерация вариантов → копирование между альбомами (0 операций S3) → перенос → окончательное удаление → сборка объектов; проверить, что объекты удаляются только при нуле ссылок
|
||||
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-10-11
|
||||
@@ -0,0 +1,30 @@
|
||||
# Proposal
|
||||
|
||||
## Why
|
||||
|
||||
Слои данных были по умолчанию внешними (`postgres` и `s3`), поэтому самодостаточный
|
||||
локальный запуск без внешних сервисов требовал явной настройки. Нужно сделать локальные
|
||||
бэкенды значением по умолчанию.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Сменить значения по умолчанию: `DB_BACKEND=sqlite` (`SQLITE_PATH=./galo.db`) и
|
||||
`STORAGE_BACKEND=fs` (`FS_ROOT=./data`).
|
||||
- Обновить `.env.example`, `project.md`, `README.md`.
|
||||
- Поведение при явном выборе `postgres`/`s3` не меняется.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
<!-- нет -->
|
||||
|
||||
### Modified Capabilities
|
||||
- `metadata-backends`: значением по умолчанию становится встроенная SQLite (ранее PostgreSQL).
|
||||
- `storage-backends`: значением по умолчанию становится локальная ФС (ранее S3-совместимая).
|
||||
|
||||
## Impact
|
||||
|
||||
- `backend/internal/config` (значения по умолчанию) и тесты.
|
||||
- `.env.example`, `project.md`, `README.md`.
|
||||
- **BREAKING (поведение):** развёртывания без явных `DB_BACKEND`/`STORAGE_BACKEND`
|
||||
переключатся на локальные бэкенды; для прежнего режима задайте `postgres`/`s3` явно.
|
||||
+23
@@ -0,0 +1,23 @@
|
||||
# Spec Delta
|
||||
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Выбор бэкенда метаданных
|
||||
Система SHALL поддерживать выбор бэкенда слоя метаданных параметром конфигурации,
|
||||
при этом встроенная SQLite SHALL быть значением по умолчанию.
|
||||
|
||||
#### Scenario: Выбор по умолчанию
|
||||
- **WHEN** бэкенд метаданных не задан в конфигурации
|
||||
- **THEN** система использует встроенную SQLite с путём по умолчанию
|
||||
|
||||
#### Scenario: Выбор PostgreSQL
|
||||
- **WHEN** в конфигурации задан бэкенд PostgreSQL
|
||||
- **THEN** метаданные читаются и пишутся во внешнюю PostgreSQL
|
||||
|
||||
#### Scenario: Выбор встроенной SQLite
|
||||
- **WHEN** в конфигурации задан бэкенд SQLite
|
||||
- **THEN** метаданные читаются и пишутся в локальный файл SQLite без внешней службы
|
||||
|
||||
#### Scenario: Неполная конфигурация
|
||||
- **WHEN** конфигурация выбранного бэкенда неполна
|
||||
- **THEN** система отказывается стартовать с явной ошибкой конфигурации
|
||||
+23
@@ -0,0 +1,23 @@
|
||||
# Spec Delta
|
||||
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Выбор бэкенда хранилища
|
||||
Система SHALL поддерживать выбор бэкенда объектного хранилища параметром конфигурации,
|
||||
при этом локальная файловая система SHALL быть значением по умолчанию.
|
||||
|
||||
#### Scenario: Выбор по умолчанию
|
||||
- **WHEN** бэкенд хранилища не задан в конфигурации
|
||||
- **THEN** система хранит и отдаёт блобы из локальной файловой системы с корнем по умолчанию
|
||||
|
||||
#### Scenario: Выбор S3-совместимого бэкенда
|
||||
- **WHEN** в конфигурации задан S3-совместимый бэкенд
|
||||
- **THEN** система хранит и отдаёт блобы через S3-совместимое хранилище
|
||||
|
||||
#### Scenario: Выбор локальной ФС
|
||||
- **WHEN** в конфигурации задан бэкенд локальной ФС
|
||||
- **THEN** система хранит и отдаёт блобы из каталога локальной файловой системы
|
||||
|
||||
#### Scenario: Неполная конфигурация
|
||||
- **WHEN** конфигурация выбранного бэкенда неполна
|
||||
- **THEN** система отказывается стартовать с явной ошибкой конфигурации
|
||||
@@ -0,0 +1,6 @@
|
||||
# Tasks
|
||||
|
||||
## 1. Дефолты и документация
|
||||
|
||||
- [x] 1.1 Сменить значения по умолчанию в `config` (`DB_BACKEND=sqlite`, `SQLITE_PATH=./galo.db`, `STORAGE_BACKEND=fs`, `FS_ROOT=./data`) и обновить тесты конфига; проверка — `go test ./internal/config` зелёный.
|
||||
- [x] 1.2 Обновить `.env.example`, `project.md`, `README.md`; проверка — запуск без переменных окружения стартует на SQLite+ФС и health показывает `db=ok, fs=ok`.
|
||||
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-10-11
|
||||
@@ -0,0 +1,107 @@
|
||||
# Design
|
||||
|
||||
## Context
|
||||
|
||||
См. `proposal.md` — Why. Текущее состояние, определяющее подход:
|
||||
|
||||
- `storage.Backend` уже введён; `storage.S3` возвращает presigned URL на Garage, а
|
||||
`storage.FS` отдаёт блобы через приложение. Presign уводит браузер напрямую в S3.
|
||||
- В проде Garage (`s3.localnet`) недоступен извне, поэтому presigned URL не работают:
|
||||
отдача обязана идти через бэкенд для обоих бэкендов.
|
||||
- Подпись URL сейчас живёт внутри `FS` (`fs.go: sign/verify/URL`).
|
||||
- Маршрут `/api/objects/*key` и `app.serveObject` уже есть и диспетчеризуются по
|
||||
необязательному интерфейсу `storage.ObjectServer`.
|
||||
- Ключи объектов непрозрачны (`objects/{id}/{role}{ext}`) и одинаковы для всех бэкендов.
|
||||
- Сборка `CGO_ENABLED=0`, один бинарник + systemd, перед Go стоит nginx.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Единая модель доставки: любой блоб клиент получает только через бэкенд.
|
||||
- Убрать presigned URL полностью; S3 отдаёт байты через прокси бэкенда.
|
||||
- Сохранить выбор бэкенда хранилища (`s3|fs`) и непрозрачные ключи.
|
||||
- Сохранить работу анонимных share-ссылок без пользовательской сессии.
|
||||
|
||||
**Non-Goals:**
|
||||
- Миграция существующих блобов между бэкендами.
|
||||
- Переключение бэкенда в рантайме.
|
||||
- Кэширующий CDN и раздача через внешний веб-сервер.
|
||||
- Изменение слоя метаданных (см. `sqlite-metadata-backend`).
|
||||
|
||||
## Decisions
|
||||
|
||||
### D1. `ServeObject` — обязательная часть `storage.Backend`
|
||||
Отдача через бэкенд нужна всем бэкендам, поэтому `ServeObject(w, r, key)` входит в
|
||||
интерфейс `Backend`, а не остаётся опциональным `ObjectServer`. Компилятор не даст
|
||||
собрать бэкенд без отдачи, а `app.serveObject` диспетчеризуется без type-assert.
|
||||
- Альтернатива: оставить `ObjectServer` опциональным — теперь это всегда реализуется,
|
||||
опциональность только маскирует ошибку.
|
||||
|
||||
### D2. `URL` всегда возвращает подписанный URL приложения
|
||||
`Backend.URL` больше не зависит от бэкенда в способе доставки: и `s3`, и `fs` возвращают
|
||||
`${APP_BASE_URL}/api/objects/<key>?e=<exp>&s=<sig>`. Presign-клиент AWS удаляется.
|
||||
- Альтернатива (dev): оставить presign для S3 — отвергнута по требованию «только через
|
||||
бэкенд» и чтобы не иметь двух режимов доставки.
|
||||
|
||||
### D3. Общий `Signer`
|
||||
HMAC-SHA256 от `key + exp` выносится в общий `Signer` (секрет `URL_SECRET` с fallback на
|
||||
`JWT_SECRET`, срок `OBJECT_URL_TTL`). Его используют и FS, и S3, и маршрут проверяет
|
||||
подпись одинаково. Внутри `FS` дублирующая подпись удаляется.
|
||||
- Альтернатива: дублировать HMAC в каждом бэкенде — риск рассинхрона проверки.
|
||||
|
||||
### D4. Отдача S3: ranged `GetObject` и стриминг
|
||||
S3-бэкенд в `ServeObject` определяет `Range` из запроса, вызывает `GetObject` с тем же
|
||||
диапазоном (Garage отдаёт `Content-Range`) и стримит тело в ответ через `io.Copy`
|
||||
(200/206/416). HEAD обслуживается `HeadObject`. Память не растёт, размер берётся из
|
||||
`Content-Length` ответа S3.
|
||||
- Альтернатива: `http.ServeContent` требует `io.ReadSeeker`; для S3 это ranged-GET-адаптер
|
||||
и лишний size-probe — отвергнуто ради простоты нативного passthrough.
|
||||
- `S3.Get` (в память) сохраняется только для воркера/backfill, отдачу не использует.
|
||||
|
||||
### D5. Отдача FS: без изменений
|
||||
`FS.ServeObject` остаётся на `http.ServeContent` (Range и Content-Type бесплатно).
|
||||
Меняется лишь источник подписи — общий `Signer`.
|
||||
|
||||
### D6. Публичный маршрут и доступ
|
||||
`/api/objects/*key` вне группы авторизации; подпись и срок — единственный контроль
|
||||
доступа, поэтому он обслуживает и авторизованных, и анонимные share-ссылки. Наружу не
|
||||
уходят ни S3-endpoint, ни AWS-подпись, ни ключ с секретом.
|
||||
|
||||
### D7. Конфигурация
|
||||
`STORAGE_BACKEND=s3|fs` без изменений (дефолт `s3`). `FS_ROOT` — для `fs`.
|
||||
`S3_PRESIGN_TTL` переименовывается в `OBJECT_URL_TTL` (семантика: срок подписи URL).
|
||||
`S3_PUBLIC_URL` удаляется как неиспользуемая. `APP_BASE_URL` становится определяющим для
|
||||
корректности ссылок; пустое значение даёт относительный `/api/objects/...` (полагается на
|
||||
nginx того же origin).
|
||||
|
||||
### D8. nginx для прокси-отдачи
|
||||
`location /api/objects/` обслуживается как `location /`, но с `proxy_buffering off` и
|
||||
`X-Accel-Buffering: no`, чтобы стриминг и Range не буферизовались. SSE-правило остаётся.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- [Все байты медиа идут через Go] → стриминг без чтения в память; `proxy_buffering off`;
|
||||
для крупных инсталляций это принятая плата за требование «только через бэкенд».
|
||||
- [Подписанный URL действует как bearer до истечения] → срок `OBJECT_URL_TTL` (1 час),
|
||||
отдельный `URL_SECRET` для отзыва.
|
||||
- [Range/HEAD через прокси] → passthrough `Range` в S3 и `HeadObject` для HEAD; покрыто
|
||||
httptest-тестами (200/206/416/404).
|
||||
- [Ломающее переименование `S3_PRESIGN_TTL`] → задокументировать в `.env.example` и
|
||||
`project.md`; при откате вернуть имя.
|
||||
- [Обход каталога через ключ FS] → нормализация пути и проверка префикса `FS_ROOT`.
|
||||
- [Смена бэкенда на существующих данных не переносит блобы] → документируем; миграция вне
|
||||
рамок.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
1. Вынести `Signer`, сделать `ServeObject` обязательным, `URL` — всегда app-signed.
|
||||
2. Реализовать S3 `ServeObject` (ranged `GetObject` + стриминг, HEAD).
|
||||
3. Удалить presign-путь и `S3_PUBLIC_URL`; переименовать TTL; обновить доки и nginx.
|
||||
4. Существующие развёртывания: дефолт `s3` сохраняется, но доставка теперь проксируется —
|
||||
внешний доступ к S3 больше не нужен.
|
||||
5. Перенос блобов между бэкендами не автоматизируется.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Нужно ли выставлять `Cache-Control` на ответы `/api/objects/` помимо браузерного кэша
|
||||
по уникальному URL.
|
||||
@@ -0,0 +1,40 @@
|
||||
# Proposal
|
||||
|
||||
## Why
|
||||
|
||||
S3-совместимое хранилище (Garage на `s3.localnet`) в проде недоступно извне, поэтому
|
||||
отдача медиа не может идти напрямую из хранилища в браузер. Нужен слой хранения,
|
||||
который всегда отдаёт блобы через бэкенд, и локальный бэкенд для инсталляций без S3.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Ввести интерфейс `storage.Backend`: Put/Delete/Get/Ping/EnsureBucket/Bucket, `URL`
|
||||
(всегда подписанный URL приложения) и обязательный `ServeObject`.
|
||||
- Реализовать бэкенд локальной ФС: блобы под корневым каталогом по непрозрачным ключам
|
||||
`objects/{id}/{role}{ext}`, защита от обхода пути, отдача через `ServeContent`.
|
||||
- Реализовать отдачу S3 через бэкенд: `ServeObject` тянет объект ranged `GetObject` из
|
||||
Garage (внутренняя сеть) и стримит браузеру; presigned URL больше не используются.
|
||||
- Публичный маршрут `/api/objects/<key>` с HMAC-подписью, сроком, Range и Content-Type
|
||||
обслуживает и авторизованных, и анонимные share-ссылки.
|
||||
- Ввести `STORAGE_BACKEND=s3|fs` и `FS_ROOT`; убрать `S3_PUBLIC_URL`; переименовать
|
||||
`S3_PRESIGN_TTL` → `OBJECT_URL_TTL`; дефолт — `s3`.
|
||||
- Сделать метку хранилища в health динамической.
|
||||
- Сделать формулировки отдачи файлов нейтральными к бэкенду в спеках `photos` и `share-links`.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
- `storage-backends`: выбор и работа объектного хранилища — S3-совместимое (Garage) и локальная ФС; блобы всегда отдаются через бэкенд по подписанному URL, хранилище клиенту не раскрывается.
|
||||
|
||||
### Modified Capabilities
|
||||
- `photos`: получение фото и скачивание не требуют presigned URL и обслуживаются бэкендом (подписанный URL).
|
||||
- `share-links`: просмотр по ссылке отдаёт подписанный URL, обслуживаемый бэкендом.
|
||||
|
||||
## Impact
|
||||
|
||||
- `backend/internal/storage` (интерфейс, общий `Signer`, FS-бэкенд, S3-proxy-отдача),
|
||||
сервисы `photos` и `albums`, `app` (маршрут отдачи, health), `cmd/*`, `config`.
|
||||
- Удаление presign-пути AWS SDK из `storage`.
|
||||
- `.env.example`, `project.md`, `deploy/nginx-sse.conf`.
|
||||
- Спеки `photos`, `share-links`, новая `storage-backends`.
|
||||
- Не затрагивает слой метаданных.
|
||||
@@ -0,0 +1,31 @@
|
||||
# Spec Delta
|
||||
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Получение фото по требованию
|
||||
Клиент SHALL иметь возможность получить актуальные данные фото с подписанным URL,
|
||||
обслуживаемым бэкендом, независимо от бэкенда хранения.
|
||||
|
||||
#### Scenario: Выдача свежего URL
|
||||
- **WHEN** клиент запрашивает фото по id
|
||||
- **THEN** система возвращает свежий подписанный URL бэкенда (срок до 1 часа) для просмотра
|
||||
|
||||
### Requirement: Скачивание одиночного фото
|
||||
Любой участник, включая viewer, SHALL иметь возможность скачать одиночное фото.
|
||||
|
||||
#### Scenario: Скачивание фото
|
||||
- **WHEN** участник запрашивает скачивание фото
|
||||
- **THEN** система возвращает подписанный URL оригинала, обслуживаемый бэкендом
|
||||
|
||||
### Requirement: Перенос и копирование фото между альбомами
|
||||
Пользователь с правом удаления в исходном альбоме и правом загрузки в целевом SHALL иметь
|
||||
возможность перенести или скопировать фото в другой альбом. Операции SHALL изменять только
|
||||
метаданные и SHALL не изменять объекты хранилища.
|
||||
|
||||
#### Scenario: Перенос
|
||||
- **WHEN** пользователь переносит фото в другой альбом
|
||||
- **THEN** фото исчезает из исходного альбома и появляется в целевом, объекты хранилища не меняются
|
||||
|
||||
#### Scenario: Копирование
|
||||
- **WHEN** пользователь копирует фото в другой альбом
|
||||
- **THEN** в целевом альбоме появляется новая запись, ссылающаяся на те же объекты хранилища
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user