File uploads

CRUD endpoints accept multipart/form-data for entities with Image or File fields. Uploads are streamed through the configured upload.Storage backend; only the resulting URL is stored on the record.

A browser sends multipart/form-data from another site with no CORS preflight and the user's cookies, so a multipart create or update must also pass the cross-site form guard (handler.IsCrossSiteRequest): a request whose Sec-Fetch-Site or Origin names another site, a sibling subdomain included, answers 403. The _batch routes and state transitions take application/json only and answer 415 to anything else.

Wire it up

14 lines
import "github.com/DonaldMurillo/gofastr/core/upload"

storage := upload.NewLocalStorage("./uploads")
app := framework.NewApp(
    framework.WithDB(db),
    framework.WithFileStorage(storage),
)
app.Entity("users", framework.EntityConfig{
    Fields: []schema.Field{
        {Name: "name",   Type: schema.String, Required: true},
        {Name: "avatar", Type: schema.Image},
        {Name: "resume", Type: schema.File},
    },
})

WithFileStorage is required if any entity declares an Image or File field. Without it, multipart requests on those entities return server has no file storage configured.

Posting an upload

4 lines
curl -X POST http://localhost:8080/users \
  -F 'name=Carol' \
  -F 'avatar=@/path/to/photo.png' \
  -F 'resume=@/path/to/cv.pdf'

The framework:

  1. Rejects the request 413 if the body exceeds crud.MaxMultipartBodyBytes = 64 MiB, then parses the multipart form (up to MaxMultipartMemory = 32 MiB in memory, spills the rest to temp files). The two limits are different things: the first caps the wire body, the second is only the in-RAM spill threshold.
  2. Coerces non-file values to the schema field's Go type (Int → int64, Bool → bool, etc.). A blank value is "not provided", as an empty string is on the JSON path, for a Bool too: an update stores NULL, a create applies the field's default, and a required field left blank answers 400. A form key repeated across parts is refused with 400 — the JSON body path refuses duplicate keys for the same reason: a proxy inspecting one of the two parts would see a payload the server never executes. The one carve-out: a schema.JSON column is the list-valued shape (an HTML multi-select submits one part per selected value), so repeats collect into a JSON array. Two file parts under one Image/File key are refused too; those columns hold a single URL. The refusals carry their own codes (duplicate_form_key, duplicate_file_part, malformed_multipart for a body that does not parse) and name the first offending key in sorted order, so one body always answers the same bytes.
  3. Streams each file part matching an Image/File field through Storage, scoped by entity name and field name. A part for a read-only or hidden field is ignored, as the write ignores a value for it, so no stored object goes unreferenced.
  4. Stores the returned URL string on the record.

The generated Go client, JS SDK and CLI send this request for you: see SDKs and App CLI.

The record persisted to the database looks like:

3 lines
{ "id": "u1", "name": "Carol",
  "avatar": "/uploads/users/avatar/abc123.png",
  "resume": "/uploads/users/resume/def456.pdf" }

Upload response (standalone handler)

upload.Handler (the standalone multipart endpoint, separate from the auto-CRUD path) responds 200 with a JSON body describing the stored file. Every key is camelCase, like the rest of the framework:

5 lines
{ "originalName": "report.txt",
  "size": 11,
  "mimeType": "text/plain; charset=utf-8",
  "uploadedAt": "2026-07-30T18:04:07.567319Z",
  "key": "report_1786943913322678000_bcdc0d8ff1d03054.txt" }

mimeType is the content type sniffed from the bytes (never the attacker-controlled multipart header), and key is the storage key the backend wrote the file under.

upload.Config.MaxSize caps the request body before the multipart form is parsed, and a larger body gets 413. Zero (the field's default) means upload.DefaultMaxSize, 32 MiB, the same cap file.MaxProcessFileSize sets on the auto-CRUD path. It does not mean "no limit". Set MaxSize to accept larger files.

The key is not the client's filename. upload.UniqueFilename sanitizes the name and appends a nanosecond timestamp and 8 random bytes, so two users uploading report.txt get two objects instead of the second silently overwriting the first. Read key from the response; do not reconstruct it from originalName.

A client cannot send that key back in a JSON write to an entity's Image/File field: the CRUD write path accepts only keys the same request uploaded (next section). Upload through the entity's own multipart route instead, or save the object in host code and write the row there with crud.WithUploadedKeys.

Storage keys on JSON and in-process writes

A relative value in an Image/File field is a storage key, and App.EraseUserData deletes the keys the erased user's rows name. So the CRUD write path (create, update, upsert, batch, nested writes, and the in-process CreateOne / UpdateOne / UpsertOne / TypedQuery.UpdateAll) accepts a relative value, in the field or in a <field>_variants storage_ref, only when:

  • this request's multipart parse saved it (the multipart routes do this for you, renditions included);
  • host code saved it and says so with crud.WithUploadedKeys(ctx, keys...);
  • it is already on the row being written, read under the caller's owner, tenant and soft-delete scope (sending an unchanged value back on update is fine); or
  • the context carries crud.WithServerWrites (trusted host code).

Anything else answers 400 with {"fields": {"<field>": ["storage key was not uploaded by this request"]}}. An absolute http(s) URL is always accepted: it is an external link, and erasure never deletes it. file.IsExternalURL(s) is the check both sides use, and file.VariantStorageRefs(raw) reads the keys a <field>_variants value names.

4 lines
ff, err := file.ProcessFileField(ctx, store, r, name, "profiles", "avatar")
if err != nil { return err }
_, err = profiles.CreateOne(crud.WithUploadedKeys(ctx, ff.StorageRef),
    map[string]any{"avatar": ff.StorageRef})

Store ff.StorageRef, the literal key, in the column. ff.URL is the key escaped for a link, segment by segment, so a name holding # or % differs between the two, and a column holding the escaped spelling fails the provenance check with key_not_uploaded.

Pass WithUploadedKeys only keys your own code just saved. A key taken from request data is what the check exists to refuse: without it, a user could copy another user's key (published in their /uploads/<key> URLs) into their own row, erase their account, and delete the other user's file.

TypedQuery.UpdateAll writes one value onto many rows, so "already on the row" cannot apply there; it needs WithUploadedKeys or WithServerWrites.

When stored objects are deleted

A row's objects (the key in each Image/File field and the renditions its <field>_variants column lists) are deleted from the handler's Storage when the row lets them go:

  • a hard delete (DELETE /{entity}/{id}, batch delete, DeleteOne, TypedQuery.DeleteAll) or a purge of a soft-deleted row deletes them all;
  • an update or upsert, including TypedQuery.UpdateAll, that replaces or clears a file deletes the old key and its unreferenced renditions.

The delete runs after the write's transaction commits; a rollback keeps the objects. A failed delete is logged, not returned, since the row change has already committed. The objects stay when:

  • another row of the same table still names the key, live or soft-deleted (one default image a seed set on many rows, say);
  • the value is an absolute http(s) URL, an external link;
  • the delete is a soft delete, so a restore gets the file back;
  • the write runs on a transaction with no commit queue (a bare db.WithTx), where nothing reports the commit. App.InTx has one.

A key that host code copied into a different entity's table is not checked: deleting the first row deletes the object the second still names. TypedQuery.UpdateAll captures objects under the same tenant, owner, soft-delete, and query predicates as its update, inside the write transaction. It preserves objects and renditions still named by the updated rows.

Field-name casing

Multipart field names are taken literally as column names; there is no JSON-case translation. If your entity's column is avatar_url, the multipart field must be avatar_url, regardless of JSONCase config. (JSON requests are reverse-cased; multipart is not.)

Field types

TypeWire formDB column
schema.Imagemultipart file partTEXT URL
schema.Filemultipart file partTEXT URL

The two differ in three ways: the UI host emits an image-aware widget for Image fields, only Image fields run the image pipeline described below, and an Image field takes only a raster image (PNG, JPEG, GIF or WebP, sniffed from the bytes; anything else answers 400 and nothing is stored). A File field is any binary, a PDF or a CSV, so decoding it as an image would fail every upload. file.AllowTypes(mimes...) is the same check for your own ProcessFileField calls; file.ImageTypes is the image list. In a multipart body a Decimal field stays the decimal string, as on the JSON path, and a Bool field's hidden "false" followed by its checked box (the HTML checkbox idiom) is one value, the box's; any other repeated scalar is still refused.

Entity screens (framework/entityui, and so the admin) draw an Image or File field as an upload when the app has file storage: the stored file (a thumbnail, or a link named by its file name) above a file input. Leaving the input empty keeps the stored file. Set entityui.Extensions.FilesURL to the path you serve stored files under (below), so the screens can draw a stored key.

Automatic renditions and placeholders

By default an upload is stored as-is: one file, one URL. Adding WithImagePipeline makes every schema.Image upload also produce resized renditions and a placeholder, with no per-entity upload handler:

17 lines
import (
    "github.com/DonaldMurillo/gofastr/framework/image"
    "github.com/DonaldMurillo/gofastr/framework/imagefield"
)

app := framework.NewApp(
    framework.WithFileStorage(store),
    framework.WithImagePipeline(imagefield.MustNew(imagefield.Config{
        Variants: []image.Variant{
            {Width: 480, Format: image.FormatWebP, Suffix: "sm"},
            {Width: 960, Format: image.FormatWebP, Suffix: "md"},
            {Width: 480, Format: image.FormatJPEG, Quality: 82, Suffix: "sm"},
            {Width: 960, Format: image.FormatJPEG, Quality: 82, Suffix: "md"},
        },
        BlurHashX: 4, BlurHashY: 3,
    })),
)

imagefield is a separate package on purpose: framework/image carries every image decoder plus the WebP encoder, so importing it from the upload path unconditionally would put all of it in the binary of every app with a CRUD handler. Only apps that ask for the pipeline link it.

Where the derived values go

Renditions are written through the same storage backend as the original, into the same directory, sharing its unique base name: an original at products/cover/photo_1730419200000000000_a1b2c3d4e5f60718.png yields products/cover/photo_1730419200000000000_a1b2c3d4e5f60718-sm.webp and so on, so two uploads of photo.png never collide. A name too long for the 255-byte filesystem limit once the suffix is added loses bytes from the front of the base, never the random token at its end. Renditions and the placeholder are derived after the upload's EXIF orientation is applied, so a portrait phone photo yields portrait renditions.

The metadata lands in sibling columns, named for the image field:

ColumnTypeContents
<field>_blurhashschema.String~28-char BlurHash
<field>_placeholderschema.StringLQIP data: URL (only if Placeholder set)
<field>_variantsschema.JSON[{storage_ref, mime, width, height}, …], ascending by width

Changing or clearing an existing image through an update or an upsert conflict clears omitted derived siblings to SQL NULL in the same write. This includes read-only or hidden derived columns and works without a configured storage backend. Explicit writable replacement metadata is kept; an unchanged image keeps its metadata. A multipart pipeline that produces only some artifacts therefore cannot retain the previous image's other artifacts. Stored renditions that the new row no longer references are released after commit.

Image updates compare the source read for that decision in the write's SQL predicate, including SQL NULL. If another writer changes the source before the update or upsert conflict, the stale write is refused without changing the current image or its metadata. Reload the row before retrying.

Columns you do not declare are skipped. Nothing errors, nothing is lost; you adopt a column by adding it to the entity. So an entity with a cover image field opts into the hash alone by declaring:

4 lines
Fields: []schema.Field{
    {Name: "cover", Type: schema.Image},
    {Name: "cover_blurhash", Type: schema.String},
}

Rendering the result

<field>_variants maps onto a responsive <picture>, and <field>_blurhash becomes the placeholder that paints before it:

15 lines
durl, _ := image.BlurHashDataURL(row.CoverBlurHash, image.BlurHashRenderConfig{})

var headers []ui.HeaderInfo
for _, v := range row.CoverVariants {
    headers = append(headers, ui.HeaderInfo{
        Name: v.StorageRef, Width: v.Width, Height: v.Height, MIME: v.MIME,
    })
}

ui.PipelineImage(ui.PipelineImageConfig{
    Fallback:    "/uploads/" + row.Cover,
    Alt:         row.Name,
    Sources:     ui.PipelineSourcesFromHeaders(headers, func(n string) string { return "/uploads/" + n }),
    Placeholder: durl,
})

See image.md for the pipeline itself and for the BlurHash-versus-LQIP trade-off.

Config reference

FieldEffect
VariantsRenditions to produce and store. Suffix (or width) plus the format's extension names each file; New refuses two that name the same one.
BlurHashX/YBlurHash component counts, 1..9. Both zero = no hash; setting only one is an error. 4×3 landscape, 3×4 portrait.
PlaceholderAlso store an LQIP data URL. Usually redundant next to a BlurHash; ~28 bytes versus a few hundred, and both render identically.
RejectAnimatedFail the upload on a multi-frame source instead of silently keeping frame one. Worth setting on avatars.
AllowUpscalePermit renditions wider than the source. Off by default, so a small upload does not fan out into pixel-multiplied files.
MaxPixelsOverride the decompression-bomb guard (default 64 MP).

New returns an error for a config that could not produce anything, or that the pipeline would reject at process time anyway, so wiring mistakes surface at startup rather than on the first upload. MustNew panics instead, for package-level wiring.

Per-field configuration

One app-wide config cannot describe every image. An avatar wants portrait BlurHash components and animated sources rejected; a hero cover wants wide renditions. Compose a default with overrides:

3 lines
framework.WithImagePipeline(covers),                        // default
framework.WithImagePipelineFor("users", "avatar", avatars),  // this field only
framework.WithImagePipelineFor("posts", "attachment", nil),  // opt out entirely

A field with no entry falls through to the default. An explicit nil opts that one field out without unpicking the app-wide config.

Doing it yourself

The declarative wiring is a convenience over interfaces that stay open; none of it is a closed system, and every layer below remains callable.

Your own deriver. WithImagePipeline takes a file.ImageDeriver, not a concrete type. imagefield is one implementation; implement the interface directly to crop to a focal point, watermark, push to a CDN, or name keys your own way:

4 lines
type ImageDeriver interface {
    DeriveImage(ctx context.Context, store upload.Storage,
        data []byte, primaryRef string) (*file.ImageDerivatives, error)
}

Whatever you return is validated and then spread across the sibling columns exactly as imagefield's output would be, so you keep the declarative half while owning the processing.

Driving the upload path directly, outside auto-CRUD: your own handler, your own storage keys, the full FileField including .Image rather than just the URL string:

3 lines
ff, err := file.ProcessFileField(ctx, store, part, filename, "products", "cover",
    file.WithImageDeriver(deriver))
// ff.Image.Variants / .BlurHash / .Placeholder

Ignoring all of it and calling the pipeline yourself. VariantSet is headless and unchanged; see image.md. Nothing about the upload path is required to use it: generated covers, imported batches, and one-off scripts have no upload to hang off.

Storing the metadata somewhere else. The sibling columns are a convention, not a requirement. Declare none of them and the derived values are dropped; do the persistence in a BeforeCreate hook, a separate table, or a JSON blob of your own shape instead.

The one thing the framework will not do is reach into framework/image from framework/file or framework/crud: that edge would link every image codec into every app with a CRUD handler, which is why the dependency is inverted through ImageDeriver in the first place.

Scope of this seam

ImageDeriver is the only per-field transform the framework has, and it is image-shaped on purpose: it runs on write, for schema.Image fields, and puts its output in sibling columns. It is not a general field hook.

For anything else, such as normalising a string on write, masking a value on read, or composing several transforms on one field, use a BeforeCreate/ BeforeUpdate hook or an entity.ValidatorFunc today. Note that schema validation runs after BeforeCreate, so a hook can normalise a value into validity.

A general per-field middleware chain is being designed in issue #144. The blocking question there is the read half: a transform applied after the query has already filtered and sorted on the stored value silently breaks ?field=, ?sort=, ?q=, cursor pagination, and export. Until that is settled, ImageDeriver stays narrow rather than growing sideways.

Failure behavior

A source that cannot be decoded fails the whole request: the row is not written and no file is kept. That is deliberate: the column is declared as an image, and a silent success would surface much later as a page with no srcset and no placeholder, with nothing in the logs pointing back at the upload. The original is stored before renditions are derived, so a derive failure never leaves a half-written primary file.

Renditions stream one at a time, so peak memory stays near a single rendition rather than all of them summed; this runs inside a request, on bytes a client chose.

Cost: this is synchronous

Deriving happens in the upload request, and it is CPU-bound. Measured on an M-series laptop, from a generated source:

RenditionsSourceDerive
JPEG ×2, 1200 px source45 KB~53 ms
JPEG ×2, 3000 px source196 KB~184 ms
WebP ×2 + JPEG ×2, 3000 px196 KB~537 ms

The jump is the WebP encoder: it is lossless-only and runs five predictor passes, shipping the smallest (see image.md → Performance notes). So:

  • For a hot upload path, prefer JPEG renditions. Two JPEG widths plus a BlurHash is under 200 ms even from a large source.
  • Reserve WebP for low-volume flows like admin uploads and one-off imports, or accept the half-second.
  • Each upload holds a request slot for that whole time and does not parallelise away. If uploads are frequent, derive out-of-band instead: store the original in the request, then produce renditions in a battery/queue job and patch the sibling columns when it finishes. The pieces are the same; only the timing moves.

There is no built-in async mode; WithImagePipeline is deliberately the simple synchronous one, because it is correct for the common case (a user uploading their own avatar or a cover image) and needs no queue.

Validation

Uploads are bounded and content-checked before anything is stored:

  • Type is checked for Image fields (above) and wherever file.AllowTypes is passed, from the sniffed type before the filename fallback, so an unknown blob named logo.png is not an image.
  • Size is capped at file.MaxProcessFileSize (32 MiB), and the multipart parser buffers at most crud.MaxMultipartMemory (32 MiB) in memory before spilling to a temp file. An oversize body returns file.ErrFileFieldTooLarge without reading past the limit.
  • Content shape is checked from the bytes, never the filename or the client's Content-Type, both of which an attacker controls. Executable magic bytes (PE, ELF, Mach-O) are rejected outright. Active-content markup is scanned in two tiers: hard tokens (<script, <svg, <iframe, <html, <!doctype, <object, <embed, <base, <math) are matched anywhere in the body, so they are caught even when buried past the leading bytes; soft tokens (<img, <?xml, <style, <link, javascript:) are matched only in the leading 512 bytes and skipped for confirmed binary types, which legitimately carry them in metadata. UTF-16 and UTF-32 text (BOM-marked, or BOM-less UTF-16 opening with <) is scanned by character as well, since an XML parser decodes it and its markup is NUL-interleaved in the raw bytes. A rejection returns file.ErrFileFieldUnsafeContent.
  • Metadata shape is enforced by FileField.Validate for callers that accept a FileField from a request body. Beyond the scheme and .. checks on URL and StorageRef, Filename is held to the same traversal rule, and a C0 control byte or DEL anywhere in URL, Filename, or StorageRef returns file.ErrFileFieldControlBytes; those fields are echoed into Content-Disposition, HTML attributes, and log lines, each of which a CR/LF splits. ProcessFileField strips control bytes from the multipart filename, so the metadata it returns always passes its own Validate.
  • Required is enforced from the field declaration, like any other field.

These are global limits, not per-field ones: there is no per-field byte cap or MIME allow-list today. To bound a specific field more tightly, use a BeforeCreate hook or an entity.ValidatorFunc, both of which run before the row is written, or set imagefield.Config.MaxPixels to tighten the decode guard for image fields.

A failing validator returns 400 Bad Request with a fields map identifying the offending field.

Uploaded filenames are sanitized to a safe storage key (path separators and control characters stripped, length capped at MaxFilenameBytes on a UTF-8 rune boundary). SanitizeFilename bounds the input it inspects to SanitizeFilenameInputBound (4 × MaxFilenameBytes) so a multi-megabyte attacker-supplied filename can't force unbounded pre-truncation work (DoS). The stripped set also covers the C1 controls and the zero-width/bidi codepoints (core/textsafe), so a served key can't flip the extension a user sees or manufacture visually identical twin names.

Metadata in stored originals

By default the stored original is byte-for-byte what was uploaded, image metadata included: a phone photo keeps its EXIF block, GPS coordinates and all, and every later download serves it. That is deliberate, not an oversight — the framework does not silently rewrite originals — but some uploads (avatars, user-generated galleries) should not carry the uploader's location. For those, pass file.StripMetadata:

2 lines
ff, err := file.ProcessFileField(ctx, store, part, filename, "products", "cover",
    file.StripMetadata(), file.WithImageDeriver(deriver))

Auto-CRUD multipart uploads opt in app-wide with framework.WithStripUploadMetadata() (every entity's file and image fields), or per handler with CrudHandler.StripUploadMetadata = true:

3 lines
app := framework.NewApp(
    framework.WithStripUploadMetadata(), // strip EXIF/XMP from every stored original
)

What it removes:

FormatRemovedHow
JPEGevery APPn segment (EXIF, XMP, ICC, Photoshop data) and COM comments; anything after EOI. The Adobe APP14 colour marker stays, cut to its 12 bytes: a CMYK JPEG does not decode without itsegments spliced out; no re-encode unless a rotation is needed
PNGtEXt, zTXt, iTXt, eXIf chunks; anything after IENDre-encoded (PNG is lossless, pixels unchanged); an animated PNG is spliced instead, every frame chunk kept; a PNG with only a trailer is cut at IEND
WebPEXIF and XMP chunks; anything past the RIFF sizecontainer rewritten, image bytes untouched

An EXIF orientation of 2–8 is applied first so the stripped photo still displays upright: JPEG and PNG bake the rotation into the pixels (the only path that re-compresses a JPEG, at quality 95), while WebP keeps an orientation-only EXIF chunk, because the WebP encoder lives above this layer and the container is all that can be rewritten here. An animated PNG does the same with an orientation-only eXIf chunk: the standard PNG decoder reads only the first frame, so re-encoding would store a still image.

Behaviour worth knowing:

  • Stripping is gated on the sniffed content, never the filename. A file named photo.jpg whose bytes are not a JPEG passes through unchanged, as does every non-JPEG/PNG/WebP upload (GIF included).
  • It runs before the image pipeline, so renditions and the BlurHash are derived from the stripped bytes, and FileField.Size reflects them.
  • It fails closed: a JPEG, PNG, or WebP whose structure cannot be walked fails the upload rather than being stored unstripped. A WebP RIFF size that omits the form tag or leaves no chunks is refused before storage.

Storage backends

upload.Storage is the interface:

6 lines
type Storage interface {
    Save(ctx context.Context, key string, r io.Reader) error
    Delete(ctx context.Context, key string) error
    Get(ctx context.Context, key string) (io.ReadCloser, error)
    Exists(ctx context.Context, key string) (bool, error)
}

Built-in implementations:

  • upload.NewLocalStorage(dir) (core/upload): writes files under a local directory. Saves are atomic — data lands in a temp file beside the target and renames into place, so a reader never sees a torn object — and containment is enforced on the symlink-resolved path at every operation, with the syscalls performed through an os.Root over the resolved root so a symlink planted after resolution is refused by the kernel, not just one present when the key was checked: a key that reaches through a symlinked directory or leaf to a file outside the storage root is refused with a 400-class key error, not followed. On a case- or normalization-insensitive filesystem (macOS default APFS), a save under a key that folds onto an existing object stored under a byte-different spelling (TenantA/x vs tenanta/x, NFC vs NFD) is refused with the same key error instead of silently overwriting it, and a delete under such a key is refused rather than removing the other spelling's object. The check lists a key's directory only where folding is possible: always on Windows, where an NTFS 8.3 short name (PHOTO~1.PNG) aliases a different long name, and for any name holding ~ or a byte that can normalize; elsewhere a case-swap probe proves the directory keeps the name apart. A key that is not its own clean form (a//b, a/./b, ./a, a trailing /), a leading / and . are refused with ErrInvalidKey on every operation, never cleaned to the object they alias. Errors never carry the absolute storage path. It only stores; it does not serve. Wire downloads with upload.ServeHandler, which sniffs the content type, blocks traversal (delegated to the backend's key sanitization), and neutralizes HTML/SVG to a forced download so an uploaded document can't execute as script in a victim's browser (stored-XSS guard). Suitable for tests and single-host deployments.
  • battery/storage: local, S3, and in-memory backends behind the same interface, plus a declarative factory registry.
  • (Add GCS / Azure adapters in your own code by implementing Storage.)

Serving stored files

LocalStorage (and the battery/storage backends) only store bytes; none of them serves files over HTTP. To download a stored file, mount upload.ServeHandler on the router with a catch-all key:

1 lines
app.Router().Get("/uploads/{key...}", upload.ServeHandler(storage))

ServeHandler resolves the key from the {key...} wildcard, sniffs the content type from the first 512 bytes (never trusting the client or the key), and sets X-Content-Type-Options: nosniff on every response. Scriptable content, HTML or SVG by sniffed type or key extension, is forced to application/octet-stream with Content-Disposition: attachment, so an uploaded document can't execute as script in a victim's browser (stored-XSS guard). Path-traversal defense is delegated to the backend's key sanitization; the handler echoes no filesystem path on any error.

Range requests and resumable downloads

Storage.Get returns an io.ReadCloser, which erases seekability, and http.ServeContent needs an io.ReadSeeker to answer a Range: header. Backends that hold their bytes locally declare the capability instead:

3 lines
type RangeGetter interface {
    GetRange(ctx context.Context, key string) (io.ReadSeekCloser, error)
}

ServeHandler type-asserts for it. When the backend implements it, range requests get a 206 with Accept-Ranges: bytes, so a download interrupted 1.8 GB into a 2 GB file resumes instead of restarting. When it doesn't, the handler serves whole bodies exactly as before; declining is legal.

BackendRangeGetter
upload.LocalStorage, storage.LocalStorageyes; already opens an *os.File
storage.MemoryStorageyes; the bytes are already resident
storage.S3Storageno; a network backend would have to buffer the whole object to satisfy Seek. Use WithPresigner so the transfer bypasses the app entirely.

Implement it on your own backend only if seeking is genuinely cheap there, and route key validation through the same code path as Get; a capability that skipped the traversal check would be a path-traversal hole with a performance justification. For filesystem-backed backends, upload.ResolveUnderRoot, upload.ScrubPath, upload.CreateTempInRoot, and upload.RefuseFoldedKey are the shared primitives for symlink-resolved containment, path-scrubbed errors, the os.Root-contained staging file for atomic writes, and the case/normalization-fold refusal; both local backends route containment through them, and RefuseFoldedKey is the one spelling of the fold walk for any backend that saves under caller-composed keys.

Content checksums

The battery/storage package provides opt-in SHA-256 helpers that wrap any Storage backend: integrity verification, dedup, or content-addressed keys without touching the backend or buffering the whole stream:

11 lines
import "github.com/DonaldMurillo/gofastr/battery/storage"

// Write once: stream is teed through a SHA-256 hasher as it saves.
res, err := storage.SaveWithChecksum(ctx, st, key, body)
// res.Size   -> bytes written
// res.SHA256 -> lowercase hex digest of the stored content

// Later: confirm the object still matches.
if err := storage.VerifyChecksum(ctx, st, key, res.SHA256); err != nil {
    log.Print(err) // wraps storage.ErrChecksumMismatch on mismatch
}

SaveWithChecksum tees the stream through a SHA-256 hasher while it is written, so the content is read exactly once and nothing is buffered. VerifyChecksum re-reads the object and compares digests; it accepts upper- or lowercase hex and returns an error wrapping storage.ErrChecksumMismatch (carrying the key and the got/want digests) on a mismatch, the underlying error if the object can't be read, and a clear validation error if the expected digest isn't 64 hex characters.

Common mistakes

  • Expecting uploads to be scrubbed by default. Originals keep EXIF/GPS metadata unless you pass file.StripMetadata on a direct ProcessFileField call; the default is preservation.
  • Forgetting WithFileStorage. Multipart requests on an Image/ File entity will error. JSON requests still work; they just can't set those fields.
  • Sending a JSON body with a base64 file. Not supported. Use multipart, or host the file elsewhere and PATCH its absolute https URL in.
  • PATCHing a storage key from a separate upload. A JSON write refuses a relative key the same request did not upload, including one upload.Handler returned. See Storage keys on JSON and in-process writes.
  • Trusting client-supplied URLs. Multipart writes the URL the server gets back from Storage.Save, not anything the client sent. A JSON request can set an absolute http(s) URL, which the server stores as an external link without fetching or uploading anything.
  • Camelcasing multipart names. They are literal column names. Use snake_case if your DB columns are snake_case.
  • Expecting sibling columns to appear on their own. WithImagePipeline fills <field>_blurhash / <field>_placeholder / <field>_variants only when the entity declares them. A missing column is skipped silently, so a hash that "isn't saving" is usually an undeclared column.
  • Wiring WithImagePipeline without WithFileStorage. Renditions are written through the same backend as the original; there is nowhere to put them otherwise.
  • Putting an image on a File field and expecting renditions. Only schema.Image runs the pipeline.
  • Reaching for framework/image from framework/file or framework/crud. Both are below it in the layering, and the edge would link every decoder into every app with a CRUD handler. That is why file.ImageDeriver is an interface and framework/imagefield implements it.