DocsServer

The server library

Every global server code has, the web’s standard ones and Sluurp’s own, with examples, compared with Node, Deno and Bun.

Functions, hooks, jobs and agents run with the same globals as Node, Deno, Bun and browsers, plus a Sluurp object for what the web platform leaves out. Nothing needs importing, and code written for those runtimes mostly runs unchanged. To use one, call it from any server file:

functions/hello.ts
export default async function () {
  const page = await fetch("https://example.com").then((r) => r.text());
  const hash = await crypto.subtle.digest("SHA-256", new TextEncoder().encode(page));
  await Sluurp.files.write("pages/example.html", page);
  return { bytes: page.length, sha256: Sluurp.encoding.hex.encode(hash) };
}

Features include:

  • fetch, streams, crypto.subtle, timers and the rest of the web’s standard globals
  • Files kept in the app’s database with Sluurp.files, or on disk with Sluurp.disk
  • WebAssembly compiled to machine code with Sluurp.wasm
  • Parsers for CSV, YAML, TOML, JSON5, JSONL and XML, and tar archives
  • Pictures resized and converted, password hashing, encodings, compression and CSRF tokens

Compared with Node, Deno and Bun

NodeDenoBunSluurp
fetch, Request, Response, HeadersYesYesYesYes; public addresses only, if Settings says so
Streams (ReadableStream and the rest)YesYesYesYes
CompressionStreamYesYesYesYes, plus brotli
URL, TextEncoder, atob, Blob, FileYesYesYesYes
crypto.subtleEvery algorithmEvery algorithmEvery algorithmThe same ones but AES-KW; see Crypto
Timers, structuredClone, consoleYesYesYesYes
Filesnode:fs, the machine’s diskDeno.readFile, with permissionsBun.file, the machine’s diskSluurp.disk, the app’s own folder on disk; Sluurp.files, in its database
YAML, TOML, CSVFrom npm@std/yaml, @std/toml, @std/csvBun.YAML, Bun.TOMLSluurp.YAML, Sluurp.TOML, Sluurp.CSV
JSON5, JSONLFrom npm@std/jsonc, from npmBun.JSON5, Bun.JSONLSluurp.JSON5, Sluurp.JSONL
XMLFrom npmFrom npmFrom npmSluurp.XML
tar archivesFrom npm@std/tarBun.ArchiveSluurp.Archive
WebAssemblyWebAssemblyWebAssemblyWebAssemblySluurp.wasm, compiled to machine code by Wasmtime
Pictures resizedFrom npm (sharp)From npmFrom npmSluurp.image
Password hashingFrom npmFrom npmBun.passwordSluurp.password
gzip in one callnode:zlibFrom npmBun.gzipSyncSluurp.compress
hex, base32, base64urlBuffer@std/encodingBufferSluurp.encoding
CSRF tokensFrom npmFrom npmBun.CSRFSluurp.CSRF
A databaseFrom npmFrom npmbun:sqlitectx, with rules; see Functions
Node’s modules (node:fs, Buffer, process)YesMostMostBun’s
Child processes, sockets, workersYesYesYesYes, as in Bun
npm packagesYesYesYesInstalled with bun add, and imported by name
How code runsOne long processOne long processOne long processBun workers beside the server, a call at a time each
What code can reachThe machineWhat you allowThe machineThe machine, as Bun’s; your data through ctx, as the caller

Server code is trusted, as the rest of your app is. The part that is designed is the data: a function reaches it through ctx, as whoever called, so it can’t read what its caller couldn’t.

Network

fetch(input, init) works as it does in browsers. Requests run side by side, and a response’s body is a stream read as it arrives; see the event loop.

const [a, b] = await Promise.all([fetch(urlA), fetch(urlB)]);
const controller = new AbortController();
setTimeout(() => controller.abort(), 10_000);
const big = await fetch(url, { signal: controller.signal });
for await (const chunk of big.body) process(chunk);
  • It reaches any address, because your server code is trusted like the rest of your app. Turn on Public addresses only under Settings, Outbound requests to keep it away from this machine and your private network: the admin on localhost, a database on your network, or a cloud server’s metadata address, which hands out credentials.

  • For an address somebody else gave you, such as a link a user posted, pass publicOnly: true. That request reaches public addresses only, whatever the server allows its own code, so a user can never point it at your server or your network:

    const page = await fetch(link, { publicOnly: true });
  • With Public addresses only on, list the hosts your code may reach anyway under Settings, Outbound requests, one per line, for one project. For every project, start the server with --fetch-allow or set SLUURP_FETCH_ALLOW. Write a name (localhost, db.internal), or *.internal for every name under a domain:

    sluurp serve --public ./app --fetch-allow localhost,*.internal
  • Redirects are followed, and each one is checked the same way.

  • Request, Response and Headers are the standard classes. Response.json(data) makes a JSON response.

  • AbortController, AbortSignal.timeout(ms) and AbortSignal.any(signals) cancel requests.

Streams

ReadableStream, WritableStream and TransformStream, with pipeThrough, pipeTo, tee, getReader, for await and ReadableStream.from(iterable).

const gzipped = new Response(text).body.pipeThrough(new CompressionStream("gzip"));
const bytes = new Uint8Array(await new Response(gzipped).arrayBuffer());

CompressionStream and DecompressionStream take gzip, deflate, deflate-raw or, beyond the standard, br for brotli.

Text, bytes and addresses

TextEncoder, TextDecoderUTF-8 to bytes and back.
atob, btoaBase64 of Latin-1 text, as in browsers. For bytes, use Sluurp.encoding.base64.
Blob, FileBytes with a type, and a name for a File: .text(), .arrayBuffer(), .bytes(), .slice().
URL, URLSearchParamsParse and build addresses and query strings.
structuredCloneA deep copy that keeps dates, maps, sets and typed arrays.

Crypto

crypto.getRandomValues(array) and crypto.randomUUID() use the operating system’s randomness.

crypto.subtle has:

AlgorithmsMethods
HashesSHA-1, SHA-256, SHA-384, SHA-512digest
MACsHMACsign, verify
SignaturesECDSA (P-256 with SHA-256, P-384 with SHA-384), Ed25519, RSASSA-PKCS1-v1_5, RSA-PSSsign, verify
EncryptionAES-GCM (128 or 256 bits, 12-byte IV), AES-CBC and AES-CTR (128, 192 or 256 bits), RSA-OAEP (SHA-256, 384 or 512)encrypt, decrypt
Key agreementECDH on P-256 and P-384, X25519deriveBits, deriveKey
Key derivationPBKDF2, HKDFderiveBits, deriveKey

Keys come in through importKey and go out through exportKey as raw, pkcs8, spki or jwk. generateKey makes every kind, RSA from 2048 to 8192 bits with the exponent 65537.

const key = await crypto.subtle.generateKey({ name: "AES-GCM", length: 256 }, true, ["encrypt", "decrypt"]);
const iv = crypto.getRandomValues(new Uint8Array(12));
const sealed = await crypto.subtle.encrypt({ name: "AES-GCM", iv }, key, new TextEncoder().encode("secret"));

Not here: AES-KW (wrapKey and unwrapKey), ECDSA with a hash other than its curve’s, and RSA-OAEP with SHA-1.

Time and the console

setTimeout, setInterval, setImmediate, their clear… twins, queueMicrotask and performance.now(). Timers fire while a call is still running; when it returns, its timers end with it.

console.log, info, warn, error and debug write to the server’s log, where the admin’s Logs screen shows them.

Sluurp.files

Files server code keeps, by path. They live in the app’s database, so they’re in its backups and replicas, and a path can’t reach anything outside the app. For files people upload, use a file field.

await Sluurp.files.write("exports/orders.csv", Sluurp.CSV.stringify(rows));
const csv = await Sluurp.files.text("exports/orders.csv");
write(path, data)A string, bytes, a Blob or a Response. Gives the size written.
read(path)The bytes, or null when there’s no such file.
text(path), json(path)Read as text, or parsed as JSON; null when missing.
exists(path)Whether there’s a file there.
stat(path){ path, size, modified }, or null.
list(folder)Every file under a folder, with sizes and times. Leave the folder out for all of them.
rename(from, to)Moves a file, or a folder with everything in it. Gives how many moved.
remove(path)Removes a file, or a folder with everything in it. Gives how many went.

Paths use /. A leading / and doubled slashes are ignored, and .. is refused.

Sluurp.disk

The same calls as Sluurp.files, on real files in a folder of the app’s own: disk/<project>/ in the data folder, or under SLUURP_DISK. Use it for big files, for files another program reads or writes, or for a folder you mount from elsewhere.

await Sluurp.disk.write("imports/prices.csv", await fetch(url).then((r) => r.text()));
for (const f of await Sluurp.disk.list("imports")) console.log(f.path, f.size);

Server code can reach nothing outside that folder. A path with .. is refused, and so is a link inside the folder that leads out of it. A write goes to a temporary file first and is moved into place, so a reader never sees half a file.

Which to use:

Sluurp.filesSluurp.disk
WhereThe app’s databaseA folder on disk
In the admin’s backupsYesNo, back it up with the data folder
On every replicaYesNo, only on the server that wrote it
Other programs can use itNoYes
Best forSmall files that belong with the dataBig files, exports, shared folders

Sluurp.wasm

WebAssembly, for the work JavaScript is slow at: image and audio processing, parsing, compression, number crunching. A module built from Rust, Go, C, Zig or AssemblyScript is compiled to machine code by Wasmtime and runs at close to native speed, inside WebAssembly’s own sandbox: it reaches only its own memory and what it’s given.

const math = await Sluurp.wasm.compile(await Sluurp.disk.read("wasm/math.wasm"));
await math.call("fib", 30); // 832040

For a program that takes bytes and gives bytes back, build it for WASI (cargo build --target wasm32-wasip1, GOOS=wasip1 GOARCH=wasm go build) and run it with its input:

const resize = await Sluurp.wasm.compile(await Sluurp.disk.read("wasm/resize.wasm"));
const { code, bytes, stderr } = await resize.run({ args: ["--width", "640"], stdin: photo });
compile(source)Compiles a module, from bytes or WebAssembly text (.wat). The same bytes are compiled once, however often you ask. Gives { id, exports, wasi, call, run }.
call(module, name, ...numbers)Calls an export that takes and returns numbers, in a new instance.
run(module, { args, stdin, env })Runs a WASI program to its end. Gives { code, stdout, stderr, bytes }, where bytes is stdout as bytes.

An instance runs as long as it needs, with as much memory as WebAssembly allows (4 GB), and time spent in it doesn’t count against the function’s own. A WASI program has no files, no network and no environment beyond what you pass. The standard WebAssembly global isn’t there; Sluurp.wasm is how to reach it.

Sluurp.CSV

const rows = Sluurp.CSV.parse(text, { header: true }); // [{ name: "Ana", city: "Cluj" }, …]
const out = Sluurp.CSV.stringify(rows);                  // with a header row from the keys
  • parse(text, { header, separator }) gives arrays of strings, or objects when header is true. Quoted fields, doubled quotes and line breaks inside quotes all work, and a byte-order mark is skipped.
  • stringify(rows, { header, separator }) takes arrays or objects. Fields that need quoting get it, and lines end with \r\n, as spreadsheets expect.

Sluurp.YAML, TOML, JSON5 and JSONL

Each has parse(text) and stringify(value):

const config = Sluurp.YAML.parse("name: shop\nports: [80, 443]");
const toml = Sluurp.TOML.stringify({ server: { port: 8080 } });
const loose = Sluurp.JSON5.parse("{ unquoted: 'keys', trailing: [1, 2,], }");
const events = Sluurp.JSONL.parse(logText); // one value per line; a bad line is named by number

Sluurp.NDJSON is the same object as Sluurp.JSONL, for code that knows the format by that name.

Sluurp.XML

parse(text) gives a plain object, as the other formats do: attributes as @name, text beside elements as #text, and an element that repeats as an array. stringify(value) is the way back, escaped correctly:

const order = Sluurp.XML.parse(ctx.body);
// { Order: { "@id": "42", Item: [{ Name: "Dune", Price: "10" }, …] } }

const feed = Sluurp.XML.stringify({ rss: { "@version": "2.0", channel: { title: "News", item: posts.map((p) => ({ title: p.title, link: p.url })) } } });

For a larger document, document(text) gives its root element to walk, with selectors of names, * and [attribute] or [attribute=value], joined by spaces (anywhere under) or > (a child):

const invoice = Sluurp.XML.document(ctx.body);
invoice.find("Invoice > ID").text;                       // "INV-42"
invoice.findAll("Line[kind=book]").map((l) => l.find("Amount").attr("currency"));

Each element has name, attrs, attr(name), text, children, elements and parent. Names are matched without their namespace’s prefix.

A function receives an XML request’s body as text (ctx.body), and answers in XML with return Sluurp.XML.respond(xml).

A document with a DTD is refused, so no entities are ever expanded, nothing outside the document is read, and a document is at most 10 MB.

Sluurp.Archive

Reads and writes tar files, gzipped or not.

const archive = new Sluurp.Archive({ "readme.txt": "Hello", "data/rows.json": JSON.stringify(rows) }, { compress: "gzip" });
await Sluurp.files.write("backups/export.tar.gz", await archive.bytes());

const read = new Sluurp.Archive(await Sluurp.files.read("backups/export.tar.gz"));
for (const [name, file] of await read.files("data/*.json")) console.log(name, await file.text());
  • new Sluurp.Archive(files, { compress: "gzip" }) makes one from an object or a Map of names to strings or bytes.
  • new Sluurp.Archive(bytes) reads one. A gzipped one is recognised.
  • files(glob) gives the files as Files by name. * matches within a folder and ** across folders.
  • bytes() and blob() give the archive.

Sluurp.compress and decompress

Whole buffers in one call: await Sluurp.compress(data, format) and await Sluurp.decompress(data, format). The format is gzip (the default), deflate, deflate-raw or br. The data may be a string, bytes or a Blob. For streams, use CompressionStream.

Sluurp.image

resize(data, options) makes a picture over: fitted into a box, in another format, smaller. It takes bytes, a Blob or a Response, and gives a Blob of the new picture.

const photo = await fetch(url);
const small = await Sluurp.image.resize(photo, { width: 640, format: "jpeg", quality: 78 });
await Sluurp.files.write("thumbs/cover.jpg", small);
Option
width, heightThe box, in pixels, up to 4,000. Leave either out to keep the picture’s shape. A picture is never made larger than it is
fitcontain (the default) fits inside the box; cover fills it, cutting off what is over; stretch takes the box exactly
formatjpeg (the default), png or webp
quality1 to 100; by default, 82

A picture of more than 50 million pixels is refused before it is read, so a crafted file can’t use up the server’s memory.

Sluurp.password

const hash = await Sluurp.password.hash(password); // "$argon2id$…"
const ok = await Sluurp.password.verify(password, hash);

Argon2id, as Sluurp’s own sign-in uses, with a new salt each time. verify takes the same time whether or not the password matches.

Sluurp.encoding

hex, base64, base64url and base32, each with encode(data) and decode(text). encode takes a string (as UTF-8) or bytes. decode gives bytes.

Sluurp.encoding.hex.encode("hi");        // "6869"
Sluurp.encoding.base64url.encode(bytes); // no padding, URL-safe
Sluurp.encoding.base32.decode("MZXW6YTBOI======");

Sluurp.CSRF

Tokens that prove a form came from your own page:

const token = Sluurp.CSRF.generate(secret, { sessionId, expiresIn: 60 * 60 * 1000 });
const valid = Sluurp.CSRF.verify(token, { secret, sessionId });

A token is signed with HMAC-SHA-256 and carries when it was made. verify checks the signature, the session and the age, and returns false rather than throwing.

View as Markdown