Skip to the content

DocsPick a model

Prompt reading

POST /v1/plan: what is read from a prompt, its tags and attachments, who reads it, and how to keep prompts on Rhumb's server.

/v1/resolve takes a need you state. POST /v1/plan takes a request as your app received it (a prompt, tags and the types of its attachments), reads the need from it, and resolves that need the same way. Use it when the need differs from one request to the next.

plan.ts
// requires /v1/plan (phase 5)
//
// Plan a request from the prompt itself, within the request's time budget.
// On a timeout or any error, use the app's own default model.
//
//   RHUMB_KEY=nnm_live_... npx tsx plan.ts
//
// RHUMB_URL overrides the base URL (default https://rhumbdata.com).
// The prompt goes only to the decision model you choose (OpenAI's by default;
// `decision: { allow_remote: false }` keeps it on our server) and is never
// stored or logged. Send tags or a summary instead if you prefer.

import { pathToFileURL } from "node:url";

type Plan = {
  model: { id: string; provider: string; provider_model_id: string };
  need: { task: string; purpose?: string };
  /** Where each part of the need came from: fact, tag, decision, request, default. */
  sources: Record<string, string>;
  decision: { model: string; fallback_used: boolean; cached: boolean };
};

export async function planFor(prompt: string): Promise<Plan | null> {
  const key = process.env.RHUMB_KEY;
  if (!key) return null;
  const base = process.env.RHUMB_URL ?? "https://rhumbdata.com";
  const res = await fetch(`${base}/v1/plan`, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${key}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      prompt,
      tags: ["usa"],
      providers: ["gemini", "openai", "openrouter"],
    }),
    signal: AbortSignal.timeout(600),
  }).catch(() => null);
  return res?.ok ? ((await res.json()) as Plan) : null; // null → use the app's own default
}

async function main() {
  const plan = await planFor("Summarize this contract in five bullet points.");
  console.log(
    JSON.stringify(
      plan
        ? {
            provider: plan.model.provider,
            provider_model_id: plan.model.provider_model_id,
            purpose: plan.need.purpose ?? null,
            purpose_from: plan.sources.purpose ?? null,
            decision: plan.decision,
          }
        : { default: "gemini-3.5-flash" },
    ),
  );
}

// Run directly (not imported): print the answer for an example prompt.
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href)
  await main();

What is read, in order

  1. Facts. Attachment types set the input capabilities the model needs, and the prompt's size sets min_context and the usage.
  2. Tags. A list of synonyms for origin, budget, capabilities and purpose aliases.
  3. The decision model, for whatever is left.

Facts and your constraints always win over the decision model, and the answer's sources says where each part came from.

A prompt is not read at all when the request leaves nothing for a reading to add: its constraints and recognised tags settle the task or purpose and the budget, it states needs (even empty), it has no tag the synonym list doesn't know, and it leaves no shape to the reading. The answer's reading is then empty and decision.model is local. Leave any of these out and the prompt is read.

Who reads the prompt

By default the prompt goes, cut to its first 4,000 tokens, to OpenAI's decision model, within a budget of 1,500 ms. On a timeout, an error or an invalid answer, Rhumb's own classifier reads it instead and decision.fallback_used is true, so the call never fails because of the decision model. Readings are cached for 24 hours under a keyed hash.

Keeping prompts on Rhumb's server

A request keeps its prompt on our server with decision: { allow_remote: false }. To make that the rule for every key, an admin or the owner turns on Keep prompts on Rhumb's server in Settings (off by default): every /v1/plan call is then read by our own classifier, its answer says decision.model: "local", and a request asking for a remote model is read locally too. Test keys always read locally. The change shows in the audit log.

Letting the reading choose the shape

With shape: "auto" the reading chooses between one model, a team and an escalation. The rule, and what a purpose with a shape of its own does to it, is in Teams and escalation.

Counting

A call counts as 5 against the quota, since a reading can call the decision model; a prompt read from the cache counts the same. Errors and no_candidate are not counted.