SDKs
Browser tasks from your own code, in TypeScript and Python, with the same names in both: whole tasks in pilotfish's cloud, or Chrome on your own machine.
Both send pages as compact outlines with refs and, after the first look, only what changed: an agent built on them reads far fewer tokens per step, and each step answers sooner. Sources: packages/sdk (TypeScript, Node 20.3+) and sdks/python (Python 3.9+, sync and async), MIT.
Install
TypeScript and JavaScript:
npm install @gaiventures/pilotfishPython:
pip install pilotfish-aiHosted: tasks in pilotfish's cloud
Make an API key in the app (Org settings → API keys) and set PILOTFISH_API_KEY. tasks.run starts a task and waits until it ends, or until it waits on a question for a person.
import { Pilotfish } from "@gaiventures/pilotfish";
const pf = new Pilotfish(); // reads PILOTFISH_API_KEY
const task = await pf.tasks.run({
task: "Find the price of the Logitech MX Keys on amazon.com",
outputSchema: {"type":"object","properties":{"product":{"type":"string"},"price":{"type":"number"},"currency":{"type":"string"}},"required":["product","price","currency"],"additionalProperties":false},
});
console.log(task.status, task.result?.data ?? task.result?.result);from pilotfish import Pilotfish
pf = Pilotfish() # reads PILOTFISH_API_KEY
task = pf.tasks.run({
"task": "Find the price of the Logitech MX Keys on amazon.com",
"outputSchema": {"type":"object","properties":{"product":{"type":"string"},"price":{"type":"number"},"currency":{"type":"string"}},"required":["product","price","currency"],"additionalProperties":False},
})
print(task["status"], task["result"].get("data") or task["result"]["result"])Steer, answer, follow up
const t = await pf.tasks.create("Book a table for two at 8 pm tomorrow at Olive, Bandra");
for await (const event of pf.tasks.stream(t.id)) console.log(event.type); // each event once, until it ends
await pf.tasks.message(t.id, "make it 8:30 instead"); // read before its next step
const waiting = await pf.tasks.wait(t.id);
if (waiting.question) await pf.tasks.answer(t.id, waiting.question.id, "yes");
// Codes and passwords never go through the API: the person enters them on waiting.answerUrl.
const next = await pf.tasks.followUp(t.id, "add a note: window seat"); // same browser, same sign-ins
if ((await pf.tasks.get(next.id)).next === "continue") await pf.tasks.continue(next.id);t = pf.tasks.create("Book a table for two at 8 pm tomorrow at Olive, Bandra")
for event in pf.tasks.stream(t["id"]): # each event once, until it ends
print(event["type"])
pf.tasks.message(t["id"], "make it 8:30 instead") # read before its next step
waiting = pf.tasks.wait(t["id"])
if waiting.get("question"):
pf.tasks.answer(t["id"], waiting["question"]["id"], "yes")
# Codes and passwords never go through the API: the person enters them on waiting["answerUrl"].
nxt = pf.tasks.follow_up(t["id"], "add a note: window seat") # same browser, same sign-ins
if pf.tasks.get(nxt["id"]).get("next") == "continue":
pf.tasks.continue_(nxt["id"])What each client has
| Resource | TypeScript | Python | What |
|---|---|---|---|
tasks | create run get wait list iterate events stream followUp continue answer message cancel pause resume rerun thread files downloadFile frames replayMp4 deleteSession | the same in snake_case (follow_up, continue_, download_file, replay_mp4, delete_session) | Tasks (the API's jobs): run, watch, steer and read them. |
batches | create run get wait list cancel | the same | Many tasks in one request: Batches. |
identities | create get list update delete usage addCredential credentials setDefaultCredential deleteCredential loginLink | the same in snake_case | Your product's end users, with an org key in a shared project. |
savedTasks / saved_tasks | list create update delete run | the same | Tasks to run again (Personal project). |
schedules | list create update delete | the same | A saved task run by itself. |
usage | get | get | Spending by kind, day, model and task. |
Python's AsyncPilotfish has the same methods as coroutines (stream and iterate as async iterators). Results are the API's JSON: objects in TypeScript (typed), dicts in Python, with the API's own field names (REST API).
Errors and retries
Each error is the API's {error, code, hint, docs, fields, retryAfterS} as an exception: BadRequestError (400, with fields), AuthenticationError (401), PaymentRequiredError (402), PermissionDeniedError (403), NotFoundError (404), ConflictError (409), RateLimitError (429), ServerError (5xx), ConnectionError, and WaitTimeoutError from wait (the task carries on); all are PilotfishErrors with code (Python: retry_after_s). A 429 is retried after its retryAfterS, POSTs included (it refused before anything was made); a 5xx or a network failure is retried for reads only; maxRetries / max_retries (default 2).
Webhooks
Pass the raw body as received, before any JSON parsing, with the Pilotfish-Signature header and your organization's webhook secret (Org settings):
import { parseWebhook } from "@gaiventures/pilotfish";
const event = await parseWebhook(process.env.PILOTFISH_WEBHOOK_SECRET!, req.headers["pilotfish-signature"], rawBody);
if (event.type === "batch.finished") console.log(event.data.counts);from pilotfish import parse_webhook
event = parse_webhook(os.environ["PILOTFISH_WEBHOOK_SECRET"], request.headers["pilotfish-signature"], raw_body)
if event["type"] == "batch.finished":
print(event["data"]["counts"])Local: Chrome on your machine
The local client starts the open MCP (@gaiventures/pilotfish-mcp) and talks to it over stdio: the installed package when there is one, else npx -y @gaiventures/pilotfish-mcp, so Node.js 22+ must be on the machine (Python included). Every browser tool is a method; your code reads each result and decides the next step.
import { LocalBrowser } from "@gaiventures/pilotfish";
const browser = await LocalBrowser.start({ headless: false });
try {
await browser.navigate("https://news.ycombinator.com");
const top = await browser.query<string[]>("[...document.querySelectorAll('.titleline > a')].slice(0, 3).map((a) => a.textContent)");
console.log(top.value);
const page = await browser.snapshot(); // the page as an outline with refs: [e12]<a>new
console.log(page.text);
} finally {
await browser.close();
}from pilotfish import LocalBrowser
with LocalBrowser.start(headless=False) as browser:
browser.navigate("https://news.ycombinator.com")
top = browser.query("[...document.querySelectorAll('.titleline > a')].slice(0, 3).map((a) => a.textContent)")
print(top.value)
print(browser.snapshot().text) # the page as an outline with refs: [e12]<a>new| Method (TypeScript / Python) | Open MCP tool |
|---|---|
navigate | browser_navigate |
snapshot | browser_snapshot |
click | browser_click |
type | browser_type |
select | browser_select |
press | browser_press |
scroll | browser_scroll |
actions | browser_actions |
query | browser_query |
find | browser_find |
getText / get_text | browser_get_text |
screenshot | browser_screenshot |
wait | browser_wait |
resume | browser_resume |
tabs | browser_tabs |
pageTool / page_tool | browser_page_tool |
close | browser_close |
call(name, args) | any tool by name |
Each returns the tool's text (the page, what changed, what needs a person), its images and structured value; a tool's error is raised as LocalToolError (throwOnError: false / throw_on_error=False returns it). Options: headless, allowedHosts, allowPrivate (localhost, for testing your own app), secrets (typed as {{name}}, never in any result), chrome, env (any of the open MCP's settings), command.
A step that looks irreversible (pay, place an order, send, delete) needs a yes, and a page that needs a person (a sign-in, a code, a CAPTCHA) is theirs to handle in the window. Pass onQuestion / on_question to answer them (kind is confirm or handoff); without it an irreversible step is refused, and a page that needs a person says NEEDS A PERSON (call resume() once it is handled).