Browser agent and the Vault

Hand the assistant work on sites that need a login. You watch the assistant operate a browser in real time, check its plan, and take over to type yourself when needed. You'll also see that the Vault holds passwords, card numbers and verification codes so the AI model never sees them, and how two layers of keys protect them.

Why a browser agent#

The assistant could read the mail in Chapter 5 right away. But issuing a tax invoice on a client portal, checking order history on a supplies site, or changing settings in an admin console can only be done in a logged-in browser, because there's no channel for programs to connect directly. The browser agent operates a browser on the server for you. It reads pages, clicks, types and sums up the results.

Two worries come up. Is it safe to give my password to the AI? And what if the AI pays for something on its own? Process GPT Lite answers like this.

A practice site — the test shop#

This chapter uses the test shop included in the repository. It's a small practice site with email and password login, a phone verification code and a card payment screen, and no real money changes hands. Open it like this from the repository folder you got during installation (you need Node.js).

node server/test/fixtures/shop.mjs 8811
# test shop on http://127.0.0.1:8811  (login demo@shop.test / otp 246810)

Try it 1 — log in to a site for the first time#

Send this in the input box.

Log in to the test shop (http://127.0.0.1:8811/login) and tell me my recent orders.

The assistant says it'll log in and check your recent orders, and a task card appears in Chat. The card shows the server browser's screen in real time. The status line above the input box shows what it's doing now, like "Opening 127.0.0.1" or "Reading the page."

When the agent reaches the password field, it stops and puts up a login card. This is a site it hasn't visited before, so there's no login saved in the Vault. An orange Action needed appears on the task card, and an orange warning button appears at the left of the input box. Click the warning button to jump to the card you need to answer.

1. Enter demo@shop.test and Correct-Horse-42 on the login card and click Login.
2. The card changes to "127.0.0.1 login saved," and the server puts the saved values into the page itself to log in.
3. When the site asks for a phone verification code, a verification code card appears this time: "What you enter is used only for this step and isn't saved." One-time values aren't kept in the Vault. Enter 246810 and click Enter code.

When the task finishes (a few minutes in our run), click the pill at the bottom of the card (the task title) to see the plan. You see the steps the agent set at the start, green dots on the finished steps, and the notes it left at each step. Below that comes a summary of the result, and at the end of the summary is a What I actually did list ("Fill in the username from the Vault," "Click 'Sign in'," "Open 127.0.0.1"…). This list isn't the agent's own words. It's the browser actions the server recorded, so you can compare it with the summary. If the agent came across a button that would buy something you didn't ask for, a Not pressed — a purchase you didn't ask for list follows it (in our run, "Click 'Order again'").

Check: The summary includes order #A-1001, Wireless earbuds, ₩89,000, Shipping, plus the line "6-digit verification code sent to your phone passed on · not saved", and a What I actually did list. If the order is missing from the summary, look for it in the plan's notes. (The test shop has no separate order-history page. In our run, the agent tried several addresses for one, left the remaining plan steps open, and reported the order shown on the account page.) Send the same request once more, and this time it logs in automatically from the Vault with no login card, asking only for the verification code.

Try it 2 — view it large and take control#

Click the expand button (↗) on the task card, or Action needed, to view it full screen.

Use Take control for screens a person has to solve, like a CAPTCHA, or when the agent keeps wandering around the same spot. Send the same request once more, click Take control on the verification code screen, then type 246810 yourself into the code field on the screen.

Confirm with Enter and click Give control back. The unused verification code card changes to "Request expired," and the agent carries on. In our runs, the page was already at My account, but the agent opened the login page again, signed in from the Vault and asked for a new verification code on a fresh card. Enter it there, and the task finishes.

When a task is already running — the queue#

There's one browser per user, so tasks run one at a time. If you ask for another browser task before the previous one finishes, the new task waits in line. The task card gets Waiting · (task title), and Chat shows a notice: "The browser does one task at a time. When the task ahead, '…', is done or canceled, I'll start '…'." When the previous task finishes, it starts on its own.

If the agent tries to finish without doing anything in the browser, the server sends it back once. If there's still no action, no recorded fact and no page change, the task is left as Needs confirmation instead of Done, with the notice "The task ended without doing anything in the browser." When that happens, send the same request again or check with Take control.

Try it 3 — pay by card and approve#

At the test shop (http://127.0.0.1:8811/checkout), buy 1 pair of wireless earbuds with my card.

There's no card in the Vault, so this time a Payment card card appears. Enter the cardholder name, card number 4242 4242 4242 4242, 12/30 and 123, then click Save card. The card changes to "Card saved · Visa •••• 4242," and the server fills in the payment form.

Right before clicking the pay button, the agent always stops and puts up an approval card. It says what it's buying, for how much, and with which card. The next time you pay, the card is already in the Vault, so no Payment card card appears and it goes straight to this approval.

Click Approve and it completes the payment and tells you the order number. Click Cancel and it doesn't pay.

Check: The approval card shows the amount and Visa •••• 4242, and after approval the summary has a new order number (#A-2002 in this run). The full card number must not appear anywhere in the chat.

Quick decisions — calling the big model less#

A browser task calls the big model for every action (it reads screenshots too, so it's expensive). So for short decisions, it asks the small routing model first (the same model as the conversation context decisions in Chapter 4).

It does this only when the routing model is sure. Otherwise, the big model decides as usual. By the development team's measurements, the number of big-model calls per task dropped by about 21%. This feature turns on only on servers with the context router connected, so it's off by default in the current install bundle.

The Vault — where secrets live, and how#

Logins and cards you saved during browser tasks are in the Vault. You can also add them yourself.

Saved passwords are never shown again. When you click a login in Settings › Accounts, you can only replace it with a new value.

A safe inside a safe — the master key and data keys#

The Vault uses envelope encryption. Here's how it looks as a safe.

1. Items such as passwords and cards are locked with a key made separately for each user, a data key (DEK on screen).
2. That data key is locked again with a master key (KEK on screen) and kept in the database.
3. The master key lives outside the database — in a key file on the server, or, with the install bundle, in the settings file .env in the install folder (Chapter 1).

So even if a whole database backup leaks, nothing can be opened without the master key. Each item is locked to that user and that site, so it won't open even if it's moved to another user's or another site's slot. Rotating the master key or creating new data keys is the operator's job (Chapter 25).

Click Check in Settings › Security and the server actually opens your items to confirm how they react to tampering and moving. It doesn't show the values.

Below the check results, Vault activity records who did what and when. In entries like "127.0.0.1 login saved · Me" and "127.0.0.1 Login Username · entered in a browser task · Browser agent," you can confirm that whoever used the value was the browser agent (the server), not the model.

Next#

In Part 1, you asked the assistant questions, split conversations, and handed it mail and website work. From the next chapter, in Part 2 Documents and tools, you open the Hangul, Word and Excel documents the assistant made in the editor beside the chat and edit them by talking (Chapter 7).