Stabilizer + Runner + BrowserContext: isStable, stabilize, runPage, createStrictBrowserContext #14

New issue

Closed

opened 2026-06-17 13:37:44 +00:00 by Artur · 1 comment

Artur commented

2026-06-17 13:37:44 +00:00

Owner

Goal

Implement the stabilization algorithm and the main runPage() function.

What to Build

src/pages/stabilizer.ts

export interface StabilityConfig {
  maxTicks: number;           // default: 10000
  stableTicksRequired: number; // default: 3
  mutationQuietPeriod: number; // default: 5 (ticks with no DOM changes)
}

export class Stabilizer {
  constructor(config?: Partial<StabilityConfig>);

  async stabilize(ctx: BrowserContext): Promise<void> {
    // Loop until stable or maxTicks reached:
    // 1. Check isStable()
    // 2. If stable for N consecutive ticks: done
    // 3. If not stable: await new Promise(r => setTimeout(r, 0))
    //    -> This yields to Bun's event loop
    //    -> JSC processes all pending microtasks, timers, I/O
    // 4. Increment tick counter
    // 5. If maxTicks reached: force-stop, log warning
  }

  isStable(ctx: BrowserContext): boolean {
    // ALL of these must hold:
    // - Parser finished: !ctx.parser.isActive()
    // - No pending blocking scripts: !ctx.scriptLoader.hasPendingBlocking()
    // - No pending modules: !ctx.scriptLoader.hasPendingModules()
    // - No pending critical network: ctx.network.hasPendingCritical()
    // - No due timers: !ctx.timerController.hasDueTimers()
    // - No pending mutation observer callbacks
    // - DOM unchanged for N ticks (mutation count check)
  }
}

src/pages/runner.ts

export interface PageResult {
  dom: string;
  mutations: unknown[];
  network: unknown[];
  cookies: Record<string, unknown>;
  storage: Record<string, unknown>;
  appState: unknown[];
  apisUsed: string[];
  timing: {
    start: number;
    parseEnd: number;
    scriptsDone: number;
    stable: number;
  };
}

export async function runPage(url: string): Promise<PageResult> {
  // 1. Create browser context
  const ctx = createStrictBrowserContext(url);

  // 2. Fetch document HTML
  const html = await ctx.fetchDocument(url);

  // 3. Start incremental parser
  ctx.parser.start(html);

  // 4. Wait for parser + scripts to settle
  //    The parser->ScriptLoader->parser cycle is automatic
  //    via the onScriptTag callback

  // 5. After parser finishes:
  //    - Execute deferred scripts
  //    - Fire DOMContentLoaded
  //    - Execute module scripts
  //    - Fire Load event

  // 6. Stabilize
  await ctx.stabilizer.stabilize(ctx);

  // 7. Collect results
  return {
    dom: ctx.document.documentElement?.outerHTML ?? '',
    network: ctx.network.log.getEntries(),
    cookies: ctx.cookies.dump(),
    storage: {
      localStorage: ctx.localStorage.entries(),
      sessionStorage: ctx.sessionStorage.entries(),
    },
    mutations: ctx.mutationLog,
    appState: ctx.appStateSnapshots,
    apisUsed: ctx.apiLogger.getEntries().map(e => e.path),
    timing: { start: ctx.timing.start, parseEnd: ctx.timing.parseEnd, ... },
  };
}

src/browser-context.ts

export interface BrowserContext {
  window: Window;
  document: Document;
  parser: IncrementalParser;
  scriptLoader: ScriptLoader;
  cookieJar: CookieJar;
  fetch: typeof fetch;
  xhr: typeof XMLHttpRequest;
  websocket: typeof WebSocket;
  networkLog: NetworkLog;
  localStorage: LocalStorage;
  sessionStorage: SessionStorage;
  apiLogger: APILogger;
  timerController: TimerController;
  stabilizer: Stabilizer;
  tolerantProxy: object;
  // ... all other components
  fetchDocument(url: string): Promise<string>;
  isStable(): boolean;
}

export function createStrictBrowserContext(url: string): BrowserContext {
  // 1. Create CookieJar
  // 2. Create NetworkLog
  // 3. Create TimerController
  // 4. Create APILogger
  // 5. Create FakeLayout with config
  // 6. Create IsolatedContext (window with strictObject + tolerantProxy)
  // 7. Install everything on window:
  //    - fetch, XMLHttpRequest, WebSocket cookies
  //    - localStorage, sessionStorage, document.cookie
  //    - navigator, screen, location, history
  //    - performance, console
  //    - canvas, WebGL
  //    - media, fonts, images
  //    - permissions, credentials, geolocation
  //    - IntersectionObserver, ResizeObserver
  //    - setTimeout, setInterval, requestAnimationFrame (via TimerController)
  // 8. Create ScriptLoader
  // 9. Create IncrementalParser (with onScriptTag -> ScriptLoader)
  // 10. Create Stabilizer
  // 11. Return context object with everything wired
}

Tests

Unit Tests

Test	Verifies
stabilizer.basic.test.ts	Stabilizer returns stable when all queues empty
stabilizer.pending-script.test.ts	Not stable while script pending
stabilizer.pending-timer.test.ts	Not stable while timer due
stabilizer.dom-change.test.ts	Not stable while DOM changing
stabilizer.stable-ticks.test.ts	Requires N consecutive stable ticks
stabilizer.max-ticks.test.ts	Force-stops after maxTicks
runner.basic.test.ts	runPage() returns PageResult with correct shape

Integration Tests

Test	Verifies
runner.simple-page.test.ts	Simple HTML page with inline script passes
runner.external-script.test.ts	Page with external script loads and executes
runner.storage.test.ts	Page sets localStorage -> storage dump contains it
runner.network.test.ts	Page makes fetch -> network log contains it
runner.timing.test.ts	Timing fields are populated

Definition of Done

src/pages/stabilizer.ts implemented
src/pages/runner.ts implemented (runPage)
src/browser-context.ts implemented (createStrictBrowserContext)
All components wired together and working
All tests pass
100% line + branch coverage

## Goal Implement the stabilization algorithm and the main runPage() function. ## What to Build ### src/pages/stabilizer.ts ``` export interface StabilityConfig { maxTicks: number; // default: 10000 stableTicksRequired: number; // default: 3 mutationQuietPeriod: number; // default: 5 (ticks with no DOM changes) } export class Stabilizer { constructor(config?: Partial<StabilityConfig>); async stabilize(ctx: BrowserContext): Promise<void> { // Loop until stable or maxTicks reached: // 1. Check isStable() // 2. If stable for N consecutive ticks: done // 3. If not stable: await new Promise(r => setTimeout(r, 0)) // -> This yields to Bun's event loop // -> JSC processes all pending microtasks, timers, I/O // 4. Increment tick counter // 5. If maxTicks reached: force-stop, log warning } isStable(ctx: BrowserContext): boolean { // ALL of these must hold: // - Parser finished: !ctx.parser.isActive() // - No pending blocking scripts: !ctx.scriptLoader.hasPendingBlocking() // - No pending modules: !ctx.scriptLoader.hasPendingModules() // - No pending critical network: ctx.network.hasPendingCritical() // - No due timers: !ctx.timerController.hasDueTimers() // - No pending mutation observer callbacks // - DOM unchanged for N ticks (mutation count check) } } ``` ### src/pages/runner.ts ``` export interface PageResult { dom: string; mutations: unknown[]; network: unknown[]; cookies: Record<string, unknown>; storage: Record<string, unknown>; appState: unknown[]; apisUsed: string[]; timing: { start: number; parseEnd: number; scriptsDone: number; stable: number; }; } export async function runPage(url: string): Promise<PageResult> { // 1. Create browser context const ctx = createStrictBrowserContext(url); // 2. Fetch document HTML const html = await ctx.fetchDocument(url); // 3. Start incremental parser ctx.parser.start(html); // 4. Wait for parser + scripts to settle // The parser->ScriptLoader->parser cycle is automatic // via the onScriptTag callback // 5. After parser finishes: // - Execute deferred scripts // - Fire DOMContentLoaded // - Execute module scripts // - Fire Load event // 6. Stabilize await ctx.stabilizer.stabilize(ctx); // 7. Collect results return { dom: ctx.document.documentElement?.outerHTML ?? '', network: ctx.network.log.getEntries(), cookies: ctx.cookies.dump(), storage: { localStorage: ctx.localStorage.entries(), sessionStorage: ctx.sessionStorage.entries(), }, mutations: ctx.mutationLog, appState: ctx.appStateSnapshots, apisUsed: ctx.apiLogger.getEntries().map(e => e.path), timing: { start: ctx.timing.start, parseEnd: ctx.timing.parseEnd, ... }, }; } ``` ### src/browser-context.ts ``` export interface BrowserContext { window: Window; document: Document; parser: IncrementalParser; scriptLoader: ScriptLoader; cookieJar: CookieJar; fetch: typeof fetch; xhr: typeof XMLHttpRequest; websocket: typeof WebSocket; networkLog: NetworkLog; localStorage: LocalStorage; sessionStorage: SessionStorage; apiLogger: APILogger; timerController: TimerController; stabilizer: Stabilizer; tolerantProxy: object; // ... all other components fetchDocument(url: string): Promise<string>; isStable(): boolean; } export function createStrictBrowserContext(url: string): BrowserContext { // 1. Create CookieJar // 2. Create NetworkLog // 3. Create TimerController // 4. Create APILogger // 5. Create FakeLayout with config // 6. Create IsolatedContext (window with strictObject + tolerantProxy) // 7. Install everything on window: // - fetch, XMLHttpRequest, WebSocket cookies // - localStorage, sessionStorage, document.cookie // - navigator, screen, location, history // - performance, console // - canvas, WebGL // - media, fonts, images // - permissions, credentials, geolocation // - IntersectionObserver, ResizeObserver // - setTimeout, setInterval, requestAnimationFrame (via TimerController) // 8. Create ScriptLoader // 9. Create IncrementalParser (with onScriptTag -> ScriptLoader) // 10. Create Stabilizer // 11. Return context object with everything wired } ``` ## Tests ### Unit Tests | Test | Verifies | |------|----------| | stabilizer.basic.test.ts | Stabilizer returns stable when all queues empty | | stabilizer.pending-script.test.ts | Not stable while script pending | | stabilizer.pending-timer.test.ts | Not stable while timer due | | stabilizer.dom-change.test.ts | Not stable while DOM changing | | stabilizer.stable-ticks.test.ts | Requires N consecutive stable ticks | | stabilizer.max-ticks.test.ts | Force-stops after maxTicks | | runner.basic.test.ts | runPage() returns PageResult with correct shape | ### Integration Tests | Test | Verifies | |------|----------| | runner.simple-page.test.ts | Simple HTML page with inline script passes | | runner.external-script.test.ts | Page with external script loads and executes | | runner.storage.test.ts | Page sets localStorage -> storage dump contains it | | runner.network.test.ts | Page makes fetch -> network log contains it | | runner.timing.test.ts | Timing fields are populated | ## Definition of Done - [ ] src/pages/stabilizer.ts implemented - [ ] src/pages/runner.ts implemented (runPage) - [ ] src/browser-context.ts implemented (createStrictBrowserContext) - [ ] All components wired together and working - [ ] All tests pass - [ ] 100% line + branch coverage