{"benchmark":"Codex Computer Use native Chrome vs ego (lite) browser automation","date":"2026-09-11","environment":{"hardwareArchitecture":"arm64","operatingSystem":"macOS 26.5.1 (25F80)","codexAppVersion":"26.903.61454","egoVersion":"0.5.1.2","egoChromiumVersion":"152.0.7977.54","targetSite":"https://the-internet.herokuapp.com"},"methodology":{"runsPerCase":3,"timerScope":"Mechanical browser execution after the target page was loaded. Includes observations, actions, waits, and final verification. Excludes model deliberation and initial navigation.","codexPath":"Codex Computer Use operating isolated native Google Chrome windows through macOS accessibility and screenshots.","egoPath":"ego-browser operating one dedicated ego (lite) Space through semantic browser locators and snapshots.","successRule":"The final page state had to match the case-specific assertion. A dispatched action without a verified state change was not counted as success.","limits":["This is a small browser-only benchmark, not a general model-quality benchmark.","The target is a public test site with deterministic controls; results do not predict arbitrary production websites.","The benchmark does not cover CAPTCHA, MFA, checkout, file upload, cross-app workflows, or multi-hour tasks.","Timings include each tool's recommended observation and verification path, so they measure the tested operator stack rather than raw browser-engine speed."]},"cases":[{"id":"form-login","task":"Fill the published test username and password, submit the form, and verify the secure-area success message.","successAssertion":"URL is /secure and the page says 'You logged into a secure area!'.","actions":3},{"id":"dynamic-controls","task":"Remove the checkbox, wait for the asynchronous completion message, enable the disabled input, enter 'benchmark-ready', and verify the value.","successAssertion":"The checkbox is gone, the input is enabled, and its value is 'benchmark-ready'.","actions":3},{"id":"add-remove-elements","task":"Add three buttons, remove two, and verify that exactly one Delete button remains.","successAssertion":"Exactly one Delete button remains.","actions":5},{"id":"table-max-due-extraction","task":"Read the first data table and return the person with the highest amount due, including email and amount.","successAssertion":"Jason Doe — jdoe@hotmail.com — $100.00.","actions":0},{"id":"session-survives-reload","task":"Reload an authenticated secure-area page and verify that the session remains authenticated.","successAssertion":"The reloaded page remains on /secure and shows the secure-area welcome text.","actions":1}],"summary":[{"case":"form-login","codexPasses":"3/3","codexMedianMs":9685,"egoPasses":"3/3","egoMedianMs":1076},{"case":"dynamic-controls","codexPasses":"3/3","codexMedianMs":13012,"egoPasses":"3/3","egoMedianMs":7431},{"case":"add-remove-elements","codexPasses":"3/3","codexMedianMs":15346,"egoPasses":"3/3","egoMedianMs":1806},{"case":"table-max-due-extraction","codexPasses":"3/3","codexMedianMs":284,"egoPasses":"3/3","egoMedianMs":9},{"case":"session-survives-reload","codexPasses":"3/3","codexMedianMs":2358,"egoPasses":"3/3","egoMedianMs":450}],"runs":[{"tool":"Codex Computer Use — native Chrome","case":"form-login","elapsedMs":[9844,9685,8417],"successes":3,"humanTakeovers":0,"note":"One run surfaced Chrome's leaked-password warning after the success state. Continuing in that window required dismissing the browser warning before link actions would work."},{"tool":"ego (lite)","case":"form-login","elapsedMs":[1076,955,1178],"successes":3,"humanTakeovers":0},{"tool":"Codex Computer Use — native Chrome","case":"dynamic-controls","elapsedMs":[13788,13012,12471],"successes":3,"humanTakeovers":0},{"tool":"ego (lite)","case":"dynamic-controls","elapsedMs":[7370,7461,7431],"successes":3,"humanTakeovers":0},{"tool":"Codex Computer Use — native Chrome","case":"add-remove-elements","elapsedMs":[15637,14411,15346],"successes":3,"humanTakeovers":0},{"tool":"ego (lite)","case":"add-remove-elements","elapsedMs":[1806,1777,3477],"successes":3,"humanTakeovers":0},{"tool":"Codex Computer Use — native Chrome","case":"table-max-due-extraction","elapsedMs":[288,284,279],"successes":3,"humanTakeovers":0,"answer":"Jason Doe — jdoe@hotmail.com — $100.00"},{"tool":"ego (lite)","case":"table-max-due-extraction","elapsedMs":[9,9,8],"successes":3,"humanTakeovers":0,"answer":"Jason Doe — jdoe@hotmail.com — $100.00"},{"tool":"Codex Computer Use — native Chrome","case":"session-survives-reload","elapsedMs":[2358,1572,2386],"successes":3,"humanTakeovers":0},{"tool":"ego (lite)","case":"session-survives-reload","elapsedMs":[450,304,582],"successes":3,"humanTakeovers":0}],"excludedPilots":["A Codex in-app Browser table pilot used an incremental accessibility snapshot that returned 'no change'; the standardized table run used the full DOM snapshot instead and is not part of the native Chrome result above.","An ego (lite) add/remove pilot used an invalid nth locator syntax. The selector was corrected before the three timed runs.","A native Chrome add/remove pilot verified too early and produced an ambiguous count. The final three runs waited for the asserted button count after every action."]}