Explore the interactive work
Evidence Guide
Ask about my work.
Explore a decision, a result or my contribution. Follow the answer to the published evidence behind it.
Ask the Evidence Guide How I evaluated the GuideMIN–MAX Living · Delivery model
Follow the work through the system.
Explore the 3D map of responsibilities, review, incidents and lessons. Use the guided story or the accessible outline.
Explore the 3D delivery model Read the MIN–MAX product caseThe delivery model and reviewed Evidence Guide run here.
The systems, their users and their current evidence
| System and task | Operating context | Evidence and next question |
|---|---|---|
| Enterprise performance investigation: help product and engineering prioritize and test | Delivered AI-assisted practice. People own technical tests and releases. | UAT average request time: 0.910s to 0.228s; separate from live-user measurement |
| Portfolio search: find a case | Browser search of reviewed content. | Find a relevant case and its evidence. |
| Evidence Guide: answer a visitor’s question with sources | Model-assisted evidence selection with application-controlled factual wording. | Agent evaluation on authored questions; visitor task comparison is next |
| MIN–MAX Living: compare rental options | Working private prototype. Deterministic comparisons; renter makes the final choice. | Prototype behavior demonstrated; renter research and external beta are next |
01 · Enterprise performance and release intelligence
I took an AI-assisted investigation and release workflow from the user-visible performance problem through delivery. I defined the evidence, candidate comparisons and conditions for release.
The LLM organized approved diagnostic material, identified gaps, proposed hypotheses and drafted communication. Engineering and product owners inspected the evidence, selected what to test and approved any production change.
From a signal to a release decision
Reconstructed from the delivered practice. The LLM organized evidence and drafted hypotheses. People controlled testing and production decisions.
Telemetry, query plans, work items and test results.
Connect patterns, identify gaps and draft testable hypotheses.
Choose a bounded candidate, compare behavior and approve the next step.
Inspect the candidate acceptance criteria
| Question | Evidence required | Product consequence |
|---|---|---|
| Can users still find the record? | Representative search and workflow scenarios | Protect the task alongside response time |
| What does the faster read cost? | Read/write comparison, maintenance and downstream checks | Evaluate the whole change |
| Can we explain and reverse it? | Traceable tested version, release scope and rollback readiness | Make a bounded release decision |
Documented team release result · UAT load tests
74.9% lower · June 28 to July 6, 2026
1,000 virtual users. Test configuration recorded as 80 CPUs. Source: July 2026 internal performance update.
Independent product · Evidence Guide
A question, an answer, a release decision.
I designed the Guide to help a visitor understand my work and inspect the source. Follow the configuration choices, the failures that changed the design and the evidence behind the release.
Guide release record · 7 October 2026
Ask. Read. Check the source.
The released configuration uses a model to interpret questions and select evidence. The application checks those selections and assembles reviewed facts with their qualifications.
Agent reviewers found useful, source-backed answers in 48/60 private-test questions and 19/22 hosted-QA questions across separate sets. Human visitor usefulness, understanding and recall are the next measures to test.
Open the Evidence GuideAsk about Ola’s workMY ROLE
Product direction and the release decision.
I own the product goal, evidence standards, priorities and release decisions. Implementation and evaluation were agent-assisted.
I compare useful completion, source accuracy, cost and delivery time when deciding what earns its place.
From exploration to release
How the release choice took shape.
Two questions shaped the work: does model selection help visitors, and which repaired configuration is ready to move through release? Each experiment answers a different part of that decision.
Decision history · local editorial draft · separate candidates and question banks
Four configurations. Then a repaired Luna path.
| Arm / configuration | Useful | Contract rejected | Controls | Disposition |
|---|---|---|---|---|
| A · MiniExisting instructions | 21/30 | 3 | 8/10 | Overall private reference |
| B · MiniRevised instructions | 13/30 | 11 | 2/10 | Not promoted |
| C · LunaRevised instructions | 20/30 | 8 | 8/10 | Luna parent for repair |
| D · LunaA’s existing instructions | 15/30 | 12 | 5/10 | Exact configuration retired |
Mini: archived gpt-5-mini-2025-08-07. Luna: archived gpt-6-luna. D’s original first-submission result was 14/30; 15/30 includes a recovered workflow submission. No challenger met the frozen rule; A remained a reference.
1 / 6 · Compare A–D
No arm qualified for release.
A: 21/30 · C: 20/30
- Decision criteria
- The frozen challenger rule required at least three net gains over A, all ten historical controls, no sampled material unsupported claim and validated P95 ≤45 seconds.
- What happened
- Keep A as the overall reference and C as the better tested Luna parent. B was not promoted; the exact D configuration was retired.
Same 30 consumed diagnostic questions: 20 historical failures and ten controls. D scored 15/30 on the first-provider path and 14/30 on original first submissions. Execution conditions changed; this was not a general model ranking.
Record identity
A–D reviewed archive · candidate 60581a0
Read the full decision path and final configuration
Compare A–D: No arm qualified for release.
A: 21/30 · C: 20/30
Criteria: The frozen challenger rule required at least three net gains over A, all ten historical controls, no sampled material unsupported claim and validated P95 ≤45 seconds.
Keep A as the overall reference and C as the better tested Luna parent. B was not promoted; the exact D configuration was retired.
Same 30 consumed diagnostic questions: 20 historical failures and ten controls. D scored 15/30 on the first-provider path and 14/30 on original first submissions. Execution conditions changed; this was not a general model ranking.
A–D reviewed archive · candidate 60581a0
Repair the contract: Fix application responsibilities before another model decision.
V4 C: 12/16 → V5 C: 14/16
Criteria: Retain application-owned question references, approved evidence, useful controls and the first exact source. Keep original failures in the ledger.
Move literal references and coordinates into application code; align requested decisions, opening facts and first sources. Carry C forward for the bounded decision-focus correction.
Reused diagnostic bank. Contemporary A scored 9/16 in V4 and 8/16 in V5; C retained 8/10 then 9/10 controls. Both comparisons remained HOLD under the all-ten rule. These were whole-configuration comparisons, not isolated causal estimates. V6 later reported 16/16 on the consumed main diagnostic, not fresh acceptance.
V4/V5 reviews · V6 verification
Test fresh questions: The private progression rule passed.
Luna: 48/60 useful first submissions
Criteria: The separately authorized progression threshold was at least 41/60 on one independently authored sealed bank, following the owner’s greater-than 40/60 decision.
Proceed to local integration and preservation work. Recruiter tasks scored 23/30; operations tasks 25/30.
One bank, independently agent-scored. Twelve original failures remained, including two unsupported-claim flags relative to the frozen corpus. The test explicitly supplied Luna settings; the default application caller still differed and needed correction.
Fresh60 · candidate 15be646 · original result unchanged
Qualify the caller: Connect the tested configuration to the actual application.
21/21 component answers and first sources
Criteria: Match the tested model, instruction lineage and effective settings through the real caller; preserve prior outcomes; close exact-source gaps; pass source and runtime checks.
Wire the C-derived Luna configuration and finish composition/source corrections. Retain 48/48 previously useful answers and 60/60 original records. Final local preservation: 2,657 pass, zero fail, three disclosed skips.
The 21 component observations combine five captured selections with 16 legal selections. They are not 21 fresh model answers. This qualification used zero new provider calls.
Local qualification · c3552ff · 43/43 bounded sources
Verify hosted QA: Deploy the qualified source, then observe the hosted path.
QA93: 19/22 joint useful · 22/22 first sources
Criteria: Explicit QA authorization, exact source/package/environment custody, bounded source readbacks, mobile/keyboard journeys, fallback presentation and retained rollback.
QA92 verified the deployment with 43/43 source checks and intercepted journeys. A separately authorized QA93 trial then compared the actual Luna path with source-only: 19/22 versus 15/22 joint useful outcomes.
QA93 was a 22-question diagnostic, with six paired gains and two losses. The source-only comparator ran offline on the same approved corpus; it was not an observed hosted fallback or a model-only causal comparison. Three Luna task omissions remained. It was not a new fresh acceptance bank or a production-usefulness score.
QA92 c3552ff · QA93 d3fa5f7 · separate native receipts
Promote to production: Protected promotion was its own release decision.
Production v57 · gpt-6-luna
Criteria: Owner authorization, protected public repository review and merge, required native CI/security checks, exact deployment/source/environment readback, spending controls and retained rollback.
Promote merge a06e287 to production environment 17. Keep validated source fallback, pause QA paid calls and retain v56 for rollback. The released configuration is owner-selector-luna-c-v6-request-meaning-v1.
Release checks verified deployment and source/navigation paths. They did not produce a new paid production answer or establish human comprehension, recall or population usefulness. Raw visual differences and original skips remain recorded.
Release closeout · 7 October 2026 · voice input off
The final application selects reviewed evidence and permitted operations using gpt-6-luna, medium reasoning, default service tier and 8192 output tokens. Application code checks and renders factual material. Legacy mini settings remain for historical reproduction; they do not select the production owner-path model.
Earlier source-only comparison · a separate experiment
The comparison below tested a different historical candidate and question bank. Its source-only recommendation applies to that experiment.
Keep the simpler path for this experiment.
The model selector added cost and latency without improving useful answers overall.
Inspect the evaluationSource-only
Recommended80%useful answers
96 of 120 questions
Deterministic retrieval · zero model calls
Optional selector
Not preferred79.2%useful answers
95 of 120 questions
Model selects reviewed material
Previously unsuccessful answers became useful
Previously useful answers became unsuccessful
useful answer across the paired evaluation
Earlier local experiment · same 120 questions per path · independent agent scoring. The source-only path left 24 questions unresolved.
Inspect the test design and remaining work
How the test worked
Two independent authors created 60-question sets. The baseline returned 48 useful answers in each; the selector returned 48 and 47. Independent agent reviewers scored the same-question first answers, with adjudication for disagreements.
What still needs improvement
24 source-only answers were not useful. Next validation covers actual visitor outcomes, the current runtime, a completed tool-agent workflow and model comparison.
The production workflow case informed my approach to states, exceptions, ownership and controlled release. Its production history is separate from this assistant’s maturity.
How it works
Each step has a clear job.
Source and configuration verified in the 7 October 2026 release record; this diagram is not a live availability check.
- 01Visitor request
The application keeps the full question and issues stable request references.
- 02Approved evidence
Reviewed public facts, exact sources and material qualifications form the selectable catalog.
- 03Luna selects
gpt-6-luna selects evidence references, a permitted operation and request annotations. Factual prose stays in the application.
- 04Application checks
Validate references, applicability, source identity, withdrawal and required qualifications.
- 05Compile + present
Render approved factual wording, evidence limits and the first exact source.
Selection cannot be used?
Provider unavailable, spending blocked or selection rejected → use the source-only fallback. Unsupported parts remain explicit gaps with source navigation.
I own the evidence and release.
Approved facts, qualifications, withdrawals, spending limits and activation remain human-controlled. Voice was off in this release.
Expand the detailed architecture · six stages
Follow each handoff from question to answer. Open a stage to inspect its inputs, responsibilities, outputs and recovery limits.
01 · Request custody and interpretation
- Input
- The visitor’s full original question.
- Responsibility
- Application code owns request identity, exact coordinates and verbatim extraction. The selector interprets requested actors, cases, phases, conditions and deliverables using application-issued references.
- Output
- Request-bound selections and annotations for the requested tasks.
- Failure / limit
- A valid whole-question reference proves custody, not completion. Missing clauses and changed premises still require semantic evaluation.
02 · Evidence catalog and source binding
- Input
- Reviewed public evidence and exact source identities.
- Responsibility
- Build selectable evidence references from approved material. Preserve source/version binding, maturity, attribution, essential qualifications and required companions.
- Output
- Eligible evidence choices tied to reviewed wording and sources.
- Failure / limit
- Private, unknown, withdrawn or incompatible evidence cannot become a public claim. A source match alone does not establish applicability.
03 · Metered Luna selection
- Input
- The original request, application-issued references, approved catalog and permitted operations.
- Responsibility
- The protected provider path applies spending and runtime controls. Luna selects evidence, a permitted operation, focal supports and first-source support. It returns structured selections, not public factual prose.
- Output
- A proposed selection plan; it has not yet earned rendering authority.
- Failure / limit
- Provider unavailability, blocked spending or unusable output leads to the validated source-only recovery path. No additional writer or judge call is part of this diagram.
04 · Contract and applicability checks
- Input
- The proposed plan and its bound catalog/source identities.
- Responsibility
- Check known references, operation compatibility, source status and withdrawal. Enforce required qualifications and companions. Match the selected evidence to the requested actor, phase and condition.
- Output
- An accepted plan or a specific rejection.
- Failure / limit
- Structural validity cannot guarantee full task completion. Wrong-story, wrong-phase and missing-consequence failures remain separate semantic evaluation targets.
05 · Compile the first answer and source
- Input
- Accepted evidence selections, approved factual text and essential qualifications.
- Responsibility
- Application code composes the first-visible answer and selects the task-relevant exact source. Related story detail can sit behind expansion; essential requested facts and limits belong in the first answer.
- Output
- Reviewed factual wording, material limits and inspectable source navigation.
- Failure / limit
- Unsupported requested parts remain explicit gaps. Expansion does not rescue an omitted essential answer or an irrelevant first source.
06 · Recovery, observation and release authority
- Input
- Provider/validation outcomes and the source-only answer path.
- Responsibility
- Retain fallback and original evaluation outcomes. Keep provider, compiler, browser delivery and source-readback observations distinct. I control evidence approval, withdrawals, spending, activation and release.
- Output
- A bounded answer or gap, with deployment and evaluation evidence recorded separately.
- Failure / limit
- A successful source check or deployment does not prove a useful visitor answer. The release record retained rollback v56, paused QA paid calls and kept voice off.
Scope: released v57 source/configuration record, 7 October 2026. These responsibilities describe the application contract; semantic completion still needs independently scored questions and first-source review.
Separate external-assistant path and configuration
External assistant ↔ read-only MCP search / fetch ↔ approved public evidence. This transport is separate from the Guide’s application-controlled answer.
Released selector: gpt-6-luna · medium reasoning · default service tier · 8192 maximum output tokens. Configuration: owner-selector-luna-c-v6-request-meaning-v1.
The model selects; application code validates and renders. Release verification established source and configuration custody, not a new production usefulness score.
Explore a real hours-saved estimate and a five-submission teaching example.
Historical architecture · visual map and runnable reference
Solid flow: source-only baseline. Separate panels: optional experiments and external access.
- 01Visitor question
Clauses, cases, exclusions and evidence limits
- 02Approved evidence
Exact + lexical retrieval; eligibility and withdrawal checks
- 03Reviewed answer
Compose approved wording; preserve limits and gaps
- 04Validation
Check coverage, citations and source status
- 05Answer + exact source
Short answer, optional story and source navigation
Between retrieval and validation
Meter reservation → model selects reviewed IDs → validate → accept or recover to baseline.
Models do not supply public factual prose.
Search → fetch → select
One search round → up to six eligible fetches → final ID selection. At most three completions; 15-second overall deadline.
Observed trial: three empty searches, then stop. Fetch and final selection were not reached.
I control evidence and release
Reviewed facts, copy, withdrawals, activation, spending and QA/production promotion.
External MCP client ↔ public tools
Initialize → discover → search / fetch approved evidence. The Guide does not call the hosted MCP transport.
Across the evaluated workflow: content-free stage observations, exact source identities and validated recovery. No persistent visitor memory. Current QA has later changes and has not been re-scored against the frozen sets.
The downloadable reference reproduces the historical source-only path and simulated recovery fixtures. It is separate from the later model-enabled release.
Explore the evidence
Choose the depth you need.
See the work behind the approach.
Production workflows, migration and team decisions.
Read the platform case →Examine the answers.
Questions, scoring, paired results and unsuccessful examples.
Inspect the evaluation →Follow an execution.
Retrieval, tool contracts, recovery and observed costs.
Explore execution traces →More research and the next study
The next visitor study asks whether people can choose a relevant case, explain its evidence and limits, and remember the contribution.
What we can conclude
Useful answers need more than a capable model.
Selection must earn its place.
In the earlier paired 120-question experiment, model selection recovered seven answers and degraded eight: 95 useful answers versus 96 for source-only. That candidate supported keeping the simpler path.
The application contract matters.
The A–D comparison exposed failures in request references, evidence selection and task completion. Later work moved literal references into application code and tightened composition and first-source alignment. A valid selection still needs to answer the right question.
The repaired Luna path earned progression.
Agent reviewers found useful answers and first sources in 80% (48/60) of fresh private-test questions. That cleared the approved private progression threshold. Hosted QA separately reached 86% (19/22), with useful first sources in all 22 cases.
My product decision
Carry the repaired Luna configuration forward.
Measure success by whether the visitor’s question gets answered with relevant evidence.
- Reviewed facts
The application controls factual wording.
- Exact sources
Check the evidence behind each answer.
- Source-only recovery
Keep evidence usable when model selection cannot be used.
Can they understand the work, inspect the right evidence and remember the difference?
Experiment limitations
What these tests tell us, and what comes next.
Agent-scored task completion
Reviewers assessed whether the first answer completed the requested task with supporting evidence and a useful first source. Human usefulness, understanding and recall remain the next study.
Separate questions, separate results
The private 60-question test and hosted 22-question diagnostic used different sets. Their scores describe each test; they do not establish an improvement trend or a production visitor rate. Reused diagnostic questions informed repairs.
Configuration matters
Results belong to the tested evidence, instructions, application and execution conditions. They do not rank models universally. Release checks verified the deployed source and configuration; ongoing production answer quality needs its own measurement.
Read the remaining measurement limits
Component checks, simulated failures and source readbacks establish specific behaviour. They do not replace fresh end-to-end answers, human accessibility review or visitor research. QA93’s source-only comparator ran offline, so its paired results are not a model-only causal estimate.
Reported provider costs exclude engineering, review and hosting. Latency applies to each recorded test population and execution setting.
Inspect methods and historical results →03 · How I organize AI-assisted delivery
My operating notes examine when another agent adds a useful capability, how a handoff should be specified and where review becomes the next constraint. These are working practices drawn from independent exploration.