NOTE 001 / TESTING
A good test can embarrass the idea.
Testing is not a ceremony performed after the design team has emotionally adopted the prototype. A useful test creates a realistic chance that the favored explanation will lose. If the procedure can only confirm success, it is a demonstration.
Start by writing the failure condition in plain language. What observation would make you revise the mechanism, architecture, assumption, or claim? Then make sure the instrument can actually observe that condition. A test that cannot distinguish between competing explanations may still produce numbers, which is how dashboards acquire religious authority.
Field question: If the opposite result appeared, would we know what it meant?
NOTE 002 / UNCERTAINTY
“We do not know yet” is a usable result.
Uncertainty becomes dangerous when a team feels socially required to replace it with a number. Estimates are useful, but only when their assumptions and ranges travel with them. A single precise-looking value can erase the most important fact in the room: several plausible futures are still alive.
Separate what is measured, what is inferred, what is assumed, and what is merely convenient for planning. Then ask which unknown could flip the decision. That unknown deserves attention before another decimal place does.
Field question: Which uncertainty can actually change what we do next?
NOTE 003 / SOURCES
Read the claim and the path that produced it.
Source quality is not a binary badge. A firsthand measurement can be poorly instrumented. A secondary explanation can be excellent. A prestigious institution can repeat an old error. The useful habit is tracing how observation became conclusion.
Look for the original source when stakes justify it. Check dates, definitions, sample selection, measurement method, missing comparison groups, incentives, revisions, and whether the headline quietly broadened what the underlying work actually showed.
Field question: What would I need to inspect to reconstruct this claim?
NOTE 004 / PROTOTYPES
Version one should buy information.
The first prototype is rarely supposed to be beautiful. Its job is to retire uncertainty cheaply. That often means exposed fasteners, ugly fixtures, temporary instrumentation, oversized safety margins, manual controls, and components chosen because they are easy to replace rather than impressive in a render.
A prototype becomes expensive when it tries to answer every question at once. Break the uncertainty apart. Geometry, control logic, load path, thermal behavior, interface assumptions, and human factors can often be tested with different rigs.
Field question: What is the cheapest object that can answer the next important question?
NOTE 005 / MAINTENANCE
Maintenance reveals the hidden architecture.
Design reviews see drawings. Maintenance sees consequences. It discovers which connector cannot be reached, which panel requires removing three unrelated assemblies, which diagnostic code means five different things, and which “replaceable” component assumes a tool nobody carries.
Use maintenance evidence as design evidence. Track recurring failure modes, repair time, parts availability, access problems, contamination paths, calibration drift, operator workarounds, and modifications people make because the original interface did not survive reality.
Field question: What does the repair history know that the design file does not?
NOTE 006 / AI OUTPUT
Fluent output still needs an evidence model.
An AI response can be useful while remaining uncertain, incomplete, or wrong. Treat fluency as interface quality, not validation. For consequential work, identify which claims are externally checkable, which are calculations, which depend on current information, and which are interpretation.
Useful systems should make verification cheaper. That means preserving sources where available, exposing assumptions, distinguishing confidence from evidence, and encouraging comparison against alternatives rather than treating the first coherent answer as closure.
Field question: Which sentence would hurt most if it were confidently wrong?
NOTE 007 / DECISIONS
Separate decision quality from outcome quality.
A good decision can produce a bad outcome when uncertainty is real. A terrible decision can get lucky. If teams judge only by outcome, they eventually reward superstition and punish sensible risk management.
Record the information available at decision time, the options considered, major assumptions, expected failure modes, and what evidence would trigger revision. Later review the process against what was knowable then, not what became obvious afterward.
Field question: Would we make the same decision again with the same information?
NOTE 008 / SYSTEMS
Local optimization can quietly damage the whole machine.
A subsystem can become lighter, faster, cheaper, or more efficient while making the total system harder to repair, less tolerant of faults, more dependent on rare parts, harder to inspect, or more fragile at interfaces. Engineering metrics become dangerous when they forget the mission.
Before optimizing, name the system-level constraints: availability, repair time, safety, energy, mass, cost, operator skill, supply chain, environment, expected life, and graceful degradation. Then ask whether the local improvement helps the actual objective.
Field question: What did we make worse while making this number better?