September 5, 2026
We ran the first structured field test of the FlyForward AI field assistant prototype with a four-person maintenance crew during a scheduled ground equipment inspection at STQ. The session lasted ninety minutes. Here is what we learned.
What worked. The assistant handled step-by-step procedure queries cleanly — crew members could ask “what’s the torque spec for the left strut assembly” and get a usable answer without navigating menus. Response latency was acceptable. The mobile interface held up under tarmac conditions: direct sun, gloves, ambient noise from equipment.
What failed. Three failure modes surfaced immediately.
First, the assistant did not gracefully handle partial queries. When a crew member said “the pressure is off” without additional context, the assistant returned a generic troubleshooting response that was not wrong but also not useful. Crew members in actual field conditions do not front-load context; the system needs to ask clarifying questions, not assume.
Second, cold-hands interaction was a problem. The touch interface worked in testing; it did not work reliably with the nitrile gloves the crew wore during inspection. This is a hardware and UX problem we should have anticipated and did not.
Third, two crew members defaulted to their own judgment when the assistant’s response conflicted with their experience, without flagging the conflict. This is correct behavior — the assistant should not override experienced workers — but it means we have no mechanism to capture those discrepancies for model improvement.
What we’re changing. Prompt architecture is being revised to support context-building across a session (assistant asks up to two clarifying questions before answering). UI team is evaluating voice input as a glove-compatible alternative. Logging protocol for crew-AI disagreements is being designed.
Next test scheduled for September 19.