Researchers fine-tune F-35 pilot-aircraft speech system

  • Published
  • By John Schutte
  • Human Effectiveness Directorate
WRIGHT-PATTERSON AIR FORCE BASE, Ohio (AFPN) --  When the first F-35 Lightning II rolls out in 2008, communications between pilot and aircraft will enter a new era thanks in part to testing and analysis conducted at the Air Force Research Laboratory's Human Effectiveness Directorate.

The F-35 will be the first U.S. fighter aircraft with a speech recognition system able to "hear" a pilot's spoken commands to manage various aircraft subsystems, such as communications and navigation.

Researchers at the directorate's Warfighter Interface Division are collecting data and recommending improvements now to ensure optimal performance when the F-35's new speech recognition system undergoes future operational tests, said David Williamson, senior crew systems engineer with over 25 years experience with speech recognition technology.

Currently pilots must press buttons, flip switches or glance at instruments for status information. The new system not only simplifies a pilot's workload but increases safety and efficiency, since pilots can remain focused on flying the aircraft and scrutinizing the combat environment.

Using digital recordings from in-flight tests aboard the NF-16D Variable-stability In-flight Simulator Test Aircraft, or VISTA, researchers pinpoint problem phrases and recommend alternatives that are more easily recognized. They also tweak error parameters and adjust the system's sensitivity for peak speech-recognition performance with minimal errors.

To collect data, AFRL electronic engineer Rob Snyder built a custom single-board computer system that plugs into the VISTA instrument panel. Researchers transfer data from the aircraft to the lab's database using a common memory stick.

The speech recognition system is integrated with the aircraft's onboard computer to access data. Communication occurs through the pilot's oxygen mask microphone with command feedback provided on the pilot's helmet-mounted display.

SRI International developed the DynaSpeak® speech recognition software as a highly accurate system for noisy environments, specifically for embedded devices like personal digital assistants, in-car navigation systems and avionics systems, Mr. Williamson said. It is speaker-independent, meaning a pilot can use it without first "training" the system to his or her voice, which took up to an hour on previous experimental systems.

SRI International is working with integrating contractor Adacel Systems, Inc., to tailor the system for the F-35 Joint Strike Fighter's airborne environment.

The AFRL team, including Mr. Williamson, Mr. Snyder and senior scientist Timothy Barry of General Dynamics Advanced Information Systems, worked with six pilots at the U.S. Air Force Test Pilot School at Edwards AFB, Calif., whose primary mission was flying VISTA test-bed sorties to collect speech recognition data.

During typical 90-minute VISTA flights, the system's digital recorder captured the pilot's ongoing dialog with the aircraft. Researchers then returned with the data to the lab where they studied it for troublesome phrases and the impact of environmental conditions, such as ambient cockpit noise or high acceleration forces, which can affect the quality of the acoustic characteristics.

Ambient cockpit noise degrades the quality of the spoken command entering the recognition system, which could cause the system to misinterpret or misunderstand the command. Acceleration forces, expressed in "g," had little impact up to about 6 g, Mr. Barry said.

If the system proves successful on the F-35, it could be applied to other platforms such as the F-22 Raptor, Mr. Barry said.

The system could be advanced enough in less than five years to enable a single ground-based controller to fly up to four unmanned aerial vehicles using voice commands supplemented with gestures or touch-panel commands, Mr. Williamson said.

But first, the Human Effectiveness team wants the F-35 system to be the best it can be, meaning an error rate of less than 2 percent to consider the baseline system optimized.

"We're going to work with the vendor to improve the baseline system for the F-35," Mr. Williamson said. "Collaboratively, we're going to make the system much better before it even gets to the initial operational test and evaluation phase."

Comment on this story (comments may be published on Air Force Link)

View the comments/letters page