Sony patent could let deaf gamers see the sounds they cannot hear

Sony Interactive Entertainment has filed a patent application for a system that converts video game sounds into on-screen symbols, controller lights, and vibrations for players who are deaf or hard of hearing. The US application, titled “Audio Processing System and Method," was published on October 22 as US 2026/0329526 A1.

The filing describes software that analyses each sound before it plays and works out the effect it is meant to have on the player. The patent's claims describe this step as “classifying an intended effect on a user of the sound.” Categories named in the filing include alerts, notifications, ambient mood, dialogue, environmental effects, and moments designed to create tension or surprise, and sounds can also be sorted as friendly, neutral, or aggressive.

Once a sound is classified, the system produces a matching cue. Examples in the filing include a warning icon placed near an attacking monster, a word such as “Roar!” written on screen, and a graphical indicator surrounding the player's character. The sound's own properties shape how each cue looks. A sudden, loud noise produces a brighter and more prominent warning, while volume, pitch, and speech content can change an indicator's color, shape, size, and transparency.

The system also works out where a sound is coming from. It reads left and right placement from stereo mixes and finer positioning from 5.1 and 7.1 surround audio, then points the player towards the source. One proposal uses directional LEDs around a controller's touchpad, the area where the PS5's DualSense controller carries its light bar, to show a sound's approximate direction. Other proposals include vibration patterns matched to each type of sound, lights around the display or the player's surroundings, and a companion app.

The filing describes cues that follow a consistent pattern, so a given icon or vibration always carries the same meaning. Sony explains that the analysis has to happen before playback because some of a sound's properties only become clear as it plays, by which point a visual warning would arrive too late. Not every sound would trigger every type of cue, and the choice depends on the sound itself and how well the system can interpret it.

The approach does not rely on developers adding support to each game. For titles whose code cannot easily be changed, the console's operating system or a helper application could scan audio files as they load, tag them, and generate the metadata needed for visual indicators. The filing also describes a shared metadata repository for existing games and allows a scripted scene to pause until a cue has appeared on screen. Games without vibration support could still use the on-screen indicators.

The US application was filed on July 2, 2026, as a continuation of an earlier application submitted on May 31, 2023. Its listed inventors are Rajeev Gupta, Mark Jacobus Breugelmans, Carl Pryke, and Lloyd Preston Stemple. Patent tracker Patentlyze reports that the application was awaiting an examiner as of mid-July.

Sony has not announced plans to bring the system to the PS5 or a future console. Some coverage has linked the filing to the PlayStation 6, but the document does not name a specific console, and companies often patent ideas that never reach a product. 

Visual sound cues already exist in some individual games. Fortnite offers a Visualize Sound Effects setting that places a radial indicator on screen to show where gunfire, vehicles, chests, and wildlife are coming from. Sony's own The Last of Us Part II launched in 2020 with more than 60 accessibility settings. The patent instead describes a system-level tool that could add similar cues to games that were never designed with them.

The World Health Organization estimates that more than 1.5 billion people, nearly 20% of the global population, live with hearing loss. Of these, 430 million have disabling hearing loss, a figure the WHO expects to exceed 700 million by 2050.