How audio-to-haptics conversion works
Sound and touch are related but not the same. A good conversion does not copy the waveform into the motor; it picks out the moments that should be felt. There are two things to find in a sound:
- Onsets: the start of a sharp sound such as a kick drum, a door slam or a gunshot. These become transient events, single crisp taps.
- Sustained energy: stretches where the sound holds steady, such as a bass note, an engine or wind. These become continuous events, with intensity following loudness.
A common approach in audio analysis is to track the signal’s loudness envelope over time and detect sudden rises in energy as onsets. Whatever the method, the output is the same kind of thing: a timeline of transient and continuous events that Core Haptics can play.
What Haptic Forge does with a sound
In Haptic Forge, my Core Haptics editor for iPhone, iPad and Mac, you import a sound and get a pattern back:
- A tap on every beat it detects.
- A continuous rumble through every steady stretch.
- The waveform sits under the chart, so you can see why an event is where it is.
- Sound and pattern play together, so you judge the result the way users will experience it.
The generated events are ordinary layers. You can delete, move, reshape or retune any of them, and export the result as a standard AHAP file.
Why generated haptics still need a designer
Automatic generation is a starting point. Some things a tool cannot know:
- Which beats matter. A busy drum track can produce a tap every 100 ms. That is tiring on the hand. Often you keep only the downbeats or the snare.
- Relative weight. The loudest sound is not always the most important one. A quiet “click” of a lock may deserve a stronger tap than a loud background crash.
- Sharpness. Loudness maps naturally to intensity, but sharpness is about character. A bass drop wants low sharpness; a hi-hat wants high sharpness.
- Silence. Leaving gaps gives the next hit more impact.
A refining checklist
- Play the sound and pattern together on an iPhone.
- Remove taps that blur into a buzz.
- Raise the intensity of the two or three moments that carry the scene.
- Set sharpness per layer: low for thumps and rumble, high for clicks and ticks.
- Add a short fade at the end of continuous layers so they do not cut off abruptly.
Use cases for audio-driven haptics
| Use case | What to keep from the audio | Typical result |
|---|---|---|
| Music player or rhythm game | Kick and snare onsets | Steady taps on the beat, light rumble under bass |
| Game sound effects | Impacts and explosions | Strong transient plus short continuous tail |
| App trailers and onboarding | Key hits in the soundtrack | A few deliberate taps on big moments |
| Meditation or breathing audio | Slow swells | Gentle continuous events that rise and fall |
| UI sounds | The click itself | One transient, matched in timing |
For game-specific patterns such as engines, weapons and UI, see haptics for game developers.
Keeping sound and haptics in sync in your app
Sync is where audio-driven haptics most often fail in production. A few rules:
- Start both together. Prepare the audio player and the haptic pattern player ahead of time, then start them in the same step.
- Or put the audio in the pattern. AHAP supports
AudioCustomevents that reference a sound file, so Core Haptics schedules both on one timeline. - Keep the engine running during playback. Handle the engine’s stopped and reset callbacks so a backgrounded app does not lose the haptics halfway through.
- Check on device. Only an iPhone plays Core Haptics; the Simulator, iPad and Mac do not.
How I built it
The tricky part was not detecting beats; it was making the output something a designer wants to edit. Generated events land as normal layers with the waveform beneath them, so the first pass and the hand-tuned result live in the same timeline. If you want to understand the two event types the generator produces, read transient vs continuous haptics.
