A speech model now fits in a 16.9 MB file
Small on-device speech models make voice features cheaper to try. The audio can stay on the phone instead of going to a cloud API.
Cactus released Whistle on October 2. It turns speech into text on the device. It runs on a CPU with no extra dependencies. It supports seven languages.
The license is Apache 2.0. The weights are public. You can install it with pip. cactuscompute.com/blog/whistle
My read: small on-device speech models make voice features cheaper to try. The audio can stay on the phone instead of going to a cloud API.
Your pick: if your app already sends audio to a cloud API, test Whistle on 20 real clips from your users. Compare the errors, the speed, and the cost.
Whistle handles about 30 seconds of audio at a time. The speed numbers come from Cactus. Whisper base still wins on some datasets.
12 months: I expect more apps to do basic transcription on the device.
24 months: I expect voice to show up as a normal input in more tools.
36 months: I expect cloud speech APIs to handle only the hard cases.
Why I may be wrong: your users may speak other languages, or the room may be noisy.
Which voice feature would you try on-device first?