Voice Typing support using AI engine

[coolswapnil]

Requirement: Could you please check if voice typing support can be added using an AI engine? Generally, most keyboards use a basic text-to-speech engine for voice typing, and due to which they lack features such as spelling, grammatical, or punctuation corrections. Is it possible to integrate Gemini, Grok, or an offline AI engine to use for voice typing?

This would be a great addition. I guess even Gboard lacks this feature (only Pixel devices have it with Gboard).

Again, thanks for all your work 🙂.

[LeanBitLab]

I think we can't directly use an AI engine for voice typing, but we can integrate it in between speech recognition and text output, like the output just got proofread before showing in the input field, but isn't that the same as clicking the proofread key after speech recognition output?

[Hugh288]

I am trying to dial this in as well.
Using Leantype + ai integration (Groq, Compound-mini) + Custom AI Keys (My own prompt) + Futo Voice Input....I have streamlined my voice to text communication in the best possible way, that is currently available.
I was switching back and forth between Clevertype (For Ai) and Futo Keyboard (For Speed)
But now with this new setup, I have everything needed in one.

The next step, that I would like to get to, is being able to use voice input and have it go through my custom prompt and then immediately in the text field. It just cuts out one more step....leaving basically only a 2 step flow...voice to ai edited text + send.

It's not like it's difficult or too time consuming to tap the custom ai key after voice typing but why not have it all in one motion?

Dictate keyboard is supposed to be able to do this...but I tried and tried and couldn't get it to work well at all. And...Dictate doesn't have an onscreen keyboard...which is a dealbreaker for me... It makes no sense to me to have a keyboard app, even with great voice typing, and not have the keyboard. The space is there, you might as well have the keyboard.

Groqboard seems to currently have the fastest transcription, but again it doesn't have a keyboard. I have messaged the developer as it would be amazing to have his tech be available as a voice input that I could use with Leantype.

I use voice typing all day to communicate with so many different people for work on multiple different platforms. And yeah, I'm a tech nerd and I just love trying to dial this in.

There is another app that I'm on the waitlist for...Wispr Flow. I'm not sure exactly how it will work, but I am thinking that it's still not gonna be this one step flow.

The prompt is important for me because when you're using voice typing, you are speaking, so you say words like "gonna" and "like" and also there's no formatting. So it just comes out in one continuous paragraph with no separation. It can do the punctuation in grammar fairly well, but it doesn't do all the formatting and/or like bullet points, etc. It just comes out in one big paragraph like this.

With my custom prompt, I have all of that set up where when I tap the button, it fixes all that for me. I would just like to have that happen automatically.

✨🙏

[mad1mystic]

I think we can't directly use an AI engine for voice typing, but we can integrate it in between speech recognition and text output, like the output just got proofread before showing in the input field, but isn't that the same as clicking the proofread key after speech recognition output?

You can look into this. Its open source https://github.com/kroko-ai,
the voice typing model is small, Plus you can integrate into the keyboard and the voice typing is pretty accurate, maybe if it can work as add on module it would be great. As for futo voice input, Dev said he wouldn't update it as futo voice would only be available in futo keyboard.

[Mokhyy]

Hello everyone,

I think that instead of adding voice capability to this keyboard, we should use a completely specialized keyboard for voice typing.

I recently discovered a voice‑typing keyboard that has numerous features, is built specifically for voice input (with AI) , and is still under development.

It would be better to focus our efforts on the core features of the keyboard rather than on adding this capability, and let the voice‑typing functionality be handled by that dedicated keyboard.

Link to the keyboard:

https://github.com/BryceWG/BiBi-Keyboard

This keyboard works with AI and is available both online (via an API) and offline (with support for only a few languages).

It’s a newly created keyboard, so it’s likely that few people are familiar with it yet.

[coolswapnil]

But it will be a better to have single app with both functions instead of two. Instead switching between apps a single app will serve both functions.

[BraveFalconfromTinyCastle]

Hi, I just wanted to contribute with the name of an app I've been using for a while for voice typing. It just appears as an overlay circular icon when a text field is selected; it's called VoiceFlow, available on the Play Store. It uses either the Groq or Gemini API. I'm not sure if it's open source or not.

[LeanBitLab]

Hi, I just wanted to contribute with the name of an app I've been using for a while for voice typing. It just appears as an overlay circular icon when a text field is selected; it's called VoiceFlow, available on the Play Store. It uses either the Groq or Gemini API. I'm not sure if it's open source or not.

I'll check it👍

[quantavil]

Hi, I just wanted to contribute with the name of an app I've been using for a while for voice typing. It just appears as an overlay circular icon when a text field is selected; it's called VoiceFlow, available on the Play Store. It uses either the Groq or Gemini API. I'm not sure if it's open source or not.

Nope it is not open source, (https://play.google.com/store/apps/details?id=com.omkar.voiceflow&pcampaignid=web_share) ,also it doesnt work without Google PlayService.

[m13v]

this would be a great addition. we built voice-to-text for a macOS app using WhisperKit (on-device, no API key needed) and the accuracy is honestly better than most cloud APIs for English. the key insight: run the transcription model locally on the device's neural engine, then pass the raw transcript through a small LLM for punctuation and grammar cleanup. the two-stage pipeline gives you the best of both worlds - fast local transcription with AI-quality corrections.

for Android, sherpa-onnx with the whisper models would be the equivalent path. the latency is sub-second on Apple Silicon and similar hardware-accelerated chips. the punctuation correction step alone makes a huge difference - raw whisper output is often grammatically correct but missing all punctuation, which makes it feel unfinished.

[m13v]

our WhisperKit-based transcription service with on-device model management, audio preprocessing, and streaming results: https://github.com/m13v/fazm/blob/main/Desktop/Sources/TranscriptionService.swift

handles model selection (distil-large-v3 for speed, large-v3 for accuracy), download management, and chunked transcription with sub-second latency on Apple Silicon.