Grammar for coding by voice using Dragonfly2

Go to file

Jeremy Hayes d9c822f4d7 Added logic to multi-edit.py to allow selecting the word at the cursor and selecting the line you are on or the current line plus a number of lines specified		2022-02-10 08:28:11 -06:00
holding	added readme,md modified gerammars to work better on linux	2022-02-10 07:19:17 -06:00
vocabulary_config	Updated grammars	2019-08-16 09:42:32 -05:00
.gitignore	added readme,md modified gerammars to work better on linux	2022-02-10 07:19:17 -06:00
LICENSE	Add LICENSE	2022-02-09 17:49:32 +00:00
README.md	added readme,md modified gerammars to work better on linux	2022-02-10 07:19:17 -06:00
_angularjs_grammar.py	added readme,md modified gerammars to work better on linux	2022-02-10 07:19:17 -06:00
_cs_grammar.py	added readme,md modified gerammars to work better on linux	2022-02-10 07:19:17 -06:00
_css_grammar.py	added readme,md modified gerammars to work better on linux	2022-02-10 07:19:17 -06:00
_custom_globals.py	added readme,md modified gerammars to work better on linux	2022-02-10 07:19:17 -06:00
_html_grammar.py	added readme,md modified gerammars to work better on linux	2022-02-10 07:19:17 -06:00
_java_grammar.py	added readme,md modified gerammars to work better on linux	2022-02-10 07:19:17 -06:00
_javascript_grammar.py	added readme,md modified gerammars to work better on linux	2022-02-10 07:19:17 -06:00
_multiedit.py	Added logic to multi-edit.py to allow selecting the word at the cursor and selecting the line you are on or the current line plus a number of lines specified	2022-02-10 08:28:11 -06:00
_namingcon.py	Added logic to multi-edit.py to allow selecting the word at the cursor and selecting the line you are on or the current line plus a number of lines specified	2022-02-10 08:28:11 -06:00
_python_grammar.py	added readme,md modified gerammars to work better on linux	2022-02-10 07:19:17 -06:00
_react_grammar.py	added readme,md modified gerammars to work better on linux	2022-02-10 07:19:17 -06:00
_terminal_grammar.py	added readme,md modified gerammars to work better on linux	2022-02-10 07:19:17 -06:00
app.py	added readme,md modified gerammars to work better on linux	2022-02-10 07:19:17 -06:00
kaldi-dfly-loader-plus.py	added readme,md modified gerammars to work better on linux	2022-02-10 07:19:17 -06:00
kaldi_dfly_loader_plus_gui.py	added readme,md modified gerammars to work better on linux	2022-02-10 07:19:17 -06:00
server_state.json	Initial commit	2019-08-02 15:25:33 -05:00

README.md

Dragonfly Grammars for Coding by Voice

A simple way to setup voice coding in Linux, and Windows using Kaldi on Linux and WSR on Windows Grammar avalible for Python, Javascript, React.js, HTML, CSS, Java, and C#

Setup an installation

Clone this repo using GIT
If you haven't installed dragonfly yet run this command -- For linux

pip install 'dragonfly2[kaldi]'

-- For windows

pip install dragonfly2

Install a Kaldi model from (https://github.com/daanzu/kaldi-active-grammar/releases) -- I recommend kaldi_model_daanzu_20211030-biglm.zip -- If you are feeling risky you can train your own model (http://jrmeyer.github.io/asr/2016/12/15/DNN-AM-Kaldi.html) -- download and place into the directory you clone the project.
On Linux you may need install some extra software

sudo apt install wmctrl xdotool xsel

You may now try it out

python kaldi_module_loader_plus.py

On some linux distributions you may need to use this command instead

python3 kaldi_module_loader_plus.py

On Windows you may need to use the python loader py

py -3.6 kaldi_module_loader_plus.py

How to use

Once it is running say wake up. If everything is working you should hear a voice prompt stating that it's awake.
Say enable [language of your choice]. You should hear a voice prompt stating that the language has been activated
You can combine some grammars just be aware that some may have conflicting rules -- For example I often combine HTML and Javascript
You can manipulate the cursor by saying up, down, left, right plus the number of times you want it to move
You can combine commands for example "shift" + "direction" + "number of moves" or "ctrl" + "up/down" + "number of moves"

Tech

This project uses:

Python - An awesome and powerful interpreted programming language
Dragonfly2 - A framework that allows you to code by voice
Festival - A tts engine
Kaldi - A open source speech recognition tool kit
WSR - Microsoft Windows built in speech recognition tool kit
Tkinter - A GUI for programming user interfaces in Python

License

MIT