01 / PROBLEM
Editing beyond a keyboard and mouse.
For people with motor impairments, conventional document controls can create barriers. Our hackathon project explored whether common editing actions could be triggered with spoken commands.
In a four-person team, we built a Python desktop prototype during a 48-hour hackathon. The goal was a working interaction approach for creating and editing text.
02 / CONTRIBUTION
My contribution: speech to editing actions.
I contributed to the core speech-to-text and command-recognition features, including selecting, copying, and saving text, as part of the four-person team.
The editor connects microphone input to recognition, command parsing, and Tkinter text operations. Standard menus remain available alongside the voice-command control.
03 / DECISIONS
Separate commands from ordinary text.
The current code uses the SpeechRecognition library's Google recognizer. It converts the result to lowercase and checks for a “ZQ” command phrase. Recognized speech without the command phrase is inserted into the document.
Use a small, explicit command vocabulary
Commands such as “ZQ copy” and “ZQ select all” map to Tkinter text-widget operations. This keeps the interaction easy to inspect and explain.
Reuse native document behavior
Cut, copy, and paste use the widget's built-in events, while selection and cursor movement use text indices and tags.
Support spoken navigation
The implementation includes moving to a word or line, with word2number converting a spoken number for line navigation.
04 / CHALLENGES
Recognition is only part of accessibility.
The prototype uses a button to start listening, and some operations still open native file dialogs. It demonstrates voice-driven editing, but end-to-end hands-free operation has not been validated.
Listening and recognition run synchronously in the current implementation. Command ordering also needs care: a general “save” check appears before “save as.” These are meaningful limitations to address before treating it as a finished accessibility tool.
05 / TAKEAWAYS
An interface is only as clear as its feedback.
Voice input introduces uncertainty that a visible button does not. The useful engineering question is how to make recognition, intent, and the resulting action understandable to the person using the editor.
Keep the interface responsive
A next iteration should move recognition off the GUI thread and provide visible listening, processing, and error states.
Make commands more robust
A structured parser with specific commands checked first would reduce ambiguity. Risky editing actions would benefit from clearer feedback and recovery.
Validate with the intended users
Testing complete tasks with people who use assistive technology would be necessary to assess accessibility beyond the prototype's design intent.