Transcribe

Transcribe

THIS IS THE FREE VERSION OF THE PLUGIN. TO GET THE FULL CAPACITY OF THE PLUGIN PLEASE CONSIDER PAYING FOR THE FULL VERSION: Get it on my official website or get it on BuiltByBit

a Paper plugin that turns short spoken words into real game actions. It piggybacks on the microphone stream that Simple Voice Chat already captures, so players don't need to press "push to talk" for a second time and no extra mod is required beyond the voice chat one.

Press a trigger (sneak + F by default), a brief listen window opens, you say a word from your server's vocabulary, and Transcribe fires an event and/or runs your commands.

Runs fully offline. Speech recognition happens on your server (or your own box). No audio ever leaves your infrastructure.

---

Features

• Two speech-to-text backends, your choice
  • INTERNAL - Vosk embedded directly in the server JVM via JNI. No extra service to run.
  • EXTERNAL - streams audio to your own STT microservice over a persistent WebSocket. Keep the Minecraft server's footprint at zero and scale/harden STT separately.
• Tiny vocabulary, huge accuracy + speed. You list a handful of words (e.g. fire, light, ice) and Vosk is grammar-constrained to exactly those. Constraining the search space is what makes this cheap and reliable instead of a fuzzy general transcriber.
• Bring your own activation. Choose any combination of eight triggers, or drive it from a PlaceholderAPI condition:
  • OFF_HAND / SNEAK_OFF_HAND (F, or sneak + F)
  • LEFT_CLICK / SNEAK_LEFT_CLICK
  • RIGHT_CLICK / SNEAK_RIGHT_CLICK
  • HOTBAR_SLOT / SNEAK_HOTBAR_SLOT (pick any of slots 1-9)
  • a polled PlaceholderAPI placeholder == value
• Gesture events can be suppressed (cancel-trigger-events) so players don't actually swap hands, drop items, or swing while casting.
• Fully configurable feedback. Action bar text and sounds on window open/close, with independent volume and pitch. Accepts vanilla sound keys and resource-pack custom sounds (mypack:spells.cast).
• Runs commands, or anything else. Every recognized word can dispatch console commands with %player% and %keyword% placeholders, so one entry can branch across your whole spell set - or write your own plugin.
• Clean API for developers. A cancellable VoiceWordSpokenEvent (player, keyword, confidence) plus TranscribeAPI to query or force-open listen windows from your own code.
• Near-zero idle cost. Microphone packets are dropped before decoding unless a listen window is actually open.
• Hot reload. /transcribe reload re-reads config without a restart, and builds the new engine before tearing down the old one - a bad config leaves the working one running.
• Four verbosity levels of [Transcribe debug] logging for diagnosing the audio pipeline, trigger detection, and recognition results.

---

Requirements

• Paper (or Purpur) 1.21+, Java 21.
• Simple Voice Chat - server and client. This is a hard dependency; Transcribe will not enable without it.
• (optional) PlaceholderAPI

Note: players must install the Simple Voice Chat mod on their client (Fabric / Forge / NeoForge). It has to be the mod, not just the server plugin, or no audio is ever captured.

---

Quick start

1. Install Simple Voice Chat server-side and on clients, and verify proximity voice works.
2. Drop Transcribe.jar into plugins/.
3. Start the server once to generate plugins/Transcribe/config.yml.
4. Choose your words:

   engine:
     internal:
       grammar:
         - fire
         - light
         - ice
   

5. (INTERNAL only) Download a small Vosk model from https://alphacephei.com/vosk/models and unzip it to plugins/Transcribe/models/vosk-model-small-en-us-0.15.
6. Point your spells at the keywords:

   recognition:
     console-commands:
       - "spellcast trigger %player% %keyword%"
   

7. /transcribe reload, then sneak + F and say "fire".

---

How it works

trigger gesture
      |
      v
Listen Window opens (configurable, default 3s)  -- audio bar + sound cue
      |
      v
Opus microphone packets decoded to 48 kHz mono PCM
      |
      v
Speech recognition (INTERNAL Vosk, or EXTERNAL WebSocket service)
      |
      v
keyword matched against your grammar?  -- no --> window closes, nothing happens
      | yes
      v
VoiceWordSpokenEvent (cancellable)  -- cancelled --> nothing happens
      | not cancelled
      v
recognition.console-commands dispatched as console

---

Performance notes

• Recognition only runs while a listen window is open. Outside of that, packets are dropped before decoding.
• A grammar of a few words collapses the decoder's search space, so even INTERNAL mode costs almost nothing at idle.
• Recognition happens off the main thread on virtual threads; only the resulting event and commands touch the server thread.
• The small Vosk models (~40 MB) are more than sufficient for keyword spotting. You do not need a large model.

---

Configuration

Everything lives in plugins/Transcribe/config.yml, which is heavily commented inline. Headlines:

Key	What it does
debug.level	0-3, how much [Transcribe debug] logging to emit
engine.mode	INTERNAL (embedded Vosk) or EXTERNAL (WebSocket service)
engine.sample-rate-hertz	PCM rate handed to the engine (default 16000)
engine.internal.model-path	path to an unpacked Vosk model, under the plugin folder
engine.internal.grammar	your vocabulary - the words players can say
engine.external.uri	ws:// / wss:// endpoint of your STT service
engine.external.request-timeout-millis	per-request ceiling before giving up
activation.window-seconds	how long the listen window stays open
activation.triggers	which of the eight gesture triggers are active
activation.hotbar-slots	slots 1-9 used by the hotbar triggers
activation.cancel-trigger-events	suppress the vanilla action behind a trigger
activation.feedback.action-bar.*	open/close action bar text, &-colour codes, '' to hide
activation.feedback.sound.*	open/close sound key, volume, pitch
activation.placeholderapi.*	polled placeholder condition trigger
recognition.console-commands	commands to run on a recognized keyword

%player% and %keyword% are substituted in every entry of recognition.console-commands. VoiceWordSpokenEvent always fires first and can be cancelled to suppress the commands - which is where cooldowns, mana costs, or silence effects belong.

Commands & permissions

Command	Permission	Default
/transcribe reload	transcribe.reload	op

For developers

@EventHandler
public void onSpell(VoiceWordSpokenEvent event) {
    if (event.getKeyword().equals("fire")) {
        event.setCancelled();              // stop the configured console commands
        castFireball(event.getPlayer());
    }
}

// open a window programmatically, e.g. from an item or a skill GUI
TranscribeAPI.get().forceStartListenWindow(player, Duration.ofSeconds(3));

if (TranscribeAPI.get().isListening(player)) { /* ... */ }

Build against it with:

compileOnly("org.eyman_:Transcribe:<version>")

Notes & limitations

• LEFT_CLICK / SNEAK_LEFT_CLICK fire for clicking air or a block, but not for attacking an entity - Bukkit emits a damage event there, which Transcribe does not hook.
• The microphone stream is 48 kHz mono (Simple Voice Chat's native rate) and is resampled to engine.sample-rate-hertz for recognition.
• The bundled external STT reference server processes one segment at a time per connection; concurrent players are served by concurrent connections.
• activation.triggers entries SWAP_HAND / SNEAK_SWAP_HAND from older configs were renamed to OFF_HAND / SNEAK_OFF_HAND. Update them or they'll be ignored with a warning.
• Requires the standard Bukkit scheduler, so Folia is not supported.

License

GPL-3.0. Simple Voice Chat is a separate, independently licensed project and is neither bundled nor redistributed here - it is a required runtime dependency.

USED CLAUDE FOR THE DESCRIPTION