Notematic Vocoder

---

[!IMPORTANT] Prerequisite: This plugin requires Notematic Player to be installed and active on the server. It uses the Notematic Player engine to schedule notes, parse structures, and handle packet playback.

Inspired by Mark Rober's "Talking Piano", the plugin uses Digital Signal Processing (DSP) to analyze Text-to-Speech audio, extract formant frequencies, and dynamically map them to Minecraft's sound system, allowing note blocks to "talk" in real-time.

---

Video Showcases

See and hear the Notematic Vocoder in action:

English Speech Demo (Speaking English)

Chinese Speech Demo (Changing Language)

---

Key Features

• Real-time In-Game Speech: Speak aloud using note blocks via /<command> <prompt>.
• 6-Octave Voice Range: Intelligently maps overlapping frequencies across 5 note block instruments (Bass, Guitar, Harp, Flute, Bell) to bypass Minecraft's default 2-octave limit.
• Full DSP Pipeline: Runs Fourier Transforms (FFT), formant-tracking filters, and sibilant/plosive consonant classifiers (using zero-crossing rate and flatness).
• Custom Accent Locales & Voices: Choose between male/female voices and various pronunciation accents (e.g. English, Chinese, French, Japanese, Australian) via Google's TTS API.
• Zero-Lag Audio Caching: Automatically caches generated speech files so repeated phrases play back instantly without contacting the API.
• 100% Server-Side: No mods or resource packs required by players. Fully compatible with vanilla clients.

---

How It Works: The 6-Octave Bypass

Vanilla Minecraft note blocks support only 2 octaves (0.5 to 2.0 pitch scale, MIDI 54-78). To make speech intelligible, Notematic Vocoder extends the playable scale to 6 full octaves (72 semitones) by crossing 5 instruments:

1. Bass (MIDI 30 - 54) — For deep vocal formants
2. Guitar (MIDI 42 - 66) — For lower midtones
3. Harp (MIDI 54 - 78) — For center vocal frequencies
4. Flute (MIDI 66 - 90) — For upper vocal range
5. Bell (MIDI 78 - 102) — For high-pitched harmonic overtones

When a prompt is sent:

• Formants are extracted and mapped to the closest overlapping instrument octave.
• Unvoiced sibilants (like /s/ and /sh/) are simulated using TNT fuse hiss.
• Plosives (like /p/ or /t/) trigger button clicks and note block snare/hats.

---

Commands & Permissions

Command Aliases

You can use any of the following base commands interchangeably:

• /vocoder (Primary)
• /speech (Alias)
• /vocode (Alias)

Command	Permissions	Description	Default
/<cmd> <prompt>	notematicvocoder.use	Generates speech for the prompt and plays it.	Everyone
/<cmd> settings	notematicvocoder.use	Displays your active preferences and global settings.	Everyone
/<cmd> settings voice <male|female>	notematicvocoder.use	Sets your personal voice preference.	Everyone
/<cmd> settings <language|lang> <langCode>	notematicvocoder.use	Sets your accent locale (e.g. en, es, fr, ja).	Everyone
/<cmd> settings default-voice <male|female>	notematicvocoder.admin	Updates the global default voice (Admin).	OP
/<cmd> settings <default-language|default-lang> <langCode>	notematicvocoder.admin	Updates the global default language accent (Admin).	OP
/<cmd> toggle	notematicvocoder.admin	Globally toggles command access for non-admins (Admin).	OP

Detailed Command Guide & Accent Codes

Customizing Voices and Accents

You can adjust how the vocoder pronounces your text.

• To change your voice: /<cmd> settings voice male or /<cmd> settings voice female
• To set a different language/accent: /<cmd> settings lang en-AU (Australian English), /<cmd> settings language es (Spanish), /<cmd> settings lang ja (Japanese).

Admin Controls

• To restrict non-admin players from generating speech (e.g. to save bandwidth), run /<cmd> toggle.
• Global defaults can be changed using /<cmd> settings default-voice <voice> and /<cmd> settings default-lang <lang>.

---

Installation & Setup

1. Make sure you have Notematic Player installed.
2. Download the latest release of Notematic Vocoder and drop it into your server's plugins folder.
3. Restart or reload the server to generate the default configuration.
4. Set up permissions (notematicvocoder.use and notematicvocoder.admin) using your favorite permissions manager.

---

FAQ

Does this require a client mod or resource pack? No. The entire audio translation and playback process happens server-side using vanilla sound packets. Any player can join and hear it. Is there any lag when generating a prompt? All audio downloads and digital signal processing run asynchronously. This ensures that your server's main tick loop is completely unaffected while a prompt is generating. Can I use this in my modpack? Yes! Feel free to include this in any Spigot/Paper server pack or modpack.