Build a Compact Synthesizer and Looper on M5Stack

What You’re Building and Why It’s Worth It

Building a compact synthesizer and DIY music looper on M5Stack means turning two small microcontroller boards into fully playable, open-source music hardware with touchscreen interfaces, real-time DSP effects, and proper audio I/O, so you can experiment with microcontroller music projects that feel like real instruments instead of test rigs.

You’ll build and set up two devices: the M5KaossLooper, a 4-track DIY music looper with 60 DSP effects running on an M5Stack Core2 with a touch XY pad, audio module, and SD-based loop storage, and the M5Stack AMY Synth, a 26-key M5Stack synthesizer on CoreS3 with illuminated keys, MIDI input, and stereo line output through an external DAC. If you’re comfortable with basic soldering, following wiring diagrams, and flashing firmware, this is squarely in reach. The real caveat: both projects push ESP32 hardware hard, so you must respect pin assignments, task timing, and audio paths or you’ll chase glitches for days.

At the end, you’ll have a one-stack looper where “an M5Stack Core2 for AWS with the Module Audio gives you the touchscreen, the audio codec and the storage — no breadboard, no USB audio interface, no external screen”, plus a CoreS3 synth that can be played from its own 26-key keyboard or an external MIDI controller. Both are open-source, so you’re free to modify firmware, tweak schematics, and add your own features as you gain confidence.

Gather the Right Hardware Before You Start

Before wiring or flashing anything, make sure you have the specific M5Stack cores and audio hardware these designs expect. For the M5KaossLooper, you need an M5Stack Core2 for AWS with its touchscreen, dual-core ESP32, PSRAM, SD slot, and integrated power management, plus the M5Stack Module Audio with the ES8388 stereo codec providing mic/line TRS input and TRRS headphone output. A FAT32-formatted micro-SD card both stores your loops and holds a config.json file for settings. The Module Audio snaps onto the Core2’s M5-Bus, so there’s no loose wiring in the audio path.

For the M5Stack AMY Synth, you use the M5Stack CoreS3 as the main controller; it runs the AMY synthesizer, drives the touch UI, scans the keyboard, and manages patch selection. You’ll need a PCM5102 I2S DAC for stereo line-level output and to connect that DAC’s outputs to an amplifier, mixer, or powered speakers. Around it, expect common parts: MCP23S17 GPIO expander, 74HC138 for keyboard column selection, diodes for the 26-key matrix, an HT16K33 driver for panel LEDs and display, a 74HC4051 for analog multiplexing, NeoPixels for each key, plus a joystick, rotary encoder, potentiometers, and UART MIDI input hardware.

The real prerequisite isn’t exotic components; it’s patience with staged testing. According to the AMY Synth project, assembling and testing the hardware in stages “made it much easier to locate wiring, power, and software problems.” Treat that as advice, not a suggestion.

Build the Synth First, Then Stack the Looper

You can do these in any order, but building the M5Stack AMY Synth first teaches you a lot about the CoreS3 and audio timing. Follow the project’s staged approach—it’s there to protect you from hard-to-track bugs. Then you’ll assemble the Kaoss-style looper, which is mechanically simpler but demanding on DSP and audio routing. Here’s a practical sequence that keeps you out of trouble.

1. Assemble power circuits and verify stable 5 V and 3.3 V rails, then boot the CoreS3 by itself to confirm it runs and the display responds.
2. Wire the PCM5102 DAC to the CoreS3 I2S pins (DATA GPIO5, BCLK GPIO6, LRCK GPIO7, MCLK GPIO8), configure AMY for external I2S output, and verify a simple test tone before adding any control hardware.
3. Build the 26-key diode matrix using MCP23S17 for rows and a 74HC138 for columns, test each row and column independently, then connect and sequence-test the NeoPixel LEDs for visual key feedback.
4. Add the HT16K33-based physical control panel, confirming buttons, patch LEDs, and the three-digit patch-number display work before tying them into the patch-selection logic.
5. Connect the UART MIDI input to GPIO44, enable AMY’s MIDI support, and test playing notes from an external MIDI keyboard with correct Note On/Note Off handling.
6. Wire in the joystick, rotary encoder, potentiometers through the 74HC4051 analog multiplexer, plus the touch UI; filter and test each input so it doesn’t interrupt continuous AMY synthesis and I2S audio output.
7. Stack the M5KaossLooper: snap the Module Audio onto the M5Stack Core2 M5-Bus, insert the FAT32 micro-SD card with config.json, and flash the open-source firmware to the Core2.
8. Connect an instrument or microphone with a TRS cable to the Module Audio input, then use the XY touchscreen pad to play the active effect, freeze slots, stack up to 8 serial FX, and record, overdub, undo, and play back up to 4 tracks streaming from SD.

Both projects demand that audio runs continuously while UI, LEDs, and inputs are active. If you add extra debug logging or heavy screen updates, do it carefully and test often. For the looper, remember that Core 0 runs the audio pipeline (I2S in → BPM detection → looper → FX chain → I2S out) while Core 1 handles the UI and SD I/O under FreeRTOS; any change that starves Core 0 will show up as glitches or resets.

Avoid the Gotchas: Audio Bleed and Interruptions

These designs aren’t fragile, but there are known failure modes you should fix before you wonder why the sound is wrong. On the M5KaossLooper, the ES8388 codec library originally left the analog bypass on, which meant dry signal bleeding through your processed audio; the project notes that a two-register write (0x90 to registers 0x27 and 0x2A) disables that bypass and restores correct wet/dry balance. The firmware also once rebooted every 8–17 seconds because the audio task ran at 100% on Core 0, starving the idle watchdog; inserting a periodic vTaskDelay(1) and moving hot DSP loops into IRAM fixed that permanently.

On the M5Stack AMY Synth side, early versions saw audio interruptions when LCD updates, serial logging, keyboard scanning, and analog input processing happened together. That’s your warning not to overload the event loop. Keep display updates lean, avoid blocking calls in input processing, and treat I2S audio as the priority. If you experiment with new features—extra animations, complex patch screens—add them one at a time and listen for clicks or dropouts. The system architecture deliberately separates I2S audio, SPI keyboard expansion, I2C panel control, UART MIDI, and ADC analog reads into clear roles so they can coexist without fighting each other.

The upside of hitting these gotchas is that both projects document their fixes as part of their open-source firmware and schematics. When something misbehaves, there’s a paper trail instead of guesswork.

What You Get When It All Works

Once everything is wired, flashed, and tuned, the payoff is big for the effort. The M5KaossLooper becomes a compact instrument where the XY touchscreen gives you direct control over 60 effect programs across filters, modulation, delays, reverbs, granular, distortion, and combo effects, arranged in an 8-slot serial FX chain with per-slot bypass and hold. You can freeze any slot at its current pad position, reposition it live, and those held effects continue to shape the loops you’re playing back. The 4-track looper records and streams raw 22050 Hz stereo from the micro-SD card using double buffering, so quantised recording, overdub, undo, and playback run without glitches while the BPM detector keeps tempo-based effects and the arpeggiator locked to what you play.

On the synth side, the current M5Stack AMY Synth prototype integrates continuous AMY synthesis with stereo PCM5102 line output, reliable 26-key scanning, Note On/Off handling, illuminated keyboard feedback, touch and physical patch selection, JUNO-style, DX7-style, and piano preset banks, UART MIDI input, pitch bend and modulation from a joystick, plus power-on visual diagnostics—all running on the CoreS3. You start it by connecting the PCM5102 outputs to an amp, powering on, watching the self-test sequence, selecting a bank and patch on the screen or panel, then playing its own keyboard or an external MIDI board, using the joystick to bend pitch or add vibrato.

Together, these projects show how modular open-source platforms can enable affordable music production hardware that sits on a table or pedalboard without cable looms, while exploring “how far a small ESP32-S3-based controller can be pushed as a real musical instrument.” They’re absolutely worth the build if you want playable microcontroller music projects rather than another blinking demo—just keep a close eye on audio paths, timing, and pin conflicts as you customize.