Four self-developed audio apps — covering functional design, on-device AI model deployment, and native engineering. From metronome training to AI stem separation, from on-device inference to cross-platform audio engines.
4Apps
3Platforms
3/4Apps with on-device AI
8+CoreML models
Overview
Four Apps, One Audio Stack
Metronome training, AI stem-separation practice, AI drum-machine production, and BPM detection — covering the full chain from basic tempo to intelligent audio analysis.
An AI stem-separation player for band rehearsal and music practice, advocating Practice with Control - not passive listening, not a bare separation tool, but practicing with control.
iPhone / iPad / Mac CatalystCoreML · On-device inferenceiOS 16+
SplitJam's core is running Meta's HTDemucs source-separation model on-device: import any song and split it into six stems - drums, bass, other, vocals, guitar, piano - wrapped in a full set of controls designed around practicing a section repeatedly. Unlike pure audio separation tools, it is video-first - the video frame stays visible, letting users rehearse, play along, and review in the real song context.
Core Features
Multi-source input: file import, photo-library picker, music-library (MediaPlayer) import, in-app audio recording, video recording, files shared from other apps, drag-and-drop import.
AI stem separation: on-device split into 6 stems (drums / bass / other / vocals / guitar / piano), with 4-Stem, 6-Stem, and Hybrid modes and graceful fallback to the 4-Stem model; long audio is processed via segmentation with overlap-add.
Per-stem control: mute (multi-select), solo (exclusive), volume, pan, color per stem.
Playback control: play / pause, scrubbing, tempo (0.5x-2.0x, Pro unlocks 0.1x-2.0x full range), pitch adjustment (semitones + cents).
A/B loop: mark loop regions, save multiple practice-section presets and jump back quickly.
Beat analysis: automatic beat detection plus a manual metronome, rendering clicks by BPM with bar alignment.
Waveform visualization: Canvas multi-stem overlaid waveforms, playback progress, and A/B loop region display.
Export: stem export / mixdown export / with-video export / with pitch-and-volume export, dispatched via the system share sheet.
Project management: pending-draft list, history list, rename, delete, WAV-to-AAC compression, reopen to continue practicing.
AI Models
Two AI model families, all on-device
Audio source separation - Meta HTDemucs (core): includes htdemucs (4-stem: drums / bass / other / vocals) and htdemucs_6s (6-stem: drums / bass / other / vocals / guitar / piano), converted to CoreML mlprogram format, Float32, running on-device. The 6-Stem uses a graph-splitting strategy - STFT / iSTFT are kept out of the CoreML graph; instead iOS computes the complex STFT spectrum via Accelerate, CoreML carries only the neural-network body, and Accelerate performs iSTFT reconstruction - bypassing CoreMLTools' failure on complex STFT nodes.
Beat detection - Madmom BLSTM: Madmom's downbeats BLSTM network, run on-device via a C bridge, for automatic beat / downbeat detection that drives metronome alignment and bar-line rendering.
Highlights
On-device AI, privacy-first: all separation and beat analysis run locally - no network, no audio upload.
6-stem separation: drums / bass / other / vocals / guitar / piano, beating common 4-stem tools, with 4 / 6 / Hybrid mode selection and fallback.
Video-first practice: unlike pure audio separation tools, the video frame stays visible so you practice within the real song scene.
Practice-oriented controls: A/B loop, tempo, pitch, metronome, bar alignment, saved section presets - built around practicing a section repeatedly.
CoreML + Accelerate engineering: a self-built graph-splitting scheme solves the HTDemucs complex-STFT-to-CoreML problem, making on-device 6-stem separation possible.
Multi-source input & full export chain: file / photo / music-library / recording / video / share import; stem / mixdown / with-video / with-pitch export plus system share.
AI Drum Machine · Rhythm MakingAI · 6 CoreML models
Drum AI: From One-Tap Drum Extraction to Step Sequencing
An AI drum machine and rhythm-practice tool for drummers and beatmakers, putting extract-drums / make-drums / practice-drums on one device. App Store brand Drum AI, subtitle: AI Drum Recognition & Rhythm Practice.
iPhone / iPad / Mac CatalystCoreML + Rust FFIiOS 17.6+
Drum AI's differentiator is a complete drum-extraction pipeline: import any song, AI first separates the drum stem and further splits it into five components - Kick / Snare / Hi-hat / Toms / Cymbals - while recognizing BPM and beat positions, then auto-quantizes and generates an editable drum-machine pattern. Behind this are 6 CoreML models and a Rust-ported Madmom beat-tracking network, all running on-device.
Core Features
AI drum recognition / separation: import any song; AI separates Kick / Snare / Hi-hat / Toms / Cymbals and auto-detects BPM and beat positions.
One-tap sequence generation: recognition results are auto-converted into an editable drum-machine pattern via BPM detection -> drum-hit detection -> quantization -> downbeat adjustment -> A-B selection -> sequence generation.
Step Sequencer: 16 / 32-step grid with lookahead scheduling for drift-free precision; supports 4/4, 3/4, 6/8, 12/8 time signatures and triplets, BPM 30-360.
24 pro drum kits: Acoustic, 808, Classic, Vintage, Analog, Electro, Techno, Hip-hop, Urban, Funk, BreakBeat, Deep House, Trap, Crunk, Percussion, and more.
TempoDrill speed practice: AB Loop with multi-segment tempo (random BPM optional), count-in, jump to next segment, loop - built for drummers.
Humanize: randomizes timing and velocity simultaneously, modeled on Logic Pro's MIDI Transform Humanize.
Full mixer: real-time Compressor / Phaser / Reverb / Saturation / Filter master chain, plus per-voice Level / Pan / Filter / Tune / Decay.
Multi-source recording: iOS microphone, ReplayKit system audio (broadcast extension), macOS ScreenCaptureKit system audio.
Pro export: MIDI (SMF Format 0, GM channel 10) / WAV (44.1kHz / 32-bit float) / AAC (.m4a 192kbps), with multi-loop and effect bypass.
Auto session restore: Preset, BPM, time signature, subdivision, Groove, and mixer settings persist automatically.
AI Models
Three AI / ML model families, all local
HTDemucs (CoreML) - source separation: Meta's Hybrid Transformer Demucs splits the mix into drums / bass / other / vocals; the drum stem is taken. Uses segmented inference with overlap-add reconstruction, running on cpuOnly.
LarsNet (5 CoreML models) - drum-component split: five models for Kick / Snare / Toms / Hi-hat / Cymbals that take the STFT magnitude spectrum and output a masked magnitude spectrum, further splitting the drum stem into 5 independent tracks; the 5 models run in 2 concurrent groups, then inverse-STFT reconstructs the audio.
Madmom beat tracking (BLSTM + DBN, Rust FFI) - BPM / beat / downbeat detection: Madmom's downbeats BLSTM network invoked via a Rust-compiled xcframework, outputting beat times, time signature, downbeats, and confidence.
Two analysis modes: Quick uses only LarsNet to split 5 components directly, suited for already drum-heavy audio; Deep uses the Demucs + LarsNet two-step approach, extracting the drum stem from a full song first - suited for mixes with vocals / other instruments. Both modes then run unified Madmom beat analysis, per-component DSP onset detection, quantization, and sequence generation.
Highlights
One-tap AI drum extraction: locally separates drum components from any song and generates an editable pattern in one tap - the core differentiator.
Pro step sequencer: 16 / 32 steps, full time-signature system (incl. compound 6/8, 12/8 and triplets), Groove styles (Straight / Swing / Shuffle / Blues), Humanize, Flam, Ratchet, per-step velocity.
24 carefully sampled pro kits: 5-parameter shaping per voice (Level / Pan / Filter / Tune / Decay), polyphonic playback (4-node pool per voice).
Full effect chain: AudioKit compressor + phaser + reverb + saturation + filter + PeakLimiter, savable to a Preset as a whole.
Multi-source recording + instant recognition: mic / system audio / screen-capture audio, dropped straight into the AI recognition flow after recording.
Cross-device consistency + auto session restore: iPhone / iPad / Mac share one project - debug on desktop, practice on mobile.
BPM Detector: Real-Time and File-Based Beat Tracking
A BPM / tempo detection app built for real workflows: detect BPM live via the microphone, or import audio / video files for analysis, with results saved per segment to history and replayable for comparison. For DJs, arrangers, instrument practice, and short-video editing.
BPM Detector turns a self-built Madmom beat-tracking library into a dedicated app: the microphone captures audio live to continuously output BPM, and imported audio or video files can be analyzed offline. It does not just give a single number; it outputs a full beat timeline, downbeats, bar lines, and time signature, all viewable against playback in history replay.
Core Features
Real-time BPM detection: microphone capture with an 8-second sliding window analyzed every 1 second, IQR quartile filtering to remove outliers, continuously outputting the current BPM.
Smart tempo selection: auto-picks among 0.5x / 1x / 2x candidates the value falling in the 60-130 BPM range (center 95), with manual override; manual selection expires back to auto after 30 seconds.
Video import & analysis: import videos from the photo library, extracting the audio track via AVAssetReader before analysis.
Time-signature detection: DBN beatsPerBar set to [2,3,4,6], inferring time signature (3/4, 4/4, 6/8) every 10 seconds from accumulated beat markers.
History & replay: JSON-file persistence for sessions and segments; replay uses AVPlayer with a BPM timeline chart, beat markers, and bar lines, draggable playhead, and tap-to-jump segments.
Beat visualization: spectrum waveform bars, ripple animation, and a real-time BPM trend chart.
BPM range selection: pick a range around the song's approximate BPM to improve accuracy.
Subscription & localization: RevenueCat subscription (free tier limits: 3-min live, 3-min import, 3 history items); Simplified Chinese / English / Japanese.
AI Models
Madmom BLSTM neural network, on-device via Rust FFI
Madmom downbeats BLSTM + DBN: Madmom's classic bidirectional LSTM beat / downbeat tracking model, with weights downbeats_blstm_weights.npz (3.3MB) and architecture downbeats_blstm.json, run on-device via a self-built madmom_beats_port_ffi.xcframework (Rust port) through a C bridge.
Inference flow: audio -> log-filtered spectrogram features (fps=100, frame sizes [1024,2048,4096]) -> BLSTM inference -> DBN post-processing -> beat times / time signature / downbeats / confidence. The same engine serves both real-time detection and offline file analysis.
Technical reuse: this Rust-ported Madmom beat-tracking library (madmom_beats_port_ffi) is also reused in Drum AI and SplitJam: Drum AI reuses the exact same xcframework and model files, and SplitJam reuses the same Madmom BLSTM model. BPM Detector is the only one of the four apps built solely around Madmom and containing no CoreML models.
Highlights
On-device AI, privacy-first: the Madmom neural network runs entirely locally, with no network and no audio upload.
Real-time + file + video, one engine: a single Madmom engine covers microphone live, audio files, and video audio tracks.
Beat-level, not just a number: outputs a full beat timeline, downbeats, bar lines, and time signature, viewable against playback in the detail page.
Smart tempo: normalizes the detected value to the musically natural 60-130 BPM range, avoiding half / double-tempo misjudgment.
A cross-platform professional metronome designed for music practice, serving drummers, guitarists, bassists, pianists, and music learners to build a steady sense of tempo and rhythmic control.
The core challenge of a metronome is precision - jitter misleads a learner's sense of beat. This app does not hand timing off to high-level framework timers; instead, playback is pushed down to the native audio layer. Flutter unifies the UI and business logic, while a self-built timing engine is invoked per platform via MethodChannel, guaranteeing sub-millisecond stability by design.
Core Features
High-precision tempo engine: BPM 1-500, time signatures of 2-16 beats with 2/4/8/16-note values; rated error under 1ms and jitter under 0.1ms.
Volume & timbre control: master volume layered with per-beat volume and strong / medium / weak / very-weak timbres, per-beat mute supported; timbres include standard, wood, digital, plus spoken count-in in Chinese / English / Japanese.
Tempo training: multiple speed stages, switch by time or bar count, loop practice, random tempo; practice schemes can be shared and imported.
Rhythm-stability training: interval-silence training and 5%-95% probabilistic random-silence training, forcing the learner to keep the beat internally.
Advanced rhythm-pattern system: built-in presets like Bossa Nova and Mambo, plus custom patterns with per-beat subdivisions, rests, and tuplets; a floating bar shows beat positions in real time.
Recording & rhythm analysis: auto-named recording with waveform visualization, per-beat logging precise to the millisecond; playback offers a record mode (true beat line reflecting BPM changes) and a manual mode (even line at fixed BPM) toggled with one tap to spot rushing / dragging.
Theme system: light / dark, Chinese-style, Nezha, concert hall, studio, stage, New Year presets, plus custom colors and background images.
AI Models
No AI models used
Tempo playback and rhythm analysis rely entirely on a traditional high-precision MIDI timing engine and audio waveform visualization, with no machine-learning / neural-network models. The project contains an ai.md research / planning doc listing potential directions such as Demucs Light separation, BeatNet / Tempo-CNN BPM detection, and RhythmNet rhythm analysis, with test notes - none have been implemented.
Highlights
Dual native audio-engine architecture: Flutter handles UI and business orchestration while beat playback sinks to the native layer - one set of business logic, two highest-precision audio paths.
FluidSynth real-time synthesis (Android): integrates FluidSynth and SoundFont (.sf2) via C++ / JNI; the project also maintains a glib-stripped FluidSynth source trimmed for mobile, used to build the native timbre-synthesis and timing library.
Apple native audio chain (iOS / macOS): uses AVAudioEngine + AVAudioSequencer + AVAudioUnitSampler to load SoundFont, with a multi-sampler scheme for strong/weak beats and timbre switching; iOS / macOS share the same native code.
Dual-mode recording beat-line comparison: record mode restores the true tempo trajectory, manual mode draws at a fixed BPM - toggling reveals rhythm problems instantly.
Multi-channel IAP & localization: Chinese / English / Japanese; Android ships four productFlavors (official / GooglePlay / GoogleLocal / Huawei) targeting Google Play Billing and Huawei IAP respectively, while iOS / macOS use StoreKit with local .p8 key subscription validation.
Online rhythm-pattern updates: local YAML config merged with online config, with cache fallback, so the pattern library keeps growing.