Audio Analysis
Raw audio → MFCC, beat detection, energy curves, structural segmentation, CLAP embeddings
The Vision
What if a lighting designer's years of experience could be encoded into a model — and used to generate a full light show from a single audio file?
MLI is a private independent non-profit research project exploring neural audio-to-light synthesis. The goal: train a model that understands the relationship between sound and light the way a professional designer does.
Phase one focuses on learning brightness, energy, and beat-synchronization from real concert footage. Phase two will introduce synthetic training data generated by an LLM trained on professional lighting design literature.
This is not a finished product! This is research only for now.
The System (for the PoC)
Feature Extraction
Think of it like a question: What is happening in this song?
Sequence Modeling
BiLSTM encoder processes temporal audio context. LSTM decoder generates frame-by-frame light grid predictions.
Grid Output
12×8 RGB grid representing stage lighting state at every frame — brightness, color, spatial distribution
Synthetic Pipeline (In Development)
LLM-based lighting designer generates training data at scale. Stage layouts, fixture distributions, music-reactive cue lists — all synthetic, all labeled. Human training data remains essential for validation and real-world progression.
Current Status
Training Data
Active
96 real concert recordings processed. Synthetic pipeline in development. Target: 10,000+ training samples.
Model
In Progress
CLAP + BiLSTM architecture. 3.3M trainable parameters. Beat-synchronization: functional. Style transfer: research phase.
Product
Planned
Private independent non-profit research project moving toward a for profit Organisation desktop application for lighting designers. Built to connect with every major professional lighting workflow and control environment. Target: late 2026 to mid 2027.
Get In Touch
Research inquiries, collaboration, data partnerships.
If you work in professional lighting design and are interested in this private independent non-profit research project — or if you have access to lighting show recordings and would like to contribute to the research — reach out.
[ LinkedIn ]External link disclaimer: LinkedIn is operated by LinkedIn Ireland Unlimited Company. The Owner of the current page is not responsible for external content, data processing, tracking, availability, or changes on LinkedIn after leaving this website.