Ah, MIDI files. The only type of music you could realistically download from the internet back in the day, and you had to wake up at ungodly hours so that your dialup modem would not rack up a massive phone bill.
killerstorm 22 minutes ago [-]
MIDI is still widely used for professional music production. It sounded goofy back in the day because synthesizer it was played on was not very good.
HPsquared 11 minutes ago [-]
I'm thinking canyon.mid on Microsoft GS Wavetable Synth.
xtracto 6 minutes ago [-]
I remember downloading MOD files. It was just a bit larger than midi, but sounded better. I had a PC but I think it was an Amiga thing
heikkilevanto 1 hours ago [-]
I would love something like that, except that I play the melody, and it produces proper 3-4 part accompaniment, preferably in good baroque style. Extra bonus if it could also write it into a file in a format suitable for music editing programs.
Naitik88 3 minutes ago [-]
This feels like a natural next step. Starting with a simple melody and having the system fill in the rest while still following your playing style could make it much more useful for experimentation than generating a complete piece from scratch.
vunderba 23 minutes ago [-]
An early attempt at this was Microsoft Songsmith [1] all the way back in 2009, which would take a melody (usually recorded by mic) and try to scaffold an accompaniment around it though obviously not realtime in any sense of the word.
The closest we've had to realtime orchestration around a melody in the "real world" is probably arranger keyboards though your left hand is still responsible for the chord progression itself.
That's a fun idea. You could start playing the piano and it kicks in with a base and drums for a jazz band.
gaya3bollineni 34 minutes ago [-]
Running a 125M model on-device at that speed is impressive. How much did you have to optimize the model to get that performance on an iPhone?
Naitik88 5 minutes ago [-]
The answer about changing the note representation was interesting. Sometimes a change in how the problem is represented ends up giving a much bigger improvement than trying to optimize the model itself.
simedw 18 minutes ago [-]
The biggest speed improvement came from changing the note representation when I switched to compound note events: roughly 5× fewer autoregressive passes per note.
For the current model I’m using Core ML, which optimizes the kernels the first time you run it. I haven’t actually spent that much time tuning performance beyond that.
speedgoose 31 minutes ago [-]
Talking about AI music with some live human MIDI inputs, Magenta Realtime 2 was released a few weeks ago and is pretty fun.
This is really fun. Scaler 3 starts with a chord progression and lets you break it down into musical performances and parts. Useful for ideation when producing.
Would be fun to get a midi clock going and play some chords on my piano and have my synth start jamming along with the bass and my keyboard doing some performance. Or any combination of the above.
Naitik88 4 minutes ago [-]
That would be a really interesting direction. At that point it starts feeling less like autocomplete and more like having another musician reacting to what you're playing in real time.
mh- 1 hours ago [-]
+1 all of this. That would be incredible (this already seems very cool - excited to get home and try it!)
davidajackson 1 hours ago [-]
How would you expand this to support elements like attack ("velocity of the key-down" in piano speak), grace notes, timing etc. Would each of those be part of this model or another model? How would you model an arbitrary element (pedal, duration, etc...)
simedw 56 minutes ago [-]
[dead]
aghilmort 8 minutes ago [-]
really incredible work! great use case, impeccable learning strategy, congrats!
evalystai 2 hours ago [-]
The idea is awesome! :) However there's definitely much room for improvement, first of all rythm and composition (so there's some sense of musical form).
simedw 2 hours ago [-]
Thank you.
Yes, I think I’ve gotten it to roughly a GPT-2 level: good enough to share, but with a lot of room left to improve. I think adding some kind of bar/measure token might help with rhythm, and perhaps some form of longer-term planning for the overall composition.
subhajeet2107 1 hours ago [-]
This is so amazing, can you improve the quality of generation at the cost of notes per seconds ? No one can play 108 notes/sec anyways, maybe you can train the model to do CoT for better quality
simedw 43 minutes ago [-]
Yes, some kind of planning step is on my TODO list. Another thing I want to try is generating a few continuations in parallel, picking the one that looks best, and then continuing from there. Maybe the picking could be automatic.
I can probably squeeze out quite a bit more than 100 notes/sec as well. I haven’t spent much time optimizing inference yet.
the_black_hand 23 minutes ago [-]
Cool work. I tried using LLMs to parse sheet music and they are really bad.
Naitik88 1 minutes ago [-]
[dead]
isoprophlex 2 hours ago [-]
Amazing idea! Gonna hook this up to my little synthesizer and blast some square wave arpeggiated ML music!
leobg 1 hours ago [-]
I don’t have MIDI. How about whistling or playing the piano via microphone? Sounds easy. Another 6 month rabbit hole? :)
pil0u 1 hours ago [-]
Even after a few years deep into AI, I find your application absolutely magic. This is very inspiring, thank you for sharing.
arkmm 52 minutes ago [-]
Very cool! Can you say a little bit about the size of the DPO training examples and how long training took?
simedw 46 minutes ago [-]
For DPO I only had around 700 preference examples, so not much data at all. That took about 12 minutes to train on a single GPU.
Pretraining was obviously a a lot slower, the 125M model took roughly half a day.
yoouareperfect 2 hours ago [-]
This is really awesome thanks for sharing
jwr 1 hours ago [-]
> Eventually I used Gemini 3.5 Flash for pairwise evaluation
But, but… wouldn't that be… (gasp) DISTILLATION?
Fun project!
1 hours ago [-]
5 minutes ago [-]
lvbyte 1 hours ago [-]
[dead]
qmmmur 2 hours ago [-]
[flagged]
kennyadam 2 hours ago [-]
Is this HN? Aren't people supposed to tinker with tech for no reason other than seeing if they can? Is everything that involves a transformer now just AI BAD? Is that what the world has devolved into? Each side screaming "Orange Man Bad" and endless variations at each other?
meerita 2 hours ago [-]
It is tiring to read these people. It feels like nothing will please them, but their own ideas, work, methods.
tescreal 2 hours ago [-]
While it is a neat parlor trick, a lot of people have specific grevience against the application to art. AI has only served to further disempower artists broadly, and arguably it pushes "art" to a lower common denominator. Try to actually situate yourself in "why" people get rankled instead of making it a thought terminating cliche.
dgellow 1 hours ago [-]
Given the AI crowd is very loudly telling us since years how humans will be replaced by LLMs and we will all be poor and left behind if we don’t join their cult, I think it’s a perfectly reasonable reaction. Anything with AI mixed with Art or other human experience is suspicious.
Also, yes, the orange fascist who attempted to coup his way to power, raped women, and is destroying democratic institutions is indeed bad.
phoghed 1 hours ago [-]
No they haven’t. It’s like two guys that nobody really believes. You specifically seek that information out to get mad, then proceed to see it where it’s not. Like when someone makes a cool personal project about their hobby and it happens to involve AI.
dgellow 1 hours ago [-]
I’m not mad and have nothing against the project.
> No they haven’t. It’s like two guys that nobody really believes.
It’s the entire leadership of the AI industry, in case you haven’t noticed
Razengan 1 hours ago [-]
> Given the AI crowd is very loudly telling us since years how humans will be replaced by LLMs and we will all be poor
Oh no, AI will take our jobs??? FUCKING LET IT!
Who even wants to do all these jobs if we don't HAVE to??
Ask politicians to give us UBI.
You want to attack the shit that makes shit easier instead of attacking the 200 year old institutions in place that ensure class divisions and perpetual debt and wage slavery? Smart bugger
dgellow 1 hours ago [-]
> You want to attack the shit that makes shit easier instead of attacking the 200 year old institutions in place that ensure class divisions and perpetual debt and wage slavery
I have bad news for you if you think that will improve with the current deployment of AI. You will also notice I didn’t mention anything about jobs
perching_aix 1 hours ago [-]
4) it's fun and quaint, like an odd, jazzy version of a player piano from an alternate universe
isoprophlex 2 hours ago [-]
man hits thing
thing go plink
machine hear plink
machine make many more plink
man happy for plink is fun
man not hit machine with club or scream on orange site
man leave cave and touch plant
Razengan 2 hours ago [-]
What is the goal here?
To eventually prove that Intelligence™ and Creativity™ isn't special?
Or maybe there is some other "spark" deeper within?
When everyone can make anything as soon as they think of it, what will set us apart?
Rendered at 14:43:31 GMT+0000 (Coordinated Universal Time) with Vercel.
https://www.francoispachet.fr/continuator/
The closest we've had to realtime orchestration around a melody in the "real world" is probably arranger keyboards though your left hand is still responsible for the chord progression itself.
[1] - https://en.wikipedia.org/wiki/Microsoft_Research_Songsmith
For the current model I’m using Core ML, which optimizes the kernels the first time you run it. I haven’t actually spent that much time tuning performance beyond that.
https://magenta.withgoogle.com/magenta-realtime-2
Would be fun to get a midi clock going and play some chords on my piano and have my synth start jamming along with the bass and my keyboard doing some performance. Or any combination of the above.
Yes, I think I’ve gotten it to roughly a GPT-2 level: good enough to share, but with a lot of room left to improve. I think adding some kind of bar/measure token might help with rhythm, and perhaps some form of longer-term planning for the overall composition.
I can probably squeeze out quite a bit more than 100 notes/sec as well. I haven’t spent much time optimizing inference yet.
Pretraining was obviously a a lot slower, the 125M model took roughly half a day.
But, but… wouldn't that be… (gasp) DISTILLATION?
Fun project!
Also, yes, the orange fascist who attempted to coup his way to power, raped women, and is destroying democratic institutions is indeed bad.
> No they haven’t. It’s like two guys that nobody really believes.
It’s the entire leadership of the AI industry, in case you haven’t noticed
Oh no, AI will take our jobs??? FUCKING LET IT!
Who even wants to do all these jobs if we don't HAVE to??
Ask politicians to give us UBI.
You want to attack the shit that makes shit easier instead of attacking the 200 year old institutions in place that ensure class divisions and perpetual debt and wage slavery? Smart bugger
I have bad news for you if you think that will improve with the current deployment of AI. You will also notice I didn’t mention anything about jobs
thing go plink
machine hear plink
machine make many more plink
man happy for plink is fun
man not hit machine with club or scream on orange site
man leave cave and touch plant
Or maybe there is some other "spark" deeper within?
When everyone can make anything as soon as they think of it, what will set us apart?