Polyphonic pitch tracking

General
#1

I’ve been looking for ways to track multiple pitches (the frequencies of multiple fundamental pitches and their relative amplitudes) such as when several voices / chords are being played on a piano. The Csound manual only mentioned monophonic pitch tracking opcodes as far as I could tell…

Does anyone happen to know of a plugin or custom opcode for Csound that would be capable of doing this?

Though I’d prefer if it were possible to do this directly within Csound, I’m of course open to different ideas, e.g. someone suggested to try connecting csound to pure data, and I’ve heard of people accomplishing something like this in python, e.g. with librosa (though I’m not sure of all the details and how well these work in real time)…

#2

It’s a while since I used Pd, but do you know the name of the object that can do this, i.e., will break a chord into its various different pitches?

#3

Me too—haven’t used it for a long time, and unfortunately I don’t know yet what the name of the object is—someone suggested it on the mailing list that I mentioned above, but for some reason I can’t post any replies there despite having signed up, so I couldn’t ask any follow-up questions…

#4

I don’t think there is anything in Pd that can do this out of the box. I think you would be faced with the same complex issues in Pd as in Csound. If there was something that did this in Pd we could look at porting the object to a Csound opcode.

#5

I’m not aware of such an object in Pd, but I used sigmund~ in the past, with decent results: you don’t have polyphonic tracking, but you can choose how many partials you want to track.
The overall results depend strongly by the characteristics of the instruments you want to track.

Separating instruments is very difficult: you can try using timbreID to recognise the instruments that are playing (you need to train the object with prerecorded data, and I don’t know how many instruments can be detected at the same time), but then you will also need to add some logic to extract each instrument from the mix and send it to sigmund~ to track it.
In the case of a piano, once extracted, you can set how many partials you want to detect. The results won’t be that accurate (it depends on how many notes you play together), but it’s a start!

EDIT:
Attached a cleaned version of the patch I was using, it can be extended to listen for any number of inputs.
It already uses timbreID to track the spread of the signal, it shouldn’t be too difficult to use its other analysis capabilities.
You will need Pd-Extended for this to work, if I remember correctly.
MIA_1.6.pd.zip (6.1 KB)

EDIT 2:
It seems that such a tool exists for SuperCollider:

maybe someone could port it to Csound?