Manually resynthesizing ATS residuals

Dear Csounders!

How to manually resynthesize resulting ATS residuals?

If I store ATS results of c_note.wav into c_note.ats using this command:

ires system_i 1,{{ atsa c_note.wav c_note.ats }}

then I can, using ATSread opcode and recursion, pretty simply manually re-synthesize ATS partials using e.g. RecursivePartialsSynth UDO (see code below).

On the other hand when I do similar thing with residuals and want recursively resynthesize 25 critical bands using ATSreadnz opcode, it doesn’t work. Resulting noise is way too strong. (see RecursiveResidualSynth UDO in the code below)

Values for 25 critical band frequency edges I took from 05K04_atsreadnz.csd example in floss manual.
It seems to me that I’m probably calculating bandwidth wrong or I’m using reson opcode wrong.

Any idea how to fix this?

Here is my code:

<CsoundSynthesizer>
<CsOptions>
-odac
</CsOptions>
<CsInstruments>

sr = 48000
ksmps = 32
nchnls = 2
0dbfs  = 1


// relevant examples:
// https://flossmanual.csound.com/sound-modification/ats-resynthesis
// https://github.com/csound/manual/issues/561#issuecomment-879122056

ires system_i 1,{{ atsa c_note.wav c_note.ats }} ; default settings

giAtsNoiseFc ftgen 0, 0, 32, -2, 0, 100, 200, 300, 400, 510, 630, 770, 920, 1080, 1270, 1480, 1720, 2000, 2320, 2700, 3150, 3700, 4400, 5300, 6400, 7700, 9500, 12000, 15500, 20000

// UDO to recursively synthesize and add the partials
opcode RecursivePartialsSynth, a, Skip
	Sfile,ktime,inparts,istart xin

	if istart < inparts then ;if inparts have not yet reached
		acall RecursivePartialsSynth Sfile, ktime, inparts, istart+1 ;call another instance of this UDO
	endif

	kfreq, kamp	ATSread  ktime, Sfile, istart ; take the istart-th partial
	aout oscili kamp, kfreq
	aout = aout + acall ;add the audio signals
	xout  aout
endop


opcode RecursiveResidualSynth, a, Skip
	Sfile,ktime,inparts,istart  xin

	if istart < inparts then ;if inparts have not yet reached
		acall RecursiveResidualSynth Sfile, ktime, inparts, istart+1 ;call another instance of this UDO
	endif

    iscal = 1                       ;reson filter scaling factor
	iband = istart                  ;energy band required
	if1     table   iband-1, giAtsNoiseFc  ;lower edge
	if2     table   iband, giAtsNoiseFc    ;upper edge
	idif    = if2-if1
	icf     = if1 + idif*.5         ;center frequency value
	ibw     = idif*0.7              ; NOTE: this is probably wrong  

	ken ATSreadnz ktime, Sfile, iband ;energy value for this band
	anoise gauss 1
	aout reson anoise*sqrt(ken), icf, ibw, iscal ;synthesize with scaling NOTE: or maybe Im using reson wrong
	aout = aout + acall ;add the audio signals
	xout  aout
endop


// Instrument that reads ats file and resynthesie it manualy usinig oscilators and noise generators
instr AtsResynth	
	Sfile   =       "c_note.ats"
	isr     ATSinfo Sfile, 0
	ifs     ATSinfo Sfile, 1
	iws     ATSinfo Sfile, 2
	inp     ATSinfo Sfile, 3
	inf     ATSinfo Sfile, 4
	ima     ATSinfo Sfile, 5
	imf     ATSinfo Sfile, 6
	id      ATSinfo Sfile, 7
	ift     ATSinfo Sfile, 8
			prints  {{
	*****c_note.ats*********************
	0. Sample rate = %d Hz
	1. Frame Size  = %d samples
	2. Window Size = %d samples
	3. contains      %d partials
	4. contains      %d frames
	5. Max. Ampl.  = %f
	6. Max. Freq.  = %f Hz
	7. Duration    = %f seconds
	8. File Type   = %d
	*********************************\\n
	}}, isr, ifs, iws, inp, inf, ima, imf, id, ift

	p3 = id ;set instrument duration to file duration
	ktime timeinsts
	aoutPartials RecursivePartialsSynth Sfile, ktime, inp, 1 ;call the UDO

	aoutResiduals RecursiveResidualSynth Sfile, ktime, 25, 1 ;call the UDO -> there are always 25 critical bands

	if p4 == 0 then
		aout = aoutPartials ;use only partials
		fout "c_note_partials_only.wav", 18, aout ;write to soundfile
	else
		aout = aoutPartials + aoutResiduals ;add the partials and residuals
		fout "c_note_partials_and_residuals.wav", 18, aout ;write to soundfile
	endif

	outs aout, aout
endin

</CsInstruments>
<CsScore>
i "AtsResynth" 0 3 0 ; p4=1 -> use resiudals also, p4=0 -> use only partials
</CsScore>
</CsoundSynthesizer>

hi lovre -

i don’t know if it helps, but i have some different ideas:

  1. what is the range of ken in the line
    ken ATSreadnz ktime, Sfile, iband
  2. would it help to scale the noise signal accordingly?
  3. i don’t think you used reson in a wrong way, but you could try butbp
    to compare.
  4. i seem to remember that ATS uses randi for resynthesizing the noise
    bands, not noise plus filter. i think i read it in …
    ah yes i found it — have a look at section 8 in
    https://dxarts.washington.edu/sites/dxarts/files/documents/wiki/ats_theory.pdf

best -
joachim

Thanks @joachim !

That was very helpful. Now I understand the ATS method better and what I’m doing wrong.
I think I need to modulate the nose in particular critical band with a corresponding partial’s sinusoidal component.

If I manage to implement something that works, I will post it here :nerd_face:

Did u find any results?
It’s interesting!

Hi @Cordelia !

Sorry for the late response; I had a (pretty) long vacation :upside_down_face:

I only partially managed to crack this issue but because it was enough for my use-case I sopped there.
Since I’m developing one ATS GUI tool/plugin, what I needed was the ability to play with noise levels in every band and continuously in time so I end up using ATSaddnz opcode since it gives me exactly that.

This my ATS tool is still in the development so here is a relatively simple example of how I’m using ATS:

<CsoundSynthesizer>
<CsOptions>
-odac
</CsOptions>
<CsInstruments>

sr = 48000
ksmps = 32
nchnls = 2
0dbfs  = 1


// relevant examples:
// https://flossmanual.csound.com/sound-modification/ats-resynthesis
// https://github.com/csound/manual/issues/561#issuecomment-879122056

ires system_i 1, "atsa -h0.1 c_note.wav c_note.ats" ; ATS analysis 

// UDO to recursively synthesize and add the partials
opcode RecursivePartialsSynth, a, Skip
	Sfile,ktime,inparts,istart xin

	if istart < inparts then ;if inparts have not yet reached
		acall RecursivePartialsSynth Sfile, ktime, inparts, istart+1 ;call another instance of this UDO
	endif

	kfreq, kamp	ATSread  ktime, Sfile, istart ; take the istart-th partial
	apartial oscili kamp, kfreq

	aout = apartial + acall ;add the audio signals
	xout  aout
endop


// UDO to recursively synthesize and add the residuals
opcode RecursiveResidualSynth, a, Skip
	Sfile,ktime,inparts,istart  xin

	if istart < inparts-1 then ;if inparts have not yet reached
		acall RecursiveResidualSynth Sfile, ktime, inparts, istart+1 ;call another instance of this UDO
	endif

	anoise  ATSaddnz ktime, Sfile, 1, istart

	aout = anoise/10 + acall ;add the audio signals -> NOTE: this division by 10 is artificially added to reduce synthesized noise level
	xout  aout
endop


// Instrument that reads ats file and resynthesie it manualy usinig oscilators and noise generators
instr AtsResynth	
	Sfile   =       "c_note.ats"
	isr     ATSinfo Sfile, 0
	ifs     ATSinfo Sfile, 1
	iws     ATSinfo Sfile, 2
	inp     ATSinfo Sfile, 3
	inf     ATSinfo Sfile, 4
	ima     ATSinfo Sfile, 5
	imf     ATSinfo Sfile, 6
	id      ATSinfo Sfile, 7
	ift     ATSinfo Sfile, 8
			prints  {{
	*****c_note.ats*********************
	0. Sample rate = %d Hz
	1. Frame Size  = %d samples
	2. Window Size = %d samples
	3. contains      %d partials
	4. contains      %d frames
	5. Max. Ampl.  = %f
	6. Max. Freq.  = %f Hz
	7. Duration    = %f seconds
	8. File Type   = %d
	*********************************\\n
	}}, isr, ifs, iws, inp, inf, ima, imf, id, ift

	p3 = id ;set instrument duration to file duration
	ktime timeinsts ; get current time

	aoutPartials RecursivePartialsSynth Sfile, ktime, inp, 1 ;synth partials
	aoutResiduals RecursiveResidualSynth Sfile, ktime, 25 ;synth residuals -> there are always 25 critical bands


	if p4 == 0 then
		aout = aoutPartials + aoutResiduals ;add the partials and residuals
		fout "c_note_partials_and_residuals.wav", 18, aout ;write to soundfile
	elseif p4 == 1 then
		aout = aoutPartials ;use only partials
		fout "c_note_partials_only.wav", 18, aout ;write to soundfile
	elseif p4 == 2 then
		aout = aoutResiduals ;use only residuals
		fout "c_note_residuals_only.wav", 18, aout ;write to soundfile
	endif

	outs aout, aout
endin

</CsInstruments>
<CsScore>
i "AtsResynth" 0 3 2 ; p4=0 -> use partials and residuals, p4=1 -> use only partials, p4=2 -> use only residuals
</CsScore>
</CsoundSynthesizer>

I still don’t know why resynthesized noise is so strong hence I artificially divide it by 10.

Here is also a wav file that I’m using (since .wav files are not supported in this forum I just changed file’s extension to .txt so just change it back to .wav before using it :wink:). If you put it in the same directory as the csound script, code should work out of the box . If you want to test it with some other (mono) wav files, just change file name in the code.

c_note.txt (202.0 KB)