LashaBukhnikashvili's blog

Time Complexity Reduction
By LashaBukhnikashvili, history, 6 years ago, In English

Hello, I'm working on Audio Algorithm: Pitch-Shifter, unfortunately I can't find Text material about it design, But I found one of the code, implemented on c#: pastebin link.

I would post code here as well, and if you have any idea how I can reduce time or implement new one, please share it, Thanks in advance.

/****************************************************************************
*
* NAME: PitchShift.cs
* VERSION: 1.2
* HOME URL: http://www.dspdimension.com
* KNOWN BUGS: none
*
* SYNOPSIS: Routine for doing pitch shifting while maintaining
* duration using the Short Time Fourier Transform.
*
* DESCRIPTION: The routine takes a pitchShift factor value which is between 0.5
* (one octave down) and 2. (one octave up). A value of exactly 1 does not change
* the pitch. numSampsToProcess tells the routine how many samples in indata[0...
* numSampsToProcess-1] should be pitch shifted and moved to outdata[0 ...
* numSampsToProcess-1]. The two buffers can be identical (ie. it can process the
* data in-place). fftFrameSize defines the FFT frame size used for the
* processing. Typical values are 1024, 2048 and 4096. It may be any value <=
* MAX_FRAME_LENGTH but it MUST be a power of 2. osamp is the STFT
* oversampling factor which also determines the overlap between adjacent STFT
* frames. It should at least be 4 for moderate scaling ratios. A value of 32 is
* recommended for best quality. sampleRate takes the sample rate for the signal 
* in unit Hz, ie. 44100 for 44.1 kHz audio. The data passed to the routine in 
* indata[] should be in the range [-1.0, 1.0), which is also the output range 
* for the data, make sure you scale the data accordingly (for 16bit signed integers
* you would have to divide (and multiply) by 32768). 
*
* COPYRIGHT 1999-2006 Stephan M. Bernsee <smb [AT] dspdimension [DOT] com>
*
* 						The Wide Open License (WOL)
*
* Permission to use, copy, modify, distribute and sell this software and its
* documentation for any purpose is hereby granted without fee, provided that
* the above copyright notice and this license appear in all source copies. 
* THIS SOFTWARE IS PROVIDED "AS IS" WITHOUT EXPRESS OR IMPLIED WARRANTY OF
* ANY KIND. See http://www.dspguru.com/wol.htm for more information.
*
*****************************************************************************/

/****************************************************************************
*
* This code was converted to C# by Michael Knight
* madmik3 at gmail dot com. 
* http://sites.google.com/site/mikescoderama/ 
* 
*****************************************************************************/


using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public static class PitchShifter
{

    #region Private Static Memebers
    private static int MAX_FRAME_LENGTH = 16000;
    private static float[] gInFIFO = new float[MAX_FRAME_LENGTH];
    private static float[] gOutFIFO = new float[MAX_FRAME_LENGTH];
    private static float[] gFFTworksp = new float[2 * MAX_FRAME_LENGTH];
    private static float[] gLastPhase = new float[MAX_FRAME_LENGTH / 2 + 1];
    private static float[] gSumPhase = new float[MAX_FRAME_LENGTH / 2 + 1];
    private static float[] gOutputAccum = new float[2 * MAX_FRAME_LENGTH];
    private static float[] gAnaFreq = new float[MAX_FRAME_LENGTH];
    private static float[] gAnaMagn = new float[MAX_FRAME_LENGTH];
    private static float[] gSynFreq = new float[MAX_FRAME_LENGTH];
    private static float[] gSynMagn = new float[MAX_FRAME_LENGTH];
    private static long gRover, gInit;

    public static int Progress = 0;
    #endregion

    #region Public Static  Methods
    public static float[] PitchShift(float pitchShift, long numSampsToProcess,
       float sampleRate, float[] indata)
    {
        return PitchShift(pitchShift, numSampsToProcess, (long)2048, (long)10, sampleRate, indata);
    }
    public static float[] PitchShift(float pitchShift, long numSampsToProcess, long fftFrameSize,
        long osamp, float sampleRate, float[] indata)
    {
        int MAX_FRAME_LENGTH = 16000;
        gInFIFO = new float[MAX_FRAME_LENGTH];
        gOutFIFO = new float[MAX_FRAME_LENGTH];
        gFFTworksp = new float[2 * MAX_FRAME_LENGTH];
        gLastPhase = new float[MAX_FRAME_LENGTH / 2 + 1];
        gSumPhase = new float[MAX_FRAME_LENGTH / 2 + 1];
        gOutputAccum = new float[2 * MAX_FRAME_LENGTH];
        gAnaFreq = new float[MAX_FRAME_LENGTH];
        gAnaMagn = new float[MAX_FRAME_LENGTH];
        gSynFreq = new float[MAX_FRAME_LENGTH];
        gSynMagn = new float[MAX_FRAME_LENGTH];
        gRover = 0; gInit=0;
        float magn, phase, tmp, window, real, imag;
        float freqPerBin, expct;
        long i, k, qpd, index, inFifoLatency, stepSize, fftFrameSize2;


        float[] outdata = indata;
        /* set up some handy variables */
        fftFrameSize2 = fftFrameSize / 2;
        stepSize = fftFrameSize / osamp;
        freqPerBin = sampleRate / (float)fftFrameSize;
        expct = 2.0f * Mathf.PI * (float)stepSize / (float)fftFrameSize;
        inFifoLatency = fftFrameSize - stepSize;
        if (gRover == 0) gRover = inFifoLatency;


        /* main processing loop */
        Debug.Log(numSampsToProcess);
        for (i = 0; i < numSampsToProcess; i++)
        {
            Progress = (int)(((i + 1) * 100) / (1.0f * numSampsToProcess));
            /* As long as we have not yet collected enough data just read in */
            gInFIFO[gRover] = indata[i];
            outdata[i] = gOutFIFO[gRover - inFifoLatency];
            gRover++;

            /* now we have enough data for processing */
            if (gRover >= fftFrameSize)
            {
                gRover = inFifoLatency;
                
                /* do windowing and re,im interleave */
                for (k = 0; k < fftFrameSize; k++)
                {
                    window = -.5f * Mathf.Cos(2.0f * Mathf.PI * (float)k / (float)fftFrameSize) + .5f;
                    gFFTworksp[2 * k] = (float)(gInFIFO[k] * window);
                    gFFTworksp[2 * k + 1] = 0.0F;
                }
                
                /* ***************** ANALYSIS ******************* */
                /* do transform */
                ShortTimeFourierTransform(gFFTworksp, fftFrameSize, -1);
                
                /* this is the analysis step */
                for (k = 0; k <= fftFrameSize2; k++)
                {

                    /* de-interlace FFT buffer */
                    real = gFFTworksp[2 * k];
                    imag = gFFTworksp[2 * k + 1];

                    /* compute magnitude and phase */
                    magn = 2.0f * Mathf.Sqrt(real * real + imag * imag);
                    phase = Mathf.Atan2(imag, real);

                    /* compute phase difference */
                    tmp = phase - gLastPhase[k];
                    gLastPhase[k] = (float)phase;

                    /* subtract expected phase difference */
                    tmp -= (float)k * expct;

                    /* map delta phase into +/- Pi interval */
                    qpd = (long)(tmp / Mathf.PI);
                    if (qpd >= 0) qpd += qpd & 1;
                    else qpd -= qpd & 1;
                    tmp -= Mathf.PI * (float)qpd;

                    /* get deviation from bin frequency from the +/- Pi interval */
                    tmp = osamp * tmp / (2.0f * Mathf.PI);

                    /* compute the k-th partials' true frequency */
                    tmp = (float)k * freqPerBin + tmp * freqPerBin;

                    /* store magnitude and true frequency in analysis arrays */
                    gAnaMagn[k] = (float)magn;
                    gAnaFreq[k] = (float)tmp;

                }

                /* ***************** PROCESSING ******************* */
                /* this does the actual pitch shifting */
                for (int zero = 0; zero < fftFrameSize; zero++)
                {
                    gSynMagn[zero] = 0;
                    gSynFreq[zero] = 0;
                }

                for (k = 0; k <= fftFrameSize2; k++)
                {
                    index = (long)(k * pitchShift);
                    if (index <= fftFrameSize2)
                    {
                        gSynMagn[index] += gAnaMagn[k];
                        gSynFreq[index] = gAnaFreq[k] * pitchShift;
                    }
                    else break;
                }


                /* ***************** SYNTHESIS ******************* */
                /* this is the synthesis step */
                for (k = 0; k <= fftFrameSize2; k++)
                {

                    /* get magnitude and true frequency from synthesis arrays */
                    magn = gSynMagn[k];
                    tmp = gSynFreq[k];

                    /* subtract bin mid frequency */
                    tmp -= (float)k * freqPerBin;

                    /* get bin deviation from freq deviation */
                    tmp /= freqPerBin;

                    /* take osamp into account */
                    tmp = 2.0f * Mathf.PI * tmp / osamp;

                    /* add the overlap phase advance back in */
                    tmp += (float)k * expct;

                    /* accumulate delta phase to get bin phase */
                    gSumPhase[k] += (float)tmp;
                    phase = gSumPhase[k];

                    /* get real and imag part and re-interleave */
                    gFFTworksp[2 * k] = (float)(magn * Mathf.Cos(phase));
                    gFFTworksp[2 * k + 1] = (float)(magn * Mathf.Sin(phase));
                }

                /* zero negative frequencies */
                for (k = fftFrameSize + 2; k < 2 * fftFrameSize; k++) gFFTworksp[k] = 0.0F;

                /* do inverse transform */
                ShortTimeFourierTransform(gFFTworksp, fftFrameSize, 1);

                /* do windowing and add to output accumulator */
                for (k = 0; k < fftFrameSize; k++)
                {
                    window = -.5f * Mathf.Cos(2.0f * Mathf.PI * (float)k / (float)fftFrameSize) + .5f;
                    gOutputAccum[k] += (float)(2.0 * window * gFFTworksp[2 * k] / (fftFrameSize2 * osamp));
                }
                for (k = 0; k < stepSize; k++) gOutFIFO[k] = gOutputAccum[k];

                /* shift accumulator */
                //memmove(gOutputAccum, gOutputAccum + stepSize, fftFrameSize * sizeof(float));
                for (k = 0; k < fftFrameSize; k++)
                {
                    gOutputAccum[k] = gOutputAccum[k + stepSize];
                }

                /* move input FIFO */
                for (k = 0; k < inFifoLatency; k++) gInFIFO[k] = gInFIFO[k + stepSize];
            }
        }

        return outdata;
    }
    #endregion

    #region Private Static Methods
    public static void ShortTimeFourierTransform(float[] fftBuffer, long fftFrameSize, long sign)
    {

        float wr, wi, arg, temp;
        float tr, ti, ur, ui;
        long i, bitm, j, le, le2, k;

        for (i = 2; i < 2 * fftFrameSize - 2; i += 2)
        {
            for (bitm = 2, j = 0; bitm < 2 * fftFrameSize; bitm <<= 1)
            {
                if ((i & bitm) != 0) j++;
                j <<= 1;
            }
            if (i < j)
            {
                temp = fftBuffer[i];
                fftBuffer[i] = fftBuffer[j];
                fftBuffer[j] = temp;
                temp = fftBuffer[i + 1];
                fftBuffer[i + 1] = fftBuffer[j + 1];
                fftBuffer[j + 1] = temp;
            }
        }
        long max = (long)(Mathf.Log(fftFrameSize) / Mathf.Log(2.0f) + .5);
        for (k = 0, le = 2; k < max; k++)
        {
            le <<= 1;
            le2 = le >> 1;
            ur = 1.0F;
            ui = 0.0F;
            arg = (float)Mathf.PI / (le2 >> 1);
            wr = (float)Mathf.Cos(arg);
            wi = (float)(sign * Mathf.Sin(arg));
            for (j = 0; j < le2; j += 2)
            {

                for (i = j; i < 2 * fftFrameSize; i += le)
                {
                    tr = fftBuffer[i + le2] * ur - fftBuffer[i + le2 + 1] * ui;
                    ti = fftBuffer[i + le2] * ui + fftBuffer[i + le2 + 1] * ur;
                    fftBuffer[i + le2] = fftBuffer[i] - tr;
                    fftBuffer[i + le2 + 1] = fftBuffer[i + 1] - ti;
                    fftBuffer[i] += tr;
                    fftBuffer[i + 1] += ti;

                }
                tr = ur * wr - ui * wi;
                ui = ur * wi + ui * wr;
                ur = tr;
            }
        }
    }
    #endregion
}
  • Vote: I like it
  • -41
  • Vote: I do not like it

»
6 years ago, # |
← Rev. 35   Vote: I like it +1 Vote: I do not like it

The window parameter in the interleaving k loop of the second PitchShift() function is independent of the sample number i. You can pre-compute and store these values in an array window[ fftFrameSize ], before the outer i loop, in a dynamic programming sense, and use window[ k ] in the interleaving loop.

A C++17 update before the outer loop should look like that:

const float pi = acos( -1.0 ), twice_pi = 2.0 * pi;

float window[ fftFrameSize ];
    
for( int k = 0; k < fftFrameSize; k++ )
   window[ k ] = 0.5 - 0.5 * cos( twice_pi *  float( k ) / float( fftFrameSize ) );

And the updated interleaving loop should look like that:

/* do windowing and re,im interleave */
                
for ( int k = 0, l = 0, m = 1; k < fftFrameSize; k++, l += 2, m += 2 )
    gFFTworksp[ l ] = gInFIFO[ k ] * window[ k ],
    gFFTworksp[ m ] = 0.0;

The same window array can be used later in the following loop.

/* do windowing and add to output accumulator */

for ( int k = 0, l = 0; k < fftFrameSize; k++, l += 2 )
    gOutputAccum[ k ] *= osamp_factor * window[ k ] * gFFTworksp[ l ];

where osamp_factor can be computed before the outer loop as:

float osamp_factor = 2.0 / ( float( fftFrameSize2 ) * float( osamp ) );

Finally, a more recent C++ version of the C# code is available on Stephan Bernsee's Blog. You may also review the source code of the Rubber Band Library for audio time stretching.

The following is a test program for the proposed updates.

→ Reply
  • »
    »
    6 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    Thank you very much for your help, Sir.

    → Reply
    • »
      »
      »
      6 years ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      With pleasure.

      Please re-check the post and the test program, as the +0.5 factor at the end of the statement in the second loop was overlooked. Actually, both loops use the same expression for window. Therefore, the second vector c_window in redundant, and it is sufficient to use the former vector only.

      → Reply