2023-02-27

sound synthesis

see also analysis and resynthesis and frequency filtering. related to digital signal processing

models

thought models for thinking about sound

  • physically: sound is made of longitudinal vibrations of matter in three dimensional space. humans have two main pathways to receive and analyse changes in air pressure and also discern its source direction to a limited extent. regularity in received signals is of particular significance and can be described by frequency
  • time/magnitude: one channel of magnitude samples over time as the representation of a vibration of a directionless sound with the values of any separate contributing events summed. can be transferred to a membrane directly to reproduce sound
  • frequency/phase/time/magnitude: imagine a kind of piano roll, that instead of being limited to piano keys and note onsets or durations, holds horizontally the information for finely spaced sine frequencies and circumferential detailed amplitude envelopes of each. this is a type of split representation of time/magnitude data
  • sound sources and instruments: sound being the excitation of matter begins with the excitation of matter. air pressed through a small opening might become to oscillate at high frequencies. traditional musical instruments usually have interfaces for excitation of matter to be induced by human motion, parameterised for example by relative striking position or intensity

non-noise

simple periodic wave forms

wave forms other than sines are mostly used for subtractive synthesis

  • square

    • alternation between two values with the same duration for each
    • frequency spectrum contains only odd harmonic frequencies
    • its stochastic counterpart is a two-state trajectory

  • triangle

    • linear change with abrupt directional change at the extremes
    • frequency spectrum contains only odd harmonics

  • sawtooth

    • like a triangle with one rectangular angle
    • "its spectrum contains both even and odd harmonics of the fundamental frequency. because it contains all the integer harmonics, it is one of the best waveforms to use for subtractive synthesis of musical sounds, particularly bowed string instruments like violins and cellos, since the slip-stick behavior of the bow drives the strings with a sawtooth-like motion"
    • creates aliasing if frequencies above half the sample rate are included

  • sine wave

    • single-frequency spectrum, no harmonics
    • doesnt change angular direction abruptly
    • the only periodic waveform that retains its wave shape when added to another wave with that form of the same frequency and arbitrary phase and magnitude
    • one full cycle in 2 * pi radians
    • every sound can theoretically be created by a sum of a possibly infinite number of sines

    • samples can be taken from

      • common sin() function. it is calculated with the taylor series and many optimizations
      • a lookup table where elements are pre-calculated samples. this saves the time to evaluate sin() but may lead to cache misses. values are pre-calculated for a specific sampling rate and frequency. if the frequency does not divide the sampling rate without rest, interpolation or re-calculation may be used
      • less precise sine approximation functions

  • some more shapes

    • pulse

      • like a square wave with two different peak durations

    • rectangle

      • like a square wave where peak duration as well as the minimum and maximum value can vary

      • a bit like an amplitude and frequency modulated square wave

frequency modulation synthesis

  • an introduction to fm
  • wikipedia: frequency modulation synthesis
  • wikipedia: frequency modulation
  • formulas for frequency modulation
  • modulating the frequency of one wave according to another wave
  • vibrato is slow frequency modulation below 20 hz
  • the time domain waveform is level and not spikey because only the frequency changes. the waveform looks distinctly different from the waveform created by summing the partials separately
  • the bandwidth increases with the amplitude of the modulator
  • the overall amplitude stays practically the same regardless of bandwidth
  • partials with negative frequencies are possible, which means they have an inverted phase. these partials may interfere with the positive frequency partials

partial frequencies

carrier-frequency - partial-index * modulator-frequency
carrier-frequency + partial-index * modulator-frequency

partial amplitudes

bessel(partial-index, modulator-amplitude / modulator-frequency)

the first parameter is the order of the bessel function and the second argument is the modulation index

online calculator

time domain waveform

fm(t) = carrier-amplitude * sin(t * carrier-frequency + modulator-amplitude * sin(t * modulator-frequency))

alternative:

fm(t) = sum(n, -infinity, infinity, bessel(n, modulator-amplitude / modulator-frequency) * sin(t * carrier-frequency + n * t * modulator-frequency))

noise

  • typically created from samples of a random number generator passed through filters to remove frequencies
  • there do not seem to be many other methods to create noise. summing many random sines is possible but computationally intensive
  • frequency modulation synthesis can create noise, sin(sin(rand))

other time series post-processing to shape sounds

  • delay: input samples are put out at a later time, possibly repeated

  • reverberation

    • can be simulated with many short delays that repeat delayed output and change amplitude and frequency with each repetition
    • convolution can also create reverb effects. convincing, but without dynamically changing parameters

  • basic operations

    • multiplication: shapes the amplitude of signals

    • division: shapes the amplitude of signals. dividing a signal by itself flattens the signal

    • addition: can be used to sum sine waves

    • convolution: multiplies frequencies. "under suitable conditions the fourier transform of a convolution of two signals is the pointwise product of their fourier transforms"

grain processing

a signal split into small chunks that are then processed

  • repetition or removal of pieces can create a time stretching effect that does not change pitch as much as removing or duplicating individual samples would
  • the repetition of larger grains can create interesting rhythmical effects similar to a repeated delay
  • example operations: randomise, repeat, reduce, effects on selected grains, swap. example parameters: grain_size, repetition.
  • the total output of a grain processor can be shorter or longer than the input. this might lead to a growing length difference between the unprocessed and processed signal and require buffering

other

  • additive vs subtractive processes: add vs subtract parts to create a desired result. for example summing sinusodials vs filtering
  • amplitude modulation: in particular, fast changes of the amplitude. related to tremolo
  • frequency modulation: changing frequency based on the values of another signal. this creates new harmonics. "in the context of audio coding, fm synthesis can be considered a "lossy compression method" for additive synthesis". related to vibrato
  • granular synthesis: grain processing

  • transformation of recorded sound vs synthesis from formulas alone

    • getting sound parameters from analysing recorded sounds, while necessarily lossy, gives access to a wide variety of sounds with complex content
    • purely synthesised sounds, while being an exact representation of their original intent and possibly creating what can not be recorded or analysed precisely, require much work to be as complex as real world recorded sounds