2020-12-30

sound synthesis

see also analysis and resynthesis and frequency filtering. related to digital signal processing

models

thought models for thinking about sound

  • physically: sound is made of longitudinal vibrations of matter in three dimensional space. humans have two main pathways to receive and analyse changes in air pressure and also discern its source direction to a limited extent. regularity in received signals is of particular significance and can be described by frequency
  • time/magnitude: one channel of magnitude samples over time as the representation of a vibration of a directionless sound with the values of any separate contributing events summed. can be transferred to a membrane directly to reproduce sound
  • frequency/phase/time/magnitude: imagine a kind of piano roll, that instead of being limited to piano keys and note onsets or durations, holds horizontally the information for finely spaced sine frequencies and circumferential detailed amplitude envelopes of each. this is a type of split representation of time/magnitude data
  • sound sources and instruments: sound being the excitation of matter begins with the excitation of matter. air pressed through a small opening might become to oscillate at high frequencies. traditional musical instruments usually have interfaces for excitation of matter to be induced by human motion, parameterised for example by relative striking position or intensity

non-noise

simple periodic wave forms

  • square

    • alternation between two values with the same duration for each
    • frequency spectrum contains only odd harmonic frequencies
    • amplitudes of the harmonics can be calculated with 2 / (pi * odd_n)
    • its stochastic counterpart is a two-state trajectory
  • triangle

    • linear change with abrupt directional change at the extremes
    • frequency spectrum contains only odd harmonics
    • triangle(t) = 8 / pi ** 2 sum(i, 0..(N - 1), -1 ** i n ** -2 sin(2 pi f n t))
  • sawtooth

    • like a triangle with one rectangular angle
    • "its spectrum contains both even and odd harmonics of the fundamental frequency. because it contains all the integer harmonics, it is one of the best waveforms to use for subtractive synthesis of musical sounds, particularly bowed string instruments like violins and cellos, since the slip-stick behavior of the bow drives the strings with a sawtooth-like motion"
    • creates aliasing if frequencies above half the sample rate are included
    • harmonics: amp / 2 - amp / pi sum(k, 1..inf, -1 ** k sin(2 pi k f t) / k)

      • -1 ** k flips the sign for each k and with that flips the sine vertically for each harmonic, so that a ramp is created and the peaks dont accumulate to one large peak
      • subtracting amp / 2, which could be considered to be a line, reverses the output. saw waves can ramp up then go down, or ramp down and go up
      • k f is the frequency of the current harmonic, k times fundamental
  • sine wave

    • single-frequency spectrum, no harmonics
    • doesnt change angular direction abruptly
    • the only periodic waveform that retains its wave shape when added to another wave with that form of the same frequency and arbitrary phase and magnitude
    • one full cycle in 2 * pi radians
    • every sound can theoretically be created by a sum of a possibly infinite number of sines
    • samples can be taken from

      • common sin() function that is based on the taylor series
      • a lookup table where elements are pre-calculated samples. this is a faster way to get sine values. values are pre-calculated for a specific sampling rate and frequency, taking other frequencies might need interpolation or re-calculation
      • less precise sine approximation functions
  • some more shapes

    • pulse

      • like a square wave with two different peak durations
    • rectangle

      • like a square wave where peak duration as well as the minimum and maximum value can vary
      • a bit like an amplitude and frequency modulated square wave
    • trapezoid

      • clipped triangle

      • the clipping would create pops

noise

  • typically created from samples of a random number generator passed through filters to remove frequencies
  • there dont seem to be many other methods to create noise. summing many random sines is possible but computationally intensive

other time/magnitude data post-processing to shape sounds

  • delay: input samples are put out at a later time
  • reverberation: can be created with many short delays that repeat delayed output and change amplitude and frequency with each repetition
  • basic operations

    • multiplication: shapes the amplitude of signals
    • division: shapes the amplitude of signals. dividing a signal by itself flattens the signal
    • subtraction: remove signals from each other
    • addition: adds signals to each other
    • convolution: multiplies frequencies. "under suitable conditions the fourier transform of a convolution of two signals is the pointwise product of their fourier transforms"
  • convergence: two signals and the output shall fall in between. for example: (b - a = 100%); (a + 0.5 * (b - a))

grain processing

a signal split into small chunks that are then processed

  • repetition or removal of pieces can create a time stretching effect that does not change pitch as much as removing or duplicating individual samples would
  • the repetition of larger grains can create interesting rhythmical effects similar to a repeated delay
  • example operations: randomise, repeat, reduce, effects on selected grains, swap. example parameters: grain_size, repetition.
  • the total output of a grain processor can be shorter or longer than the input. this might lead to a growing length difference between the unprocessed and processed signal and require buffering

other

  • additive vs subtractive processes: add vs subtract parts to create a desired result. for example summing sinusodials vs filtering
  • amplitude modulation: in particular, fast changes of the amplitude. related to tremolo
  • frequency modulation: changing frequency based on the values of another signal. this creates new harmonics. "in the context of audio coding, fm synthesis can be considered a "lossy compression method" for additive synthesis". related to vibrato
  • granular synthesis: grain processing
  • transformation of recorded sound vs synthesis from formulas alone

    • getting sound parameters from analysing recorded sounds, while necessarily lossy, gives access to a wide variety of sounds with complex content
    • purely synthesised sounds, while being an exact representation of their original intent and possibly creating what cant be recorded or analysed precisely, require much work to be as complex as real world recorded sounds