2019-06-14

sound synthesis

see also analysis and resynthesis and frequency filtering. related to digital signal processing

models

thought models for thinking about sound

  • physically: sound is made of longitudinal vibrations of matter in three dimensional space. humans have two main pathways to receive and analyse changes in air pressure and also discern its source direction to a limited extent. regularity in received signals is of particular significance and can be described by frequency
  • time/magnitude: one channel of magnitude samples over time as the representation of a vibration of a directionless sound with the values of any separate contributing events summed. can be transferred to a membrane directly to reproduce sound
  • frequency/phase/time/magnitude: imagine a kind of piano roll that, instead of being limited to piano keys and note onsets or durations, holds information for finely spaced sine frequencies and detailed amplitude envelopes of each. this is a type of split representation of time/magnitude data
  • sound sources and instruments: sound being the excitation of matter begins with the excitation of matter. air pressed through a small opening might become to oscillate at high frequencies. traditional musical instruments usually have interfaces for excitation of matter to be induced by human motion, parameterised for example by relative striking position or intensity

non-noise

simple periodic wave forms

  • square: amplitude alternates between only two values, with the same duration at each. spectrum contains only odd harmonic frequencies. the amplitudes of the harmonics can be calculated with (2 / (pi * harmonic_odd_n)). its stochastic counterpart is a two-state trajectory
  • sawtooth: triangle with one rectangular angle. "its spectrum contains both even and odd harmonics of the fundamental frequency. because it contains all the integer harmonics, it is one of the best waveforms to use for subtractive synthesis of musical sounds, particularly bowed string instruments like violins and cellos, since the slip-stick behavior of the bow drives the strings with a sawtooth-like motion"
  • triangle: linear change with fast change of direction. spectrum contains only odd harmonics with amplitudes maybe (2 / (pi * (harmonic_odd_n ** 2))) (verification needed)
  • rectangle: like a square wave but without the requirement of the same duration at every value. minimum and maximum values could also vary
  • trapezoid: clipped triangle
  • sine wave

    • single-frequency spectrum, no harmonics

    • doesnt change angular direction abruptly

    • the only periodic waveform that retains its wave shape when added to another wave with that form of the same frequency and arbitrary phase and magnitude

    • one full cycle in (2 * pi) radians

    • every sound can theoretically be created by a sum of a possibly infinite number of sines

    • samples can be taken from

      • common sin() function that is based on the taylor series

      • a lookup table where elements are pre-calculated samples. this is a faster way to get sine values. values are pre-calculated for a specific sampling rate and frequency, taking other frequencies might need interpolation or re-calculation

      • less precise sine approximation functions

noise

  • typically created from samples of a random number generator passed through filters to remove frequencies
  • there dont seem to be many other methods to create noise. summing many random sines is possible but computationally intensive

other time/magnitude data post-processing to shape sounds

  • delay: input samples are put out at a later time
  • reverberation: can be created with many short delays that repeat delayed output and change amplitude and frequency with each repetition
  • basic operations

    • multiplication: shapes the amplitude of signals
    • division: shapes the amplitude of signals. dividing a signal by itself flattens the signal
    • subtraction: remove signals from each other
    • addition: adds signals to each other
    • convolution: multiplies frequencies. "under suitable conditions the fourier transform of a convolution of two signals is the pointwise product of their fourier transforms"
  • convergence: two signals and the output shall fall in between. for example: (b - a = 100%); (a + 0.5 * (b - a))

grain processing

a signal split into small chunks that are then processed

  • repetition or removal of pieces can create a time stretching effect that does not change pitch as much as removing or duplicating individual samples would
  • the repetition of larger grains can create interesting rhythmical effects similar to a repeated delay
  • example operations: randomise, repeat, reduce, effects on selected grains, swap. example parameters: grain_size, repetition.
  • the total output of a grain processor can be shorter or longer than the input. this might lead to a growing length difference between the unprocessed and processed signal and require buffering

other

  • additive vs subtractive processes: add vs subtract parts to create a desired result. for example summing sinusodials vs filtering
  • amplitude modulation: in particular, fast changes of the amplitude. related to tremolo
  • frequency modulation: changing frequency based on the values of another signal. this creates new harmonics. "in the context of audio coding, fm synthesis can be considered a "lossy compression method" for additive synthesis". related to vibrato
  • granular synthesis: grain processing. additive synthesis: summing sines to create sounds
  • transformation of recorded sound vs synthesis from formulas alone

    • getting sound parameters from analysing recorded sounds, while necessarily lossy, gives access to a wide variety of sounds with complex content
    • purely synthesised sounds, while being an exact representation of their original intent and possibly creating what cant be recorded or analysed precisely, require much work to be as complex as real world recorded sounds