2023-04-05

see also analysis and resynthesis and frequency filtering. related to digital signal processing

thought models for thinking about sound

- physically: sound is made of longitudinal vibrations of matter in three dimensional space. humans have two main pathways to receive and analyse changes in air pressure and also discern its source direction to a limited extent. regularity in received signals is of particular significance and can be described by frequency
- time/magnitude: one channel of magnitude samples over time as the representation of a vibration of a directionless sound with the values of any separate contributing events summed. can be transferred to a membrane directly to reproduce sound
- frequency/phase/time/magnitude: imagine a kind of piano roll, that instead of being limited to piano keys and note onsets or durations, holds horizontally the information for finely spaced sine frequencies and circumferential detailed amplitude envelopes of each. this is a type of split representation of time/magnitude data
- sound sources and instruments: sound being the excitation of matter begins with the excitation of matter. air pressed through a small opening might become to oscillate at high frequencies. traditional musical instruments usually have interfaces for excitation of matter to be induced by human motion, parameterised for example by relative striking position or intensity

- compound sounds created by the summing of sinusoid waves
- additive synthesis waveform generator interactive visual tool for demonstrating additive synthesis
- additive synthesis notes with examples for the nord modular synthesizer
- wikipedia: additive synthesis

wave forms other than sines are mostly used for subtractive synthesis

- alternation between two values with the same duration for each
- frequency spectrum contains only odd harmonic frequencies
- its stochastic counterpart is a two-state trajectory

- linear change with abrupt directional change at the extremes
- frequency spectrum contains only odd harmonics

- like a triangle with one rectangular angle
- "its spectrum contains both even and odd harmonics of the fundamental frequency. because it contains all the integer harmonics, it is one of the best waveforms to use for subtractive synthesis of musical sounds, particularly bowed string instruments like violins and cellos, since the slip-stick behavior of the bow drives the strings with a sawtooth-like motion"
- creates aliasing if frequencies above half the sample rate are included

- single-frequency spectrum, no harmonics
- doesnt change angular direction abruptly
- the only periodic waveform that retains its wave shape when added to another wave with that form of the same frequency and arbitrary phase and magnitude
- one full cycle in
`2 * pi`

radians - every sound can theoretically be created by a sum of a possibly infinite number of sines
samples can be taken from

- common sin() function. it is calculated with the taylor series and many optimizations
- a lookup table where elements are pre-calculated samples. this saves the time to evaluate sin() but may lead to cache misses. values are pre-calculated for a specific sampling rate and frequency. if the frequency does not divide the sampling rate without rest, interpolation or re-calculation may be used
- less precise sine approximation functions

some more shapes

- like a square wave with two different peak durations

rectangle

like a square wave where peak duration as well as the minimum and maximum value can vary

a bit like an amplitude and frequency modulated square wave

- timbre created by fast frequency variation
- fm synthesis

- typically created from samples of a random number generator passed through filters to remove frequencies
- there do not seem to be many other methods to create noise. summing many random sines is possible but computationally intensive
- frequency modulation synthesis can create noise, sin(sin(rand))

- delay: input samples are put out at a later time, possibly repeated
reverberation

- can be simulated with many short delays that repeat delayed output and change amplitude and frequency with each repetition
- convolution can also create reverb effects. convincing, but without dynamically changing parameters

basic operations

multiplication: shapes the amplitude of signals

division: shapes the amplitude of signals. dividing a signal by itself flattens the signal

addition: can be used to sum sine waves

convolution: multiplies frequencies. "under suitable conditions the fourier transform of a convolution of two signals is the pointwise product of their fourier transforms"

a signal split into small chunks that are then processed

- repetition or removal of pieces can create a time stretching effect that does not change pitch as much as removing or duplicating individual samples would
- the repetition of larger grains can create interesting rhythmical effects similar to a repeated delay
- example operations: randomise, repeat, reduce, effects on selected grains, swap. example parameters: grain_size, repetition.
- the total output of a grain processor can be shorter or longer than the input. this might lead to a growing length difference between the unprocessed and processed signal and require buffering

- additive vs subtractive processes: add vs subtract parts to create a desired result. for example summing sinusodials vs filtering
- amplitude modulation: in particular, fast changes of the amplitude. related to tremolo
- granular synthesis: grain processing
- waveshaping synthesis: mapping signal values using a table
transformation of recorded sound vs synthesis from formulas alone

- getting sound parameters from analysing recorded sounds, while necessarily lossy, gives access to a wide variety of sounds with complex content
- purely synthesised sounds, while being an exact representation of their original intent and possibly creating what can not be recorded or analysed precisely, require much work to be as complex as real world recorded sounds