Introduction to Speech Technology (Holmes & Holmes, 2001)

December 04, 2022

Ch 4. Digital coding of speech

3 things to consider in speech coding: data rate, speech quality, and algorithm complexity
Human cognitive processes cannot take account of an information rate in excess of a few tens of bis per second, thus implying a ratio of information transmitted to information used of between 1,000 and 10,000
2 properties of speech communication: the restricted capacity of the human auditory system and the physiology of the speaking mechanism (based on the fact that the signal is known to be produced by a human talker)
3 coding methods: simple waveform coders, analysis/synthesis systems, and intermediate systems

copy the actual shape of the waveform produced by the microphone and its associated analogue circuits
consist of a band limiting filter, a sampler, and a device for coding the samples
types of simple waveform coders
1. Pulse code modulation (PCM)i) used for feeding analogue signals into computers or other digital equipment for subsequent processingii) not normally used due to the high required digit rateiii) does not exploit the above two properties (i.e., speech production and/or auditory perception), except for the limited bandwidth
2. Deltamodulation: uses its transmitted digital codes to generate a local copy of the input waveform and chooses successive digital codes

analyze the speech signal in terms of parameters
the output does not need to resemble the the original waveform in appearance, but should be perceptually similar
based on a model of speech production
the data are coded into frames representing speech spectra measured at 10-30 ms
types of vocoders
1. channel vocoders i) principle: the spectrum is represented by the response of a bank of contiguous variable-gain bandpass filters, and the control signals for the channels are derived by measuring the short-term-average power from a similar set of filters fed with the input speech signal in the transmitter, ii) limitation: need a large number of channels, iii) performance: 15-20 channels reasonable for communication purposes; operating at data rates of around 2,400 bits/s or lower
2. sinusoidal coders i) representing the short-term spectrum of a speech signal as a sum of sinusoids specified in terms of frequency, amplitude, and phase (called sinusoidal transform coding (STC), ii) voiced speech and unvoiced speech represented by harmonically related sinusoids for the former and with random phases for the latter, iii) Multi-band excitation (MBE) coding: voiced as a combination of the relevant set of harmonic sinusoids; unvoiced using a frequency-domain method with regard to a whitenoise excitation signaliv) 2,000-4,000 bits/s range
3. linear predictive coding (LPC) vocoders
4. formant vocoders

early methods (e.g., Hyde (1972): rule-based approaches, which was not very successful due to co-articulation and difficulty of phone identification
pattern-matching techniques: one way is to store example acoustic patterns (called templates) for all the words

Filter-bank analysis: describes the speech as a sequence of feature vectors, which can be compared with stored templates for all the words in the vocabulary using a suitable distance metric
Normalization
1. adding a small consonant to the measured level before taking logarithms
2. the square of the Euclidean distance in the multi-dimensional space
Dynamic time warping (DTW)
- can deal with sequences of connected words ( ⇒ can solve the end-point detection problem)
- by matching one word on to another in a way which applies the optimum non-linear timescale distortion to achieve the best match at all points
- Score pruning: not allowing paths from relatively badly scoring points to propagate further in the DTW calculation

Speech recognition: store the representation and choose the best matching
3 broad kinds of synthesis
1. Parametric: manipulate the acoustic?
2. Physical (Frankenstein):
3. Concatenative (cheating): using pre-recorded units
The synthesis stage
1. a decoding process: the waveform of the acoustical units must be reconstructed from their coded version
2. a concatenation process: the sequence of acoustical units must be concatenated after an appropriate modification of their intrinsic prosody
Types of synthesis
Concatenative synthesis: synthesize the sounds by concatenating such elementary speech units as diphones and demi-syllables→ Pitch adjustment is a problem
Some pitch adjustment techniques
1. Linear Predictive Coding (LPC): an encoding technique. Do multiple regression on the wave samples to predict any particular sample from the n samples that precede it.Example) w0, w1, w2, w3, w4, w5, w6, w7, w8, w9 (10-sample windows)c1w0 + c2w1 + c3w2 = w3c1w1 + c2w2 + c3w3 = w4...c1w6 + c2w7 + c3w8 = 29→ calculate c1, c2, and c3 by the autoregression→ can be used to reconstruct waves
2. Pitch Synchronous Overlap and Add (PSOLA): a digital signal processing technique. The speech waveform is divided into small overlapping segments, and then they are moved to further apart or closer together, in order to change pitch.

Length
Filter banks: take a wave, break it into time windows, make a spectrum of each window, and then quantize those spectra into a vector of machines→ But how to deal with the data with the different duration? Dynamic time warping (finding an optimal path between two given (time-dependent) sequences)

Things to consider: i) naturalness, ii) computational efficiency, and iii) results (performance assessment)
with regard to filter banks: i) overlapping time windows, ii) amplitude scaling, and iii) frequency scaling→ about iii) frequency scaling:
1. Mels: the idea is that low-frequency differences are precisely than high-frequency differences to humans, so scaling is done accordingly)
2. Cepstrum: a way of frequency decomposition for a time window—creating a spectrum via Fourier analysis, log-transform it, and do the Fourier again
3. MEL frequency cepstral coefficients (MFCC)

A generative model that models sequential data stochastically
Markov model: represents sequence that consists of states as state transition probability matrix
Markov assumption: the observation at the time t is dependent on the latest r observations
The basic structure: a finite number of states, a designated start state, a finite alphabet, a finite number of arcs, a probability associated with each arc, and for each state, a probability associated with each symbol
It is supposed that the current state is hidden and draws out it based on the shown information. The direct cause that results in it is unobservable; in other words, only what resulted from the state(s) can be seen. Keep in mind that what are hidden are not parameters, but continuous states the model goes through.
P(W|A) = P(A|W)P(W)/P(A), A ⇒ O, W ⇒ lambda, P(A|W) ⇒ P(O|lambda)
Forward and backward procedures: to calculate the probability given the sequence of the status and the model (lambda)
The Viterbi algorithm: to find the optimal status sequence to explain the data most well
Baum-Welch re-estimation procedure: to optimize the lambda (parameters of the model) for maximizing P(O|lambda)

Finite state transducers (FST): a finite state automation (FSA), which produces output as well as reading input, meaning it is useful for parsing while a bare FSA can only be used for recognizing (i.e., pattern matching)

Categories: Speech Technology, Book Summary