Hi,

I'm not really into DSP. I'm interested in this topic, but I'm lacking a
lot of necessary basic knowledge. However, I'm trying to process an
audiosignal and receiving some unwanted sideffects, which I would like to
get rid of. So I would be glad for any hints, which things I need to
consider.

Here's what I'm doing:
First I analyze an audiosignal A with an FFT (and then smooth the spectrum
with a so called "cepstral method"). Doing this with a continuous
audiosignal I receive a number of overlapping spectra (since I'm using a
overlapping Hamming Window performing the FFT).
Then I compare each smoothed spectrum with the following spectrum (of the
next window) by an algorithm (similar to optical flow for motion detection
in video processing) and receive vectors for each frequency bin, telling me
either that a frequency bin "seemed" to have moved up or down in
frequency.
These data of frequency movements I write into a file.
Then I try to use this data from the file to manipulate another soundfile -
an sudiosignal B.

I apply an FFT to audiosignal B and then move the frequency bins according
to the frequency shifts calculated from audiosignal A. E.g. according to my
file frequency bin 5 should move up 3 frequency bins (on a linear scale).
Then I would take the amount in frequency bin 5 and write it into frequency
bin 8 and afterwards set frequency bin 5 to 0.
Afterwards I interpolate the zero-values by simply averaging them with
their neighbour values.

So far so good, but if I continuesly (and repeatedly) apply frequency
shifting data of a long durated signal A to a short signal B, it starts to
produce click sounds and very strong peaks in the beginning and at the end
of each time domain signal of the frequency shifted material.

So, I wonder what could be the major reasons for this. I understand that a
strong amplitude difference in the time domain signal produces click-like
sounds, but how about a spectrum? Do rapid value differences between
neigboured frequency bins have a similar effect? And if, would there be any
kind of smoothing possible to avoid the peaks in the resulting time domain
signal?
Or might this be an issue which is more related because of messing with the
phase?

I know that this whole thing probably sounds like complete nonsense to most
of you, but I really would like it to work and would appreciate any
suggestions, which could help.

thanks a lot in advance,

Peter K.

What is the purpose of this? Tell what you try to accomplish instead of 
telling what you do.

VLV


onep wrote:
> Hi,
> 
> I'm not really into DSP. I'm interested in this topic, but I'm lacking a
> lot of necessary basic knowledge. However, I'm trying to process an
> audiosignal and receiving some unwanted sideffects, which I would like to
> get rid of. So I would be glad for any hints, which things I need to
> consider.
> 
> Here's what I'm doing:
> First I analyze an audiosignal A with an FFT (and then smooth the spectrum
> with a so called "cepstral method"). Doing this with a continuous
> audiosignal I receive a number of overlapping spectra (since I'm using a
> overlapping Hamming Window performing the FFT).
> Then I compare each smoothed spectrum with the following spectrum (of the
> next window) by an algorithm (similar to optical flow for motion detection
> in video processing) and receive vectors for each frequency bin, telling me
> either that a frequency bin "seemed" to have moved up or down in
> frequency.
> These data of frequency movements I write into a file.
> Then I try to use this data from the file to manipulate another soundfile -
> an sudiosignal B.
> 
> I apply an FFT to audiosignal B and then move the frequency bins according
> to the frequency shifts calculated from audiosignal A. E.g. according to my
> file frequency bin 5 should move up 3 frequency bins (on a linear scale).
> Then I would take the amount in frequency bin 5 and write it into frequency
> bin 8 and afterwards set frequency bin 5 to 0.
> Afterwards I interpolate the zero-values by simply averaging them with
> their neighbour values.
> 
> So far so good, but if I continuesly (and repeatedly) apply frequency
> shifting data of a long durated signal A to a short signal B, it starts to
> produce click sounds and very strong peaks in the beginning and at the end
> of each time domain signal of the frequency shifted material.
> 
> So, I wonder what could be the major reasons for this. I understand that a
> strong amplitude difference in the time domain signal produces click-like
> sounds, but how about a spectrum? Do rapid value differences between
> neigboured frequency bins have a similar effect? And if, would there be any
> kind of smoothing possible to avoid the peaks in the resulting time domain
> signal?
> Or might this be an issue which is more related because of messing with the
> phase?
> 
> I know that this whole thing probably sounds like complete nonsense to most
> of you, but I really would like it to work and would appreciate any
> suggestions, which could help.
> 
> thanks a lot in advance,
> 
> Peter K.
> 
> 

On 09/24/2010 08:17 AM, Vladimir Vassilevsky wrote:
>
> What is the purpose of this? Tell what you try to accomplish instead of
> telling what you do.
>
> VLV

Agreed.  You have described a whole bunch of trees in loving detail, but 
you haven't told us where the forest is, yet.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" was written for you.
See details at http://www.wescottdesign.com/actfes/actfes.html

Well, that's probably the reason why I felt like explaining the details. It
might sound strange, but I basically want to do what I have described. The
overall project I'm working on is about exploring an algorithm, which is
the Lukas-Kanade Optical Flow algorithm. Regarding Video this algorithm can
be used to compare the movement of pixel groups over time. So I'm trying to
do sth similar with sound (well, as analog as it could get since video and
audio are obviously quite different things).

This is the purpose - technically, I want to apply repeatedly frequency
shifting to a soundfile. Frequency shifting in which I shift different bins
for different amounts, and this over and over. Maybe I could call it
Morphing without a final destination?

Sorry, if this all sounds confusing, maybe I should mention that I work in
a media art department of a University?


>On 09/24/2010 08:17 AM, Vladimir Vassilevsky wrote:
>>
>> What is the purpose of this? Tell what you try to accomplish instead of
>> telling what you do.
>>
>> VLV
>
>Agreed.  You have described a whole bunch of trees in loving detail, but 
>you haven't told us where the forest is, yet.
>
>-- 
>
>Tim Wescott
>Wescott Design Services
>http://www.wescottdesign.com
>
>Do you need to implement control loops in software?
>"Applied Control Theory for Embedded Systems" was written for you.
>See details at http://www.wescottdesign.com/actfes/actfes.html
>

On Sep 25, 4:44=A0pm, "onep" <P@n_o_s_p_a_m.dimmsumm.com> wrote:

>
> Sorry, if this all sounds confusing, maybe I should mention that I work i=
n
> a media art department of a University?
>

Why didn't you tell them earlier so they could have ignored you more
quickly.


Hardy

@Hardy
well, I admit that I was suprised that I couldn't find any of those
comments for the pure sake of commenting. At least not in the threads I
read so far of this forum... seems I should have checked better. 
thanks

>On Sep 25, 4:44=A0pm, "onep" <P@n_o_s_p_a_m.dimmsumm.com> wrote:
>
>>
>> Sorry, if this all sounds confusing, maybe I should mention that I work
i=
>n
>> a media art department of a University?
>>
>
>Why didn't you tell them earlier so they could have ignored you more
>quickly.
>
>
>Hardy
>

onep wrote:

> So, I wonder what could be the major reasons for this. I understand that a
> strong amplitude difference in the time domain signal produces click-like
> sounds, but how about a spectrum? Do rapid value differences between
> neigboured frequency bins have a similar effect? And if, would there be any
> kind of smoothing possible to avoid the peaks in the resulting time domain
> signal?
> Or might this be an issue which is more related because of messing with the
> phase?

If you manipulate a result of FFT of a piece of a signal, and then 
convert it back to the time domain by means of IFFT, you will get the 
discontinuites at the edges of the block. The way to mitigate this 
effect is overlapping and adding the consequitive windowed FFT frames.

Vladimir Vassilevsky
DSP and Mixed Signal Design Consultant
http://www.abvolt.com

On 09/25/2010 01:39 AM, HardySpicer wrote:
> On Sep 25, 4:44 pm, "onep"<P@n_o_s_p_a_m.dimmsumm.com>  wrote:
>
>>
>> Sorry, if this all sounds confusing, maybe I should mention that I work in
>> a media art department of a University?
>>
>
> Why didn't you tell them earlier so they could have ignored you more
> quickly.

Oooh, snarkity snark.  Or maybe taken that into account and not beat up 
on him for not asking his prof, instead?

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" was written for you.
See details at http://www.wescottdesign.com/actfes/actfes.html

On 09/24/2010 09:44 PM, onep wrote:
(top posting fixed)
>> On 09/24/2010 08:17 AM, Vladimir Vassilevsky wrote:
>>>
>>> What is the purpose of this? Tell what you try to accomplish instead of
>>> telling what you do.
>>>
>>> VLV
>>
>> Agreed.  You have described a whole bunch of trees in loving detail, but
>> you haven't told us where the forest is, yet.
>>
 > Well, that's probably the reason why I felt like explaining the > 
details. It
 > might sound strange, but I basically want to do what I have 
described. The
 > overall project I'm working on is about exploring an algorithm, which is
 > the Lukas-Kanade Optical Flow algorithm. Regarding Video this 
algorithm can
 > be used to compare the movement of pixel groups over time. So I'm 
trying to
 > do sth similar with sound (well, as analog as it could get since 
video and
 > audio are obviously quite different things).
 >
 > This is the purpose - technically, I want to apply repeatedly frequency
 > shifting to a soundfile. Frequency shifting in which I shift 
different bins
 > for different amounts, and this over and over. Maybe I could call it
 > Morphing without a final destination?
 >
 > Sorry, if this all sounds confusing, maybe I should mention that I 
work in
 > a media art department of a University?

I'm not sure that you'd get any results that would be at all related to 
the original.  The Fourier transform makes a heck of a lot of sense 
mathematically, but the relationship between sound and perception is 
considerably more complicated and subtle than just the bins in a Fourier 
transform.

If you're just trying to take a sound file and make a complete mishmash 
of it, while still being able to say "the sounds that you are hearing 
are a computer-processed version of a recording of baby harp seals being 
clubbed to death with precious Stradivarius violins"*, then you may well 
be on the right track.

If you are trying to _do_ something -- like preserve the meaning of a 
spoken or sung text, while completely changing the character either 
randomly or in some directed fashion -- then tell us what you _really_ 
want to do and someone will have a suggestion.

* Engineers and scientists like to criticize artists for using public 
funds to generate absolute crap.  Any time they start doing this in 
conversation, just bring up Cold Fusion or the Tacoma Narrows bridge or 
the number of gallons of oil it takes to make a gallon of biofuel -- 
that'll shut them up.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" was written for you.
See details at http://www.wescottdesign.com/actfes/actfes.html

On Sep 27, 7:18=A0am, Tim Wescott <t...@seemywebsite.com> wrote:

> * Engineers and scientists like to criticize artists for using public
> funds to generate absolute crap. =A0Any time they start doing this in
> conversation, just bring up Cold Fusion or the Tacoma Narrows bridge or
> the number of gallons of oil it takes to make a gallon of biofuel --
> that'll shut them up.

Who are 'they' you want to shut up? The artists? The engineers?

Rune

I like the idea with the clubbed baby harp seals! :)

What I want to do is shifting frequencies gradually in different
directions. Actually, I'm already doing it. But I would like to understand
why I am getting peaks at the beginning and end of each bit of audio signal
I process that way.

I understand the issue with the discontinuities, but I assume it is not the
main problem (even though another one). The problem particular appears when
I'm setting frequency bins to zero. 

So, I would like to know/understand:

1. How do rapid fluctuations in a fft-spectrum relate to the resulting time
domain signal.

2. Would it make sense to apply a smoothing algorithm to the spectrum
before I convert it back?

3. What kind of method would be suitable for smoothing?

4. Why is it so bad to be curious about sth. I didn't learn before?

On Sep 29, 5:43=A0am, "onep" <P@n_o_s_p_a_m.dimmsumm.com> wrote:
> I like the idea with the clubbed baby harp seals! :)
>
> What I want to do is shifting frequencies gradually in different
> directions. Actually, I'm already doing it. But I would like to understan=
d
> why I am getting peaks at the beginning and end of each bit of audio sign=
al
> I process that way.
>
> I understand the issue with the discontinuities, but I assume it is not t=
he
> main problem (even though another one). The problem particular appears wh=
en
> I'm setting frequency bins to zero.
>
> So, I would like to know/understand:
>
> 1. How do rapid fluctuations in a fft-spectrum relate to the resulting ti=
me
> domain signal.
>
> 2. Would it make sense to apply a smoothing algorithm to the spectrum
> before I convert it back?
>
> 3. What kind of method would be suitable for smoothing?
>
> 4. Why is it so bad to be curious about sth. I didn't learn before?

Question 4 first: It's not the curiosity, but the topics. Your
questions are typical for the amateur who have no experience with
technical topics in general, and DSP in particular. DSP professionals
spend a lot of time learning

- Which questions make sense (and by induction, which don't)
- How to phrase a question on a sensible topic for it to make
  sense (which, of course, means that not all questions make
  sense even if the topic does)
- What technical effects influence the answers to the above
  questions

As for your question 1, it relates to basic knowledge about what
a spectrum is and what it tells the analyst. This is stuff that
just has to be learned, if you want to find out what is going on,
or if you want help here. You need to show comp.dsp that you want
to learn, and not just stick with your present misconceptions.

What questions 2 and 3 are concerned, it depends entirely on
what you want to do and why. DSP is not much different from
most other fields in that different tools do different things,
and different tasks call for different things to be done.

If you just want to mess around for curiosity or fun, then use
whatever you like and see what happens. If you have a particular
purpose in mind (and yes, we *can* tell the difference! Most of
us have worked with students and newbies for years or decades)
then don't ask about how to get your preferred tool to do the
job, but how to solve the task at hand.

Rune

>
>Question 4 first: It's not the curiosity, but the topics. Your
>questions are typical for the amateur who have no experience with
>technical topics in general, and DSP in particular. 

thanks and very informative, especially since it was the first thing I
introduced, when posting here ...


>
>- Which questions make sense (and by induction, which don't)
>- How to phrase a question on a sensible topic for it to make
>  sense (which, of course, means that not all questions make
>  sense even if the topic does)
>- What technical effects influence the answers to the above
>  questions

"sensible topic"? maybe you should really find some hobbies, which are not
DSP related.


What's wrong with you guys? I don't have much of a clue, I'm not denying
this, simply because I am not from this field. I also don't have the time
to learn it or study it in such depth. (How is it actually, you go to
Universities for this or you learn it all by yourself and that's why you so
proud about it).
Is it a crime not to have the time to become a Professional, but trying to
learn more anyways?

I mean let's take a synthesizer, well I mean the instrument ... this thing
with some white and black keys on the front and a lot of funny knobs on the
plastic or metal case ....

I don't need to know what is a cutoff frequency, an LFO etc. in order to
use it. It might definitely help, especially if I have a particular aim.
But I assume in many cases it's rather a matter of taste. If not than I
would like to know what kind of music you listen to.

>As for your question 1, it relates to basic knowledge about what
>a spectrum is and what it tells the analyst. This is stuff that
>just has to be learned, if you want to find out what is going on,
>or if you want help here. 

a-ha ... so why should I post a question, if I know the answer already, or
is this the "extremely cleverly phrased questions" section?
And if you think I really should learn something particular, why not send
me some links with related material for DSP dumbasses like me, instead of
analyzing how stupid my questions are.

- because that would helpfull (and by induction, what is not)


> You need to show comp.dsp that you want
>to learn, and not just stick with your present misconceptions.

is it sth. like a gang or so?

>What questions 2 and 3 are concerned, it depends entirely on
>what you want to do and why. 

why?! because I find it interesting. Do I need a driving license for
touching anything related to DSP or what?
Some guys over here have pretty bad websites ... so, should we ban them
from the internet, because they don't have a clue about that?

Whats the purpose of a electric guitar then?! ooops sorry, just realized
that's sth analogue so you probably won't know ...


>If you just want to mess around for curiosity or fun, then use
>whatever you like and see what happens. If you have a particular
>purpose in mind (and yes, we *can* tell the difference! Most of
>us have worked with students and newbies for years or decades)
>then don't ask about how to get your preferred tool to do the
>job, but how to solve the task at hand.


and ... I didn't realize that I had asked for a tool.

On 09/28/2010 08:43 PM, onep wrote:
> I like the idea with the clubbed baby harp seals! :)
>
> What I want to do is shifting frequencies gradually in different
> directions. Actually, I'm already doing it. But I would like to understand
> why I am getting peaks at the beginning and end of each bit of audio signal
> I process that way.
>
> I understand the issue with the discontinuities, but I assume it is not the
> main problem (even though another one). The problem particular appears when
> I'm setting frequency bins to zero.

Setting just one frequency bin to zero has the effect of subtracting out 
a sine wave at just one harmonic of the base period of the FFT (i.e. a 
sine at one multiple of the total FFT time).  If there were "in phase" 
content, then you'd get peaks at the ends and at a bunch of places in 
the middle, too.

> So, I would like to know/understand:
>
> 1. How do rapid fluctuations in a fft-spectrum relate to the resulting time
> domain signal.

In general they relate to long ringing in the time domain.  Try this: 
Make a boxcar in the frequency domain, then take it's inverse.  See what 
it looks like.  Now soften the edges of the boxcar (by making it into a 
raised cosine, or something).  See what it looks like now.

> 2. Would it make sense to apply a smoothing algorithm to the spectrum
> before I convert it back?

Possibly.

> 3. What kind of method would be suitable for smoothing?

That's a good question.  You might try convolution, but I'm not sure 
you'd get the results you wanted.  A better way might be to use a 
rounded profile to make your holes -- like poking your spectrum with a 
thick rounded stick, instead of a needle.

> 4. Why is it so bad to be curious about sth. I didn't learn before?

Well, first: it's the beginning of the school term, when we 
traditionally get flooded by students who slept through their first 
three years and suddenly realize that all that nonsense that they 
cribbed from their frat brothers is actually stuff they need to know to 
do their senior projects and graduate so that they can become 
incompetent managers and inflict themselves on the professionals who 
actually paid attention.

So some of us get cranky.

Why it's _good_ to learn this material is because many questions answer 
themselves if you do, because if you learn it from a well structured 
book (or set thereof) or a class then you'll have the same knowledge 
base as the rest of us, and because if you have that knowledge base then 
when we answer questions you'll understand the answers.

Basically, if you try to learn the basics by asking random questions on 
newsgroups and trying to understand the answers you'll end up with 
inferior knowledge and it'll take an inordinate amount of your time and 
ours.  There are just times when you want to learn the stuff out of a 
book, or let one prof to teach a class of which you are a member.

Perhaps you should audit some of the signal processing courses in your 
university's EE program, and see if you find the subject approachable.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" was written for you.
See details at http://www.wescottdesign.com/actfes/actfes.html

On Sep 29, 10:02=A0am, "onep" <P@n_o_s_p_a_m.dimmsumm.com> wrote:
> >Question 4 first: It's not the curiosity, but the topics. Your
> >questions are typical for the amateur who have no experience with
> >technical topics in general, and DSP in particular.
>
> thanks and very informative, especially since it was the first thing I
> introduced, when posting here ...
>
>
>
> >- Which questions make sense (and by induction, which don't)
> >- How to phrase a question on a sensible topic for it to make
> > =A0sense (which, of course, means that not all questions make
> > =A0sense even if the topic does)
> >- What technical effects influence the answers to the above
> > =A0questions
>
> "sensible topic"? maybe you should really find some hobbies, which are no=
t
> DSP related.

Yes and no. DSP used to be my vocation, one I worked hard to become
good at. By now it's merely a hobby, mostly because I encountered
far too many ought-to-be-professionals who weren't even amateurs.

> What's wrong with you guys? I don't have much of a clue, I'm not denying
> this, simply because I am not from this field. I also don't have the time
> to learn it or study it in such depth. (How is it actually, you go to
> Universities for this or you learn it all by yourself and that's why you =
so
> proud about it).

I am one of those who spent 3 years in undergrad and then 5 years
in grad school with DSP as my main topic. Not because I had nothing
better to do, but because that's what it took to become half-decent
with the topic.

As I said, all you need to do is to show a willingness to learn.
DSP is a lot harder than it seems.

> Is it a crime not to have the time to become a Professional, but trying t=
o
> learn more anyways?

It's a crime not making the effort to learn the basics. Read
one of the intro books. Learn the maths. It's necessary.

> I mean let's take a synthesizer, well I mean the instrument ... this thin=
g
> with some white and black keys on the front and a lot of funny knobs on t=
he
> plastic or metal case ....

> I don't need to know what is a cutoff frequency, an LFO etc. in order to
> use it. It might definitely help, especially if I have a particular aim.
> But I assume in many cases it's rather a matter of taste. If not than I
> would like to know what kind of music you listen to.

It's not necessary to know DSP if you want to use, say, a cell phone
which usees a lot of DSP internally. It *is* necessary to know DSP if
you want to play around and poke with the tools yourself.

> >As for your question 1, it relates to basic knowledge about what
> >a spectrum is and what it tells the analyst. This is stuff that
> >just has to be learned, if you want to find out what is going on,
> >or if you want help here.
>
> a-ha ... so why should I post a question, if I know the answer already, o=
r
> is this the "extremely cleverly phrased questions" section?

No. This is the 'show that you have made an effort to understand
the basics' section.

> And if you think I really should learn something particular, why not send
> me some links with related material for DSP dumbasses like me, instead of
> analyzing how stupid my questions are.

Because you need to learn the generals before you can learn the
particulars: Read the intro books or take the intro classes.

> - because that would helpfull (and by induction, what is not)
>
> > You need to show comp.dsp that you want
> >to learn, and not just stick with your present misconceptions.
>
> is it sth. like a gang or so?

No. It's a bunch of (used to be) DSP professionals.

> >What questions 2 and 3 are concerned, it depends entirely on
> >what you want to do and why.
>
> why?! because I find it interesting. Do I need a driving license for
> touching anything related to DSP or what?

No. But you need to know what you are trying to do. Referring
to your own example with the synth, there is no music merely by
poking the keys that make sounds. One needs to know how to
put the individual sounds together.

> Some guys over here have pretty bad websites ... so, should we ban them
> from the internet, because they don't have a clue about that?

> Whats the purpose of a electric guitar then?! ooops sorry, just realized
> that's sth analogue so you probably won't know ...
>
> >If you just want to mess around for curiosity or fun, then use
> >whatever you like and see what happens. If you have a particular
> >purpose in mind (and yes, we *can* tell the difference! Most of
> >us have worked with students and newbies for years or decades)
> >then don't ask about how to get your preferred tool to do the
> >job, but how to solve the task at hand.
>
> and ... I didn't realize that I had asked for a tool.

I know. As I have said numerous times: Make the effort to learn
the basics. We can tell. The same way you can tell whether it is
a newbie or seasoned pro like, say, Mark Knopfler who handles a
guitar, merely by listening to a couple of seconds worth of music.

Rune

Thank you for the reply,

I have some new amateur questions though.
I am using a simple triangle window with 50% overlapping, in order to avoid
the strong peaks at the beginning and end of each timedomain signal frame
and to reduce the mismatch.
But, there is still a small mismatch and I'm still having similar
mismatches at other places of the generated time domain signal.
They look like this :
http://dimmsumm.com/mismatch01.jpg

If I move a group of (neighboured) frequency bins with their full amounts
(like boxcar function) or if I smooth the amounts I move, with something
like a hamming window function doesn't prevent the mismatches.

Is there a common way to interpolate the local mismatch of two functions? 
(I'm not expecting anybody to explain me such method, I just would need
some keywords, which I could look up to figure out more)

onep wrote:

> I am using a simple triangle window with 50% overlapping, in order to avoid
> the strong peaks at the beginning and end of each timedomain signal frame
> and to reduce the mismatch.
> But, there is still a small mismatch and I'm still having similar
> mismatches at other places of the generated time domain signal.
> They look like this :
> http://dimmsumm.com/mismatch01.jpg
> 
> If I move a group of (neighboured) frequency bins with their full amounts
> (like boxcar function) or if I smooth the amounts I move, with something
> like a hamming window function doesn't prevent the mismatches.
> 
> Is there a common way to interpolate the local mismatch of two functions? 
> (I'm not expecting anybody to explain me such method, I just would need
> some keywords, which I could look up to figure out more)

Use the overlapping higher then 50%. That is, overlap more then two FFT 
frames. Ideally, you would like to do all of the calculations at every 
sample.

Vladimir Vassilevsky
DSP and Mixed Signal Design Consultant
http://www.abvolt.com