Hi DSP experts,
I want to do spectral analysis of a vector. I want to divide the vector
into equal length block, multiply each block to a window, then do FFT on
each block, then at each frequency, add the magnitude from each block’s
FFT magnutide spectrum, divide by number of blocks, to obtain the averaged
magnitude spectrum.

I have two questions:
1. Can the blocks overlap? 
2. How can I get the averaged power spectrum? Get sum of the square of
magnitude at each frequency then divide by number of block? 

Thank you for your input.

On Oct 5, 5:39=A0pm, "shinchan75034"
<shinchan75034@n_o_s_p_a_m.gmail.com> wrote:
> Hi DSP experts,
> I want to do spectral analysis of a vector. I want to divide the vector
> into equal length block, multiply each block to a window, then do FFT on
> each block, then at each frequency, add the magnitude from each block=92s
> FFT magnutide spectrum, divide by number of blocks, to obtain the average=
d
> magnitude spectrum.
>
> I have two questions:
> 1. Can the blocks overlap?
> 2. How can I get the averaged power spectrum? Get sum of the square of
> magnitude at each frequency then divide by number of block?
>
> Thank you for your input.

Yes and yes.

http://openlibrary.org/books/OL2736151M/Statistical_spectral_analysis

On Oct 5, 2:39=A0pm, "shinchan75034"
<shinchan75034@n_o_s_p_a_m.gmail.com> wrote:
> Hi DSP experts,
> I want to do spectral analysis of a vector. I want to divide the vector
> into equal length block, multiply each block to a window, then do FFT on
> each block, then at each frequency, add the magnitude from each block=92s
> FFT magnutide spectrum, divide by number of blocks, to obtain the average=
d
> magnitude spectrum.
>
> I have two questions:
> 1. Can the blocks overlap?
> 2. How can I get the averaged power spectrum? Get sum of the square of
> magnitude at each frequency then divide by number of block?
>
> Thank you for your input.

Rick Lyons recently reminded me of a paper on flattop windows that
begins with a nice explanation of the issues you have asked about.
Look at pages 5-21 of the paper at:

http://www.rssd.esa.int/SP/LISAPATHFINDER/docs/Data_Analysis/GH_FFT.pdf

The rest of the paper may also be useful to you.

Dale B. Dalrymple
There is a paper you can

On Oct 5, 11:05=A0pm, dbd <d...@ieee.org> wrote:
> On Oct 5, 2:39=A0pm, "shinchan75034"
>
> <shinchan75034@n_o_s_p_a_m.gmail.com> wrote:
> > Hi DSP experts,
> > I want to do spectral analysis of a vector. I want to divide the vector
> > into equal length block, multiply each block to a window, then do FFT o=
n
> > each block, then at each frequency, add the magnitude from each block=
=92s
> > FFT magnutide spectrum, divide by number of blocks, to obtain the avera=
ged
> > magnitude spectrum.
>
> > I have two questions:
> > 1. Can the blocks overlap?
> > 2. How can I get the averaged power spectrum? Get sum of the square of
> > magnitude at each frequency then divide by number of block?
>
> > Thank you for your input.
>
> Rick Lyons recently reminded me of a paper on flattop windows that
> begins with a nice explanation of the issues you have asked about.
> Look at pages 5-21 of the paper at:
>
> http://www.rssd.esa.int/SP/LISAPATHFINDER/docs/Data_Analysis/GH_FFT.pdf
>
> The rest of the paper may also be useful to you.
>
> Dale B. Dalrymple
> There is a paper you can

Wow Dale,

For the life of me I can't figure out why you would recommend a
"measurement" flat-top window for the OP's application. A
"measurement" flat-top produces  a lot of smearing in the spectrum,
and has the property that it gets rid of the scalloping loss in the
measurement of an isolated sinusoid/sinusoidal component, where the
smearing doesn't efect the measurement value, but certainly effects
the computed spectrum (see page 38 of your recommended article).
Doesn't sound like the OP wants that.

Why did you pick a "measurement" flat-top window to suggest?

Where you thinking of a "time doman" flat-top window (quite
different)?

Maybe you are just poking fun at the OP?

Dirk

On Oct 5, 11:05=A0pm, dbd <d...@ieee.org> wrote:
> On Oct 5, 2:39=A0pm, "shinchan75034"
>
> <shinchan75034@n_o_s_p_a_m.gmail.com> wrote:
> > Hi DSP experts,
> > I want to do spectral analysis of a vector. I want to divide the vector
> > into equal length block, multiply each block to a window, then do FFT o=
n
> > each block, then at each frequency, add the magnitude from each block=
=92s
> > FFT magnutide spectrum, divide by number of blocks, to obtain the avera=
ged
> > magnitude spectrum.
>
> > I have two questions:
> > 1. Can the blocks overlap?
> > 2. How can I get the averaged power spectrum? Get sum of the square of
> > magnitude at each frequency then divide by number of block?
>
> > Thank you for your input.
>
> Rick Lyons recently reminded me of a paper on flattop windows that
> begins with a nice explanation of the issues you have asked about.
> Look at pages 5-21 of the paper at:
>
> http://www.rssd.esa.int/SP/LISAPATHFINDER/docs/Data_Analysis/GH_FFT.pdf
>
> The rest of the paper may also be useful to you.
>
> Dale B. Dalrymple
> There is a paper you can

Wow Dale,

For the life of me I can't figure out why you would recommend a
"measurement" flat-top window for the OP's application. A
"measurement" flat-top window produces a lot of smearing in the
computed spectrum.  It has the property that it gets rid of the
scalloping loss in the measurement of an isolated sinusoid/sinusoidal
component, where the smearing doesn't effect the measurement value,
but certainly effects the computed spectrum (see page 38 of your
recommended article). Doesn't sound like the OP wants that.

Why did you suggest a "measurement" flat-top window?

Were you thinking of a "time domain" flat-top window (quite
different)?

Were you are just poking fun at the OP?

Dirk

On Oct 5, 11:05=A0pm, dbd <d...@ieee.org> wrote:
> On Oct 5, 2:39=A0pm, "shinchan75034"
>
> <shinchan75034@n_o_s_p_a_m.gmail.com> wrote:
> > Hi DSP experts,
> > I want to do spectral analysis of a vector. I want to divide the vector
> > into equal length block, multiply each block to a window, then do FFT o=
n
> > each block, then at each frequency, add the magnitude from each block=
=92s
> > FFT magnutide spectrum, divide by number of blocks, to obtain the avera=
ged
> > magnitude spectrum.
>
> > I have two questions:
> > 1. Can the blocks overlap?
> > 2. How can I get the averaged power spectrum? Get sum of the square of
> > magnitude at each frequency then divide by number of block?
>
> > Thank you for your input.
>
> Rick Lyons recently reminded me of a paper on flattop windows that
> begins with a nice explanation of the issues you have asked about.
> Look at pages 5-21 of the paper at:
>
> http://www.rssd.esa.int/SP/LISAPATHFINDER/docs/Data_Analysis/GH_FFT.pdf
>
> The rest of the paper may also be useful to you.
>
> Dale B. Dalrymple
> There is a paper you can

Dale,

You might want to steer the OP away from "measurement" flat-top
windows for the OP's application. I don't think that part of the rest
of the paper will be useful to the OP.

Dirk

On Oct 6, 11:03=A0am, Dirk Bell <bellda2...@cox.net> wrote:
> On Oct 5, 11:05=A0pm, dbd <d...@ieee.org> wrote:
> ...
> > The rest of the paper may also be useful to you.
>
> > Dale B. Dalrymple
>
> Dale,
>
> You might want to steer the OP away from "measurement" flat-top
> windows for the OP's application. I don't think that part of the rest
> of the paper will be useful to the OP.
>
> Dirk

I might want to know more about the OP's signals, noise and
applications before jumping to conclusions.

Dale B. Dalrymple

On Tue, 05 Oct 2010 16:39:33 -0500, "shinchan75034"
<shinchan75034@n_o_s_p_a_m.gmail.com> wrote:

>Hi DSP experts,
>I want to do spectral analysis of a vector. I want to divide the vector
>into equal length block, multiply each block to a window, then do FFT on
>each block, then at each frequency, add the magnitude from each block’s
>FFT magnutide spectrum, divide by number of blocks, to obtain the averaged
>magnitude spectrum.

Hi shinchan,

  When you perform an N-point FFT on real-world signals
(contaminated with some amount of noise) 
and measure a single FFT bin's spectral magnitude 
you'll get some number, say M1.  If you repeat that 
N-point FFT on a new block of N time samples and 
measure the same bin's spectral magnitude 
you'll get a new number, say M2.  If you repeat 
this ten times you'll have ten magnitudes, M1, M2, 
M3,..., M9, and M10.

Those values in the 10-sample M sequence will all be 
different.  That is, the 10-sample M sequence will have 
some non-zero variance.  (That's the inherent nature of 
FFTs.  And by the way, increasing the FFT size does *NOT* 
reduce that variance.)  So then you'll wonder, 
"Sheece.  What in the heck is the "true" magnitude value 
of that single FFT bin?"  What people do is average those 
ten M values to obtain a single "averaged magnitude"
value that is statistically closer to the "true" 
FFT bin magnitude. 

>I have two questions:
>1. Can the blocks overlap?

Yes.  In fact, if you're sure that you must 
window your time samples before performing 
FFTs, then overlapping the individual blocks 
time samples is a good thing to do.

If you don't overlap the blocks of time samples, 
then the time samples at the beginning and end 
of each block are attenuated, due to multiplication 
by the window, and those time samples 
don't have much effect on your FFT results.
You don't want that to happen.

Sorry that I don't have a reference for you but,
somewhere I read the writing of DSP guru fred harris 
and he said that a time overlap of up to 50% 
yields acceptable FFT results in reducing the 
variance when averaging multiple FFTs. 

Dale Dalrymple gave you a reference that discussed 
overlapped spectrum analysis.  Another place to read about 
this process is under the heading of "Welch's Method" 
at: 

http://www.mathworks.com/help/toolbox/signal/f12-6587.html  

>2. How can I get the averaged power spectrum? Get sum of the square of
>magnitude at each frequency then divide by number of block? 

If I understand your question, the answer is yes.
As far as I know, averaging spectral magnitudes 
or averaging spectral powers (magnitudes squared) 
will both give you the variance reduction that you 
seek.

Good Luck,
[-Rick-]

On Wed, 6 Oct 2010 10:49:37 -0700 (PDT), Dirk Bell
<bellda2005@cox.net> wrote:

   [Snipped by Lyons]
>>
>> Rick Lyons recently reminded me of a paper on flattop windows that
>> begins with a nice explanation of the issues you have asked about.
>> Look at pages 5-21 of the paper at:
>>
>> http://www.rssd.esa.int/SP/LISAPATHFINDER/docs/Data_Analysis/GH_FFT.pdf
>>
>> The rest of the paper may also be useful to you.
>>
>> Dale B. Dalrymple
>> There is a paper you can
>
>Wow Dale,
>
>For the life of me I can't figure out why you would recommend a
>"measurement" flat-top window for the OP's application. A
>"measurement" flat-top produces  a lot of smearing in the spectrum,
>and has the property that it gets rid of the scalloping loss in the
>measurement of an isolated sinusoid/sinusoidal component, where the
>smearing doesn't efect the measurement value, but certainly effects
>the computed spectrum (see page 38 of your recommended article).
>Doesn't sound like the OP wants that.
>
>Why did you pick a "measurement" flat-top window to suggest?
>
>Where you thinking of a "time doman" flat-top window (quite
>different)?
>
>Maybe you are just poking fun at the OP?
>
>Dirk

Hi Dirk,
  I don't think Dale was recommending that the OP 
used flat-top windowing.  Rather, I think Dale 
was merely giving an easily obtained refeence that 
discussed "overlapped-in-time" spectrum analysis.

Another place to read about "overlapped-in-time" 
spectrum analysis is under the heading of 
"Welch's Method" at: 

http://www.mathworks.com/help/toolbox/signal/f12-6587.html  

See Ya',
[-Rick-]

On Oct 9, 12:20=A0pm, Rick Lyons <R.Lyons@_BOGUS_ieee.org> wrote:
....
>
> Sorry that I don't have a reference for you but,
> somewhere I read the writing of DSP guru fred harris
> and he said that a time overlap of up to 50%
> yields acceptable FFT results in reducing the
> variance when averaging multiple FFTs.
>

Try section 5.2 "How much overlap" in:

http://www.signumconcepts.com/download/paper033.pdf

WINDOWS: Finite Aperture Effects and Applications in Signal Processing
by fred j. harris

Dale B. Dalrymple

On Oct 8, 11:03=A0am, dbd <d...@ieee.org> wrote:
> On Oct 6, 11:03=A0am, Dirk Bell <bellda2...@cox.net> wrote:
>
> > On Oct 5, 11:05=A0pm, dbd <d...@ieee.org> wrote:
> > ...
> > > The rest of the paper may also be useful to you.
>
> > > Dale B. Dalrymple
>
> > Dale,
>
> > You might want to steer the OP away from "measurement" flat-top
> > windows for the OP's application. I don't think that part of the rest
> > of the paper will be useful to the OP.
>
> > Dirk
>
> I might want to know more about the OP's signals, noise and
> applications before jumping to conclusions.
>
> Dale B. Dalrymple

Dale,

The OP said that he/she wanted to find the spectrum of a signal, not
the magnitude of a test tone (a single one or several tones further
apart than the flat-top window's spectral width).  Implying that a
flat-top window, which smears the spectrum, but is good for accurate
tone magnitude measurement, would be useful for spectral estimation is
certainly jumping to conclusions that contradict the OP's original
problem statement. So I agree with you, you don't want to do that.

Dirk

On Oct 9, 3:24=A0pm, Rick Lyons <R.Lyons@_BOGUS_ieee.org> wrote:
> On Wed, 6 Oct 2010 10:49:37 -0700 (PDT), Dirk Bell
>
> <bellda2...@cox.net> wrote:
>
> =A0 =A0[Snipped by Lyons]
>
>
>
>
>
> >> Rick Lyons recently reminded me of a paper on flattop windows that
> >> begins with a nice explanation of the issues you have asked about.
> >> Look at pages 5-21 of the paper at:
>
> >>http://www.rssd.esa.int/SP/LISAPATHFINDER/docs/Data_Analysis/GH_FFT.pdf
>
> >> The rest of the paper may also be useful to you.
>
> >> Dale B. Dalrymple
> >> There is a paper you can
>
> >Wow Dale,
>
> >For the life of me I can't figure out why you would recommend a
> >"measurement" flat-top window for the OP's application. A
> >"measurement" flat-top produces =A0a lot of smearing in the spectrum,
> >and has the property that it gets rid of the scalloping loss in the
> >measurement of an isolated sinusoid/sinusoidal component, where the
> >smearing doesn't efect the measurement value, but certainly effects
> >the computed spectrum (see page 38 of your recommended article).
> >Doesn't sound like the OP wants that.
>
> >Why did you pick a "measurement" flat-top window to suggest?
>
> >Where you thinking of a "time doman" flat-top window (quite
> >different)?
>
> >Maybe you are just poking fun at the OP?
>
> >Dirk
>
> Hi Dirk,
> =A0 I don't think Dale was recommending that the OP
> used flat-top windowing. =A0Rather, I think Dale
> was merely giving an easily obtained refeence that
> discussed "overlapped-in-time" spectrum analysis.
>
> Another place to read about "overlapped-in-time"
> spectrum analysis is under the heading of
> "Welch's Method" at:
>
> http://www.mathworks.com/help/toolbox/signal/f12-6587.html=A0
>
> See Ya',
> [-Rick-]

Hi Rick,

I didn't say Dale was stating that a flat-top window was appropriate,
but with his description of the paper as being on flat-top windows
(which are not appropriate to the OP's problem) and the large amount
time spent in the article on flat-top windows, I think Dale should
have steered the OP away from the irrelevant part and towards the
relevant part.  Since the OP is asking such a question, I don't think
it helpful to let him/her just sort it out. If Dale didn't want to
take the time to do this, he could have picked a far better reference.

Dirk

On Oct 9, 3:20=A0pm, Dirk Bell <bellda2...@cox.net> wrote:
>...
> Dale,
>
> The OP said that he/she wanted to find the spectrum of a signal, not
> the magnitude of a test tone (a single one or several tones further
> apart than the flat-top window's spectral width). =A0Implying that a
> flat-top window, which smears the spectrum, but is good for accurate
> tone magnitude measurement, would be useful for spectral estimation is
> certainly jumping to conclusions that contradict the OP's original
> problem statement. So I agree with you, you don't want to do that.
>
> Dirk

I stated the section I thought useful for the immediate question. As
to the rest of the reference, most of it deals with non-flattop
windows. As to the OP's future needs and applications I lack your
crystal ball, so I didn't claim that all of it will be useful. The
reference is a good review of the use of windows of a variety of
forms, whatever the content of it's title.

I think that if you were interested in generating light for the OP
instead of heat for me you would have explained your concern in your
first post and supplied a better reference, if you think there is one
as freely available. You are still free to do that now.

Dale B. Dalrymple

On Oct 9, 7:54=A0pm, dbd <d...@ieee.org> wrote:
> On Oct 9, 3:20=A0pm, Dirk Bell <bellda2...@cox.net> wrote:
>
> >...
> > Dale,
>
> > The OP said that he/she wanted to find the spectrum of a signal, not
> > the magnitude of a test tone (a single one or several tones further
> > apart than the flat-top window's spectral width). =A0Implying that a
> > flat-top window, which smears the spectrum, but is good for accurate
> > tone magnitude measurement, would be useful for spectral estimation is
> > certainly jumping to conclusions that contradict the OP's original
> > problem statement. So I agree with you, you don't want to do that.
>
> > Dirk
>
> I stated the section I thought useful for the immediate question. As
> to the rest of the reference, most of it deals with non-flattop
> windows. As to the OP's future needs and applications I lack your
> crystal ball, so I didn't claim that all of it will be useful. The
> reference is a good review of the use of windows of a variety of
> forms, whatever the content of it's title.
>
> I think that if you were interested in generating light for the OP
> instead of heat for me you would have explained your concern in your
> first post and supplied a better reference, if you think there is one
> as freely available. You are still free to do that now.
>
> Dale B. Dalrymple

I think you are overly sensitive Dale. What possible real problem
could you have had with my post? You called it a flat-top article and
now you say it has little about that subject in it.  I think calling
it that might make the OP think that is something he/she needs to
address their stated problem. It isn't. As for the future needs of the
OP, apparently your crystal balls can anticipate that, but I was just
responding to their stated present need. No crystal ball needed to see
the present.

Actually your reference explains why flat-top windows would be bad if
the OP reads and understands the article. And information to largely
answer the questions is contained in the article. I was suggesting
that since it has a mix of OP usable and a significant amount of OP
unusable information, you might help him/her sort it out better since
you pointed the OP to it. This is sounding a little repetative, but
some people need that.

Dirk

On Oct 9, 10:14=A0pm, Dirk Bell <bellda2...@cox.net> wrote:
> On Oct 9, 7:54=A0pm, dbd <d...@ieee.org> wrote:
>
>
>
> > On Oct 9, 3:20=A0pm, Dirk Bell <bellda2...@cox.net> wrote:
>
> > >...
> > > Dale,
>
> > > The OP said that he/she wanted to find the spectrum of a signal, not
> > > the magnitude of a test tone (a single one or several tones further
> > > apart than the flat-top window's spectral width). =A0Implying that a
> > > flat-top window, which smears the spectrum, but is good for accurate
> > > tone magnitude measurement, would be useful for spectral estimation i=
s
> > > certainly jumping to conclusions that contradict the OP's original
> > > problem statement. So I agree with you, you don't want to do that.
>
> > > Dirk
>
> > I stated the section I thought useful for the immediate question. As
> > to the rest of the reference, most of it deals with non-flattop
> > windows. As to the OP's future needs and applications I lack your
> > crystal ball, so I didn't claim that all of it will be useful. The
> > reference is a good review of the use of windows of a variety of
> > forms, whatever the content of it's title.
>
> > I think that if you were interested in generating light for the OP
> > instead of heat for me you would have explained your concern in your
> > first post and supplied a better reference, if you think there is one
> > as freely available. You are still free to do that now.
>
> > Dale B. Dalrymple
>
> I think you are overly sensitive Dale. What possible real problem
> could you have had with my post? You called it a flat-top article and
> now you say it has little about that subject in it. =A0I think calling
> it that might make the OP think that is something he/she needs to
> address their stated problem. It isn't. As for the future needs of the
> OP, apparently your crystal balls can anticipate that, but I was just
> responding to their stated present need. No crystal ball needed to see
> the present.
>
> Actually your reference explains why flat-top windows would be bad if
> the OP reads and understands the article. And information to largely
> answer the questions is contained in the article. I was suggesting
> that since it has a mix of OP usable and a significant amount of OP
> unusable information, you might help him/her sort it out better since
> you pointed the OP to it. This is sounding a little repetative, but
> some people need that.
>
> Dirk

You still haven't addressed anything to the OP in this thread. You
continue to misrepresent what I have posted.No light yet, just more
trolling.

Dale B. Dalrymple

On Oct 10, 1:41=A0am, dbd <d...@ieee.org> wrote:
> On Oct 9, 10:14=A0pm, Dirk Bell <bellda2...@cox.net> wrote:
>
>
>
>
>
> > On Oct 9, 7:54=A0pm, dbd <d...@ieee.org> wrote:
>
> > > On Oct 9, 3:20=A0pm, Dirk Bell <bellda2...@cox.net> wrote:
>
> > > >...
> > > > Dale,
>
> > > > The OP said that he/she wanted to find the spectrum of a signal, no=
t
> > > > the magnitude of a test tone (a single one or several tones further
> > > > apart than the flat-top window's spectral width). =A0Implying that =
a
> > > > flat-top window, which smears the spectrum, but is good for accurat=
e
> > > > tone magnitude measurement, would be useful for spectral estimation=
 is
> > > > certainly jumping to conclusions that contradict the OP's original
> > > > problem statement. So I agree with you, you don't want to do that.
>
> > > > Dirk
>
> > > I stated the section I thought useful for the immediate question. As
> > > to the rest of the reference, most of it deals with non-flattop
> > > windows. As to the OP's future needs and applications I lack your
> > > crystal ball, so I didn't claim that all of it will be useful. The
> > > reference is a good review of the use of windows of a variety of
> > > forms, whatever the content of it's title.
>
> > > I think that if you were interested in generating light for the OP
> > > instead of heat for me you would have explained your concern in your
> > > first post and supplied a better reference, if you think there is one
> > > as freely available. You are still free to do that now.
>
> > > Dale B. Dalrymple
>
> > I think you are overly sensitive Dale. What possible real problem
> > could you have had with my post? You called it a flat-top article and
> > now you say it has little about that subject in it. =A0I think calling
> > it that might make the OP think that is something he/she needs to
> > address their stated problem. It isn't. As for the future needs of the
> > OP, apparently your crystal balls can anticipate that, but I was just
> > responding to their stated present need. No crystal ball needed to see
> > the present.
>
> > Actually your reference explains why flat-top windows would be bad if
> > the OP reads and understands the article. And information to largely
> > answer the questions is contained in the article. I was suggesting
> > that since it has a mix of OP usable and a significant amount of OP
> > unusable information, you might help him/her sort it out better since
> > you pointed the OP to it. This is sounding a little repetative, but
> > some people need that.
>
> > Dirk
>
> You still haven't addressed anything to the OP in this thread. You
> continue to misrepresent what I have posted.No light yet, just more
> trolling.
>
> Dale B. Dalrymple- Hide quoted text -
>
> - Show quoted text -

I just made a suggestion that you point out something I thought was
misleading in your response to the OP.  My initial response to you was
informative to the OP wether it was addressed to you or not.  I have
posted enough here over the years that I don't think I qualify as a
troll.  You are proving to be boring and childish and a waste of time
and bandwidth.  You may have the last word.

Dirk

On Oct 10, 10:30=A0am, Dirk Bell <bellda2...@cox.net> wrote:
....
>
> I just made a suggestion that you point out something I thought was
> misleading in your response to the OP. =A0My initial response to you was
> informative to the OP wether it was addressed to you or not. =A0I have
> posted enough here over the years that I don't think I qualify as a
> troll. =A0You are proving to be boring and childish and a waste of time
> and bandwidth. =A0You may have the last word.
>
> Dirk

Anyone interested in evaluating current behavior can do so by reading
the posts in this thread.

Dale B. Dalrymple