Just to summarize what I have learned so far:
******************************************************************
efficent code of roll/expanding mean => efficient code of roll/expanding stdev (with small risk of cancellation error mentioned by Derek)
expanding mean is implemented by cumsum function already built-in.
rolling mean is implemented either by filter or cumsum trick.
expanding max is fast enough using simple loop
******************************************************************
Now, two more questions:
******************************************************************
How to efficiently implement the rolling/expanding percentile (special case would be median)
******************************************************************
also how about rolling max??
Thanks in advance for your advice!
|
|
0
|
|
|
|
Reply
|
andy
|
1/12/2010 5:22:04 AM |
|
"andy Lu" <luyi36@gmail.com> wrote in message <hih0ts$b2a$1@fred.mathworks.com>...
This is the result of a search on "sliding window" in the file exchange
http://www.mathworks.com/matlabcentral/fileexchange/?term=sliding+window
It should have a number of options for you.
|
|
0
|
|
|
|
Reply
|
Matt
|
1/12/2010 7:20:04 AM
|
|
"andy Lu" <luyi36@gmail.com> wrote in message <hih0ts$b2a$1@fred.mathworks.com>...
> Now, two more questions:
>
> ******************************************************************
> How to efficiently implement the rolling/expanding percentile (special case would be median)
> ******************************************************************
For rolling percentile, it would look something like this
x=sort(x);
x(round(p*(1:length(x)))); %p is percentile
|
|
0
|
|
|
|
Reply
|
Matt
|
1/12/2010 7:35:20 AM
|
|
As Matt has pointed out, one way to perform median filtering is sorting on running window. One of the implementation on 2D is here
http://www.mathworks.com/matlabcentral/newsreader/view_thread/251787
This apply as well on 1D array (just provide the appropriate input).
In term of algorithmic, one can do better by inserting/removing elements in a sorted FIFO windowed array. If the data structure is well designed, that task can be carried out in O(log(window size)) complexity. Such complexity is meet by algorithms such as dichotomy search or Downheap of a heap structure.
But I guess for most applications where window size is not too big a brute force sorting is fast enough.
Bruno
|
|
0
|
|
|
|
Reply
|
Bruno
|
1/12/2010 9:33:04 AM
|
|
"Matt J " <mattjacREMOVE@THISieee.spam> wrote in message <hih8no$icd$1@fred.mathworks.com>...
> "andy Lu" <luyi36@gmail.com> wrote in message <hih0ts$b2a$1@fred.mathworks.com>...
>
> > Now, two more questions:
> >
> > ******************************************************************
> > How to efficiently implement the rolling/expanding percentile (special case would be median)
> > ******************************************************************
>
> For rolling percentile, it would look something like this
>
> x=sort(x);
>
> x(round(p*(1:length(x)))); %p is percentile
Thanks Mat.
I have tried following, which is very slow. Did I do something bad?
function r=runmed3(x,p,wind)
[m, n]=size(x);
r = zeros(m-wind+1,n);
pos = round(p*wind);
for c=wind:m
x2 = sort(x(c-wind+1:c, :));
r(c-wind+1,:) = x2(pos,:);
end;
end
tic
x=rand(30000, 30);
runmed3(x,0.5,200);
toc
Elapsed time is 7.845210 seconds.
|
|
0
|
|
|
|
Reply
|
andy
|
1/12/2010 3:49:04 PM
|
|
"Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <hihfkg$4er$1@fred.mathworks.com>...
> As Matt has pointed out, one way to perform median filtering is sorting on running window. One of the implementation on 2D is here
>
> http://www.mathworks.com/matlabcentral/newsreader/view_thread/251787
>
> This apply as well on 1D array (just provide the appropriate input).
>
> In term of algorithmic, one can do better by inserting/removing elements in a sorted FIFO windowed array. If the data structure is well designed, that task can be carried out in O(log(window size)) complexity. Such complexity is meet by algorithms such as dichotomy search or Downheap of a heap structure.
>
> But I guess for most applications where window size is not too big a brute force sorting is fast enough.
>
> Bruno
Thanks Bruno.
I will try the 2D version. Your max/min filter is great!
|
|
0
|
|
|
|
Reply
|
andy
|
1/12/2010 3:52:02 PM
|
|
|
5 Replies
315 Views
(page loaded in 0.055 seconds)
Similiar Articles: 52-week high rolling window -- HELP!!!!! - comp.soft-sys.sas ...I didn't think the data would take this long to run, is there an easier/more efficient ... wanting to sell HP 42S and 16C - comp.sys.hp48 52-week high rolling window -- HELP ... TCP MSS issue - comp.unix.programmerHow TCP packtes data is done to make transmission more efficient, that application has no need to ... But this doesn't mean that all kind of Streams functionality is ... How to set DISPLAY env. in script. - comp.sys.sun.admin... cat /tmp/TMP_DISP` > As a side note, this is more efficient ... in ooRexx & XP? - comp.lang.rexx Does this mean there ... if in a script I execute a set command, I can create ... Visual Query Builder - comp.soft-sys.matlabWhat I mean to ask is does Visual Query Builder support OLE databases? I would like to try to use the toolbox to help me learn to build more efficient queries. Problems to calculate sin - comp.lang.java.programmerBy log table I just mean a Table for Sine. Log tables are more familiar than Log Tables. ... of 10,000 independent cubic fits, but it's more reasonable and efficient to ... How to generate random number without replacement? - comp.lang ...This does not scale, although Perl is so efficient that ... between 1 and 1000,000,000, shuffle will take more ... Could you please explain what you mean by "with/without ... Comparison of a Simple Join done by EG and Hand Coded - 22 times ...I have sql code to get the median. I will compare running that SQL program ... SKILLED non-EG SQL programmer will =A0always > (99%of the time) create more efficient code ... SQLCI LOAD and BLOCKIN parameter - comp.sys.tandemIn order to make the process efficient, I would like to use the ... my target table, I interpret this > to mean ... parallelizing the work on the table could make for more ... How to write testbench file? - comp.lang.vhdlhi KJ, Do u mean that i can just create a testbench and then include whatever ... giving it permission to do something else, especially if it is more efficient. JTable - How To Loop Through Looking For Selected Rows? - comp ...Is there a more efficient way to do this than loop= ing through ... I need a row vector ... Find the median value of an array. - comp.lang.fortran 2) Pass through the array ... how to make the rolling median more efficient - Newsreader ...File exchange, MATLAB Answers, newsgroup access, Links, and Blogs for the MATLAB & Simulink user community 22 Ways to Make Your Car More Fuel Efficient | Ririan Project... fuel efficiency as such, but it does mean your ... intermittently, as needed to keep the car rolling ... anything I can do to a diesel engine to make it more efficient, ill ... 7/22/2012 1:11:26 PM
|