COMPGROUPS.NET | Search | Post Question | Groups | Stream | About | Register

### what's a good way of filtering/smoothing 4D data?

• Email
• Follow

```Lets say you obtain some sampled data from function f:

f is a function of parameters p1, p2, p3, i.e. f(p1, p2, p3)

where p1 takes value on a set of n1 values, p2 takes value on a set of
n2 values, and p3 takes value on a set of n3 values.

Therefore, you obtain the f's which is of size n1 x n2 x n3...

What's the best way to filter/smooth these data.

I am looking for something like "automatical" filtering/smoothing...

i.e. because it's hard to visualize the data, therefore, I just want
something smoother than the raw data...

usually if it is image data, it's easy to see the effect of
smoothness;

but here we are talking about 4D data, therefore some sort of
"automatic" smoothing/filtering is needed...

Thanks a lot!
```
 0
Reply lunamoonmoon (258) 9/9/2010 6:31:08 PM

See related articles to this posting

```
Luna Moon wrote:

> Lets say you obtain some sampled data from function f:
>
> f is a function of parameters p1, p2, p3, i.e. f(p1, p2, p3)
>
> where p1 takes value on a set of n1 values, p2 takes value on a set of
> n2 values, and p3 takes value on a set of n3 values.
>
> Therefore, you obtain the f's which is of size n1 x n2 x n3...
>
> What's the best way to filter/smooth these data.
>
> I am looking for something like "automatical" filtering/smoothing...
>
> i.e. because it's hard to visualize the data, therefore, I just want
> something smoother than the raw data...
>
> usually if it is image data, it's easy to see the effect of
> smoothness;
>
> but here we are talking about 4D data, therefore some sort of
> "automatic" smoothing/filtering is needed...
>
> Thanks a lot!
```
 0

```You can program up a 4D median filter.  Just loop over all dimensions,
just like you'd do for a 2D or 3D image.  I'm not saying it's the
"best" way, but it is a way and it might work fine for your needs.

(Not sure what Vladimir's issue with you is, calling you a
"stupident", as this seemed like a legitimate question to me.)
-ImageAnalyst
```
 0

```ImageAnalyst <imageanalyst@mailinator.com> wrote in message <a8151beb-f401-4cf5-a780-c85544ee090c@r10g2000vbc.googlegroups.com>...
> You can program up a 4D median filter.  Just loop over all dimensions,
> just like you'd do for a 2D or 3D image.  I'm not saying it's the
> "best" way, but it is a way and it might work fine for your needs.
========

Not sure how that's "just like a 2D or 3D image" actually. Median filtering is not a separable with respect to the image dimensions.
```
 0

```On Sep 9, 5:34=A0pm, "Matt J " <mattjacREM...@THISieee.spam> wrote:
> Not sure how that's "just like a 2D or 3D image" actually. Median filteri=
ng is not a separable with respect to the image dimensions.
---------------------------------------------------------------------------=
-------------
I know it's not separable but who cares?  Why would that be a
necessity?

Well I haven't done it in 4D but for 1, 2, and 3 D, you just collect
all the pixels in the kernel into an array, sort them, and take the
middle one.  Why is this not valid for higher dimensions?  I'm not
saying it will produce the best or desirable results but it should get
rid of impulsive noise, like it does for dimensions 1-3, wouldn't it?
For example, wouldn't median filtering a noisy speck in a color movie
using all 4 dimensions get rid of the speck?  Sure you might also be
able to get rid of the speck by median filtering one of more of the
other dimensions too but it also should work in 4D as far as I know.
If it doesn't work, maybe you can give a counter example.

```
 0

```I'll also add that filtering routines applicable to arrays of any dimension include convn() and, if you have time Image Processing Toolbox, imfilter()

Supposedly, imfilter is also smart enough to exploit separability if it detects that the filter kernel is separable, although I don't see it in the documentation.

In any case, if you don't have the Image Processing Toolbox, the example below shows how to implement a 4D separable filter using these FEX tools

http://www.mathworks.com/matlabcentral/fileexchange/26611-on-the-fly-definition-of-custom-matrix-objects

http://www.mathworks.com/matlabcentral/fileexchange/25969-efficient-object-oriented-kronecker-product-manipulation

A=rand(30,30,30,30);  % hypothetical data

N=3; %filter width

kernel=ones(N,1)/N; %1D averaging filter Nx1

Filter1D=MatrixObj;  %represent "kernel" as a MatrixObj object

Filter1D.Ops.mtimes=@(obj,X) convn(X,kernel,'same');
Filter1D.Ops.size=[30,30];

Filter4D=KronProd({Filter1D},[1,1,1,1]); %separable 4D averaging filter NxNxNxN

A_filtered=Filter4D*A;   %Apply the filter
```
 0

```"Matt J " <mattjacREMOVE@THISieee.spam> wrote in message <i6bmb4\$d71\$1@fred.mathworks.com>...

> Supposedly, imfilter is also smart enough to exploit separability if it detects that the filter kernel is separable, although I don't see it in the documentation.
======

forget this bit. I don't know why I thought imfilter exploited separability, but some quick timing tests indicate otherwise...
```
 0

```On 09/09/2010 11:31 AM, Luna Moon wrote:
> Lets say you obtain some sampled data from function f:
>
> f is a function of parameters p1, p2, p3, i.e. f(p1, p2, p3)

Where's the fourth dimension?

> where p1 takes value on a set of n1 values, p2 takes value on a set of
> n2 values, and p3 takes value on a set of n3 values.
>
> Therefore, you obtain the f's which is of size n1 x n2 x n3...
>
> What's the best way to filter/smooth these data.
>
> I am looking for something like "automatical" filtering/smoothing...
>
> i.e. because it's hard to visualize the data, therefore, I just want
> something smoother than the raw data...
>
> usually if it is image data, it's easy to see the effect of
> smoothness;
>
> but here we are talking about 4D data, therefore some sort of
> "automatic" smoothing/filtering is needed...

You use any of the smoothing algorithms that work for 1D or 2D data,
extended to however many dimensions your data actually is.

the noise's autocorrelation?  Is the noise stationary with respect to
the three dimensions in question?

If the answers are "error squared", "Gaussian", "known" and "yes" then
some sort of weighted average is best.  If it isn't, then some other
answer is best.  There's a whole world of possibilities out there, but
they depend on specifics well beyond the number of dimensions you have
or think you have.

--

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" was written for you.
See details at http://www.wescottdesign.com/actfes/actfes.html
```
 0

```I agree with Tim's comments on your other, related topic.
Explain how this is 4D data.

Let's say that p1, p2, and p3 were x,y, and z (spatial coordinates).
And your f was a value there at that 3D location - it could be
temperature, CT density, the MRI value, or whatever.  Regardless, this
is 3D data, not 4D data.
```
 0

```On Sep 10, 6:31=A0am, Luna Moon <lunamoonm...@gmail.com> wrote:
> Lets say you obtain some sampled data from function f:
>
> f is a function of parameters p1, p2, p3, i.e. f(p1, p2, p3)
>
> where p1 takes value on a set of n1 values, p2 takes value on a set of
> n2 values, and p3 takes value on a set of n3 values.
>
> Therefore, you obtain the f's which is of size n1 x n2 x n3...
>
> What's the best way to filter/smooth these data.
>
> I am looking for something like "automatical" filtering/smoothing...
>
> i.e. because it's hard to visualize the data, therefore, I just want
> something smoother than the raw data...
>
> usually if it is image data, it's easy to see the effect of
> smoothness;
>
> but here we are talking about 4D data, therefore some sort of
> "automatic" smoothing/filtering is needed...
>
> Thanks a lot!

I have a 5 and 6D result - will that do?
```
 0

```Tim Wescott <tim@seemywebsite.com> writes:

> On 09/09/2010 11:31 AM, Luna Moon wrote:
>> Lets say you obtain some sampled data from function f:
>>
>> f is a function of parameters p1, p2, p3, i.e. f(p1, p2, p3)
>
> Where's the fourth dimension?

A function f:R^N -> R can always be thought of as describing a point in
an (N+1)-dimensional since it requires N+1 values, e.g., an (N+1)-tuple
(x_1, x_2, ..., x_N, f(x_1, x_2, ..., x_N)), to describe a point. Take the
simple case of the Cartesian plane, where y = f(x).

Not sure why you're making this an issue.
--
Randy Yates                      % "She has an IQ of 1001, she has a jumpsuit
Digital Signal Labs              %            on, and she's also a telephone."
mailto://yates@ieee.org          %
http://www.digitalsignallabs.com %        'Yours Truly, 2095', *Time*, ELO
```
 0

```ImageAnalyst <imageanalyst@mailinator.com> writes:
> [...]
> Let's say that p1, p2, and p3 were x,y, and z (spatial coordinates).
> And your f was a value there at that 3D location - it could be
> temperature, CT density, the MRI value, or whatever.  Regardless, this
> is 3D data, not 4D data.

No, this is 4D data. To take one of your examples, (x, y z, temperature).
--
Randy Yates                      % "Ticket to the moon, flight leaves here today
Digital Signal Labs              %  from Satellite 2"
mailto://yates@ieee.org          % 'Ticket To The Moon'
http://www.digitalsignallabs.com % *Time*, Electric Light Orchestra
```
 0
Reply yates (3949) 9/11/2010 10:34:37 PM

```Randy Yates <yates@ieee.org> wrote in message <m31v8zgbww.fsf@ieee.org>...
> Tim Wescott <tim@seemywebsite.com> writes:
>
> > On 09/09/2010 11:31 AM, Luna Moon wrote:
> >> Lets say you obtain some sampled data from function f:
> >>
> >> f is a function of parameters p1, p2, p3, i.e. f(p1, p2, p3)
> >
> > Where's the fourth dimension?
>
> A function f:R^N -> R can always be thought of as describing a point in
> an (N+1)-dimensional since it requires N+1 values, e.g., an (N+1)-tuple
> (x_1, x_2, ..., x_N, f(x_1, x_2, ..., x_N)), to describe a point. Take the
> simple case of the Cartesian plane, where y = f(x).
>
> Not sure why you're making this an issue.

People, including my self call it N-D data not N+1. An image with grayscale is 2D data, temperature in a room is 3D data (your case).

Why it's important? Because when the problem is described, it's always relevant to know the approximation size/dimension of the data so as to advice an appropriate algorithm (memory constraints, run time, etc...).

Bruno
```
 0

```"Bruno Luong" <b.luong@fogale.findmycountry> writes:

> Randy Yates <yates@ieee.org> wrote in message <m31v8zgbww.fsf@ieee.org>...
>> Tim Wescott <tim@seemywebsite.com> writes:
>>
>> > On 09/09/2010 11:31 AM, Luna Moon wrote:
>> >> Lets say you obtain some sampled data from function f:
>> >>
>> >> f is a function of parameters p1, p2, p3, i.e. f(p1, p2, p3)
>> >
>> > Where's the fourth dimension?
>>
>> A function f:R^N -> R can always be thought of as describing a point in
>> an (N+1)-dimensional since it requires N+1 values, e.g., an (N+1)-tuple
>> (x_1, x_2, ..., x_N, f(x_1, x_2, ..., x_N)), to describe a point. Take the
>> simple case of the Cartesian plane, where y = f(x).
>>
>> Not sure why you're making this an issue.
>
> People, including my self call it N-D data not N+1. An image with
> grayscale is 2D data, temperature in a room is 3D data (your case).

> Why it's important? Because when the problem is described, it's always
> relevant to know the approximation size/dimension of the data so as to
> advice an appropriate algorithm (memory constraints, run time,
> etc...).

I accede the point. I was translating "data" in my head to what I think
the OP had in mind, which was "space". I agree that in this context, the
"dimension of the data" is the dimensionality of the array required to
access it.
--
Randy Yates                      % "She has an IQ of 1001, she has a jumpsuit
Digital Signal Labs              %            on, and she's also a telephone."
mailto://yates@ieee.org          %
http://www.digitalsignallabs.com %        'Yours Truly, 2095', *Time*, ELO
```
 0

```On Sep 9, 11:31=A0am, Luna Moon <lunamoonm...@gmail.com> wrote:
> Lets say you obtain some sampled data from function f:
>
> f is a function of parameters p1, p2, p3, i.e. f(p1, p2, p3)
>
> where p1 takes value on a set of n1 values, p2 takes value on a set of
> n2 values, and p3 takes value on a set of n3 values.
>
> Therefore, you obtain the f's which is of size n1 x n2 x n3...
>
> What's the best way to filter/smooth these data.
>
> I am looking for something like "automatical" filtering/smoothing...
>
> i.e. because it's hard to visualize the data, therefore, I just want
> something smoother than the raw data...
>
> usually if it is image data, it's easy to see the effect of
> smoothness;
>
> but here we are talking about 4D data, therefore some sort of
> "automatic" smoothing/filtering is needed...
>
> Thanks a lot!

The median filter is obvious, but what about a multi dimensional
Savistky-Golay/least squares filter?
http://pubs.acs.org/doi/abs/10.1021/ac00006a015
It doesn't cost too much to find out if it works well enough.

This is a worthy problem no matter what the source.

Peter Nachtwey
```
 0

14 Replies
324 Views

Similar Articles

12/19/2013 3:14:00 AM
page loaded in 38875 ms. (0)

Similar Artilces:

what's a good way of filtering/smoothing 4D data?
Lets say you obtain some sampled data from function f: f is a function of parameters p1, p2, p3, i.e. f(p1, p2, p3) where p1 takes value on a set of n1 values, p2 takes value on a set of n2 values, and p3 takes value on a set of n3 values. Therefore, you obtain the f's which is of size n1 x n2 x n3... What's the best way to filter/smooth these data. I am looking for something like "automatical" filtering/smoothing... i.e. because it's hard to visualize the data, therefore, I just want something smoother than the raw data... usually if it is image da...

Data Smoothing Filter
I am looking for an open-source smoother to down-sample data. I know there are many such as Hull Moving average, but I have data logged at the ms scale at non-fixed time intervals and I want to sample it on the second range at fixed time intervals. Does anyone know of a good smoother to do this? The key is that the input data is not logged at a constant frequency. Jessica wrote: > I am looking for an open-source smoother to down-sample data. I know > there are many such as Hull Moving average, but I have data logged at > the ms scale at non-fixed time intervals and I want to...

Have: Apple IIe, box of good 5.25 floppies with software. Need: good way to preserve the data
Ok, so here's my situation. I recently added my old Apple IIe from storage to my wall-o-vintage computer collection. I've hooked up the necessary Apple 5.25" Drives (x2 daisy chained). I have software that I got from a friend that I'd like to back up. I have a box of blank 5.25" floppies. Preferably I'd like to back up not to other floppies, but to something like thumb drives or at least a directory on a hard drive. I've read about (and subsequently lost the URL) a way to turn a Linux box into a floppy image server for an Apple IIe by using special software and a...

is there a good way to save data of different types ?
Hi there, for a long time I've been trying to think of way of saving different data of different types using one single class (well 2 in reality, a class for the data, and 1 class for a list of data). The problem that I try to solve is the following. Imagine a class "Attribute" that needs to save multiple "Parameters" (the number of parameters for 1 attribute will vary each time we run the application). Each parameter can be of a different type (float, integer, double...) and we can have 1 value per parameter or an array of values. So I tried to create a template class...

what's a good way of visualizing 4-D data?
Lets say you are optimizing three parameters, by grid searching, you obtain: f as a function of parameters p1, p2, p3, i.e. f(p1, p2, p3) where p1 takes value on a set of n1 values, p2 takes value on a set of n2 values, and p3 takes value on a set of n3 values. Therefore, you obtain the f's which is of size n1 x n2 x n3... What's the best way to visualize these data, suppose our goal is to check the smoothness of the surface? Thanks a lot! Luna Moon wrote: > Lets say you are optimizing three parameters, > by grid searching, > you obtain: > &...

Would this be a good way to prove CRD (compress *random* data) to the public?
Random or psuedo-random, I don't care...that's not the point I am focusing on here. If I was going to release a new kind of compression, I know that I must be able to prove it to the general masses. Here are my thoughts on what an acceptable proof would be. Step 1) Allow the public to download a "decompressor" program. Step 2) Allow the public to upload a file to a server to be compressed. Step 3) The server compresses the file and displays the output in plain Base64 encoded string (on a web page or whatever) to be *COPIED AND PASTED* in the decompressor program. All s...

Smooth data over 3D data set
I have triangulated a set of points so that the corresponding triangles create an irregular shape in three dimensions (think of, for example, a person's head created from thousands of triangles). Each of these triangles has a different value associated with it, ranging from 0-2.0. Right now, I am using trisurf to create the surface and each triangle is a color corresponding to its value, after the values are linearly scaled to the colormap. I was hoping to create a more uniform surface that could transition over the entire surface between the colors of the triangles (the sort of image you...

Filtering vibration data
I have laser displacement data measuring vibrations (3s at 4000hz). I want to apply a high-pass filter (>2hz) to reduce noise that I know is related to other things not of interest. I've been doing this as follows flt = fir1(1000,2/2000,'high'); dataFilt = filtfilt(flt,1,data); The resulting FFT spectrum looks as expected but I'm unsure of what filter order I should be using, e.g. data fitting with a higher order function always fits the sampled data better than low order functions but isn't necessarily the best description of the physics....am I doing that here and ...

Taking 4d data and making it into 3d data?
Here's some code exerpts of what I am trying to do: colorImage = mov(1,i+vidOffset).cdata; imageBuffer(frameNumber,1:480,1:720,1:3) = colorImage(1:480,1:720,1:3); tempFrame = imageBuffer(frameNumber,1:480,1:720,1:3); for some reason, tempFrame is a 4d array, as opposed to a 3d array. I am essentially trying to temporarily save a series of images, then extract them 1 at a time. "Eric Siegfried" <paranoiid1@aol.com> wrote in message news:eee28e3.-1@webx.raydaftYaTP... > Here's some code exerpts of what I am trying to do: > colorImage = mov(1,i+vidOffset...

Is there a way to implement 2D filter thru 1D filter?
Hi all, I am trying to figure out if there is a way to do meaningful 2D filtering thru 1D filter functions... Assuming we don't have 2D filters in Matlab and I only have 1D filters in Matlab. How can I achieve the results of 2D filtering via calling 1D filter functions? Thanks a lot! On Wed, 11 Jan 2012 15:33:33 -0800 (PST), Lucy <comtech.usa@gmail.com> wrote: >Hi all, > >I am trying to figure out if there is a way to do meaningful 2D >filtering thru 1D filter functions... > >Assuming we don't have 2D filters in Matlab and I only have 1D filters >in Mat...