Question for maximum likelihood estimation in matlab

  • Follow


Hello all,

I am relatively new to matlab and I came up with a project that I have 39 numbers and I have to find their distribution. I performed a liliefors test and I found that the distribution is not Gaussian. From an example in bibliography I know that the specific data if it isn't Gaussian it would be gamma so I did the following :

[phat, pci] = mle(X,'distribution','gamma');

but there are negative numbers included in these 39 numbers and it can't be gamma with negative numbers.How can I possibly found which is the right distribution?Is it right to use abs(X)?When I did that, the distribution was both gamma and Poisson so something must be wrong.

I don't know if what I am asking is easy or difficult but any help would be really apreciated.

below i display the numbers:

X: -0.3316
    0.1165
    0.2818
   -0.2029
    0.0515
    0.0778
    0.2692
   -0.1148
    0.5615
    0.2595
   -0.0297
    0.2991
   -0.6717
    0.0023
   -0.4716
   -0.4233
    0.2758
   -0.2753
    0.0679
    0.2122
    0.1093
   -0.2319
   -0.2437
    0.3562
   -0.1394
   -0.0367
   -0.1376
   -0.3024
   -0.1279
   -0.3643
   -0.0186
   -0.0984
    0.3753
   -0.1998
    0.2876
    0.1289
    0.4190
    0.2406
   -0.0942
0
Reply naimead (4) 5/5/2010 1:14:46 AM

Below I display a 1000 numbers (instead of the previous 39). Would it be easier now to define its most probable distribution?

   -0.0500
    0.5857
   -0.2129
   -0.3287
    0.1451
   -0.2810
   -0.1610
    0.2444
    0.3963
    0.4522
    0.2518
   -0.3134
   -0.0975
    0.1074
   -0.2323
    0.2471
   -0.5196
   -0.4689
    0.4993
    0.6399
    0.3385
    0.3555
   -0.0881
   -0.2061
   -0.2089
   -0.0916
   -0.3092
    0.0809
    0.5978
   -0.4791
    0.1867
    0.5309
    0.0192
    0.2427
   -0.0522
   -0.0730
    0.4258
    0.2091
    0.3874
   -0.5026
   -0.1474
   -0.1753
   -0.1106
    0.3257
    0.1955
    0.4947
    0.0442
   -0.4863
   -0.2658
    0.0636
   -0.1925
    0.1063
   -0.3940
   -0.2828
    0.1978
   -0.2984
    0.4389
    0.2906
   -0.3637
   -0.3113
   -0.0779
   -0.5110
    0.3791
    0.1347
   -0.0853
    0.4493
    0.1110
   -0.2984
   -0.1244
    0.2586
   -0.0958
    0.1091
    0.3337
   -0.5419
   -0.3059
    0.2340
   -0.1312
   -0.2053
    0.4643
    0.5076
    0.2439
   -0.3676
   -0.0505
    0.3402
    0.2201
    0.3640
    0.0304
    0.5138
   -0.1367
   -0.0832
    0.0059
    0.4071
   -0.3512
   -0.2422
    0.2529
    0.1135
    0.0551
   -0.0180
   -0.7036
    0.1424
    0.1384
   -0.0413
    0.2729
   -0.0304
    0.5066
   -0.4058
   -0.4479
   -0.3964
   -0.2245
   -0.3672
   -0.2727
   -0.0766
   -0.1703
   -0.1028
   -0.2231
   -0.4303
   -0.4066
   -0.2313
   -0.4432
   -0.4299
    0.7068
   -0.2530
    0.3363
    0.2483
    0.0800
   -0.0168
    0.0696
   -0.1677
   -0.0458
   -0.3364
    0.0093
    0.0184
    0.0493
    0.0918
   -0.3683
   -0.0408
   -0.3828
   -0.1746
   -0.0416
   -0.4568
    0.1223
   -0.0345
    0.0876
   -0.2374
   -0.2915
    0.1505
   -0.1617
   -0.3007
   -0.3576
    0.1966
   -0.1077
    0.3377
   -0.2648
    0.2742
    0.3650
    0.4071
   -0.2141
    0.1860
   -0.1219
   -0.0813
    0.6635
   -0.1265
   -0.0430
   -0.1706
    0.0512
    0.1513
    0.3988
   -0.0964
    0.2774
    0.0659
   -0.0445
    0.6305
    0.2347
   -0.3240
    0.0609
   -0.3575
   -0.3571
    0.1228
   -0.3235
    0.2770
    0.4217
   -0.0420
    0.1653
    0.2501
   -0.2904
    0.2357
    0.0049
    0.1001
    0.3569
   -0.1428
    0.3006
   -0.2325
    0.1118
    0.3419
   -0.1637
    0.0911
    0.2294
   -0.1265
    0.0541
   -0.0902
   -0.2129
   -0.2765
   -0.4022
   -0.1721
    0.1957
    0.0857
   -0.3768
   -0.5656
   -0.4093
    0.1755
   -0.0304
   -0.3568
   -0.4296
   -0.0748
    0.0620
   -0.2743
    0.0072
   -0.3873
   -0.1825
   -0.5636
   -0.3393
   -0.1743
   -0.0461
    0.2552
   -0.0681
   -0.1869
    0.0692
    0.5684
   -0.4149
    0.5942
   -0.2579
   -0.1597
   -0.2502
    0.3157
    0.0080
   -0.2718
   -0.2656
   -0.0871
   -0.0239
   -0.2460
    0.0596
    0.1607
    0.2901
   -0.1018
    0.1400
   -0.2277
   -0.2764
   -0.3992
   -0.5305
    0.2127
    0.0354
    0.1122
   -0.2351
    0.0972
    0.1655
    0.1597
   -0.3565
    0.4430
   -0.0141
    0.2894
    0.2524
    0.0312
   -0.2844
    0.0475
   -0.3427
    0.2105
   -0.0264
    0.1253
    0.2169
    0.0962
    0.2946
    0.1732
   -0.3831
    0.3555
    0.2173
   -0.2022
   -0.3147
    0.1079
    0.2854
    0.1610
   -0.0292
   -0.0855
    0.2851
   -0.1872
   -0.1745
    0.1792
   -0.0129
   -0.1725
   -0.2166
   -0.4359
   -0.2371
   -0.6623
   -0.3350
   -0.2063
   -0.1764
    0.0880
   -0.4739
   -0.2582
    0.4864
    0.3874
    0.1427
    0.4902
   -0.2284
    0.5522
   -0.1941
   -0.1954
    0.4256
    0.0224
    0.2498
    0.0523
   -0.0143
    0.1373
   -0.1380
    0.2129
    0.1556
    0.3395
   -0.1482
   -0.7696
   -0.1080
    0.1780
    0.2903
    0.3611
    0.0907
    0.1399
   -0.5644
    0.1910
   -0.0891
    0.1940
    0.0768
   -0.5719
   -0.0535
    0.3163
   -0.2201
    0.3701
    0.0266
    0.2033
    0.2749
    0.5515
   -0.2181
   -0.0636
    0.2014
    0.2154
   -0.2989
   -0.1684
   -0.2339
   -0.0414
    0.1713
   -0.5239
    0.1478
   -0.0204
    0.1948
    0.3139
    0.1459
   -0.3905
    0.0803
    0.2467
    0.2409
   -0.0054
    0.1366
    0.0764
   -0.1878
    0.1420
   -0.3472
    0.1221
   -0.2943
    0.0483
   -0.0242
    0.0609
    0.0136
   -0.6534
   -0.0497
   -0.2613
    0.1078
    0.1980
    0.3361
    0.2087
    0.4597
   -0.2752
    0.3154
    0.2798
    0.0378
   -0.3863
    0.2210
   -0.1425
    0.0672
   -0.4913
    0.1273
   -0.2913
    0.2304
   -0.1982
   -0.2457
    0.2288
   -0.0634
   -0.1994
   -0.4007
   -0.0433
    0.3475
    0.4620
   -0.1591
   -0.4129
    0.5009
    0.0160
   -0.0421
   -0.2296
    0.4123
   -0.1701
   -0.3350
   -0.4271
    0.2332
   -0.6900
    0.4174
   -0.0032
    0.0574
   -0.3552
   -0.5816
   -0.3363
    0.0184
    0.2602
   -0.4333
   -0.1803
   -0.2148
   -0.4856
   -0.4868
    0.2702
   -0.1646
   -0.3056
   -0.1144
    0.2431
   -0.0175
    0.2886
   -0.1350
    0.1862
    0.2744
   -0.4767
   -0.5332
    0.5562
    0.1476
    0.0555
   -0.0343
    0.0859
    0.0405
    0.1435
    0.4105
    0.1046
    0.2737
   -0.1150
    0.2957
   -0.0664
    0.4569
   -0.1179
    0.2256
    0.3211
   -0.3690
   -0.4033
   -0.1920
   -0.3068
   -0.4030
    0.0218
   -0.3400
    0.0445
    0.1086
   -0.2372
    0.5111
    0.4245
    0.0017
    0.3522
   -0.4650
   -0.0724
    0.5205
   -0.1055
    0.1584
    0.2240
   -0.0764
   -0.3609
    0.0778
   -0.0570
   -0.5022
    0.3179
    0.0192
    0.5660
   -0.3658
   -0.2970
   -0.4678
   -0.2193
   -0.2386
    0.0106
    0.1531
    0.1295
   -0.4333
    0.4436
   -0.0470
   -0.2079
    0.1989
   -0.3296
   -0.0906
    0.1450
    0.2432
   -0.5019
    0.0608
    0.0753
   -0.2529
   -0.2215
    0.2326
   -0.4024
    0.1340
    0.3813
   -0.0463
   -0.3825
   -0.0219
    0.2351
   -0.4219
   -0.1849
    0.1772
   -0.2413
   -0.1475
   -0.2105
    0.1889
    0.4507
    0.0048
    0.1193
    0.0601
   -0.2401
   -0.3257
    0.0833
    0.2564
   -0.5918
    0.0773
   -0.1297
   -0.4001
   -0.3909
    0.3706
    0.3269
    0.3293
   -0.2296
    0.4345
   -0.0098
   -0.3214
    0.3126
   -0.2127
   -0.5130
    0.6533
   -0.0685
    0.0736
   -0.3252
    0.0009
   -0.0449
   -0.5275
    0.5077
   -0.0068
    0.1244
   -0.1771
    0.3349
   -0.2409
    0.0470
    0.2418
   -0.5523
    0.1649
   -0.3140
    0.2265
   -0.2564
    0.3155
    0.0527
   -0.4456
    0.3628
   -0.3740
   -0.1635
   -0.0779
   -0.2803
   -0.5194
    0.1997
    0.1592
   -0.1286
   -0.4682
    0.1415
    0.2542
    0.1340
   -0.7261
    0.1376
    0.0992
   -0.0507
    0.2290
   -0.3880
   -0.3531
    0.2661
    0.4541
    0.2292
    0.3438
   -0.5629
    0.0409
    0.1156
    0.3721
    0.0313
   -0.1394
   -0.4140
   -0.1833
    0.3979
    0.2364
    0.1966
    0.4082
   -0.4538
    0.6537
   -0.1918
   -0.3170
    0.3946
    0.2460
    0.4101
    0.1732
    0.3022
    0.0395
   -0.3214
    0.2902
    0.2539
    0.3927
    0.2938
    0.5648
    0.2387
   -0.1656
    0.4226
   -0.0092
    0.1905
    0.4122
   -0.2998
    0.0204
   -0.1473
    0.5296
    0.3630
   -0.2257
    0.1165
    0.5580
    0.2878
    0.0119
    0.0586
    0.2194
   -0.0124
   -0.1387
    0.1686
    0.2277
    0.1564
    0.0102
    0.4251
    0.0859
    0.2634
   -0.0153
   -0.1753
    0.1722
   -0.0964
   -0.1158
   -0.2498
    0.2654
   -0.2704
    0.1111
   -0.0650
   -0.2697
   -0.1724
    0.1017
    0.3705
    0.0006
    0.5171
   -0.1612
    0.3102
    0.5424
    0.1605
    0.0088
    0.4244
    0.4107
    0.1925
    0.2565
   -0.4141
   -0.3105
   -0.1067
    0.3661
   -0.2227

    0.5498
   -0.1189
    0.5289
   -0.1515
    0.1332
    0.4335
    0.3425
    0.0254
    0.0899
    0.0763
   -0.1599
   -0.3607
   -0.1322
   -0.1513
   -0.0395
   -0.2495
    0.1585
   -0.2293
   -0.3072
   -0.0923
    0.3069
    0.0872
    0.3438
    0.3954
    0.0446
   -0.1905
   -0.1659
    0.1394
    0.1032
    0.1503
    0.1332
    0.2881
    0.1055
   -0.3516
   -0.1638
    0.1777
   -0.2679
    0.0600
   -0.2839
    0.0069
   -0.0388
   -0.2648
    0.0769
    0.1616
    0.0853
   -0.0410
   -0.0681
    0.3208
    0.3234
   -0.2045
    0.1647
   -0.5506
    0.1244
    0.3929
   -0.2065
    0.0072
    0.0455
    0.0284
    0.0384
    0.2640
   -0.0053
   -0.0804
   -0.1662
    0.2177
    0.0581
    0.2308
   -0.5733
   -0.0042
   -0.0369
    0.1472
   -0.0099
    0.2067
   -0.0647
    0.0308
    0.0644
   -0.2285
   -0.4131
    0.5552
   -0.1658
    0.1451
   -0.2843
   -0.2375
    0.2521
   -0.1874
    0.4691
   -0.1745
    0.1392
    0.3453
   -0.6268
    0.2952
   -0.3441
    0.2989
    0.1763
    0.1670
   -0.0140
   -0.4950
    0.2555
   -0.0627
    0.1735
    0.2029
    0.3507
    0.2763
    0.3881
   -0.1227
    0.2863
   -0.2115
    0.3119
   -0.3719
    0.0306
   -0.2502
    0.4879
    0.2258
   -0.2697
   -0.4892
    0.7059
    0.2744
    0.1732
   -0.1364
   -0.1245
    0.1761
    0.0372
    0.3152
    0.2744
   -0.7069
    0.5419
   -0.2126
    0.2585
    0.2947
   -0.4385
   -0.1818
    0.1087
    0.6450
   -0.4595
   -0.4602
   -0.4767
    0.0894
    0.2836
   -0.1743
    0.2090
   -0.1554
    0.1648
    0.2427
   -0.1885
    0.2183
   -0.0275
    0.0649
   -0.0912
   -0.1946
    0.3548
   -0.5483
   -0.2755
   -0.0088
    0.4420
    0.1041
   -0.1266
    0.3565
   -0.4420
   -0.0120
   -0.3048
    0.0317
    0.1423
    0.1670
    0.2781
   -0.1344
    0.0874
    0.1239
    0.3031
    0.2755
   -0.5421
   -0.3902
    0.0463
    0.2181
    0.3776
    0.4193
    0.2454
   -0.1579
   -0.1644
   -0.1680
    0.2992
    0.0319
    0.5294
    0.0728
   -0.2344
   -0.2426
   -0.4329
    0.3045
    0.1372
    0.5111
   -0.4545
   -0.3033
    0.1124
   -0.2215
    0.0691
   -0.4153
    0.3714
   -0.0755
    0.0240
    0.1887
    0.2018
    0.0632
    0.2901
    0.2483
    0.7030
    0.3534
    0.0700
    0.1722
   -0.2650
    0.4865
    0.2742
    0.4454
   -0.0833
   -0.2294
   -0.3833
   -0.1452
   -0.2369
    0.2558
   -0.2609
    0.4673
    0.1938
   -0.3080
    0.1339
   -0.3202
    0.0779
    0.2421
    0.1940
   -0.0289
   -0.3653
    0.2912
   -0.3518
   -0.1446
   -0.0121
    0.3532
   -0.0425
    0.3188
    0.3071
    0.3904
   -0.1671
   -0.1855
   -0.4654
   -0.1433
   -0.2522
    0.2538
   -0.2643
    0.0288
   -0.4103
    0.0685
    0.2189
   -0.2710
   -0.2239
    0.2596
   -0.1794
    0.1580
   -0.1094
    0.3454
   -0.3332
    0.1266
    0.3138
   -0.1453
    0.3978
    0.1750
   -0.1230
    0.0033
   -0.1753
   -0.5553
    0.4880
   -0.0970
    0.3764
   -0.3115
    0.3757
    0.4346
   -0.0073
    0.3365
    0.7188
    0.0125
   -0.1516
   -0.1929
    0.1435
    0.4059
   -0.0833
    0.2814
    0.4240
    0.1354
    0.1393
    0.3847
    0.0510
   -0.1620
   -0.0946
   -0.4418
    0.0690
    0.0555
   -0.3754
    0.1879
   -0.3024
    0.0463
    0.0085
   -0.3846
    0.1938
    0.1731
   -0.0293
   -0.0187
   -0.1479
    0.0269
   -0.1775
   -0.3001
   -0.2148
    0.5348
    0.2202
    0.2752
    0.3425
    0.4637
    0.1748
    0.1870
   -0.0974
    0.1375
   -0.4023
   -0.1953
    0.4534
    0.3857
    0.1362
   -0.2736
   -0.0312
    0.4063
   -0.0642
   -0.1021
    0.1535
   -0.3857
    0.0852
    0.3385
0
Reply naimead 5/5/2010 3:30:47 PM


On 5/5/2010 1:14 AM, naimead wrote:
> Hello all,
>
> I am relatively new to matlab and I came up with a project that I have 39 numbers and I have to find their distribution. I performed a liliefors test and I found that the distribution is not Gaussian. From an example in bibliography I know that the specific data if it isn't Gaussian it would be gamma so I did the following :
>
> [phat, pci] = mle(X,'distribution','gamma');
>
> but there are negative numbers included in these 39 numbers and it can't be gamma with negative numbers.How can I possibly found which is the right distribution?Is it right to use abs(X)?When I did that, the distribution was both gamma and Poisson so something must be wrong.

Naimead, if this is a random sample, then clearly it did not come from a 
Gaussian distribution, as LILLIETEST confirms.  Clearly they did not 
come from a gamma distribution, because as you point out, something like 
half of them are negative, and even if you add a constant to all of the 
values, the shape is completely wrong.  If your theory says that it must 
be normal or gamma, then either your theory is wrong, or you've 
misinterpreted it, or your data have not been collected according to the 
assumptions of your theory.

Without meaning to sound unhelpful, noone is going to be able to 
reconcile the above for you.  Arbitrarily modifying 50% of your data to 
meet your theoretical expectations probably isn't the right path to take.

There are other distributions you could investigate.  One of them is the 
extreme value, which happens to be supported by LILLIETEST (and in fact, 
the p-value is not significant at 5%).  I am NOT saying that this is the 
distribution you should use, only that it is a possibility.  There are 
other possibilities, such as the generalized extreme value.  Both the EV 
and the GEV are supported by the Distribution Fitting Tool GUI, 
DFITTOOL.  This tool will help you look at the different fits against 
the data, both the PDFs and the CDFs.  It turns out that the GEV is not 
such a bad fit (look at the CDF plot in DFITTOOL).  Again, I am NOT 
saying that this is the distribution you should use.

There are other more complicated distributions, such as a three 
parameter gamma.  That is not explicitly supported by the Statistics 
Toolbox, but you could perhaps use MLE to fit it.  You are on your own 
there.

My advice would be to revisit your assumptions, because clearly they are 
wrong.  Hope this helps.
0
Reply Peter 5/5/2010 3:30:59 PM

naimead <naimead@yahoo.com> wrote in message <1597394486.73822.1273036518211.JavaMail.root@gallium.mathforum.org>...
> Hello all,
> 
> I am relatively new to matlab and I came up with a project that I have 39 numbers and I have to find their distribution. I performed a liliefors test and I found that the distribution is not Gaussian. From an example in bibliography I know that the specific data if it isn't Gaussian it would be gamma so I did the following :
> 


What if the distribution is neither Gaussian or gamma? Plot your data (e.g., hist(x) or hist(x,20)) and you'll see you probably don't have enough points to sensibly fit a distribution in the first place.
0
Reply Mark 5/5/2010 3:53:04 PM

naimead <naimead@yahoo.com> wrote in message <162255079.77888.1273087878102.JavaMail.root@gallium.mathforum.org>...
> Below I display a 1000 numbers (instead of the previous 39). Would it be easier now to define its most probable distribution?
> 

Yes, or at least getting there. The larger the sample the more you can work with it. Without running any metrics on it, it does not seem to be a single Gaussian or Lorentzian distribution (but could be two or more overlapping ones...).

Try fitting your data to some of the various (more or less) symmetrical two-tailed distributions that are out there. I don't have or use the statistics toolbox and don't work with distributions enough to help you fit this.
0
Reply Mark 5/5/2010 8:57:04 PM

I have used DFITTOOL to fit EV, GEV and normal distributions. I agree that EV distribution fits relatively well with my data comparing with the others. But is there any quantitative way to find which fits better than the other one without relying just on the images?(I think that when I compare PDF and CDF fitting I tend to choose different distribution...)

naimead
0
Reply naimead 5/5/2010 11:22:01 PM

5 Replies
323 Views

(page loaded in 0.241 seconds)

Similiar Articles:













7/24/2012 12:40:52 AM


Reply: