f



The core of the core of the big data solutions -- Map

Title:       The core of the core of the big data solutions -- Map
Author:      pengwenwei
Email:       =20
Language:    c++
Platform:    Windows, linux
Technology:  Perfect hash algorithm
Level:       Advanced
Description: Map algorithm with high performance
Section      MFC c++ map stl
SubSection   c++ algorithm
License:     (GPLv3)

    Download demo project - 1070 Kb
    Download source - 1070 Kb

Introduction:
For the c++ program, map is used everywhere.And bottleneck of program perfo=
rmance is often the performance of map.Especially in the case of large data=
,and the business association closely and unable to realize the data distri=
bution and parallel processing condition.So the performance of map becomes =
the key technology.

In the work experience with telecommunications industry and the information=
 security industry, I was dealing with the big bottom data,especially the m=
ost complex information security industry data,all can't do without map.

For example, IP table, MAC table, telephone number list, domain name resolu=
tion table, ID number table query, the Trojan horse virus characteristic co=
de of cloud killing etc..

The map of STL library using binary chop, its has the worst performance.Goo=
gle Hash map has the optimal performance and memory at present, but it has =
repeated collision probability.Now the big data rarely use a collision prob=
ability map,especially relating to fees, can't be wrong.

Now I put my algorithms out here,there are three kinds of map,after the bui=
ld is Hash map.We can test the comparison,my algorithm has the zero probabi=
lity of collision,but its performance is also better than the hash algorith=
m, even its ordinary performance has no much difference with Google.

My algorithm is perfect hash algorithm,its key index and the principle of c=
ompression algorithm is out of the ordinary,the most important is a complet=
ely different structure,so the key index compression  is fundamentally diff=
erent.The most direct benefit for program is that for the original map need=
 ten servers for solutions but now I only need one server.
Declare: the code can not be used for commercial purposes, if for commercia=
l applications,you can contact me with QQ 75293192.
Download:
https://sourceforge.net/projects/pwwhashmap/files

Applications:
First,modern warfare can't be without the mass of information query, if the=
 query of enemy target information slows down a second, it could lead to th=
e delaying fighter, leading to failure of the entire war. Information retri=
eval is inseparable from the map, if military products use pwwhashMap inste=
ad of the traditional map,you must be the winner.

Scond,the performance of the router determines the surfing speed, just repl=
ace open source router code map for pwwHashMap, its speed can increase ten =
times.
There are many tables to query and set in the router DHCP ptotocol,such as =
IP,Mac ,and all these are completed by map.But until now,all map are  using=
 STL liabrary,its performance is very low,and using the Hash map has error =
probability,so it can only use multi router packet dispersion treatment.If =
using pwwHashMap, you can save at least ten sets of equipment.

Third,Hadoop is recognized as the big data solutions at present,and its mos=
t fundamental thing is super heavy use of the map,instead of SQL and table.=
Hadoop assumes the huge amounts of data so that the data is completely unab=
le to move, people must carry on the data analysis in the local.But as long=
 as the open source Hadoop code of the map changes into pwwHashMap, the per=
formance will increase hundredfold without any problems.


Background to this article that may be useful such as an introduction to th=
e basic ideas presented:
http://blog.csdn.net/chixinmuzi/article/details/1727195

0
Wenwei
4/21/2015 3:04:25 AM
comp.os.linux.development.apps 5216 articles. 1 followers. Post Follow

0 Replies
436 Views

Similar Articles

[PageSpeed] 59

Reply:

Web resources about - The core of the core of the big data solutions -- Map - comp.os.linux.development.apps

Solution - Wikipedia, the free encyclopedia
In chemistry , a solution is a homogeneous mixture composed of only one phase . In such a mixture, a solute is a substance dissolved in another ...

Granny flats: a cheap housing solution?
Current laws ban Victorians from renting out their backyard homes, but that might change.

Granny flats: a cheap housing solution?
Current laws ban Victorians from renting out their backyard homes, but that might change.

US 'prepared for military solution' against Islamic State in Syria
With the latest round of Syria peace talks due to start in Geneva on Monday, US vice-president issues warning to participants.

US prepared for military solution against Islamic State in Syria, says Biden
US vice-president Joe Biden said on Saturday that the United States and Turkey were prepared for a military solution against Islamic State in ...

HRC report on anti-LGBT religious exemptions under Title IX ignores most obvious solution
... and pointing out the holes in accountability and oversight, but comes up painfully short in the policy recommendations it offers as a solution. ...

D-Link Demonstrates Innovative Networking Solutions at CES
... interesting aspect here is the core platform. While vendors such as Amped Wireless have gone in for the integrated Qualcomm Atheros 2x2 solution ...

Latvian Bike Parking Solution Bindio Launches On Kickstarter
... of all active riders have had their bicycles stolen, and only 2.4% are ever retourned. Young entrepreneurs from Druagiem Group propose a solution ...

Film mogul has solution for making Oscars controversy "go away"
Tyler Perry, Whoopi Goldberg and George Clooney among stars weighing in on lack of diversity on Hollywood's biggest nights

FATHOM Combines Big Data And Software Technology To Drive Water Solutions
... same time struggling with reliable clean water delivery. Enter [&hellip FATHOM Combines Big Data And Software Technology To Drive Water Solutions ...

Resources last updated: 1/25/2016 6:15:19 PM