by Angela Guess
Jack Vaughan recently wrote in Search Data Management, “As people accumulate big data and then look to do something useful with it, one of the first things they do is put it on a map. It’s a simple first step, but it can be a challenging one. We asked Mansour Raad to shed some light on geographic data as it applies in the days of big data. Raad is a senior software architect at ESRI, and as such, he is at the center of a lot of activity. He walks geospatial software veterans through the ins and outs of Hadoop, and he walks Hadoop users through the ins and outs of geospatial data.”
Raad told Vaughan, “No pun intended, but geographic things are now on the map. Everything is becoming geographically relevant. What happens in Boston is different than what happens in Georgia, for example. Location is very important, and people are connecting data from all over the place. If you are a retailer, people are coming to your website, and implicitly or explicitly, you know where they are and you find that there is a certain product that is relevant in that certain market. Making the association between that product and that location is starting to become very important.”
Raad also said, “I am seeing something of a sunset for Hadoop because of Spark and because of different ways of storing data. Hadoop has a role, but some of the components are being phased out. In geographic data, there are other ways to store these things. The biggest problem we see is that traditional Hadoop and the key value stores, such as HBase, Cassandra and Accumulo, and all their friends, are relying on a sequential ordering of things — that is, one-dimensional ordering. That makes them amazingly fast, but that creates a single-dimension style of search. The problem with geospatial data is that it is not about a single dimension. It is multidimensional. Things that are in space are not sequentially ordered. To solve that, people are introducing an abstraction layer.”
photo credit: Flickr/ roundedbygravity