HOME  |  BLOGS  |  MESSAGE BOARDS  |  AUDIO  |  VIDEO  |  WEBCASTS  |  RESOURCES  |  ABOUT US
Cassimir Medford

Two Big Problems for Big Data

NO RATINGS
1 saves
View Comments: Newest First | Oldest First | Threaded View
Page 1 / 2   >   >>
Nasimson
User Rank
Business Processor
Re: Big Data defined
Nasimson   1/31/2012 7:00:01 AM
NO RATINGS
A great bit of work should be done using Data Mining techniques to overcome some aspects of the bitter problems of the big data. Since Data mining helps mine the minute information that is needed for making reasonable decisions.

Overall, a great article!

kdawson
User Rank
Blogger
Big Data defined
kdawson   1/13/2012 2:29:04 PM
NO RATINGS
Came across an excellent post on O'Reilly Media with a working definition of Big Data. In Edd Dumbill's view, it's Big Data if it has volume, velocity, and variety.

Chez
User Rank
Agile Expert
Re: Re : Two Big Problems for Big Data
Chez   12/27/2011 1:15:48 AM
NO RATINGS
The issue of college graduates coming into the enterprise world actually brings up some pretty interesting parallels to the enterprise world itself. Although Cassimir's tone here is very neutral (which is commendable), many would be quick to lament the choices that these college students are making. To me, this is not unlike lamenting the way large companies outsource all of their labor when it's cheaper, or maximize tax loopholes to gain an upper hand. That is, we shouldn't really act surprised that they're going where the money is.

If we want them to make different choices, which of course is subjective to begin with, the system (that is, the education system) needs to provide the proper incentives to make it happen, and we need to make sure the way things work in the real world supports that system. It's easier said than done, but if we want to, we can be a part of that change.

Chez
User Rank
Agile Expert
Re: Mr. Nie : A Real Pioneer
Chez   12/27/2011 12:19:15 AM
NO RATINGS
"We've reached an interesting turning point in the Information Age when the pace of generating new information has begun to exceed the computational power of computer chips."

@
Tom,
That's the way it always is - the more we have, the more we want. It always bears mentioning that if we go back in our time machine, there's a company in the 90s that would bankrupt their whole operation to get their hands on a tenth of the analytics we have readily available now. Going along those lines, it might be easy to say that companies who are pushing beyond the bountiful resources at their disposal are overreaching and getting ahead of themselves - and some of them probably are.

It all depends on what industry you're in, and what scale of business you're talking about, though. If you don't want to take that next step, you'd better believe that someone else will... and most would rather be ahead of themselves than behind their opponent.


Chez
User Rank
Agile Expert
Re: Mr. Nie : A Real Pioneer
Chez   12/26/2011 11:03:23 PM
NO RATINGS
The industry is all hot about a BI architecture called Hadoop which can store and process both structured and unstructured data.

It's funny, I actually listened to a webcast not too long ago which talked about unstructured data and Hadoop. The consensus I heard from the experts there was that for dealing with 'big data', Hadoop was (at least a little) overhyped and overrated. They weren't being overly negative, but they were quick to point out that they thought others might be overselling it. That caught me off guard, since I'd scarcely even heard of Hadoop before then.

It's interesting how 'new' can simply mean 'something that doesn't impact my job yet', and in a different circle it's already yesterday's news. No matter how hard we try, there's always something we're not keeping up on.


Agile
User Rank
Agile Expert
Re: Re : Two Big Problems for Big Data
Agile   12/26/2011 10:54:18 PM
NO RATINGS
Great point Seth, Colleges and Universities will soon be offering specialized degrees to supply this void of skilled worker.  Much like what happened with the Management Information Systems Degree that was created 15 or so years ago which sought develop business majors who could streamline business processes through the use of computing and technology.

WriteSeth
User Rank
Agile Expert
Re: Re : Two Big Problems for Big Data
WriteSeth   12/24/2011 6:28:43 PM
NO RATINGS
Universities are starting to catch up and offering courses such as Data Analysis with Computer Applications.  We may see Decision Science majors transform into a data scientist degree of see schools creating another major or offering certificate programs like they do for project management and paralegal studies. 

Until then, companies may have to have their own special training such as financial companies often train employees to get a secruities brokers liscense in order to fill in the gap of experience. 

Anand
User Rank
Business Processor
Re: Mr. Nie : A Real Pioneer
Anand   12/23/2011 1:13:50 AM
NO RATINGS
Now the reverse is true again, suggesting the need for a increase in speed that is, by an order of magnitude, far faster than Moore's Law would predict.

@Tom, I agree with you. Already the trasistor's are so small that its very hard to shrink them further. I guess in coming years transistro shink will not follow Moore's law. New technologies like quantum computing or nano-technology might totally change the way we do computing itself. I think only way to improve the computing speed is by using lot of parallel processing techniques.


Anand
User Rank
Business Processor
Re : Two Big Problems for Big Data
Anand   12/23/2011 1:05:48 AM
NO RATINGS
The US is currently experiencing an acute shortage of mathematicians and others trained in related fields such as statistics.

@Cassimir, very informative blog. I totally agree with your observation that US is experiencing an acute shortage of mathematicians and others trained in related fields such as statistics. Infact this trend can be observed in India as well. Many people prefer engineering/Medical careers over core subjects like Maths, Chemistry and Physics. One of the major reasons people prefer engineering is because high salaries assosciated with IT field. And those students who dont get engineering/medical seats opt for core subjects.

Cassimir Medford
User Rank
Blogger
Re: Mr. Nie : A Real Pioneer
Cassimir Medford   12/21/2011 12:50:32 PM
NO RATINGS
@ Agile Expert - Thank you. Norman Nie has all the bases covered: academics, business, and broad real-world experience in behavioral data analytics, which is the centerpiece of Big Data. I learned a lot from him.

I suspect what we will see initially in the enterprise is the splitting of functions between the traditional BI, data warehousing hub and the Big Data, unstructured hub. Those with traditional BI needs will query the former database while the trained Big Data crowd will do the large-volume computations on the newer systems. Ulimately the Big Data tools will probably get better and replace the traditional SQL legacy systems. The industry is all hot about a BI architecture called Hadoop which can store and process both structured and unstructured data.

Page 1 / 2   >   >>
More Blogs from Cassimir Medford
Hadoop is stirring talk of replacing the relational database. But for now, small steps are in order.
It may be a game-changing big data tool, but Hadoop's adoption in the IT mainstream has been slowed by the fact that it's a solution to a still-emerging problem.
In addition to reinventing TV as we know it, cable companies are transforming advertising by reaching into the cloud.
If they embrace the potential of cloud technology and eschew the failures of the Tech TV movement, cable companies will have a chance to reinvent TV as we know it.
Business Agility Twitter Feed
HOME  |  BLOGS  |  MESSAGE BOARDS  |  AUDIO  |  VIDEO  |  EXECUTIVE FORUMS  |  WEBCASTS  |  RESOURCES  |  ABOUT US