Build data – Explore the fun of big data with the new generation of intelligent cloud crawlers
Big Data
The word is said to come from Alvin Toffler’s 1970s book the Third Wave. The dead | alvin toffler: how to resolve the impact of the future
Although big data is a general concept, the topic about big data and big data processing and analysis continues to heat up recently, and now it has basically become a new round of topics at the level of industrial revolution.
What is big data? As a data acquisition team, we have been thinking about what is big data and what are the prospects and values of big data for a long time. In this article, I will share my thoughts and interesting content and resources on:
- What is big data
- Big data practices
- Application scenarios of big data
With all the talk of layoffs these days, if you want to know if the wave of Internet layoffs is really having a lasting negative impact on payrolls, you can take a look at our tool to generate a list of lists compiled several times a day on a regular basis.
(I) What is big data
Listen to the cognoscenti: Big data is more, it is more. The original equipment can not be saved or calculated. ———— Pop pineapple · Picasso
Big data, not random samples, but all data; Not precision, but promiscuity; Not causation, but correlation. — — Schonberger
Ted: Kenneth Cukier: Big Data is Better Data
America’s favorite pie is? Audience: Apple. Kenneth Cukier: Apple. Of course it is. How do we know it? Because of data. You look at supermarket sales. You look at supermarket sales of 30-centimeter pies that are frozen, and apple wins, no contest. The majority of the sales are apple. But then supermarkets started selling smaller, 11-centimeter pies, and suddenly, apple fell to fourth or fifth place. Why? What happened? Okay, think about it. When you buy a 30-centimeter pie, the whole family has to agree, and apple is everyone’s second favorite. (Laughter) But when you buy an individual 11-centimeter pie, you can buy the one that you want. You can get your first choice. You have more data. You can see something that you couldn’t see when you only had smaller amounts of it.
Apple pie was once thought to be the most popular pie, but when you have more detailed data, it turns out that its popularity is actually a compromise: apple pie is everyone’s second favorite flavor.
And when you look at the numbers for smaller pies, the apple pie actually comes in fourth or fifth.
You have more data, you can see information that you couldn’t see before.
What is the core value of big data? The structure of this post is very clear and it offers a very clear discussion of the positive implications of big data.
Big data sounds great, but is it really great? – Artificial intelligence – Zhihu @Chen Mengmeng here also said very well, I wonder if she is really an AI.
What is the core value of big data? – Business – Zhihu, this is still the problem, @Liu Fei’s article.
Big data is the collection of big data
The big data industry itself is a service industry that relies on data sources.
The most fundamental aspect of big data lies in the significant changes and innovations in the way information is collected. The emergence of big data is closely related to the direct presentation of a large amount of information on the network.
Joel Selanikio:Transcript of ” The big-data revolution in healthcare”
There’s a concept that people talk about nowadays called “big data.” And what they’re talking about is all of the information that we’re generating through our interaction with and over the Internet, everything from Facebook and Twitter to music downloads, movies, streaming, all this kind of stuff, the live streaming of TED. And the folks who work with big data, for them, they talk about that their biggest problem is we have so much information. The biggest problem is: how do we organize all that information?
Now everyone talks about big data, but what they’re really talking about is facebook, Twitter, streaming and so forth, and the big data people think we have too much data.
(Organizing information is still the hardest problem.)
I can tell you that, working in global health, that is not our biggest problem. Because for us, even though the light is better on the Internet, the data that would help us solve the problems we’re trying to solve is not actually present on the Internet. So we don’t know, for example, how many people right now are being affected by disasters or by conflict situations. We don’t know for, really, basically, any of the clinicsin the developing world, which ones have medicines and which ones don’t. We have no idea of what the supply chain is for those clinics. We don’t know — and this is really amazing to me — we don’t know how many children were born — or how many children there are — in Bolivia or Botswana or Bhutan. We don’t know how many kids died last week in any of those countries. We don’t know the needs of the elderly, the mentally ill. For all of these different critically important problems or critically important areas that we want to solve problems in, we basically know nothing at all.
Much of the data available is still completely off the web and relies on primitive methods to collect it. There are fundamental problems with data that are evident in many areas.
What are the “magic” ways to get data? – Liu Cao’s answer – Zhihu see here a recommended answer @Liu Cao. Lanceyan’s blog – Technology Sharing Framework Exchange Big data processing architecture building robot strongly recommended: How to describe the technology ecology of big data with image metaphor? What is the relationship between Hadoop, Hive, and Spark? The @ Xiaoyu Ma
(2) Practical tools of big data
See here: What tools are commonly used for big data analysis? – JavaScript – Zhihu recently saw an example of how Pokemon Go has changed the amount of exercise players do:
1. Examples of data analysis in the application:
After six months, the majority of Pokemon Go players are doing the same amount of exercise as non-players. It seems to be a pretty effective game.
2. Examples of big data analysis of traffic conditions:
Susan Etlinger: What do we do with all this big data?
Now, there’s a group of data scientists out of the University of Illinois-Chicago, and they’re called the Health Media Collaboratory, and they’ve been working with the Centers for Disease Control to better understand how people talk about quitting smoking, how they talk about electronic cigarettes, and what they can do collectively to help them quit. The interesting thing is, if you want to understand how people talk about smoking, first you have to understand what they mean when they say “smoking.” And on Twitter, there are four main categories: number one, smoking cigarettes; number two, smoking marijuana; number three, smoking ribs; and number four, smoking hot women.
This is very interesting
(3) Application scenarios of big data
First post two news observations:
Beijing-tianjin-hebei industry development status report | | data view | China big observation data industry _ data portal view | China big observation data industry _ data portal
Nowadays, big data is getting more and more attention in terms of policy and national strategy.
- Application scenarios are now distributed in:
- Supply chain and channel analysis & optimization
- Pricing analysis and optimization
- Fraud analysis & Detection
- Equipment management
- Social Media Analytics & Customer Analytics
Victor, the author of the book The Age of Big Data, believes there are three major changes in the age of Big data:
“One, we can analyze more data, and sometimes even process all the data related to a particular phenomenon, rather than relying on random sampling. Greater accuracy allows us to discover more details. Second, there is so much data that we are no longer keen on accuracy. A modest neglect of precision at the micro level leads to better insight and greater business benefits. Third, instead of looking for causality, we should find correlations between things. For example, don’t explore why airline ticket prices change, but focus on the best time to buy a ticket.” Big data breaks the boundary of traditional enterprise data and changes the situation that business intelligence only relies on internal business data of enterprises in the past, while big data makes data sources more diversified, including not only internal data of enterprises, but also external data of enterprises, especially data related to consumers
According to wild records, khwarazm, an ancient central Asian kingdom, had a strange custom in which messengers who brought good news to the king were promoted and those who brought bad news to the king were fed to tigers. People used to criticize the king for his naivete, thinking that he would encourage good news by rewarding those who brought it, and eliminate bad news by killing those who brought it.
In today’s age of information explosion, we can’t always ask messengers to deliver good news, but you can ask our crawlers to send you the most useful and appropriate information on a regular basis. Zoom-a new generation of intelligent cloud crawlers