THE NEW WORLD OF BIG DATA
Dr. Hongfei Guo ( 郭鸿飞 博士)
Senior Program Manager, Microsoft SQL Server Big Data Team
微软 SQL Server 大数据研发组 高级项目经理
大数据的新世界
价格低廉的存储
价格低廉的计算
设备爆炸
主要趋势 KEY TRENDS
社交网络
无处不在的连接 传感器网络
BIG DATA
Audio / Video
Log Files
Text/Image
Social Sentiment
Data Market Feeds
eGov Feeds
Weather
Wikis / Blogs
Click Stream Sensors / RFID / Devices
Spatial & GPS Coordinates
WHAT IS BIG DATA?
WEB Mobile
Advertising Collaboration eCommerce
Digital Marketing
Search Marketing
Web Logs
Recommendations
ERP / CRM
Sales Pipeline
Payables
Payroll
Inventory
Contacts
Deal Tracking
Terabytes
(10E12)
Gigabytes
(10E9)
Exabytes
(10E18)
Petabytes
(10E15)
复杂性： 种类和速度 Complexity: Variety & Velocity
V
o
lu
m
e
数
据
量
一系列新问题 A NEW SET OF QUESTIONS
社交网络和互联网分析
What’s the social
sentiment
for my brand or
products ?
我的品牌或产品
情绪
实时数据源
How do I optimize my
fleet based on weather
and traffic patterns?
如何优化我的车队运行
(基于天气和交通趋势）
高级分析功能
How do I better
predict future
outcomes?
如何更好预测未来
结果？
大数据， 大商机
BIG DATA, BIG OPPORTUNITY
49% of top CEOs and
CIOs are currently using Big
Data for customer analytics1
软件增长Software Growth
0
5
2012 2013 2014 2015
B
il
li
o
n
s
$
34%
compound
annual growth
rate
服务增长Services Growth
0
5
10
2012 2013 2014 2015
B
il
li
o
n
s
$
39%
compound
annual growth
rate
1. McKinsey&Company, McKinsey Global Survey Results, Minding Your Digital Business, 2012
2. IDC Market Analysis, Worldwide Big Data Technology and Services 2012–2015 Forecast , 2012
大数据生命周期 THE BIG DATA LIFECYCLE
管理 Manage 丰富 Enrich 洞察力 Insight
管理任何种类、任何大小、 任何来源的数据
MANAGE ANY DATA, ANY SIZE, ANYWHERE
非关系型
Non-Relational
数据流
Streaming
010101010101010101
1010101010101010
01010101010101
101010101010
统一监控、 管理和安全
Unified Monitoring, Management & Security
数据移动 Data Movement
关系型
Relational
HADOOP 集成 HADOOP INTEGRATION
企业级安全，高可靠性，管理 Enterprise class security, HA & management
与微软商业智能工具无缝集成 Seamlessly integrated with Microsoft BI tools
SQL Server 数据平台的一部分 Delivered as part of the SQL Server Data Platform
在Windows Azure上几分钟内完成部署 Provisioned in minutes on Windows Azure
开放和灵活 OPEN & FLEXIBLE
与Apache Hadoop 100%兼容
100% compatible with Apache
Hadoop
工具由丰富的合作伙伴生态系
统提供 Tools from a rich
ecosystem of partners
与社区的紧密合作
Built with close community
collaboration
Accelerating the delivery of
Hadoop for Windows
Hadoop for Windows
JavaScript libraries
Hive ODBC drivers
The Apache Software Foundation
大数据生命周期 THE BIG DATA LIFECYCLE
管理 Manage 丰富 Enrich 洞察力 Insight
连接数据集市产生更多价值
数据整合带来的价值
POWER OF COMBINING THE WORLDS DATA
Personal Data
个人数据
Organizational
Data
组织数据
Community
Data
社区数据
World Data
世界数据
Value
价值
数据集市 DATAMARKET
WINDOWS AZURE MARKETPLACE
大数据生命周期 THE BIG DATA LIFECYCLE
管理Manage 丰富Enrich 洞察力Insight
对任何种类、任何大小、任何来源数据的洞察力
INSIGHTS ON ANY DATA, ALL USERS, WHEREVER THEY ARE
Streaming
Non-Relational
010101010101010101
1010101010101010
01010101010101
101010101010
Relational
数据科学家
Data Scientists
商业智能专业人员
BI Professionals
业务分析人员
Business Analysts
通过熟悉的工具，为所有用户提供对数据的洞察力
INSIGHTS FOR ALL USERS THROUGH FAMILIAR TOOLS
数据科学家
Data Scientists
PB
商业智能专业人员
BI Professionals
TB
业务分析人员
Business Analysts
GB
Advanced Analytics
from Microsoft and
3rd parties
Self Service Analysis
with PowerPivot &
Power View
Interactivity &
exploration with
Hadoop data in Excel
“
”
客户示例 Customer Example
Connects to more than 1 billion signals
连接到超过 10 亿的信号/数据源
Across 15 leading social networks, including Facebook
排名前15位的社交网络，包括Facebook
Generates a ‘Klout’ score for individual people, brands & partners
为个人、 品牌及合作伙伴生成一个 'Klout' 分数
Enables analysis, targeting and social graphs
提供分析、 目标和社交图
端到端的大数据解决方案
BIG DATA REQUIRES AN END-TO-END APPROACH
洞察力
INSIGHTS
数据管理
DATA
MANAGEMENT
NON-RELATIONAL
1
0
0
1 1
1
RELATIONAL STREAMING
丰富数据
DATA
ENRICHMENT SHARE
AND GOVERN
DISCOVER
AND RECOMMEND
TRANSFORM
AND CLEAN
• SELF-SERVICE
• COLLABORATIVE
• MOBILE
• REAL-TIME
微软大数据
MICROSOFT BIG DATA
洞察力
INSIGHTS
数据管理
DATA
MANAGEMENT
丰富数据
DATA
ENRICHMENT
PowerPivot Power View
Hadoop on Windows
端到端架构案例
WELL-HEAD ANALYSIS WORKLOAD
10101 SCOOP
Cold
Stream
HDFS
Hot
Stream
E=MC2
HQL
Learned
Limits
SQL
Distributed Storage
(HDFS)
HADOOP 架构HADOOP ARCHITECTURE
Distributed Processing
(Map Reduce)
