Cloud ComputingCloud Computing
@Yahoo!
Raghu Ramakrishnan
@Yahoo!
Raghu Ramakrishnan
Yahoo! Fellow
Chief Scientist, Search and Cloud Platforms
1
,
(Many slides courtesy of others at Yahoo!)
Cloud Services @Y!: Use Cases
Content Search Index
Optimization
Search Index
Machine
Learning
Ads
Optimization
Learning
(. Spam filters)
Attachment
Storage
Image/Video
Storage &
Delivery
g
Yahoo! Data Scale
Massive user base and engagement
• 640M+ unique users, 11B page visits/monthq p g
• Hundreds of petabytes of storage
• Hundreds of billions of objects
• Hundreds of thousands of requests/sec, 200B events/day, 200 PB/day
Global
• Tens of globally distributed data centers
• Serving each region at low latencies
Challenging Users
• Rapidly extracting value from voluminous data
• Downtime is not an option (outages cost $millions)
• Variable usage patterns
“Just look at our homepage, for example. Since we began pairing our
content optimization technology with editorial expertise, we’ve seen
click-through rates in the Today module more than double. And
we’re making additional improvements to this technology that will make
the user experience ever more personally relevant.”
Carol Bartz, Analyst Call, January 27, 2010
CONTENT OPTIMIZATION
FOR PORTALSFOR PORTALS
4
Yahoo! Front Page
Product Objective Prioritize small pool of editorially programmed packages to optimize engagement in real-timep g g
Key Features
Package Ranker (CORE)g ( )
Ranks packages by expected CTR based on data
collected every 5 minutes
Dashboard (CORE)
Provides real-time insights into performance by package,
t d tsegment, and property
Mix Management (Property)
Ensures editorial voice is maintained and user gets a
variety of content
Package rotation (Property)Package rotation (Property)
Tracks which stories a user has seen and rotates them
after user has seen them for a certain period of time
Key Performance Indicators
Lifts in quantitative metricsRecommended links News Interests Top Searches
Editorial Voice Preserved+160% clicks
vs. one size fits all
+79% clicks
vs. randomly selected
+43% clicks
vs. editor selected
Content Optimization & Cloud
Online Learning
O li i d l
Offline Modeling
• Online regression models,
time-series models
Model the temporal dynamics
g
• Exploratory data analysis
• Regression, feature selection,
collaborative filtering (factorization)
• Model the temporal dynamics
• Provide fast learning for per-item models
• Seed online models & explore/exploit
methods at good initial points
f
Near real-time user feedback
• Reduce the set of candidate items
Explore/Exploit
• Multi-armed bandits
• Find the best way of collecting real-
time user feedback (for new items)
Large amount of
historical data
(user event streams)(user event streams)
Data Management in CORE
1) User click history logs
stored in HDFS
2) Hadoop job builds
d l fmodels of user
preferences 3) Hadoop reduce
writes models to
Sherpa user table 4) Models read fromp 4) Models read from
Sherpa influence users’
frontpage content
HDFS
C did tCandidate
content
Matching Users to Content
We learn how user attributes correlates with engagement in each item
Default Male Female 18-24 25-34 Heavy
Sports
+ + +
6 8 +1 0 -1 0 +0 2 +0 3 +2 + + + +
+ + +
0 0
+
We compute rankings for each user based on his/her attributes
CORE Dashboard: Overall CTR
Compare performance of models and historical benchmarksCompare performance of models and historical benchmarks
Compare
buckets and
models over
time
See which
content was
promoted mostpromoted most
across time
Compare Co pa e
bucket
metrics
COKE Dashboard: Segment Heat Map
Examples
• ACQUISITION: A “Star Trek” package was #3 with 18-20 demo, #2 with 21-24 demo, p g , ,
but #9 overall. We can acquire younger audiences with targeted content like this.
• ENGAGEMENT: “Kobe’s astonishing shot” was #25 with women, but #5 with men. We
can better engage men (or sports fans) by showing more like this, women by showing
less.
• REACH: A package about a hair-pulling soccer player was just plain interesting to
everyone (#1-3). We can maintain reach by programming content for the mass
diaudience.
11
NEXT-GEN SEARCH
12
Web of Concepts
conceptrich, aggregated data
madonna
mumbai
restaurant
jsan jose
Aggregated KB INDEX SERP
The “index” is keyed by concept instance, and organizes all
relevant information (data describing the concept instance
d i l i hi h i ) h i i dand its relationship to other instances), wherever it is drawn
from, in semantically meaningful ways
Web IE: Surfacing, Extraction, Integration
Traditional
Integration
Traditional
g
End to Traditional
Extraction
End-to-
End
Surfacing
WWW
15
DATA MANAGEMENT IN
THE CLOUD
Requirements for Cloud Servicesq
• Multitenant A cloud service must support multiple organizationallyMultitenant. A cloud service must support multiple, organizationally
distant customers.
• Elasticity. Tenants should be able to negotiate and receive
resources/QoS on-demand up to a large g
• Resource Sharing. Ideally, spare cloud resources should be
transparently applied when a tenant’s negotiated QoS is insufficient,
., due to spikes.
• Horizontal scaling. The cloud provider should be able to add cloud
capacity in increments without affecting tenants of the service.
• Metering. A cloud service must support accounting that reasonably
ib ti l d it l dit t h f th t tascribes operational and capital expenditures to each of the tenants
of the service.
• Security. A cloud service should be secure in that tenants are not
made vulnerable because of loopholes in the cloudmade vulnerable because of loopholes in the cloud.
• Availability. A cloud service should be highly available.
• Operability. A cloud service should be easy to operate, with few
operators Operating costs should scale linearly or better with the
19
operators. Operating costs should scale linearly or better with the
capacity of the service.
Yahoo! Cloud Stack
EDGE
Horizontal Cloud Services …YCS YCPI Brooklyn
EDGE
i
t
y
WEB
l
f
‐
s
e
r
v
e
)
n
g
/
S
e
c
u
r
i
Horizontal Cloud ServicesVM/OS yApache
WEB
PHP App Engine
o
n
i
n
g
(
S
e
g
/
M
e
t
e
r
i
n
Horizontal Cloud ServicesVM/OS …
APP
H
i
g
h
w
a
y
Serving Grid
P
r
o
v
i
s
i
o
M
o
n
i
t
o
r
i
n
g
Horizontal Cloud Services…PNUTS/Sherpa MOBStor
OPERATIONAL STORAGE
D
a
t
a
H
M
Horizontal Cloud Services…Hadoop
BATCH STORAGE
A Data-Centric View
Cloud Serving
Applications
Services
Sherpa, MobStorData
C ll i
p ,
Structured, Unstructured
Storage
Collection
Hadoop
Analytics
21
Cloud Data Management
•CRUDSherpa •CRUD
•Point lookups and short
scans
I d i d t bl d
p
Structured
Record
Storage •Index organized table and
random I/Os
Storage
Yahoo!
CloudHadoop MobStorHadoop
Large Data
Analysis
MobStor
Large Blob
Storage
•Scan oriented workloads
•Focus on Sequential disk I/O
•Object retrieval and streaming
•Scalable file storage
22
q g
22
The Yahoo! Data Cloud
application
1234323,
transportation
5523442,
childcare
DECLARE DATASET Listings AS
( ID String PRIMARY KEY,
Category String,
32138,
camera
ALTER Listings
transportation,
For sale: one
bicycle, barely
used
childcare,
Nanny
available in
San Jose
g y g
Description Text )
camera,
Nikon
D40,
USD 300
Simple Web Service API’s
MAKE CACHEABLE
MObStorForeign keyH d
Data
Serving
Sherpa
Search
Vespa
Media
Storage
MObStorForeign key
photo → listingData
Analytics
Hadoop
Caching
memcached
23
Messaging
Tribble
Batch export
1 illi j b th
Hadoop Core
(Core, Pig, Oozie,
Hive, Howl)
Ad BT and Inventory prediction, Content
Agility, UDA, COKE, Mail Spam, Search,
APG, Labs, Insights, Analytics
1+ million jobs per month
PB processed daily
90B events and 120 TB daily
70+ PB of Data
Map-Reduce and more …
70+ PB of Data
HADOOP:
SCALABLE ANALYTICS
Map Reduce and more …
SCALABLE ANALYTICS
One Slide Hadoop Primer
Data file HDFS
HDFS
Good for analyzing (scanning) huge files
Not great for serving (reading or writing individual objects)
Reduce tasks
Map tasks
Reduce tasks
Industry Challenge: Massive Data Sets
Being AccumulatedBeing Accumulated
“It’s now the industrial revolution of data” –
Joe Hellerstein, UC Berkeley
Massive datasets of highly-
leveragable data amassed by
relatively few players
1 “By 2020, the Digital Universe will be 44
times as large as it was in 2009” [IDC]
Deriving insights from the newly
available mountains of data still 2
challenging
Datasets become competitiveDatasets become competitive
differentiators
Search query history,
Advertising/click through data
3
Advertising/click-through data,
Surfing histories, Social graphs,
News, Finance, Sports and other
data feeds
4/1/201126Yahoo!’s technology solution: Hadoop
Hadoop: Stability at Scale
Hadoop powers the Yahoo! Network: must be rock‐solid
90
80
250
38K Ser ers80
70
200
38K Servers
170 PB Storage
1M+ Monthly Jobs
We fix bugs before you see them
• We run very large clusters
• We have a large QA effort
60
50
150
f
S
e
r
v
e
r
s
t
e
s
Hadoop Servers
Hadoop Storage (PB)
• We run a huge variety of workloads
40
30
100
o
u
s
a
n
d
s
o
f
P
e
t
a
b
y
t
Science Impact
The Yahoo! Distribution of Hadoop
• We contribute our work to Apache
• We share the exact code we run
20
10
50
T
h
o
Research
We share the exact code we run
• We don’t sell software or service
27
0 0
2006 2007 2008 2009 2010
Today
CASE STUDY
YAHOO! MAILYAHOO! MAIL
Enabling quick response in the spam arms race
• 450Mmail boxes• 450M mail boxes
• 5B+ deliveries/day
• Antispam models retrained
SCIENCE
• Antispam models retrained
every few hours on Hadoop
40% less spam than
Hotmail and 55% less
th G il
“PRODUCTION
spam than Gmail
“
28
Example: User Activity Modeling
Large dimensionality vector describing possible user activities
But a typical user has a sparse activity vectorBut a typical user has a sparse activity vector
Att ib t P ibl V l T i l lAttribute Possible Values Typical values per
user
Pages ~ MM 10 – 100g
Queries ~ 100s of MM Few
Ads ~ 100s of thousands 10s
Hadoop pipeline to model user interests from activities
29
Feature and Target Windows
T
Query Visit Y! finance Event of interest
T0
Time
Event of interest
Moving Window
Feature Window Target Window
30
30
Feature Window Target Window
User Modeling Pipeline
Component Data Processed TimeComponent Data Processed Time
Data Acquisition ~ 1 Tb per time 2 – 3 hoursData Acquisition 1 Tb per time
period
2 3 hours
Feature and Target ~ 1 Tb * Size of 4 - 6 hoursFeature and Target
Generation
1 Tb Size of
feature window
4 6 hours
Model Training ~ 50 - 100 Gb 1 – 2 hours for 100’s
of models
Scoring ~ 500 Gb 1 hour
31
A Growing User Base
Year: 2007
Year: 2008
Year: 2009 - 2010Year: 2009 2010
32
Y!OS, COKE, LocDrop, Video, Media
Search history, Answers, Messenger,
BOSS, Image Search, Blog Search
15K requests per second
Over records; 10sTB of data
ACID or BASE? Litmus tests are colorful, but the picture is cloudy
PNUTS:
SCALABLE DATA SERVING
ACID or BASE? Litmus tests are colorful, but the picture is cloudy
SCALABLE DATA SERVING
Requirements for Cloud Servicesq
• Multitenant A cloud service must support multiple organizationallyMultitenant. A cloud service must support multiple, organizationally
distant customers.
• Elasticity. Tenants should be able to negotiate and receive
resources/QoS on-demand up to a large g
• Resource Sharing. Ideally, spare cloud resources should be
transparently applied when a tenant’s negotiated QoS is insufficient,
., due to spikes.
• Horizontal scaling. The cloud provider should be able to add cloud
capacity in increments without affecting tenants of the service.
• Metering. A cloud service must support accounting that reasonably
ib ti l d it l dit t h f th t tascribes operational and capital expenditures to each of the tenants
of the service.
• Security. A cloud service should be secure in that tenants are not
made vulnerable because of loopholes in the cloudmade vulnerable because of loopholes in the cloud.
• Availability. A cloud service should be highly available.
• Operability. A cloud service should be easy to operate, with few
operators Operating costs should scale linearly or better with the
34
operators. Operating costs should scale linearly or better with the
capacity of the service.
The World Has Changed
Web serving applications need:g pp
• Scalability!
– Elastic on demand, commodity boxes
• Flexible schemasFlexible schemas
• Geographic distribution/replication
• High availability
• Low latency• Low latency
Web serving applications willing to doWeb serving applications willing to do
without:
• Complex queries
• ACID transactions
– But still benefit from support for data consistency
Typical Y! Applications
User logins and profilesg p
• Including changes that must not be lost!
– But single-record “transactions” suffice
Events
• Alerts (., news, price drops)
• Social network activity (., user goes offline)
• Ad clicks, article clicks
Application specific dataApplication-specific data
• Postings in message board
• Uploaded photos tagsUploaded photos, tags
• Shopping carts
640M+ unique users, 11B pages/month
Hundreds of petabytes of storage
Hundreds of billions of objects
Hundred of thousands of requests/secHundred of thousands of requests/sec
Global, rapidly evolving workloads
What is PNUTS/Sherpa?
CREATE TABLE Parts (
A 42342 E
B 42521 W
C 66354 W
ID VARCHAR,
StockNumber INT,
Status VARCHAR
…A 42342 EB 42521 W
E 75656 C
C 66354 W
D 12352 E
F 15677 E
)
h
Structured, flexible
h
B 42521 W
C 66354 W
D 12352 E
E 75656 C
F 15677 E schemaschema
E 75656 C
A 42342 E
B 42521 W
C 66354 W
D 12352 E
database
Parallel
database replication
Geographic
replication
E 75656 C
F 15677 E
Hosted, managed infrastructure
37
PNUTS: Key Components
• Maintains map from C h th f th TC
VIP
T bl t
p
-to-
tablet-to-SU
• Provides load balancing
• Caches the maps from the TC
• Routes client requests to
correct SU
Table: FOO
Tablet
ControllerRouters
1 2 n
Key JSON
Tablet 1
Tablet 2
Table: FOO
1
3
Key JSON
Key JSON
Key JSON
Key JSON
Key JSON
Key JSON
Tablet 3
Tablet 4
Tablet 5
5
2
9
Storage
Units
Key JSON Key JSON Key JSON
Tablet 5
Tablet M
9
n• Stores records in tablets
Units• Services get/set/delete
requests
38
Architecture
Local region Remote regions
Clients
REST API
Routers
Tablet Controller
Tribble
StorageStorage
units
39
Flexible Schema
Posted date Listing id Item Price
6/1/07 424252 Couch $570
Color Condition
Good6/1/07 424252 Couch $570
6/1/07 763245 Bike $86
6/3/07 211242 Car $1123 Red
Good
Fair
6/5/07 421133 Lamp $15
Updates
1
Write key k
8
Sequence # for key k ySequence # for key k
Routers
Message brokers
2
Write key k7
3
Write key k
4Write key kSequence # for key k
SU SU SU
5
SUCCESS
6 Write key k
41
Tablets—Ordered Table
Name Description Price
Apple
Avocado
Apple is wisdom
But at what price?
$1
$3
Name Description Price
A
Banana
Grape
Avocado
Grapes are good to eat
But at what price?
The perfect fruit
$
$2
$12Grape
Kiwi
Lemon
Grapes are good to eat
How much did you pay for this lemon?
New Zealand
$
$8
$1
H
Orange
Lime
Lemon
Limes are green
Arrgh! Don’t get scurvy!
How much did you pay for this lemon? $1
$9
$2Orange
Strawberry
Tomato
Strawberry shortcake
Arrgh! Don t get scurvy!
Is this a vegetable?
$
$900
$14
Q
42
Tomato g ? $
Z
Range Queries in YDOT
Clustered, ordered retrieval of records,
Storage unit 1Apple
A dG f ?
Storage unit 1
Canteloupe
Storage unit 3
Lime
Storage unit 2
Avocado
Banana
Blueberry
Canteloupe
Grapefruit…Pear?
Grapefruit…Lime?
Canteloupe
Storage unit 3
Lime
Storage unit 2
Strawberry
Storage unit 1
Router
Canteloupe
Grape
Kiwi
Lemon
Li
Lime…Pear?
Strawberry
Storage unit 1
RouterLime
Mango
Orange
StrawberryApple CanteloupeLimeStrawberryy
Tomato
Watermelon
Apple
Avocado
Banana
Blueberry
Canteloupe
Grape
Kiwi
Lemon
Lime
Mango
Orange
Strawberry
Tomato
Watermelon
Storage unit 1 Storage unit 2 Storage unit 3
ELASTICITY, OPERABILITY,
HORIZONTAL SCALINGHORIZONTAL SCALING
44
Distribution
Bike $866/2/07 636353
Couch $5706/1/07 424252
Car $11236/1/07 256623
Distribution for parallelismData shuffling for load balancing
Bike $866/2/07 636353
Chair $106/5/07 662113
Lamp $196/7/07 121113
Bike $566/9/07 887734
Scooter $186/11/07 252111
Hammer $80006/11/07 116458 Hammer $80006/11/07 116458
45
Server 1 Server 2 Server 3 Server 4
Tablet Splitting and Balancing
Each storage unit has many tablets (horizontal partitions of
the table)
g y ( p
the table)
hotspot
Storage unit may become a
hotspot
St itStorage unit Tablet
time
Tablets may grow over
timesplit
Overfull tablets
split
46
pp
versShed load by moving tablets to other servers
ASYNCHRONOUS REPLICATION
AND CONSISTENCY
47
Asynchronous Replication
48
Consistency: Social Alice
West East Record Timeline
User Status
West East
Busy
Record Timeline
Alice Busy Busy
Network disruption:
Alice redirected to East
User Status
Alice FreeUser Status
Free
User Status
Alice Busy
busy
User Status User Status Free
y
free
Alice ??? Alice ???
Free
PNUTS Consistency Model
W iWrite
Current
version
Stale versionStale
version
Timev. 1 v. 2 v. 3 v. 4 v. 5 v. 7Generation 1 v. 6 v. 8
A hi d i d i t lAchieved via per-record primary copy protocol
(To maximize availability, record masterships automatically
transferred if site fails)
Can be selectively weakened to eventual consistency
50
Can be selectively weakened to eventual consistency
(local writes that are reconciled using version vectors)
Record Master
A 42342 E
B 42521 W
C 66354 WC 66354 W
B 42521 E
A 42342 E
D 12352 E
E 75656 C
F 15677 E A 42342 E
B 42521 W
C 66354 W
A 42342 E
B 42521 E
C 66354 W
D 12352 E
E 75656 C
F 15677 E
A 42342 E
B 42521 W
C 66354 W
D 12352 E
E 75656 C
F 15677 E
C 66354 W
D 12352 E
E 75656 C
F 15677 E
B 42521 W
C 66354 W
D 12352 E
E 75656 C
F 15677 E
51
Consistency Techniques
Per-record mastering
• Each record is assigned a “master region”
– May differ between records
• Updates to the record forwarded to the master region
• Ensures consistent ordering of updates
Tablet-level mastering
• Each tablet is assigned a “master region”
• Inserts and deletes of records forwarded to the master region
• Master region decides tablet splits
These details are hidden from the application
• Except for the latency impact!
Consistency Techniques
Primary Key Constraint + Record Timeline
o Each tablet is assigned a “master region”
o Inserts of records forwarded to the master region
o Inserts and updates could fail during outages*
e
n
c
y
Record Timeline Consistency
C
o
n
s
i
s
t
o Each record is assigned a “master region”
o Updates to the record forwarded to the master region
i
l
i
t
y
C
o Inserts succeed, but updates could fail during outages*
Eventual ConsistencyA v
a
i
l
a
b
i
Eventual Consistency
o Low latency updates and inserts done locally
o Per field timestamp used to merge updates
A
o Per field timestamp used to merge updates
53
In case of SU or data center failure. We have failover tools!
Reads always will be sent to another region
Tablet Master
Key2: 42521
Region W
Region C Region E
Key1 42342 E
Tablet master
Region C g
Key3 66354 W
Key4 12352 E
Key5 75656 C K 1 42342 E
Key1 42342 E
Key3 66354 WKey5 75656 C
Key6 15677 E
Key1 42342 E
Key3 66354 W
Key4 12352 E
Key3 66354 W
Key4 12352 E
Key5 75656 C
Key6 15677 EKey4 12352 E
Key5 75656 C
Key6 15677 E
y
54
Tablet Mastership
Tablet master
Step 1: Forward
Region W Region C Region E
Key1 42342 E Key1 42342 E Key1 42342 E
Step 1: Forward
Req to Tablet MasterKey3 66354 W
Key4 12352 E
Key5 75656 C
Key6 15677 E
Key3 66354 W
Key4 12352 E
Key5 75656 C
Key6 15677 E
Key3 66354 W
Key4 12352 E
Key5 75656 C
Key6 15677 E
Step 2: Apply
Insert to Tablet Master
y y y
Key1 42342 E
Key2 42521 W
Key3 66354 WKey3 66354 W
Key4 12352 E
Key5 75656 C
Key6 15677 E
Key1 42342 E
Key2 42521 W
Key1 42342 E
Key2 42521 W
Step 4: Apply
Insert at Rec Master
Step 3: Replicate
Insert to Other Sites
Key3 66354 W
Key4 12352 E
Key5 75656 C
Key6 15677 E
Key3 66354 W
Key4 12352 E
Key5 75656 C
Key6 15677 E
55
Key6 15677 E Key6 15677 E
Generalizing Record Timelines to
P titi Ti liPartition Timelines
Record Partitition of records with same key
T bl t lit t t titi b d i• Tablet splits must respect partition boundaries
• Intra-partition ACID transactions can be done easily now
Si l hi t ti !– Single machine transactions!
– With composite keys, this captures Azure and Google AE
modelsmodels
• Each partition is assigned a “master region”
– May differ between partitionsy p
• Updates to the partition forwarded to the master region
• Ensures consistent ordering of updates across nodesg p
AVAILABILITY
57
Possible Failure Modes
Failure type
Storage unit
Consistency impact Storage
None
Availability impact
units
X
Degraded service (forwards) for some data.
Updates and inserts fail for some records
ResolutionResolution
If data not lost: Reboot machine
If data lost: Copy lost tablets from a remote replicapy p
Time to resolve
If data lost, hours or less (depending on tablet size
and colo location). If no data lost, minutes.
Coping With Failures
A 42342 E
B 42521 W
C 66354 W
XX
D 12352 E
E 75656 C
F 15677 E A 42342 E
B 42521 W
C 66354 W
OVERRIDE W → E
C 66354 W
D 12352 E
E 75656 C
F 15677 E
A 42342 E
B 42521 WB 42521 W
C 66354 W
D 12352 E
E 75656 C
F 15677 E
59
Possible Failure Modes
Failure typeFailure type
Router
Routers
X
Consistency impact
None
Routers
Availability impact
None
Resolution
Boot router
Time to resolve
Minutes
Possible Failure Modes
Failure
Tablet controller XConsistency impact
None
Tablet controllerX
Availability impact
Some actions (., tablet
copy) will be blockedcopy) will be blocked
Resolution
Start secondary controllerStart secondary controller
Time to resolve
MinutesMinutes
Possible Failure Modes
Failure
One msg hub node
X
Consistency impact
None
Msg Hubs
Availability impact
Writes fail for some records until a new
secondary node takes oversecondary node takes over
Resolution
Create new primary or secondary for lost p y y
topics
Time to resolve
Minutes
Possible Failure Modes
Failure
Colo power outage or partition
Clients
Consistency impact
Option to allow “relaxed consistency”
Routers
Tablet map
WS API
p y
to improve availability
Availability impact
Some inserts pdates and
Tablet controller
Load balancer
Server monitor
Tribble
Some inserts, updates and
deletes cannot succeed
Some critical reads fail
Storage
units
SU API
Option to allow updates to proceed in
“relaxed consistency mode”
Resolution units
Major overrides to force mastership
transfer; discard conflicting updates
Time to resolveTime to resolve
Hours
YCS Benchmark Tool
Java application
Man s stems ha e Ja a APIs• Many systems have Java APIs
• Other systems via HTTP/REST, JNI or some other solution
Command-line parameters
• DB to use
T t th h t• Target throughput
• Number of threads
• …
Workload
parameter file
• R/W mix
YCSB client
l
i
e
n
t
Client
u
d
D
B
• Record size
• Data set
• …
D
B
c
Client
threads
Stats
Workload
executor
C
l
o
u
Extensible: plug in new clients
Extensible: define new workloads
Further Reading on PNUTS
Efficient Bulk Insertion into a Distributed Ordered Table (SIGMOD 2008)
Adam Silberstein, Brian Cooper, Utkarsh Srivastava, Erik Vee,
Ramana Yerneni, Raghu Ramakrishnan
PNUTS: Yahoo!'s Hosted Data Serving Platform (VLDB 2008)
Brian Cooper, Raghu Ramakrishnan, Utkarsh Srivastava,
Adam Silberstein Phil Bohannon Hans Arno JacobsenAdam Silberstein, Phil Bohannon, Hans-Arno Jacobsen,
Nick Puz, Daniel Weaver, Ramana Yerneni
Asynchronous View Maintenance for VLSD Databases (SIGMOD 2009)y ( )
Parag Agrawal, Adam Silberstein, Brian F. Cooper, Utkarsh Srivastava and
Raghu Ramakrishnan
Cl d St D i i PNUTSh ll (B tif l D t O’R ill M di 2009)Cloud Storage Design in a PNUTShell (Beautiful Data, O’Reilly Media, 2009)
Brian F. Cooper, Raghu Ramakrishnan, and Utkarsh Srivastava
Adaptively Parallelizing Distributed Range Queries (VLDB 2009)Adaptively Parallelizing Distributed Range Queries (VLDB 2009)
Ymir Vigfusson, Adam Silberstein, Brian Cooper, Rodrigo Fonseca
A Batch of PNUTS: Experiences Connecting Cloud Batch and
Serving Systems (SIGMOD 2011)
Adam Silberstein et al.
Summary
Y h ! h t i l d ti• Yahoo! has an extensive cloud computing
environment
– Major open source contributor
– Currently, focused on internal y,
developers
Foundation layer for all Yahoo!– Foundation layer for all Yahoo!
products, ., Mail, Front Page,
Search GroupsSearch, Groups …
4/1/2011Yahoo! Presentation Template, Confidential
