Wednesday, July 10, 13
a history 1 Wednesday, July 10, 13
serving billions
a history of requests
with Node.js 2 Wednesday, July 10, 13
I write JavaScript, ActionScript & Ruby. hola.
MY NAME IS TOM I work in advertising tech in NYC.
3 Wednesday, July 10, 13
Not a talk about Node.js performance.
4 Wednesday, July 10, 13
5 Wednesday, July 10, 13
6 Wednesday, July 10, 13
headline call-to-action
a static ad. 7 Wednesday, July 10, 13
dunce headline call-to-action
static ads are stupid. 8 Wednesday, July 10, 13
Help Scarecrow!
Click to donate brain »
static ads have no brain. 9 Wednesday, July 10, 13
Dynamic creative provides turn-key solutions based on geography, weather, time-of-day ... 10 Wednesday, July 10, 13
headline
USA
ESP
call-to-action
football
fútbol
on sale now!
¡ya a la venta! 11
Wednesday, July 10, 13
headline
USA
UK
call-to-action
football
football
on sale now!
on sale now! 12
Wednesday, July 10, 13
headline call-to-action
bathing suit
rain jacket
on sale now!
on sale now! 13
Wednesday, July 10, 13
headline
7:00 a.m.
7:00 p.m.
call-to-action
wake up!
happy hour!
1 € café »
1 € cañas » 14
Wednesday, July 10, 13
Dynamic creative provides turn-key solutions based on geography, weather, time-of-day ... 15 Wednesday, July 10, 13
DynamicSpeedy creative provides
caching
turn-key solutions
GloballyHighlydistributedbased on available* geography, weather, time-of-day ... * Like most businesses, HA is generally important for ad tech; missed impressions are expensive. 16 Wednesday, July 10, 13
Static Content
Dynamic Content Geo-distribute
Serve static files off of a global CDN.
Wednesday, July 10, 13
thousands of requests per second.
DO. NOT. CRASH.
headline
1
call-to-action
2 CDN
18 Wednesday, July 10, 13
TL;DR 19 Wednesday, July 10, 13
Start small.
TL;DR New is hard. JIFASNIF*
* Javascript Is Fun And So Node Is Fun 20 Wednesday, July 10, 13
Start small. 21 Wednesday, July 10, 13
Node + Redis is fast! It all makes sense!
This stuff is fun! 22 Wednesday, July 10, 13
“Seed” dataset over HTTP
23 Wednesday, July 10, 13
csv.on('data', writeToRedis);
24 Wednesday, July 10, 13
“Seed” dataset over HTTP
BUFFERING crashed app 25 Wednesday, July 10, 13
“Seed” dataset over HTTP n > REDIS_QUEUE_LIMIT
STREAMS to the rescue 26 Wednesday, July 10, 13
csv.on('data', writeToRedis);
27 Wednesday, July 10, 13
function handleDrain(){ csv.resume(); } function handleCSVData(d, i) { if (redis.queue < LIMIT) writeToRedis(d, i); else csv.pause(); } csv.on('data', handleCSVData); redis.on('drain', handleDrain); Resources: https://github.com/substack/stream-adventure https://github.com/substack/stream-handbook 28 Wednesday, July 10, 13
29 Wednesday, July 10, 13
whoa. 30 Wednesday, July 10, 13
with a relatively
LIMITED
whoa. hardware stack
NODE
allows us to build
valuable things. 31
Wednesday, July 10, 13
New is hard. 32 Wednesday, July 10, 13
the v1 application
33 Wednesday, July 10, 13
v2: improve workflow. alpha
beta
staging
34 Wednesday, July 10, 13
production
v2: decrease latency.
35 4 Wednesday, July 10, 13
CDN
CDN
36 Wednesday, July 10, 13
The product is used by everyone. harder, better, faster, ... complex
DOCUMENT & EXPLAIN Existing team doesn’t have more hours. 37 Wednesday, July 10, 13
... test, Test, TEST! v1 to v2 was a complete re-write — impossible without tests. As team grew, tests selfdocumented and allowed new developers to acclimate quickly. 38 Wednesday, July 10, 13
The v1 app was just plain JS. But as a heavy Rails shop, CoffeeScript makes a lot of sense. Embrace what makes TEAM most efficient — we moved to CoffeeScript for v2.
=> 39 Wednesday, July 10, 13
JIFASNIF. * Javascript Is Fun And So Node Is Fun† † Javascript Is Frustrating And So Node Is Frustrating
40 Wednesday, July 10, 13
wild child 41
Wednesday, July 10, 13
wild child
Error handling matters. Some issues we encountered:
• Sockets timing out on API requests • Redis write-to-slave Example:
Pass errors around and deal with them! 42
Wednesday, July 10, 13
app.get(‘/route’, cb); function cb(req, res, next) { try { var s = JSON.stringify(foo); res.send(s); } catch (e) { return next(e); } } 43 Wednesday, July 10, 13
var maybeNot = function(cb) { // ... stuff ... if (e) throw new Error(e); // ? // ... more stuff ... }; var better = function(cb) { // ... stuff ... if (e) return cb(e, null); // :) // ... more stuff ... }; 44 Wednesday, July 10, 13
wild child
Visibility is important. Think a lot about monitoring, performance testing — you’ll thank yourself later. 45
Wednesday, July 10, 13
package.json is great $ pull_latest_from_git $ npm install $ npm start
46 Wednesday, July 10, 13
$ pull_latest_from_git $ npm install $ npm start
... “dependencies” : { “bad” : “ >= 1.0.0”, “worst” : “*”, “best” : “2.5.3” }
match versions exactly! 47 Wednesday, July 10, 13
C 48
Wednesday, July 10, 13
C
int main(void) { printf(“Oh hai.”) // :( return 0; }
49
Wednesday, July 10, 13
radical 50 1 Wednesday, July 10, 13
Start small.
radical New is hard. JIFASNIF 51 1 Wednesday, July 10, 13
thank you. [email protected] @tbuchok
52 Wednesday, July 10, 13
? Questions & comments 53
Wednesday, July 10, 13
serving billions
a history of requests
with Node.js 2 Wednesday, July 10, 13
I write JavaScript, ActionScript & Ruby. hola.
MY NAME IS TOM I work in advertising tech in NYC.
3 Wednesday, July 10, 13
Not a talk about Node.js performance.
4 Wednesday, July 10, 13
5 Wednesday, July 10, 13
6 Wednesday, July 10, 13
headline call-to-action
a static ad. 7 Wednesday, July 10, 13
dunce headline call-to-action
static ads are stupid. 8 Wednesday, July 10, 13
Help Scarecrow!
Click to donate brain »
static ads have no brain. 9 Wednesday, July 10, 13
Dynamic creative provides turn-key solutions based on geography, weather, time-of-day ... 10 Wednesday, July 10, 13
headline
USA
ESP
call-to-action
football
fútbol
on sale now!
¡ya a la venta! 11
Wednesday, July 10, 13
headline
USA
UK
call-to-action
football
football
on sale now!
on sale now! 12
Wednesday, July 10, 13
headline call-to-action
bathing suit
rain jacket
on sale now!
on sale now! 13
Wednesday, July 10, 13
headline
7:00 a.m.
7:00 p.m.
call-to-action
wake up!
happy hour!
1 € café »
1 € cañas » 14
Wednesday, July 10, 13
Dynamic creative provides turn-key solutions based on geography, weather, time-of-day ... 15 Wednesday, July 10, 13
DynamicSpeedy creative provides
caching
turn-key solutions
GloballyHighlydistributedbased on available* geography, weather, time-of-day ... * Like most businesses, HA is generally important for ad tech; missed impressions are expensive. 16 Wednesday, July 10, 13
Static Content
Dynamic Content Geo-distribute
Serve static files off of a global CDN.
Wednesday, July 10, 13
thousands of requests per second.
DO. NOT. CRASH.
headline
1
call-to-action
2 CDN
18 Wednesday, July 10, 13
TL;DR 19 Wednesday, July 10, 13
Start small.
TL;DR New is hard. JIFASNIF*
* Javascript Is Fun And So Node Is Fun 20 Wednesday, July 10, 13
Start small. 21 Wednesday, July 10, 13
Node + Redis is fast! It all makes sense!
This stuff is fun! 22 Wednesday, July 10, 13
“Seed” dataset over HTTP
23 Wednesday, July 10, 13
csv.on('data', writeToRedis);
24 Wednesday, July 10, 13
“Seed” dataset over HTTP
BUFFERING crashed app 25 Wednesday, July 10, 13
“Seed” dataset over HTTP n > REDIS_QUEUE_LIMIT
STREAMS to the rescue 26 Wednesday, July 10, 13
csv.on('data', writeToRedis);
27 Wednesday, July 10, 13
function handleDrain(){ csv.resume(); } function handleCSVData(d, i) { if (redis.queue < LIMIT) writeToRedis(d, i); else csv.pause(); } csv.on('data', handleCSVData); redis.on('drain', handleDrain); Resources: https://github.com/substack/stream-adventure https://github.com/substack/stream-handbook 28 Wednesday, July 10, 13
29 Wednesday, July 10, 13
whoa. 30 Wednesday, July 10, 13
with a relatively
LIMITED
whoa. hardware stack
NODE
allows us to build
valuable things. 31
Wednesday, July 10, 13
New is hard. 32 Wednesday, July 10, 13
the v1 application
33 Wednesday, July 10, 13
v2: improve workflow. alpha
beta
staging
34 Wednesday, July 10, 13
production
v2: decrease latency.
35 4 Wednesday, July 10, 13
CDN
CDN
36 Wednesday, July 10, 13
The product is used by everyone. harder, better, faster, ... complex
DOCUMENT & EXPLAIN Existing team doesn’t have more hours. 37 Wednesday, July 10, 13
... test, Test, TEST! v1 to v2 was a complete re-write — impossible without tests. As team grew, tests selfdocumented and allowed new developers to acclimate quickly. 38 Wednesday, July 10, 13
The v1 app was just plain JS. But as a heavy Rails shop, CoffeeScript makes a lot of sense. Embrace what makes TEAM most efficient — we moved to CoffeeScript for v2.
=> 39 Wednesday, July 10, 13
JIFASNIF. * Javascript Is Fun And So Node Is Fun† † Javascript Is Frustrating And So Node Is Frustrating
40 Wednesday, July 10, 13
wild child 41
Wednesday, July 10, 13
wild child
Error handling matters. Some issues we encountered:
• Sockets timing out on API requests • Redis write-to-slave Example:
Pass errors around and deal with them! 42
Wednesday, July 10, 13
app.get(‘/route’, cb); function cb(req, res, next) { try { var s = JSON.stringify(foo); res.send(s); } catch (e) { return next(e); } } 43 Wednesday, July 10, 13
var maybeNot = function(cb) { // ... stuff ... if (e) throw new Error(e); // ? // ... more stuff ... }; var better = function(cb) { // ... stuff ... if (e) return cb(e, null); // :) // ... more stuff ... }; 44 Wednesday, July 10, 13
wild child
Visibility is important. Think a lot about monitoring, performance testing — you’ll thank yourself later. 45
Wednesday, July 10, 13
package.json is great $ pull_latest_from_git $ npm install $ npm start
46 Wednesday, July 10, 13
$ pull_latest_from_git $ npm install $ npm start
... “dependencies” : { “bad” : “ >= 1.0.0”, “worst” : “*”, “best” : “2.5.3” }
match versions exactly! 47 Wednesday, July 10, 13
C 48
Wednesday, July 10, 13
C
int main(void) { printf(“Oh hai.”) // :( return 0; }
49
Wednesday, July 10, 13
radical 50 1 Wednesday, July 10, 13
Start small.
radical New is hard. JIFASNIF 51 1 Wednesday, July 10, 13
thank you. [email protected] @tbuchok
52 Wednesday, July 10, 13
? Questions & comments 53
Wednesday, July 10, 13
