Building Software at Google Scale Tech Talk

Google Tech Talk
March 21, 2012

Presented by
Michael Barnathan, Software Engineer, Google
Greg Estren, Software Engineer, Google
Pepper Lebeck-Jobe, Software Engineer, Google

ABSTRACT

At past Google NYC Tech Talks, we learned about tools that helped Google engineers automate quality testing, so that Google products could be released frequently without extensive manual testing phases or manual invocations of tools like JUnit, WebDriver, or JavaScript Test Driver.

This talk covers the Google Build System, which Google engineers use to build software from a unified, language-agnostic, continuously integrated code base, quickly and at scale. When a developer initiates a build, the build system automatically computes the minimal number of artifacts that need to be built and determines the optimal strategy for producing them as fast as possible using the resources of many worker machines. On average, each build request triggers thousands of source file compilations, while still completing within seconds.

At Google, all software components are compiled from source, in a highly parallelized fashion, possibly across thousands of machines dedicated to software compilation. Build artifacts that compose software components are also shared across build requests, such that if a developer builds a component and another developer builds a similar component, the artifacts in common between them are not built twice.

This talk will discuss in detail how all this "magic" works. More than just a time-saver for our developers, the high performance and unprecedented scale of our build system make possible far-reaching quality goals, such as continuously testing Google's entire codebase at every revision.

This talk will be given by three engineers on the build system team.

• Category

  Science & Technology

[Bionuker]

Seriously google, fix the audio of your talks! So many incredibly interesting talks are being ruined by horrible audio. Hire some guys who know recording, mics, compressors, gates, equalizers. It's worth the investment.

[enforce007]

the audio is very poor + its very nice that people as questions in the middle of the talk and we cant hear them... so annoying

[slvher Liu]

build large targets under 20 seconds? very cool !

[riz chelsea]

Thanks for sharing this video.

[U6D65]

The last question, about open sourcing it, was not answered. And its interesting that they show gcc in examples while i was thinking that google now uses Clang as a C++ compiler.

[Tim Lesher]

I'd rather see a poor video of a good talk than a perfect recording of a bad one.

[rubbernozed]

Is there a way to assign a portion of the transcribing to viewers... that would be a great way to increase the accuracy of audio captioning, yes?

[tigerex]

PPT Please ! i'll take it from there

[enforce007]

on the other hand its very very interesting :D

[pa1pa1]

how can you possibly mess up the audio? my god !

[bobdeuel]

apples and oranges dude