Rating is available when the video has been rented.
This feature is not available right now. Please try again later.
Published on Sep 19, 2019
RustConf 2019 - Rust for Weld, a High Performance Parallell JIT Compiler by Shoumik Palkar
Weld is an open source Rust project that accelerates data-intensive libraries and frameworks by as much as 100x. It does so by JIT-compiling a custom parallel intermediate representation and optimizing across functions within a single library as well as across different libraries, so developers can write modular code and still get close to bare metal performance without incurring expensive data movement costs. We first describe our use of Rust to build an extensible, performance sensitive compiler for Weld. In particular, we discuss challenges in writing a native code generator with LLVM in Rust, challenges in reducing JIT compilation times within our compiler, and architecting the compiler for both extensibility using Rust's powerful trait system, but also portability of our library across other languages, such as Python and C. Finally, we discuss our experiences with building a parallel runtime in Rust, and the unique challenges associated with calling unsafe Rust from JIT'd code.