Subscription preferences
Andrew Ross
BSDCan 2012
-
1
Solaris Boot Environments for FreeBSD
by Andrew Ross 598 views
reboot into different kernels and worlds
Solaris boot environments are a painless way for sysadmins to revert changes and upgrades. Solaris boot environments are a painless way for developers to test multiple kernels and worlds. FreeBSD does not offer this functionality natively.
I will describe the procedure of using boot environments on FreeBSD in detail and the due considerations. See the general idea at the link provided. -
2
Optimizing ZFS for Block Storage
by Andrew Ross 1,634 views
Optimizing ZFS for Block Storage
The ZFS file system has been heavily tuned for workloads where file rewrite activity is minimal or is aligned and sized to match ZFS's native record size. Exporting ZFS storage to block consumers, however, presents a situation where every write is rewriting an existing block, and unaligned writes incur a performance killing synchronous read.
This paper and talk presents Spectra Logic's optimizations to ZFS's data management layer (DMU) to convert the majority of these synchronous reads to be asynchronous and, for sequential access patterns, to avoid them entirely. We also describe a new scheme that allows concurrent reads to be issued through the DMU without the need to allocate a thread context for each I/O. The result, as implemented and tested using the FreeBSD operating system, is up to a five fold performance increase for unaligned write workloads and a three fold improvement for random read workloads. -
3
BSDCan 2012: Work in progress - Short stories from projects around the world
by Andrew Ross 407 views
For the sixtth year running, BSDCan will have a WIP (Works In Progress) session, with presentations on diverse topics.
The format remains essentially the same: in a one hour period, audiences are entertained and informed by a rapid fire series of short talks on interesting new or on-going work by individuals or groups. Slides aer permitted, but not obligatory; pictures are highly recommended. Topic areas include new open source software projects, works in progress for future releases of existing projects, student projects, etc. WIP topics this year may make good conference papers next year!
The number of slots is limited, and experience suggests there will be more takers than slots. Sign up well in advance to be assured a spot. Please e-mail wip@bsdcan.org to sign up. Send a one or two paragraph summary of the topic to be presented, and the names of the person(s) presenting it. Also, please give a time estimate -- typically times will be one to five minutes. The time limit will be strictly enforced -- you will be cut off if you try to run over! The WIP e-mail registration deadline is May 9, after which remaining slots (if any) may be signed up for in person. Any slides must be received by the WIP session chair by, at latest, May 12 at 11:59pm GMT. The session chair this year is George Neville-Neil. -
4
Modernising FreeBSD package management
by Andrew Ross 4,108 views
pkgng is a new package manager for FreeBSD, it aims at bringing modern package management features for FreeBSD
Pkgng is a completely new package manager rewritten from scratch. It aims at replacing the old pkg_install. It is developed on top of new libpkg which is the high level library that does all the package management, it brings new features such as safe upgrade, (multi) repository support, integrity checking and more. It has been designed to be extensible while remaining fully compatible with the current FreeBSD ports tree. -
5
Go SCTP!
by Andrew Ross 817 views
SCTP implemented in the language Go
Two technologies; both new, exciting, with lots of new features, so why not put them together and have even more fun?
This talk is about my effort to combine two (relatively) new technologies; Stream Control Transmission Protocol (SCTP) and the computer language Go, by implementing the functionality of SCTP as a library in Go. SCTP is a reliable message oriented transport protocol, has resistance against flooding and masquerade attacks and includes congestion avoidance procedures. First standerdized in October 2000 by the Internet Engineering Task Force (IETF) in RFC 2960 and later updated by RFC 4960. Go is a concurrent, statically typed compiled and garbage collected language with a syntax broadly similar to C. The initial design of the programming language Go started as an internal Google Inc. project in 2007 and was officially announced and open sourced in 2009. FreeBSD has the reference implementation for SCTP and Go is also available on this platform, so it made sense to do the first implementation on this OS.
I'm currently finishing my Masters in Computer Science (part time) and my dissertation subject is the implementing of SCTP in Go. I've already started to implement the library and hope to finish my dissertation in about 2/3 months time. The talk will be based on this work.
Outline of the talk (this roughly follows the outline of my dissertation): * Give some extra background information about Go and SCTP, describe characteristics etc. * Explain my approach on how to implement a new network library in the Language Go. * Do a comparison on how network programming is done (first TCP) in C and Go. * Show how TCP is implemented in Go. * Show how I implemented SCTP in Go. * Do a comparison on how network programming is done with SCTP in C and Go. * Demonstrate the performance differences between similar data transfer techniques of TCP, UDP and SCTP * Demonstrate the performance differences between the same data transfer techniques implemented in C and Go. * Question time -
6
Building a FreeBSD based Virtual Appliance
by Andrew Ross 331 views
The netmap framework for fast packet I/O
Dealing with millions of packets per second, as it can happen on 10 Gbit interfaces, puts under stress both hardware and software.
Most OSes (*BSD, Linux, Windows) are unable to handle more than 1Mpps per core, barely enough for MSS-sized traffic at 10 Gbit/s. After years of incremental improvements (interrupt mitigation, polling, multiqueue NICs, nic offloading) we recently took a very radical approach with the netmap framework, which completely redefines the device driver API and the interface with applications. Netmap provides 10-20x speedups for certain tasks (generators, traffic monitors, packet forwarding), and its use can help improving more common tasks (TCP) and point out other performance bottlenecks that are usually hidden by the current low speed I/O subsystems.
This talk will discuss challenges existing in the current packet I/O frameworks, describe the key ideas used by Netmap, and present the current status and future goals. -
7
OpenBSD network stack evolution
by Andrew Ross 1,060 views
Dealing with the IP checksum and the protocol checksums (foremost TCP and UDP) in the network stack is surprisingly complex. Having stumbled over an unexpected performance penalty from the IP checksum, I always had this area on my mental todo - and when we stumbled over a really nasty piece of code in pf dealing with these checksums, I re-evaluated and changed the IP checksumming in our stack, for performance and to make better use of checksum offloading to network cards. Changing the protocol checksums in the same way is harder and in the works.
ALTQ has been with us for more than a decade - last not least Kenjiro Cho and myself merged it with pf in 2003. ALTQ has always been a research project, and tought us and the entire community a lot of important lessons. Now it is time to re-evaluate - the entire "glue" between the actual queueing disciplines (of which just two remain, prio and bandwidth shaping) gets redesigned and -implemented. -
8
Building a FreeBSD based Virtual Appliance
by Andrew Ross 259 views
How we built the Razorback appliance
Razorback is a framework for an intelligence driven security solution. It consists of a large number of components and dependencies that make the barrier to deployment quiet large for the uninitiated. This talk aims to shed some light on the process of creating a virtual appliance that enabled us to reduce the barrier for people that want to test the system.
Lowering the barrier to entry for a complex project is key for improving deployment of your project, by building a virtual appliance you can cut the setup time from over a day to just a few minutes.
This tutorial aims to cover setting up a VM build environment that will allow you to create custom virtual appliances for you projects that are easy for people to deploy.
We will cover: * Setting up the build host for PXE based installation of the appliance. * Tuning the installer to install only the components that we need to the vm to function. * Deploying tinderbox to build the systems dependencies. * Installing the dependencies via the installer * Deploying freebsdadmin on the VM to provide a management interface. * Customizing the base freebsdadmin package. * Adding custom applications to freebsdadmin to manage your application.
The aim is provide a hands on experience so attendees should bring a laptop capable of running 2 small FreeBSD virtual machines. Attendees should also have some basic FreeBSD systems administration experience.
By the end of the session attendees should have a firm grasp on the process of creating a virtual appliance using the freebsdadmin project as the management interface. -
9
Ethernet Switch Framework
by Andrew Ross 380 views
Fully utilize your WLAN router
Designing and developing the Ethernet Switch Framework for FreeBSD.
FreeBSD is making great strides to be fully functional on many typical WLAN routers. Furthest along is support for devices based on the Atheros series of System-on-a-Chip products. Thanks to Adrian Chadds relentless work, many devices can be used with FreeBSD-current for routing between LAN and WLAN interface. The Ethernet Switch Framework closes one of the last remaining driver gaps to fully enable build an embedded FreeBSD version for such devices.
Currently under development, the Ethernet Switch framework enables configuration of built-in ethernet switch controllers. This allows users to create powerful networking setups without any additional hardware. Even though these routers are typically not very expensive, the switch controllers offer a number of features typically only found in more expensive enterprise equipment. This allows users to create interesting and powerful network setups at home or in small offices.
This talk will present the current state of development, the architecture of the driver framework and will detail the implementation of a typical switch driver. It will also go into some of the architectural challenges that needed to be solved to deal with hardware configurations typical for embedded systems that are uncommon in the world of regular desktop and server systems. -
10
FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices
by Andrew Ross 270 views
This paper describes the design and implementation of the FreeBSD operating system port for the QorIQ Data Path Acceleration Architecture, a family of communications microprocessors from Freescale.
These chips are a modern, multi-core, PowerPC based SoCs, which feature a number of specifically designed peripherals, addressed for the high performance networking devices, which are increasingly common in modern communication infrastructure.
The primary focus is the Data Path Acceleration Architecture (DPAA) with the new approach to network interface architecture. It has significant influence on the FreeBSD device drivers design and implementation. The paper describes how the full network functionality was brought forward, and also covers other major development tasks like the e500mc quad-core SMP bring-up and support for other integrated devices. -
11
Recent Advances in IPv6 Security
by Andrew Ross 504 views
During the last few years, the UK CPNI (Centre for the Protection of National Infrastructure) carried out the first comprehensive security assessment of the Internet Protocol version 6 (IPv6) and related technologies (such as transition/co-existence mechanisms). The result of the aforementioned project is a series of documents that provide advice both to programmers implementing the IPv6 protocol suite and to network engineers and security administrators deploying or operating the protocols. Part of the results of the aforementioned project have been recently published, leading to a number of improvements in many IPv6 implementations.
Fernando Gont will discuss the results of the aforementioned project, introducing the attendees to the "state of the art" in IPv6 security, and providing advice on how to deploy the IPv6 protocols securely. Gont will also discusss recent advances in IPv6 security areas such as Denial of Service attacks, firewall circumvention, and Network Reconnaissance, and will describe other IPv6 security areas in which further work is needed. Additionally, he will demonstrate the use of some attack/assessment tools that implement new network reconnaissance techniques or that exploit a number of vulnerabilities found in popular IPv6 implementations.
The IPv6 protocol suite was designed to accommodate the present and future growth of the Internet, and is expected to be the successor of the original IPv4 protocol suite. It has already been deployed in a number of production environments, and many organizations have already scheduled or planned its deployment in the next few years. Additionally, a number of activities such as the World IPv6 Day in 2011 and the upcoming World IPv6 Launch Day (scheduled for June 2012) have led to an improvement in IPv6 awareness and an increase in the number of IPv6 deployments.
There are a number of factors that make the IPv6 protocol suite interesting from a security standpoint. Firstly, being a new technology, technical personnel has much less confidence with the IPv6 protocols than with their IPv4 counterpart, and thus it is more likely that the security implications of the protocols be overlooked when the protocols are deployed. Secondly, IPv6 implementations are much less mature than their IPv4 counterparts, and thus it is very likely that a number of vulnerabilities will be discovered in them before their robustness matches that of the existing IPv4 implementations. Thirdly, security products such as firewalls and NIDS's (Network Intrusion Detection Systems) usually have less support for the IPv6 protocols than for their IPv4 counterparts, either in terms of features or in terms of performance. Fourthly, the security implications of IPv6 transition/co-existence technologies on existing IPv4 networks are usually overlooked, potentially enabling attackers to leverage these technologies to circumvent IPv4 security measures in unexpected ways.
During the last few years, the UK CPNI (Centre for the Protection of National Infrastructure) carried out the first comprehensive security assessment of the Internet Protocol version 6 (IPv6) and related technologies (such as transition/co-existence mechanisms). The result of the aforementioned project is a series of documents that provide advice both to programmers implementing the IPv6 protocol suite and to network engineers and security administrators deploying or operating the protocols. Part of the results of the aforementioned project have been recently published, leading to a number of improvements in many IPv6 implementations.
Fernando Gont will discuss the results of the aforementioned project, introducing the attendees to the "state of the art" in IPv6 security, and providing advice on how to deploy the IPv6 protocols securely. Gont will also discusss recent advances in IPv6 security areas such as Denial of Service attacks, firewall circumvention, and Network Reconnaissance, and will describe other IPv6 security areas in which further work is needed. Additionally, he will demonstrate the use of some attack/assessment tools that implement new network reconnaissance techniques or that exploit a number of vulnerabilities found in popular IPv6 implementations. -
12
Automated Documentation Proofreading
by Andrew Ross 127 views
igor: Making Documentation Easier
Automated Documentation Proofreading
Making documentation easier and better by automating tests for errors in language, formatting, and usage.
Few people like to work on documentation. There are numerous rules for wildly-varying documentation formats, many rarely used and hard to remember. An automatic proofreader to check for errors ranging from spelling to meeting all the arcane formatting rules of the different toolchains would relieve much of the stress. Not only will this encourage improving the documentation, it helps to prevent errors in the first place, and detect those that have slipped through already. Clean, consistent files are easier to maintain, expand, and convert to new formats. The automated proofreader, named "igor" after a famous lab assistant, helps the writer focus on improving the content of their document. -
19
BSD Multiplicity
by Andrew Ross 1,201 views
An applied survey of BSD multiplicity and virtualization strategies from chroot to BHyVe
Ever since the University of California, Berkeley CSRG implemented the chroot(8) command and system call in its BSD operating system in 1982, the community-developed BSD Unix derivatives have set the standard for the introduction of plurality to the conventionally-singular Unix computing model. Today's system operators and developers have an array of BSD-licensed multiplicity strategies at their disposal that offer various degrees of both isolation and virtualization when introducing plurality. This paper will survey current and experimental BSD multiplicity strategies including chroot, FreeBSD jail, NetBSD/Xen, Amazon EC2, compatlinux, GXemul and SIMH, plus experimental strategies such as FreeBSD BHyVe, compatmach, Usermode NetBSD, Dragonfly BSD vkernel, OpenBSD sysjail and NetBSD mult. As an applied survey, this paper will both categorize each multiplicity strategy by the Unix environment to which it introduces plurality and demonstrate the usage of the utilities relating to each solution.
The survey criteria consist of five distinct Unix environments to which plurality is introduced:
• Machine Multiplicity, as distinguished by the introduction of native and foreign hardware system and their distinct Instruction Set Architectures defined by distinct physical hardware systems, virtualized instances of them, or software emulattions of them (NetBSD-Xen/EC2, GXemul, SIMH and BHyVe)
• Kernel Multiplicity, as distinguished by a plurality of executing kernels (Usermode NetBSD, Dragonfly BSD vkernel)
• Init Multiplicity, as distinguished by distinct kernel-spawned init processes and their descendant processes (mult)
• Userland Multiplicity, as distinguished by distinct userlands with optional process tables and their descendant processes (chroot, FreeBSD jail and sysjail)
• API Multiplicity, as distinguished by distinct foreign Application Programming Interface compatibility layers (compatlinux, compatmach)
The reader will thus come away with a set of working examples for each solution that they can implement on their own.
In addition, this survey addresses key host and guest administrative considerations applicable to each multiplicity solution:
• Storage Device considerations: Are they hardware or software-based? What image and file system formats are supported?
• Network Device considerations: Are they configured by the host? From within the guest?
• Console Device considerations: Does the guest appear on the host console? Is it redirected to a network-aware solution such as VNC or X11 over SSH?
• Kernel considerations: Is the native or foreign guest kernel modified? Does it reside within or outside the guest userland?
• Userland considerations: Is the guest userland modified? Does it lend itself to customization through additive or subtractive techniques? -
20
Intro to DNSSEC
by Andrew Ross 1,967 views
Jeremy C. Reed serves on the board of directors of The NetBSD Foundation and the BSD Certification Group.
As a software packager, he has created and maintained hundreds of packages. He is employed by Internet Systems Consortium (ISC) as the BIND 10 Release Engineer and technical writer. He has taught over 60 workshops and classes covering various open source or Unix administration topics. -
22
FreeBSD Unified Deployment and Configuration Management
by Andrew Ross 334 views
Andrew Pantyukhin is a clandestine hacker at large with affinities to FreeBSD, large--scale heterogeneous computing, unconventional architectures, — currently changing life, the universe and everything in a team of like minds.
Since 2010, Andrew has been the CTO with Dream Industries, a disruption lab launching next-generation cloud media services in emerging markets around the world. His responsibility and interest is rethinking development and operations to better suit current demands and those looming on new horizons, where he draws much value from his FreeBSD experience.
He has also been an aspiring Unix hacker for the last decade, a latent FreeBSD ports committer since 2006, an architect behind Gubkin University IT operations, and enjoying occasional gigs speaking about open--source and BSD at conferences and teaching FreeBSD classes. -
23
An Overview of Locking in the FreeBSD Kernel
by Andrew Ross 280 views
The FreeBSD kernel uses seven different types of locks to ensure proper access to the resources that it manages. This talk describes the hierarchy of these locks from the low-level and simple to the high-level and full-featured.
The functionality of each type of lock is described along with the problem domain for which it is intended. The talk concludes by describing the witness system within the FreeBSD kernel that tracks the usage of all the locks in the system and reports any possible deadlocks that might occur because of improper acquisition ordering of locks. -
24
auditdistd - Secure and reliable distribution of audit trail files
by Andrew Ross 138 views
Security Event Audit is a facility to provide fine-grained, configurable logging of security-relevant events. Audit events are stored in trail files that can be used for postmortem analysis in case of system compromise. Once the system is compromised, an attacker has access to audit trail files and can modify or delete them. The auditdistd daemon's role is to distribute audit trail files to a remote system in a secure and reliable way.
The talk will provide background to the Security Event Audit facility in FreeBSD and will describe auditdistd daemon in detail. The auditdistd daemon is a good example of using modern sandboxing mechanisms, like capsicum. During the talk audit subsystem and auditdistd daemon will be presented live. -
25
NetBSD/mips
by Andrew Ross 407 views
Since NetBSD 5 was released, the support for MIPS on NetBSD has been completely revamped. It is now one of the more advanced ports of NetBSD. This talk is an overview on what has changed and what the current state of MIPS support and a brief look forward to what else is coming.
Subjects to be covered:
Why? (Big Embedded space, large amounts of memory, etc). Quick Introduction to the MIPS architecture
Overview of XLR/XLS/XLP
Overview of what changed (toolchain, SMP, pmap, PCU, compat32, new cpu support, use of MIPS features, fast softint) Design decisions
why N32 by default?
why no separate mips64?
Major features
64-bit address space
cpu abstraction
dynamic fixups (changing indirect calls to direct calls)
splsw
UVM changes
Fast software interrupts
SMP (for NetLogic XLR/XLS/XLP)
mostly lockless pmap
Choosing a new page size
COMPAT_NETBSD32 networking filesystem mounting
32-bit systems
N32 Kernels
Effects on the NetBSD in general
PCU
direct-mapped UAREAs
COMPAT_NETBSD32
common pmap for TLB based MMUs
What coming?
MIPS MTE support (MultiThread Extension)
Support for other MP MIPS chips (maybe) -
26
pfSense 2.1: IPv6 and more
by Andrew Ross 3,956 views
pfSense is a BSD licensed customized distribution of FreeBSD tailored for use as a firewall and router. In addition to being a powerful, flexible firewalling and routing platform, it includes a long list of related features and a package system allowing further expandability without adding bloat and potential security vulnerabilities to the base distribution. This session is being presented by the founders of the pfSense project, Chris Buechler and Scott Ullrich.
At the time of BSDCan 2012, pfSense 2.1 will be newly released. This release adds IPv6 support to nearly every portion of the system, as well as some other smaller changes. This session will cover all of the changes in the 2.1 release, primarily focusing on adding IPv6 support to your existing deployments. With IPv4 address space dwindling and World IPv6 Launch coming up in June, now is the time to bring up IPv6 on your networks. While IPv6 brings new benefits, it also poses new security and connectivity considerations, which will be covered. Attendees will come away with all the latest on the project, as well as knowledge to securely bring their networks onto the 21st century Internet with IPv6. -
27
An Introduction to Verifiedexec in NetBSD
by Andrew Ross 149 views
The verifiedexec feature has been part of NetBSD for some years now. It seems that a lot of people are unaware of the feature or do not know the full capabilities of verifiedexec. This talk will introduce the feature, what it can do and also what it could be capable of with some kernel changes.
The verified execuction feature is a unique extension to the NetBSD kernel that allows an administrator to ensure the binaries and files that are being accessed have not been modified by comparing the fingerprint of the on-disk file with a "known good" copy of the fingerprint kept in kernel memory. This allows very fine grain control over what will be executed on the machine, even by root, and can provide assurance that files have not been modified. In this talk I will go over some of the history of verified execution, how it works and what it can do, then finally move on to what the next steps I want to take in the development of veriexec. Verified execution has been in NetBSD for a long time but it seems to be a feature that that is not widely known about, hopefully this talk can raise its profile somewhat.
