Cilium, with Thomas Graf from Cisco - a podcast by Ben Pfaff

Interview with Thomas Graf of Cisco, regarding the Cilium project.

Cilium is a “science project” that Thomas and others at Cisco and
elsewhere are hacking on, to address the question of how to addresspolicy in a legacy-free container environment that scales to millions of
endpoints. It's an experiment because the outcome isn't yet certain, andit's a question that hasn't seen much work outside of hyperscale
providers.

Cilium is based oneBPF, a
Linux kernel technology that introduces the ability for userspace toinject custom programs into the kernel using a bytecode analogous to Java
virtual machine bytecode. Cilium uses eBPF-based hooks can interceptpackets at various places in their path through the kernel to implement a
flexible policy engine.

Topics include:

• How Chris Wright encouraged Thomas to become involved with OpenvSwitch, when Thomas was at Red Hat.
• The important differences between containers and VMs (quantity,
  duration of workloads, and frequency of state changes) and how Ciliumaddresses these issues.
• The toolchain that Cilium uses to generate a eBPF program customized
  for the policy of each container in a minimal and complete way.
• The potential to bypass kernelsk_buffoverhead by
  intercepting packets before these data structures are constructed, viaExpress Data Path (XDP), leading to the potential for DPDK-like packet
  forwarding performance without ever leaving the kernel. What's thedrawback? We don't know yet how it will play out.
• The main benefit of getting XDP-based early access to packets is to
  avoid the main Linux networking code paths, which are optimized fordelivery to socket buffers (taking advantage of segmentation offload)
  as opposed to forwarding.
• Dropping packets quickly is important because many operators are under
  attack all the time.
• Importance of performance for small and large packets.
• Languages available for writing eBPF: C via LLVM/Clang and Python
  (among others?).
• Limitations due to the verifier (primarily restrictions on loops), and
  how to work around them.
• How Cilium generates its eBPF programs: a base C program, plus an agent
  in Go that generates a C header file. Cilium compiles the C program toeBPF bytecode, using LLVM, then loads it into the running kernel with
  thetcutility.
• Potential for difficulty in getting a compiler toolchain into
  production deployments. Is the simplicity of supplying Cilium as acontainer image that builds in the toolchain an advantage?
• How often eBPF programs need to be recompiled in Cilium.
• eBPF “map” data structures that can be shared among eBPF programs,
  the rest of the kernel, and userspace.
• Policy in Cilium. The basic idea is that whoever specifies Cilium
  policy should not have to understand traditional networking conceptslike IP addresses ands port. Instead, abstract labels specify which
  classes of containers can talk to each other.
• Lessons learned from policy in Cilium.
• How Cilium does datapath packet processing, how it passes labels from
  source to destination, and where it applies policy.
• The direction in which Cilium points for eBPF support in Open vSwitch:
  first, it shows that it is possible; second, it shows that the tracingbuffer mechanism available from eBPF is a potential replacement for
  Open vSwitch “upcalls” currently implemented via Netlink; third, itpoints out an alternative for the flow-based model. (Does it makes
  sense to implement OVN directly via code generation?)
• Connection tracking in eBPF.
• eBPF helper functions in Linux, and the limitations of the current
  ones.
• Potential for applying eBPF to other targets such as DPDK or the OVS
  port to Hyper-V.
• Performance penalty for eBPF versus native code.
• Early controversy in the kernel community over eBPF when it was
  introduced.
• What's next for Cilium: load balancing, IPsec, IPv4 .

More information about Cilium:slidesand thecode repository.

You can find Thomas on theovs-dev mailing
list,@tgraf__on Twitter,
or on Facebook.

OVS Orbit is produced byBen Pfaff. The
intro and bumper music isElectro
Deluxe, featuring Gurdonack, copyright 2014 by My Free Mickey. The
outro music isGirls like
you, featuring Thespinwires, copyright 2014 by Stefan Kartenberg.
All content is licensed under a Creative CommonsAttribution 3.0
Unported (CC BY 3.0)license.

Further episodes of OVS Orbit

Further podcasts by Ben Pfaff

Website of Ben Pfaff