APIs, Actors, and Abstractions in a Distributed System

Find Session: http://sched.co/71Se

Orchestration is the act of managing services. Before 2005 it was mostly by hand using a combination of scripts and commands.

  • A system that provides control over high level abstractions.
  • Has nothing to do with containers/VMs/processes. All about abstraction.
  • Copes with flexibility and complexity

After 2005 Configuration management takes off with Puppet, Chef, Ansible, eat.

Abstrations

Force Multiplying ideas

Control loops are the platform's automata, they react to the state and change on notification.
Event driven agents coordinate changes in the system. The goal being to maintain a desired state.

Patterns

Event driven agents that coordinate changes
  • Sate Observation
  • Entity Lifecycle graph
  • Registration and Discovery
    • Route to an IP
    • Engine in a cluster
    • Replica of a service
    • Endpoint of a service

What breaks

Common issues

  • Still need high-low entropy service discovery and routing
  • Needs to be able to tie into existing infrastructure

Networking Stress

  • N machines and M containers on an overlay or virtual network. The traffic can be overwhelming.
  • Container attachment/detachments
  • Node attachment/detachment
  • Heartbeats/monitoring (Nagios)

Platform Lifecycle

  • Upgrade machines?
    • Carefule Dance? Will upgrading a host break the orchestration?
    • Is rollback possible?
    • Blue/Green?

For a orchestrated container testing. Force latency delay, network outages, ensure the container can take a beating.
HTTPS://github.com/buildertools/entropy