For the RaspberryPI Kubernetes deployment experiment that’s serving this page, I wanted to use Varnish for a simple static asset cache. I’m not sure why I decided to go down this path, because this site isn’t exactly high traffic, but ¯\_(ツ)_/¯.

After about 30 seconds of looking around I couldn’t find a pre-built ARM64 docker image for Varnish… so I built one.

Building the image

This was about as much as creating a image repository on DockerHub, cloning this, and running:

$ docker buildx build --platform linux/arm64 . -t zackkitzmiller/varnish-arm:alpine --push

The image is now here on DockerHub.

update: Here’s the Elixir configuration file with websocket support

vcl 4.0;

import directors;
import std;

backend z19rpw {
  .host = "z19rpw-service";
  .port = "80";

sub vcl_recv {
  if (req.http.upgrade ~ "(?i)websocket") {
    return (pipe);

  # normalize the port for testing/staging
  set req.http.Host = regsub(req.http.Host, ":[0-9]+", "");

  # pass on the real IP address here as X-Forwarded-For
  if (req.restarts == 0) {
      if (req.http.X-Forwarded-For) {
          set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
      } else {
      set req.http.X-Forwarded-For = client.ip;

  if (req.http.Host ~ "") {
    std.log("setting backend to z19rpw");
    set req.backend_hint = z19rpw;

  if (req.method != "GET" && req.method != "HEAD") {
    return (pass);

  if (req.url == "/") {
    std.log("attempting to cache home");
    unset req.http.Cookie;
    return (hash);

  if (req.url ~ "\.(png|jpg|jpeg|css|js)") {
    std.log("processing for z19r assets");
    unset req.http.Cookie;
    return (hash);

  std.log("not an asset request");
  return (pass);

sub vcl_deliver {
  if (obj.hits > 0) {
    set resp.http.X-Cache = "HIT";
  } else {
    set resp.http.X-Cache = "MISS";
  return (deliver);

sub vcl_backend_response {
  if (bereq.url ~ "\.(png|jpg|jpeg|css|js)" || bereq.url == "/") {
    unset beresp.http.set-cookie;

  if (bereq.url ~ "^/$") {
    # Software pages are purged explicitly, so cache them for 48h
    set beresp.http.Cache-Control = "max-age=3600";
    set beresp.ttl = 1h;

  return (deliver);

sub vcl_pipe {
  if (req.http.upgrade) {
    set bereq.http.upgrade = req.http.upgrade;
    set bereq.http.connection = req.http.connection;

  return (pipe);

The lengths New Yorkers will go for a drink, the Raine’s Sandwich.

By the following weekend, a vanguard of downtown saloon-owners were gleefully testing the law’s limits. A suspicious number of private “clubs” were founded that April, and saloons started handing out membership cards to their regulars.

Less than a week this lasted. Everything great about NYC goes back as far as NYC is old.

Law enforcement declared itself satisfied. “I would not say that a cracker is a complete meal in itself, but a sandwich is,” an assistant D.A. in Brooklyn told an assembly of police captains as the first Raines hotels sprouted up. Remarkably, the courts upheld these definitions of “meal” and “guest.”

This attitude is still prevalent in NYC. It will never be impossible to find a bar in the five boroughs to find a drink. Whisky Town has frozen Hot Pockets, but no operational microwave, BGH has their assortment of potato chips, and of course, McSorley’s, which has to try pretty hard to get shut down as it is.

If you need to use the Mnesia library for cluster syncing make sure that you change the backing store for Pow here.

defmodule Z19rpwWeb.Credentials do
  @moduledoc "Authentication helper functions"

  alias Z19rpw.Users.User
  alias Phoenix.LiveView.Socket
  alias Pow.Store.CredentialsCache

  @doc """
  Retrieves the currently-logged-in user from the Pow credentials cache.
  @spec get_user(
          socket :: Socket.t(),
          session :: map(),
          config :: keyword()
        ) :: %User{} | nil

  def get_user(socket, session, config \\ [otp_app: :z19rpw])

  def get_user(socket, %{"z19rpw_auth" => signed_token}, config) do
    conn = struct!(Plug.Conn, secret_key_base: socket.endpoint.config(:secret_key_base))
    salt = Atom.to_string(Pow.Plug.Session)

    with {:ok, token} <- Pow.Plug.verify_token(conn, salt, signed_token, config),
         {user, _metadata} <-
           CredentialsCache.get([backend: Pow.Store.Backend.MnesiaCache], token) do
      _any -> nil

  def get_user(_, _, _), do: nil

I’m putting this here because I can never find it when I google it.

did you just tell me to go fuck myself

gpg-agent --daemon --enable-ssh-support --write-env-file $HOME/.gpg-agent-info

if test -e \"$HOME/.gpg-agent-info\"
     set arrayName (cat \"$HOME/.gpg-agent-info\")
     for line in $arrayName
         set environment_var (echo $line | cut -d= -f1)
         set environment_val (echo $line | cut -d= -f2)
         set -x $environment_var $environment_val

set -x GPG_TTY (tty)    
  • Fast Food
  • Papa Johns, Dominos, Little Ceasers, Pizza Hut
  • Standard Midwest
  • Linos, Ittza Pizza, etc. Find more examples. Almost never a chain
  • Brick Oven
  • Tobys, Motorino, Stone Eagle Tavern
  • NYC
  • Thin, sold by the slice (probs shouldnt include the dollar slice)
  • Chicago
  • Uno, Giadarno
  • Deep Dish
  • Old Chicago

.travis.yml ended up looking like this:

language: go

go: 1.3.1

  - export PATH=$PATH:$HOME/gopath/bin
  - go get
  - godep restore

  - go test
  - test -z \"$(go fmt ./...)\"

edit: Pull Request to travis-build here.<br> edit 2: Pull Request Accepted! TravisCI will now install Godeps without editing a .travis.yml

Things at SeatGeek

@zack  0 2014-02-13 12:12

These are some of the things we use, most of them daily:

  • PHP
  • Symfony
  • Python
  • Tornado
  • Flask
  • Ruby
  • Rails
  • Nginx
  • Sass
  • Brunch
  • Bower
  • node.js (assets only)
  • CoffeeScript
  • Javascript
  • Vagrant
  • VMWare
  • HA Proxy
  • ElasticSearch
  • RDS
  • S3
  • EC2
  • Sixpack
  • Graphite
  • StatsD
  • MongoDB (defunct)
  • MySQL
  • Redis
  • TOR
  • RabbitMQ
  • Leaflet
  • backbone.js
  • underscore.js
  • lodash
  • Logstash
  • Kibana
  • NewRelic
  • Varnish
  • Circus
  • Postgres

Im sure there are more, Ill update this list.

I dip everything. Always. Everytime. You can ask anyone that has ever eaten a meal with me. So, naturally, I make dips all the time and bring them to parties. This is my most requested dip.


  • 12 oz dark beer<sup>1</sup>
  • 1 stick butter
  • 1 package cream cheese
  • 1 Kielbasa, or other smoked sausage
  • Hot Sauce
  • 1 tbsp brown mustard
  • 1 or 2 Jalepeno (or habenaro) - optional<sup>2</sup>
  • 4 cups extra sharp cheddar cheese
  • Some bread or something for dipping<sup>3</sup>


  1. Preheat oven to 350 degress
  2. Slice and dice kielbasa and hot peppers, pan fry.
  3. Drain the fat from the kielbasa, turn heat down to simmer/low.
  4. Mix beer, butter and cream cheese in a large pot over medium heat until smooth. Stiring constantly
  5. Mix in mustard and hot sauce to taste <sup>4</sup>
  6. Slowly mix in the sharp cheddar cheese, continue stiring, again, untill smooth. <sup>5</sup>
  7. Mix in kielbasa and peppers
  8. Pour entire mixture into a large pyrex baking dish, and bake at 350 for 30-40 minutes
1 2 3 4 5

Just pushed and tagged Sixpack version 1.1. This build includes the following:

  • BUG CSV export should respect KPI setting
  • ENHANCEMENT Better handling of traffic fractioning
  • FEATURE Configuation by environment variables
  • NEW iOS Client
  • DOCS Multi-armed bandit information.

You can get the new version via PyPi, or GitHub Releases. For more information checkout the official Sixpack site.


Yeah, I know. Thats a mouth full, perils of webscale, I guess.

I was recently having a conversation with Taylor Otwell on twitter about consuming Laravel queue messages in other languages, and figured Id quickly throw together a little proof of concept.

You can see all of the source code here.

There is an almost fresh laravel install, with a single artisan command email:send. The command simply uses Queue:push to put a message in a beanstalkd queue. The app does nothing else, but one could image its a fully functional app undergoing a refactor.

There is also a small go application here. This worker listens to the beanstalk queue with one goroutine, and sends email with another. The goroutines communicate through a single channel. Ive included the complied go binary in case someone wants to play with this without getting a go environment setup.

The source for the go application is relativly small, and should be pretty readable. Feel free to ask questions in the comments if something isnt entirely clear.

If you want to play with this, youll need a valid sendgrid account and Beanstalkd running and the following env vars set:

  • BEANSTALKD: location of the beanstalk installation (e.g. localhost:11300)
  • SENDGRID_USER: username for sendgrid account
  • SENDGRID_PASS: password for sendgrid account

Update: Extra special thanks to the SendGrid team for provisioning a test account early on a saturday morning.

Tiny PHP

@zack  0 2013-11-30 18:06

Today, Id like to annouce my new PHP Micro-Framework, Tiny.

Ha. Just kidding.

When I was building Forrst with Kyle, we knew that we didnt want to expose various primary keys, like post_ids, and user_ids, etc, on the site. The solution was to make URLs that looked like

We used a library from our friend, Jacob, that allowed us to obfuscate primary keys simply by base62 encoding them. I find myself using this library constantly, constantly copying and pasting from Kyles github account, where the code has been sitting untouched for literally years. I finally made it composer-installable.

You can find it here. I hope you can find it as useful as I do.

edit: Just to clarify, the tiny-ified ID wasnt stored anywhere, it was always calculated. We used methods something like:

class Post {
    public function getUrl() {
       return $this->getSlug() . - . $this->getTinyId();

edit 2: This could also be used to build something like a URL shortener, as $tiny->to(99999999); would actually output jpWDw

Turns out upgrading to Mavericks breaks a few things, afterall. One error ran into today was #include \"libxml/xmlversion.h not found trying to install lxml.

Simple Fix: reinstall XCode Developer Tools with

$ xcode-select –install

You should probably just do that anyway.

words in word

@zack  0 2013-10-16 11:11

My son, Jayden (8 years old), had a homework assignment yesterday to find words made up of the letters of another word (e.g. pat in part). So, instead of thinking, we wrote this:

from itertools import permutations
import sys

word = sys.argv[1]

words = open(/usr/share/dict/words).read().split(\"\n\")
valid = set()

for k in range(1, len(word)):
    for combination in permutations(word, k + 1):
        to_check = \"\".join(combination)
        if to_check in words:

 print list(valid)

Use like:

$ python part

He also now has some understanding of strings and lists in python.

close enough

go fish

@zack  0 2013-10-13 14:02

go fish

Ive been playing with Go quite a bit lately, and found there to be quite a bit of boilerplate to get started with a new project. So I thew together gofish. Its a virtualenvwrapper inspired wrapper for working on Go projects in the Fish shell.

If youve used virtualenvwrapper (or virtualfish) the commands will be familiar to you.

To create a new go workspace:

$ mkgoworkspace project-name

This will created a project structure that looks like this:

/bin /pkg /src /zackkitzmiller /project-name main.go

The default namespace will be the currect users username, but that can be overridded by setting the GOFISH_NAMESPACE environment variable:

$ set -x GOFISH_NAMESPACE; and mkgoworkspace project-name

To switch the GOPATH:

$ goon project-name

To deactive a workspace completely:

$ gooff

The various commands all emit events, so you can listen for them in your fish config. The available events to listen to are:

  • gofish_workspace_will_be_created
  • gofish_workspace_created
  • gofish_workspace_will_activate
  • gofish_workspace_activated
  • gofish_workspace_will_deactivate
  • gofish_workspace_deactivated

The following example would send a growl notification on workspace activation

function announce –on-event gofish_workspace_activated command growlnotify -m \”Workspace: \” (basename $GO_CURRENT_WORKSPACE) \” activated\” -n gofish end

The following global environment variables will be available after a successful workspace activation:

  • $GOPATH - as expected

This is still just a work in progress. Let me know what you think. And as always, pull requests welcome.


  • multiple modules in same workspace
  • auto git?

git legit

@zack  0 2013-10-10 16:04

Seems that Kenneth Reitz nailed it again. Its been around for > 2 years, but this is such a neat little tool. Wish I would have found it earlier.


@zack  0 2013-09-30 12:12

Last week I was hanging out at the OnDeck Hackathon in NYC. After not doing much for a while, I decided Id give myself an hour to build something, and unlike all of my previous hackathon projects, I wanted to actually deploy this to the internets, so I did.

So, what I built was venuestagram. Its a silly little app that pulls in all popular venues from the SeatGeek API and use the lat/lng I got to query the instagram API. I built it with Flask, and this python instagram wrapper I found.


Heres a cute little pattern I threw together when writing sixpack. Im assuming youre using Werkzeug, but you could do the same thing with Flask, or pure WSGI.

First step is to write a little decorator that youre going to decorate your route handlers with. Its pretty straight forward. Here I used the decorator package to simplify things, but you can just do it in pure python if you dont need the additional dependency.

import decorator

from redis import ConnectionError

def service_unavailable_on_connection_error(f, *args, **kwargs):
    return f(*args, **kwargs)
except ConnectionError:
    return json_error({\"message\": \"redis is not available\"}, None, 503)

Its simply going to try to run the method you decorated, and catch a Redis ConnectionError. You could do this for any datastore, or anything else thats prone to complete failure. In this example, json_error() is a simple helper method that returns a JSON response with the correct Content-Type, status, etc. headers.

Then, in your server you can have something like this:

def __init__(self, redis_conn):
    self.redis = redis_conn

    self.url_map = Map([
        Rule(/_status, endpoint=status),

… more werkzeug boilerplate

def on_status(self, request):
    return json_success({version: __version__}, request)

If throws ConnectionError the decorator will catch it, and return the failing JSON. So, like the title says, nothing groundbreaking, but cute and might save you a few lines.


@zack  1 2013-08-28 14:02


I think Im going to contribute back to the world, and write somethings once in a while. Probably mostly dev/tech/whatever.

Maybe Ill talk about RabbitMQ.


As you may have heard, Ive spent some time at work working on an open source, language-agnostic a/b testing framework.

Well, today, Im happy to announce its public launch. You can read the format announcement post here, and view the website here.


sixpack-server and sixpack-web are built with Python and Werkzurg, with Redis as the primary datastore. It also make use of a little bit of Lua script for interacting with Redis.

Go ahead an browse the source at github.

Client libaraies are available for Python, PHP, JavaScript, and Ruby.


We had originally planned to launch Sixpack several months ago, but decided to let it run in production to gather quite a bit more data. This turned out to be an incredible good idea, as it helped up to squash several more subtle, yet show-stopping bugs, such as bad robot detection, and some bugs related to sixpack-web and the client libraries.

hacker news discussion: here

2022 zack kitzmiller. this website is built using elixir w/ pheonix, cockroachdb, memcached. hosted on a 10 node RPi kubernetes clusters in my apartment. load tested to 12,000 concurrent users. print your venmo name on the receipt printer sitting on my desk and i'll venbro you $100.

served by z19rpw-5f4f9f8b87-tq6xg [prod] in 1 second, 474.331 milliseconds — [email protected]