software engineer. former new yorker — current chicagoan

Yeah, I know. Thats a mouth full, perils of webscale, I guess.

I was recently having a conversation with Taylor Otwell on twitter about consuming Laravel queue messages in other languages, and figured Id quickly throw together a little proof of concept.

You can see all of the source code here.

There is an almost fresh laravel install, with a single artisan command email:send. The command simply uses Queue:push to put a message in a beanstalkd queue. The app does nothing else, but one could image its a fully functional app undergoing a refactor.

There is also a small go application here. This worker listens to the beanstalk queue with one goroutine, and sends email with another. The goroutines communicate through a single channel. Ive included the complied go binary in case someone wants to play with this without getting a go environment setup.

The source for the go application is relativly small, and should be pretty readable. Feel free to ask questions in the comments if something isnt entirely clear.

If you want to play with this, youll need a valid sendgrid account and Beanstalkd running and the following env vars set:

  • BEANSTALKD: location of the beanstalk installation (e.g. localhost:11300)
  • SENDGRID_USER: username for sendgrid account
  • SENDGRID_PASS: password for sendgrid account

Update: Extra special thanks to the SendGrid team for provisioning a test account early on a saturday morning.

Tiny PHP

@zackkitzmiller 2013-11-30 18:06

Today, Id like to annouce my new PHP Micro-Framework, Tiny.

Ha. Just kidding.

When I was building Forrst with Kyle, we knew that we didnt want to expose various primary keys, like post_ids, and user_ids, etc, on the site. The solution was to make URLs that looked like http://forrst.com/posts/Poly_Mantis-GFC.

We used a library from our friend, Jacob, that allowed us to obfuscate primary keys simply by base62 encoding them. I find myself using this library constantly, constantly copying and pasting from Kyles github account, where the code has been sitting untouched for literally years. I finally made it composer-installable.

You can find it here. I hope you can find it as useful as I do.

edit: Just to clarify, the tiny-ified ID wasnt stored anywhere, it was always calculated. We used methods something like:

class Post {
    public function getUrl() {
       return $this->getSlug() . - . $this->getTinyId();
    }
}

edit 2: This could also be used to build something like a URL shortener, as $tiny->to(99999999); would actually output jpWDw

Upgrading to Mavericks breaks lxml

@zackkitzmiller 2013-10-26 14:02

Turns out upgrading to Mavericks breaks a few things, afterall. One error ran into today was #include \”libxml/xmlversion.h not found trying to install lxml.

Simple Fix: reinstall XCode Developer Tools with

$ xcode-select –install

You should probably just do that anyway.

words in word

@zackkitzmiller 2013-10-16 11:11

My son, Jayden (8 years old), had a homework assignment yesterday to find words made up of the letters of another word (e.g. pat in part). So, instead of thinking, we wrote this:

from itertools import permutations import sys

word = sys.argv[1]

words = open(/usr/share/dict/words).read().split(\”
\”) valid = set()

for k in range(1, len(word)):

 for combination in permutations(word, k + 1):
     to_check = \"\".join(combination)
     if to_check in words:
         valid.add(to_check)

print list(valid)

Use like:

$ python words-in-word.py part

He also now has some understanding of strings and lists in python.

close enough

go fish

@zackkitzmiller 2013-10-13 14:02

go fish

Ive been playing with Go quite a bit lately, and found there to be quite a bit of boilerplate to get started with a new project. So I thew together gofish. Its a virtualenvwrapper inspired wrapper for working on Go projects in the Fish shell.

If youve used virtualenvwrapper (or virtualfish) the commands will be familiar to you.

To create a new go workspace:

$ mkgoworkspace project-name

This will created a project structure that looks like this:

/bin /pkg /src /zackkitzmiller /project-name main.go

The default namespace will be the currect users username, but that can be overridded by setting the GOFISH_NAMESPACE environment variable:

$ set -x GOFISH_NAMESPACE github.com/zackkitzmiller; and mkgoworkspace project-name

To switch the GOPATH:

$ goon project-name

To deactive a workspace completely:

$ gooff

The various commands all emit events, so you can listen for them in your fish config. The available events to listen to are:

  • gofish_workspace_will_be_created
  • gofish_workspace_created
  • gofish_workspace_will_activate
  • gofish_workspace_activated
  • gofish_workspace_will_deactivate
  • gofish_workspace_deactivated

The following example would send a growl notification on workspace activation

function announce –on-event gofish_workspace_activated command growlnotify -m \”Workspace: \” (basename $GO_CURRENT_WORKSPACE) \” activated\” -n gofish end

The following global environment variables will be available after a successful workspace activation:

  • $GOPATH - as expected
  • $GOFISH_WORKSPACE
  • $GOFISH_NAMESPACE

This is still just a work in progress. Let me know what you think. And as always, pull requests welcome.

Todo:

  • multiple modules in same workspace
  • auto git?

git legit

@zackkitzmiller 2013-10-10 16:04

Seems that Kenneth Reitz nailed it again. Its been around for > 2 years, but this is such a neat little tool. Wish I would have found it earlier.

did you just tell me to go fuck myself

@zackkitzmiller 2013-09-25 14:02

Im putting this here because I can never find it when I google it.

did you just tell me to go fuck myself

venuestagram

@zackkitzmiller 2013-09-30 12:12

Last week I was hanging out at the OnDeck Hackathon in NYC. After not doing much for a while, I decided Id give myself an hour to build something, and unlike all of my previous hackathon projects, I wanted to actually deploy this to the internets, so I did.

So, what I built was venuestagram. Its a silly little app that pulls in all popular venues from the SeatGeek API and use the lat/lng I got to query the instagram API. I built it with Flask, and this python instagram wrapper I found.

Enjoy

Cute little WSGI error handling pattern

@zackkitzmiller 2013-08-29 14:02

Heres a cute little pattern I threw together when writing sixpack. Im assuming youre using Werkzeug, but you could do the same thing with Flask, or pure WSGI.

First step is to write a little decorator that youre going to decorate your route handlers with. Its pretty straight forward. Here I used the decorator package to simplify things, but you can just do it in pure python if you dont need the additional dependency.

import decorator

from redis import ConnectionError

@decorator.decorator
def service_unavailable_on_connection_error(f, *args, **kwargs):
try:
    return f(*args, **kwargs)
except ConnectionError:
    return json_error({\"message\": \"redis is not available\"}, None, 503)

Its simply going to try to run the method you decorated, and catch a Redis ConnectionError. You could do this for any datastore, or anything else thats prone to complete failure. In this example, json_error() is a simple helper method that returns a JSON response with the correct Content-Type, status, etc. headers.

Then, in your server you can have something like this:

def __init__(self, redis_conn):
    self.redis = redis_conn

    self.url_map = Map([
        Rule(/_status, endpoint=status),
    ])

… more werkzeug boilerplate

@service_unavailable_on_connection_error
def on_status(self, request):
    self.redis.ping()
    return json_success({version: __version__}, request)

If self.redis.ping() throws ConnectionError the decorator will catch it, and return the failing JSON. So, like the title says, nothing groundbreaking, but cute and might save you a few lines.

Whatever

@zackkitzmiller 2013-08-28 14:02

whatever

I think Im going to contribute back to the world, and write somethings once in a while. Probably mostly dev/tech/whatever.

Maybe Ill talk about RabbitMQ.

Sixpack Public Launch

@zackkitzmiller 2013-08-27 21:09

sixpack

As you may have heard, Ive spent some time at work working on an open source, language-agnostic a/b testing framework.

Well, today, Im happy to announce its public launch. You can read the format announcement post here, and view the website here.

technology

sixpack-server and sixpack-web are built with Python and Werkzurg, with Redis as the primary datastore. It also make use of a little bit of Lua script for interacting with Redis.

Go ahead an browse the source at github.

Client libaraies are available for Python, PHP, JavaScript, and Ruby.

notes

We had originally planned to launch Sixpack several months ago, but decided to let it run in production to gather quite a bit more data. This turned out to be an incredible good idea, as it helped up to squash several more subtle, yet show-stopping bugs, such as bad robot detection, and some bugs related to sixpack-web and the client libraries.

hacker news discussion: here