While I would happily proclaim that my ur language is Ruby, I spend increasingly large amounts of time these days using Python. And while there are many things that I don't like about Python (the syntax makes my eyes want to weep and then die; thanks Tim Curry / Psych; around 20 seconds in), the str...

Yesterday I was stuck in a hellish debugging cesspool – you know what I mean – when you are debugging a mysterious crash right down embedding line by line print statements and nothing is making sense. And I don't mean that its not making a lot of sense; I mean that its making NO DAMN SENSE. I was de...

I recently had an interesting experience with Python dictionaries that made question one of Python's foundational principles i.e. TOOWTDI: There should be one– and preferably only one –obvious way to do it. Although that way may not be obvious at first unless you're Dutch. More… The application was ...

I recently had the odd situation where it seemed like I had outbound Internet connectivity issues under Ruby powered by Docker but not with Python powered by Docker. Here's how I disproved this: docker run -it ruby:2.7.1 ping google.com 64 bytes from ord30s26-in-f238.1e100.net (216.58.192.238): icmp...

I am currently involved in leaving a gig where I've been for a year now and I find myself breaking one of my fundamental engineering beliefs, in effect, engaging in an anti-pattern. If a pattern is what you should do for something then an anti-pattern is what you should not do. The anti-pattern I fi...

Normally when I'm trying to validate that an Internet service is available, well, I ping. Ping is one of those Internet bedrock protocols that I'm firmly in the camp of "SHOULD ALWAYS, ALWAYS, ALWAYS BE TURNED ON". But there are ping flooding attacks, etc so there are reasons why it isn't always a d...

layout: post title: So That One Time You Played with Docker for hte So that one time, at band camp – wait; wrong movie, this is a technical blog post about Docker … I recently helped a friend doing some federal work in the medical field get a better understanding of Docker and how Docker can mess ab...

I am in the process of wrapping up a year long engagement where I: Built a high performance data pipeline Capable of processing all of Twitter in real time / near real time Applied multiple tools to the data at different stages of the pipeline Applied one or more Machine Learning models at different...

This one is a short one but a good one. I've been using SQS (disclaimer – I love SQS) lately for a large scale data pipeline. We are processing literally billions of small JSON blobs and in our cost modeling, we initially came up with an estimate that made SQS something that we couldn't use because ...

I recently took a Python and Ruby (but mostly Python) Machine Learning data pipeline live and, almost immediately, hit a pretty significant problem. The issue was this line of code: model_input = tokenizer.encode_plus(text_to_classify, add_special_tokens=True, return_tensors='pt').to(device_to_use) ...

Some computing problems fall into the category of evergreen – no matter what you do, they will always, always, always occur – they are evergreen just like a pine tree. Today's version of this is storage. I've been running an EC2 node which has a SystemD service (no flames; I actually like SystemD al...

I have been, since my very first foray into startups, a huge advocate of the tag line. A tag line is that one to 10 word phrase that encapsulates exactly what you are building. I talk about this in my document on Opportunity Profiles where I identify it as one of the first things you write down when...

In the age of the iPhone, it is really, really hard to imagine just how significant Palm used to be. Here are some examples: Palm used to sell over a billion dollars worth of organizers annually There were multiple compatible versions of the Palm from different companies Both Palm and Handspring, a ...

Right now I have a directory with about 750,000 odd files in it and any attempt to manipulate the files results in this: root@ip-172-31-1-219:~/datastreamer/data# mv * /home/ubuntu/data/data bash: /bin/mv: Argument list too long My first challenge was to do this: chown ubuntu:ubuntu * And the fix fo...

Part of speech tagging is a classic NLP (natural language parsing) where you give a sentence of sentence fragment to a bit of software and ask it to tell you the parts of speech. If this sounds like a 7th Grade English exercise, well, you aren't wrong … I'm experimenting with this for an upcoming us...

Tonsky recently had a great post about 4K monitors, fonts and scaling. The magic in this article are his suggestions around how to set fonts on your OSX machine. Apparently Apple does not set the defaults correctly (and, I'm writing this on a brand new OSX box that is NOTABLY improved by following h...

I recently had to setup an NGINX server to serve a directory of files for processing via an AWS Lambda function. And while there are other ways to do this, NGINX offered an easy, convenient way to do this albeit at the cost of some configuration. The trick to this is the autoindex on directive but i...

I have an interesting technical situation facing me: A docker container which runs persistently and writes JSON files to a directory Each JSON file contains short form messages with JSON encoding along a variety of metadata fields Each file is idempotent i.e. once it exists in the directory will nev...

I recently had to look at a complex JSON structure in Python. The way that I wanted to do this was to look at a sorted list of the keys. Here's the core of what I came up with: keys = my_json.keys() keys = list(keys) keys.sort() I keep a common_json.py library and this is what I came up that I can c...

I recently had the situation where even the simplest docker command: docker run -it bash gave this error: docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.40/containers/create: dial unix /var/r...

One of the better lessons that you learn as a programmer is that you never want to work too large, in short, that laziness can be a virtue. I'm speaking, of course, of doing things manually that you can automate. I recently wanted to download a set of images from github that represented anonymous an...

My wife recently got kittens and wanted to show pictures of them with our kids via our ChromeCast. The problem is that there seems to be no simple way to do that. Here's the work around I came up with: Create an album on your iPhone and put all the pictures in it. Go into that album and look at the ...

CI / CD or "Continuous Integration / Continuous Deployment" is the process of: taking a source code base running tests against it compiling it (if applicable) deploying it CI / CD is an ongoing process that runs continuously and responds to pull requests on your source code repo. In this blog post w...

Ever since OSX Catalina, every time I open a new shell window I get this error: zsh compinit: insecure directories, run compaudit for list. Ignore insecure directories and continue [y] or abort compinit [n]? y (eval):32: command not found: complete The issue with complete isn't the problem here – it...

So I'm at the very early learning stage of Rust, that stage where almost nothing you type in works. And this, for me, usually means "make the learning stick by writing it down". So here goes … Note: I wrote this code using the Rust playground, a web based repl, which was invaluable. A Simple Program...

I recently had the situation where I needed to run an Ansible task on a user which I sudo su'd into i.e.: sudo su jenkins I then ran my Ansible task and I repeatedly got this error: TASK [zzet.rbenv : update apt cache] *********************************************************************************...

need to fix the CovidNearMe News page. It's using target="_new" and it should be target="_blank". It keeps opening stuff over and over in the same browser tab. (this is a thing I can do but I'm at the PC)

A friend mentioned to me "I'd like to see my folks but they live far away and I don't want to fly and they are immune compromised". And that made me think and then write this. Is Flying Safe? Here's what I know about flying right now: Airlines, even when they promised first responders to New York Ci...

I would argue that even in the age of the app, a domain for your side project is still absolutely important. Now, despite that, one of the truths of side projects is that your domain is important but you aren't going to be dogmatic about it in the way that a funded company approaches their naming. W...