For those who lived through cloud, mobile, and crypto, this all feels really familiar. But neither Cloud nor mobile nor crytpo went away. They just sort of found their place in the world and they’re now standard in a lot of industries and workflows.

I wanted to put some thoughts down about AI, how I’m using it, and how I think organizations should be embracing it.

AI For The Individual Contributor

For the IC AI is both amazing and horrifying. On the one hand, it can spit out an insane amount of mostly workable code. On the other hand, it can spit out an insane amount of mostly workable code.

When I first started to play with AI I was throwing it at ambiguous problems and then watching it hallucinate and spiral out of control. And while that’s kind of fun and hilarious, it put a bad taste in my mouth about the first interaction with it. On top of that, as an IC I am drawn to deterministic outputs and LLMs are just not that! Chaotic systems are those that given a very slightly different variance in input give wildly different outputs. AI isn’t really that, but sort of does feel that way. It was challenging to find a reasonable way to use it.

I’ve been using CoPilot for a long time, and I quite liked the small context sensitive suggestions for boilerplate I didn’t want to write. I probably would have killed for CoPilot when I was at Stripe working in Go before generics. There’s only so many for loops to map data from A to B I can write in my life and I’ve long exceeded that amount.

Incorporating something like Claude into my workflow has been a learning curve. What I’ve come away with is it’s great, for me, in two scenarios:

Discovery Engine

LLMs are a fantastic way to explore new topics using NLP as a human interface. I mean thats what LLMs are best at: NLP (natural language processing). I often ask Claude about syntax I don’t know, topics I am learning about, and as a general entrypoint into knowledge systems.

But, and this is the big but, probing ends mostly there. After that I will often try and go to primary documents or other vetted knowledge bases to dig further. LLMs are prone to giving close-but-not-quite answers but that’s kind of OK if you’re trying to frame a mental picture and squint at a fuzzy thing to understand its shape.

I’ve taken to using LLM entrypoints as a first pass for the unknown. The moment I encounter things that are clearly wrong or it’s starting to break down I give up on it. It’s not time well spent to try and get an LLM to give you an answer you want. That’s signal that you don’t know enough to ask or that the information is more nuanced and its back to using your grey matter to figure it out.

LLMs can be an amazing way to facilitate learning and learning faster. But you can’t short circuit actual learning, even if the LLM looks like it can do all the work for you.

As an example of this in a non technical area, my partner and I recently bare boat chartered in Croatia and sailed the Adriatic for a week. We’ve been sailing for 7 years and done a lot of long trips here in the PNW and into the Canadian Gulf Islands, so we’re not strangers to life on a boat. But this was the first time we were planning an overseas adventure. As part of planning I spent a lot of time poring over charts, pilot books, and forums trying to piece together routes, anchorages, and provisioning areas. I was feeling overwhelmed by all the new information and it was hard to figure out what was a good starting point. To help narrow the focus down I opted to try ChatGPT to help me out and at first glance it pulled some great information. I asked it to get me context on prevailing winds, backup routes, weather forecasts, all sorts of stuff I have to do when planning a trip.

While I was impressed with what it came up with it, a lot of it turned out to be nonsense. However, it did give me a great starting point for further analysis and it helped frame my mental model of what was common in the area. When I finally got the plan together it was nothing like what the LLM gave me, but I think this is a good example of using LLMs as discovery engines and probing through ambiguities.

Boilerplate Code Monkey

LLMs are pretty good at writing code, and a lot of it. However, most code in the wild is absolutely fucking awful and lines of code written is probably one of the worst metrics you can ever include as a measure of success. Anyone can write poorly factored, poorly encapsulated, poor architected code. I should know, I spent years doing it as a junior engineer! Well written code is an art and requires deep understanding of the direction you’re going, the systems you’re in, and the trade off’s you’re making.

I’ve had a lot of success using LLMs to do semi-complex work when I direct them with clear examples and the codebase I’m in is resilient to these kinds of changes. What I mean is that code in the codebase is consistent, has lots of good tests and test fixtures, linting, and obviously easy to consume patterns.

In areas of code that are more chaotic, the LLM creates more chaos. Left to its own devices it creates tests that are mocking everything and testing effectively nothing, imagining endpoints that don’t need to exist, special casing workflows, duplicating code it doesn’t know exists already, etc.

Recently I had a Claude interaction that was very successful for me:

I like these kinds of tasks for an agent. They are reasonably scoped, annoying for a human to automate, and move the needle in a net positive way. However I also at some point tried to use it for a task that would touch hundreds of files and it broke down really quickly.

I know it’s possible to get LLMs to do more but the cost of guiding and reviewing their work isn’t worth it to me. In fact I think most people suck at explaining what they want. I’ve read enough JIRA tickets to know that even knowing the final outcome of an ask is often ambiguous.

I am probably not going to be someone who spends ages customizing agent workflows, creating ultra detailed CLAUDE.md files, etc. Just like I don’t spend a lot of time making vimrc files. I firmly believe tools should be easy to use, after all the easy thing should be the right thing. I’m too lazy I guess. That said, this kind of work, like showcased above is tedious, painful, and I just hate it. Having the LLM do it is actually a massive time saver but it was successful because I did the example that it follows myself. This is kind of no different to how I’d lead a team of people, good patterns propagate and bad patterns propagate too.

It’s critical though that I review all of it’s output and validate that it does the right thing. If I let it write code that I don’t understand then it’s ralph-meme territory.

Pitfalls Of Vibing

Letting LLM’s vibe their way through code is becoming such an issue that new patterns of code engagement are being explored, like knowing if LLMs were used to create code. It makes sense, if you’re reviewing things it does actually matter if someone thought the code through or not. Even if you use an LLM did you actually review it with the same rigor as if you wrote it by hand? I can tell you for sure if I see a PR with 1k+ changes I’m not reading any of that too carefully and I trust a coworker has thought it all through or we’re going to go back to the drawing board.

It’s funny to think that code review as a skill is more important than ever now, and many people struggle with reviewing code. It’s hard, and it takes years to practice. Often code review as a skill only happens when you have a mental model to map things to. I expect code to follow specific shapes, patterns, and similarities. That comes with lots of experience and practice though.

In any setting, reviewing thousands of lines of effectively auto generated code is never going to happen. How many of us have scaffolded a project using some new framework and just ¯\_(ツ)_/¯’d our way through the generated code. We’ve learned a long time ago that small changes are easier to follow, keeping cognitive load low and minimizing bugs.

As IC’s we should be using LLMs in the same way and free ourselves up to focus on the important things. Things like setting up codebases and systems for success. If you don’t know what success is your LLM agents will run rampant making their own decisions for better or worse.

I recently spent some time in a heavily LLM coded codebase and very quickly found inconsistencies that actually slowed me down and led my own agent use down rabbit holes that created more issues than they actually fixed. I had to pull back, refactor, and backtrack to undo a lot of cruft that worked but wasn’t extensible. I was tempted by the vibe-dragon and it bit back.

AI In Products

Just about every company in the world is trying to figure out how to incorporate LLMs into their product, not just their employee workflows.

Lot of new products are coming out that are 100% LLM driven and it will be interesting to see how many of those succeed over time. These companies are leveraging RAG, LangChain, and in general finding ways to break down their problem domain into smaller sub-contexts to have agents excel at and then chain them together to create more interesting flows.

I think using NLP as an interface is really cool and having a Claude style prompt in scenarios where complex onboarding occurs is actually slick as hell. Using LLMs as tiny “nice to haves” in places is a joy. Some good examples are places in systems that ask for some sort of textual boilerplate: “title” or “description” are classic examples. Let some AI slop auto gen that based on other localized context and that’s a loveable moment in any product. I’d be surprised if this wasn’t even baked into new UI frameworks honestly. I worked on an NLP system in 2017 trying to categorize text into groups and it’s funny that we built hand rolled models to do this when an LLM excels super well at this.

However there is a limit to how much people want to interact with a non deterministic engine that starts to hallucinate over time. Eventually people want determinism and control. Adding LLMs on systems that already have well defined APIs and SDKs is a great way to gracefully extend products. Building escape hatches is critical. I almost feel like the escape hatch should be the first way you build things - SDK and API first, and then let LLM’s drive using structured schemas and MCP tools as constraints to your system.

I see this the same way mobile happened. Do you like it when a company ONLY has a mobile app? It can be nice to have a desktop app, or even an in person moment (depending on industry). It’s great when it has a mobile app, but forcing people into specific workflows often kind of sucks.

Customer success has been hit super hard with bots even long before LLMs came out in force but if you’ve ever interacted with an LLM in chat (or even worse voice) the lack of humanity comes out quickly and it is degrading in a moment when you need actual help! I think companies should tread carefully in this space and there is a lot of value in having an actual real human who can help. Weirdly enough this is actually a differentiating feature now.

AI In Organizations

Where I think LLMs shine in an organization is not in replacing anyone but in simplifying the complexity of communication and helping reduce cognitive load for people enabling them to think and work on things that matter the most.

LLMs that summarize project statuses and can provide realtime stats help project managers focus on the inter personal and strategic decisions they actually want to be focusing on.

LLMs that integrate into slack that can create overviews of long text chains helps welcome new people into a complicated conversation and lowers the barrier and improves communication equity. We’ve all seen a 100+ thread and nope’d out because aint nobody got time for that.

LLMs that can act as a souped up first pass triage for inter team questions like a Zapier on steroids is great for lowering team’s necessity to spend hand holding first touch interactions between teams.

There’s endless uses for NLP to remove annoyances. However I firmly believe that these flows don’t replace anyone. If anything they should free people up to think and act and decide on more important things.

As an example, I pasted a draft of this article into Claude and I tried to give it a non biased request for value

And it certainly stroked my ego

But this is a great way of getting information quickly and if I want to dig in more I can.

The 10^n fallacy

We’ve all heard the “everyone is gonna be a 10000x engineer” but in reality I think LLMs make a 10x engineer a 12x engineer by freeing up their time to focus on things that matter.

I think LLMs make a 1x engineer a 0.5x engineer because they aren’t getting better and learning when relying heavily on LLMs. Junior and mid engineers become seniors and staff and principals. And unlike LLMs, humans do learn and get better. Time invested in a person pays off over time. Re prompting an LLM doesn’t.

Even in my example above, if I consistently and only use LLMs to summarize, or write for me, then I’m going to atrophy that muscle. I often mentor my engineers to practice writing and reading comprehension.

There’s been a lot of toxic hype about executives frothing to get rid of all their employees in favor of massive agentic webs, but every organization is only as good as it’s people. This has been true for all time, pre-industrial revolution, post, the computing age, etc. You need to invest in people.

I think organizations are coming around. It was refreshing to hear at least one executive see through the hype veil, but it seems that most executives are taking an extremely heavy micro-managing hand here, which just breeds resentment and skepticism about their actual incentives.

Contrary to how Coinbase handled their AI policy, I think this GitHub post is actually an interesting and healthy approach to AI centric tooling. This snippet is key to me:

Their mistake is treating AI adoption as a technology problem when it is, in fact, a change management problem. Companies fail at AI adoption because they treat it like installing software when it's actually rewiring how people work. The difference between success and failure isn't buying licenses. It's building the human infrastructure that turns skeptical employees into power users.

Places that have created a major sense of distrust failed because it’s unusual for leadership to demand specific tool usage on employees! I’ve never seen it before. As an example, an IDE with semantic refactoring support saves an enormous amount of time and yet nobody has ever looked a vimlord in their dead eyes and told them to :wq! into IntelliJ. I’m curious what incentives are at play that managers and executives who are many levels removed from day to work find it reasonable to dictate specific tooling?

The way to get people to use tools is to show them how it can make them more powerful and find more joy in their work. And the goal isn’t to replace them, it’s to augment them. Sometimes that means answering questions that might be hard to google. Sometimes that means generating unit tests. Sometimes that means upgrading react native, and other times it means writing me a for loop because it’s boring.

That said I think removing engineers from coding is going to backfire. Engineers are good because they know the systems they are in. If an mechanical engineer doesn’t know how torque works, can they design an engine? If an aerospace engineer isn’t living and breathing the thermo coefficients of materials can they actually build a re-entry hull for spacecraft? Abstractions can remove someone from certain levels of work, few of us work in assembly for example. But we still need to know what is going on at all levels and sometimes we do need to drop in that low!

Conclusion

I think LLMs and AI are here to stay but the level at which people interact with them or find them helping in their life varies wildly. For professionals who are using LLMs to speed up their work we need to be disciplined and diligent to not offshore our own brains to the agent. We need to invest in people, like we always have, and let people discover what helps them and embrace those patterns. We need to carve time out for people to improve themselves and deep understanding of things so that they can actual be useful in managing LLMs. If we fall into the fallacy that everyone is working at 100x the speed how are we actually going to manage the 100x output?

We’re at an interesting point in time but in the end all you have is your brain so use it or lose it.

I talk about testing quite a bit in my book but all that maters is that tests are readable and that they fail when you break your code.

Here are a few guidelines to go by when thinking about your tests.

Subscribe now

Single tests

Tests should always run and fail in isolation. You never want test setup to be dependent on other tests, or rely on any particular order of tests. Each tests should effectively work on a blank slate of the world. That either means its testing pure stateless code or if its interacting with the local word (via sqlite or docker or something) then the dependencies should be wiped between tests and set-up fresh for each test. Alternatively you can namespace and create new scoped spaces for long running dependencies - a new schema in sqlite, a new bucket in localstack s3, etc. The underlying context stays up (db/docker) but the namespace/scoping changes per test.

Setup should be clear

Test setup should be clear so that you can tell what the test needs.

Here’s an example of a bad test:

it "validates context" do 
  expect(call).to eq(result)
end

This is a bad test because

• It’s unclear what setup is happening (the setup happens outside the scope of the test)

• It’s unclear where the result value is, or why it’s set up this way? Again the scope is set up outside the test

A better version of this would be

it "validates context" do
  sample = 123

  insert_data(sample)

  expect(call).to eq(sample)
end

Regardless of what may or may not be happening here you can clearly see that we set something up by inserting our data as part of the test, then validating that we got something back related to the insertion.

The example here is silly but these kinds of tests happen all the time.

Test setup in the real world can be very non-trivial, so ensure you wrap test setup in classes or utilities that make it easier to re-use. These test setup utilities can and should be tested themselves too! If you need to set up a complicated API interaction wrap it up! If you need to generate sample data and insert it into in memory db wrap it up! The more you create your own internal testing API the more you can re-use this and then run faster, have stronger tests, and the tests themselves stay short but clear - the entire setup, call, validation is obvious.

I’ve talked about internal APIs before and test setup tooling falls into that same bucket.

Make sure it breaks

Not a lot to say here other than make sure if you write tests make sure you break the code under test and validate that the test itself breaks! You might be surprised to find that sometimes your tests pass and even if you break your code… they still pass. That’s not a good test.

This is especially important when over-using mocks. I encourage teams I mentor to use as much real resources as possible. Actually talk to a database, actually make an in memory http call to your API, etc. When you over-use mocks you might find that you are only testing what the mock says is doing, and that nowhere is the actual interaction between logical components is tested.

Avoid cargo culting

This brings to me another point to not fixate on cargo cult definitions. It doesn’t matter if something is an integration test or a unit test, its just a test. Some tests are small and stateless, other tests require more setup. If the test provides value then its a valuable test!

This is technically an integration test because it talks to a database, but … who cares?

Notice that it

• Sets is data up

• Is isolated per test

• Clearly asserts the contract under tests

• Is obvious what the expectation

This a recipe for a successful test.

Conclusion

Tests are first class code. Treat them as such. Writing a good test is just like writing good non-tests - prefer readability, keep it simple, re-use and componetize shared dependencies, and don’t be afraid to refactor!

I’ve long talked about writing a book, I mean it’s basically just like 100 blog posts chained together sort of, but I didn’t get the chance to really sit down and write it till last summer when I was between jobs. I decided to finally do a brain dump of all the things I’ve been evangelizing and putting into practice for years with the intent to be able to share this with new teams I join and people I mentor.

For those interested in the book writing process it was surprisingly accessible. I wrote my original manuscript in google docs, about 100 pages worth, and then hired a wonderful development editor off of reedsy who helped me flesh the book out into something cohesive. Then I hired a proof reader (again off reedsy) who polished the book and made it ready for publication. At that point I handed the book off to a great graphic designer, Christian Arnder, who beautifully designed the cover and inside layout of the book.

For e-pub we used the kindle create tool to upload a docx file and then do some tweaking and used the kindle previewer to make sure it formatted well and then it was easy to publish and release via amazon KDP.

I’d love for anyone who reads the book to drop a comment or review, and I’m always happy to hear feedback if there’s any questions or comments.

Enjoy!

I’ve played with Ruby on and off for 20 years (I wrote a rudimentary early version of instagram back in 2003 in Ruby for friends and I really regret not pursuing that more ha). Most often though I use Ruby for CI scripts because it’s installed on machines by default and is better than shell scripting when it comes to slightly non trivial tooling.

For many years I shied away from taking jobs that used Ruby because… well … I like types! Why work in a language without them? My entire blog is basically a paean of types. However, I don’t like being afraid of something and I made a very conscious choice to work in an untyped language, be it Python or Ruby, for my next main job.

I’ll be honest, I don’t hate it as much as I thought I would. There are some things I really like about it actually and just much as that I don’t like - some expected and some new and unexpected!

Let’s start with the things I like.

Pros

terse and expressive

This is Ruby’s biggest sell - the fact that the language is terse, concise, and extremely expressive. In the hands of a power user it’s actually fun. I love the simple module and class syntax, writing abstractions feels almost like when I’m sketching code out… but it’s runnable! This reminds me a lot of F#. I wrote an F# example once that read like like a sentence and a co-worker of mine was amazed that it wasn’t just pseudocode, it was actual code that did what the sentence said.

I recently had to write some AST processing to transform a higher level DSL into an elasticsearch query and Ruby made it extremely easy to build a nice little API to model boolean and query logic. It felt joyful!

def build(schema = nil)
  case @tree
  when And
    operator = @tree.not ? :must_not : :must

    {
      bool: {
        operator => @tree.terms.map do |term|
          QueryBuilder.new(term).build(schema)
        end,
      },
    }
  when Or
    {
      bool: {
        should: @tree.terms.map do |term|
          QueryBuilder.new(term).build(schema)
        end,
      },
    }
  ...

transforming data is easy

Ruby comes with a lot of modern collection tooling. Dealing with maps, arrays, lazy iterators, it’s all built in and easy to use. I love that you can make any class enumerable just by implementing the each function. Most of my day to day involves mapping, filtering, and slicing of collections - simple but obvious tasks that not every language makes easy to do. It’s quite nice to just have it available, and I remember why Ruby made a big splash in the tech world in the early 2000’s and 2010’s. It was a language that embraced modernity and gave you the boilerplate free tooling to do what you need to do without a lot of overhead.

meta programming is wildly powerful

Message passing is a concept that a lot of people struggle with, but I find it makes more sense when you imagine it as an actor model of delegation. You can call class A with method “foo” and if it decides “I don’t want to handle this” it can capture that and re-invoke something else instead for you. Neat. But why would you want to do this?

Well, imagine you want to proxy all methods that start with elastic_ to the Elastic processor and all methods that start with rds_ to the rds handler but have it be composed and auto updated anytime anyone adds new methods? This is really easy now

class Proxy
  def initialize(elastic, rds)
   @elastic = elastic
   @rds = rds
  end

  def method_missing(method_name, *args)
      case method_name
      when /elastic_.*/
         @elastic.send(method_name, args)
      when /rds_.*/
         @rds.send(method_name, args)
    end
 end
end

Wild! You get basically a method invocation for any missing method and can do whatever you want there.

In the right hands this is crazy powerful and allows for very expressive and interesting code.

zeitwerk and rails

I really like in the zeitwerk and rails model of not having to include imports. The forced naming convention of module namespaces map to directories and classes map to file names (all snake case) makes it really easy to find where things should be. Granted this isn’t a problem in literally any other language because IDE’s manage all the imports and lots of languages require convention based file locations, but its still nice in Ruby too.

Rails also has a lot of great extensions and utilities that I sort of don’t even think about. I honestly don’t know sometimes if methods I’m calling are because of rails monkey patching or not, and to a certain extent I’m not sure I care. Rails mostly just works and stays out of the way and that’s fine with me.

pry and the REPL

Normally I prefer to do all my debugging in my IDE using attached breakpoints. I’m a sucker for gutter icons and right clicking on a test to step into it. But my work has moved to a model where we run all code in k8s remotely. This is kind of cool cause they can set up a bunch of pre-baked infra, and ensure consistent runtime environments. However, remote debugging is often complex here and in other languages this might be more frustrating. In Ruby though we can just debug… via Ruby! You can drop into an effective debugger via the pry shell by putting a binding.pry statement where you want the debugger to pop up.

You can customize actions and script things as well, because it’s all just Ruby. The debugger is fully fledged, you can inspect locals, step into classes, add more breakpoints, etc. I’ve found it to be a pleasant, albeit different, experience to what I’m used to.

library ecosystem

Ruby is like stefon - it’s got everything. There seems to be a Gem that does just about anything you want, which is kind of nice. It reminds me a bit of the Node ecosystem where there’s a zillion packages to do a zillion things. Ruby’s seems a bit more slimmed down and curated, and the packages themselves seem to be higher quality than a majority of Node packages. I haven’t wanted for packages that do what I need in Ruby and it’s been easy and painless to consume them.

sorbet

I couldn’t help myself, I needed types anyways. Thankfully sorbet (from Stripe) adds a meta layer of runtime and static checking to ruby projects. I remember when they were working on this back when I was at Stripe and I’m glad that it’s been open sourced and is pretty widely used. Chime, Spotify, Stripe, and many others use Sorbet in production.

Sorbet is a pretty weak guarantee, and all things the same a bad type engine, but for a language with zero types… sorbet is a life saver. Teams that embrace sorbet gain a lot of benefit if anything not having to do manual runtime checks of input arguments is a big time saver anyways.

rubocop

I love linters. I used ESLint aggressive in the past and Rubocop is reasonably robust for what it does. It has quite a few false positives though, but overall it’s better than nothing. Also being able to format code with the linter settings is a huge win, as I hate having formatting arguments. Just automate it and be done.

delegation for composition

This is just such a cool language feature, to be able to compose objects together and automatically proxy methods from one to another via

delegate :<the method to expose>, to: <some other object>

Which replaces writing

def proxy_method
   @some_object.proxy_method
end

It’s a small touch but this kind of composition is really effective as you can start with a proxy and then later implement the method yourself and it’s drop in compatible. Creating composition layering like this is super easy and avoids a lot of the noisy boilerplate that other languages might have.

rspec

RSpec is a test runner tool for Ruby and it reminds me a lot of Jest. For the most part it’s just easy and stays out of the way which is why I like it. I’ve had to write custom matchers a few times and it was trivial to extend RSpec (much simpler than Jest was) and overall does what I want with aplomb.

Cons

Fair warning, there is a lot I don’t like. ¯\_(ツ)_/¯

no types

I knew this one going in, so I can’t really blame Ruby. But types are just so important to software it still hurts my brain to think that actual languages don’t have it. I get why Ruby happened, if you time travel to the last 90’s the prevailing languages were Java and C/C++. Talk about verbose monsters! Getting even a basic “hello world” required tons of files, compilation, etc. Then forget about modern utilities like map or filter. Hell Java didn’t have anything like that till their streams release with Java 8 till 2014!

But in the modern world this is a huge travesty to be missing types. Types are a part of the meta information about the problem domain you are solving. They are information, and to wantonly discard it is short sighted. I could probably write a whole novel about why types are good, but I will just say that missing types (and bolting it on with Sorbet) makes for a very painful experience in the real world, especially with larger projects, multiple developers, and actual real money at stake.

hash obsessed

Back to the lack of types, Ruby loves its untyped hash bag. So much so that there are tons of collection utilities to deal with it. I worked in Node for years and the amount of times I used an untyped bag of object I could count on one hand. It really just shouldn’t need to be done. Granted Ruby has structs which allow you to create immutable paired data but a lot of people just don’t use them. But there are multiple flavors of structs - sorbet, dry-struct, ruby Data object, and they all … are slightly different!

This leads to people passing around untyped mutable object bags everywhere that has god knows what in it, sometimes adding fields (or even worse removing fields!) through a call chain.

Reasoning about systems that overuse hashes is an exercise in futility. I’m just too stupid to be a compiler, and so you have to step through code via unit tests which eats up endless amounts of limited time.

Hashes are fine when they are used as an informal object pairing inside of a function, or closed inside of a class (maybe) but exposing them as contract results of an API bleeds more and more loose shenanigans through the system which begets brittle systems that are hard to maintain.

fractured type system (rbs vs sorbet/rbi)

But there is hope! In Ruby 3 they introduced type annotations via the RBS syntax, similar to .h files in the C world. While this apparently isn’t meant to be user-generated (it’s supposed to be machine used) it’s the start of the ruby world moving to a place where types can at least be opt-in.

However, this now creates a fractured world where a lot of the big players are using sorbet but the language is moving to rbs. This isn’t great for unification of things, and in fact I find that this fracturing is endemic in the ruby world. There’s 500 versions of things and everyone does stuff their own way.

dynamic exploration

Due to the dynamic nature of Ruby (see some of the things I listed in pros) the language service is not that great. Discovery of what methods or fields are available via just hitting the .<tab> in your IDE is often non-existent. Granted you sort of learn the fields and methods you use on the regular, but self discovery doesn’t exist. You instead have to do a lot of reading of source code, poking around in a REPL, and hoping that the human managed documentation is correct (often it’s not).

Since there’s no compiler and there’s no way to validate that anything you write is correct or not, the only way forward it to demand 100% code coverage. Any line not covered is a line that could blow up. This is a very stark contrast to type checked languages where enforcing 100% coverage is a fools errand - you end up writing tests that are mostly garbage just to satisfy the requirement… oh wait, that happens in Ruby too! In fact, because you can literally mock out the .new method on a class object people tend to skirt inversion of control and often mock (needlessly) just to get the coverage to pass.

Unfortunately this creates a false sense of security. Either you trust your tests or you don’t, and if you don’t then you end up afraid to make changes. Teams that are afraid to make changes won’t be incentivized to make improvements and the whole thing is self fulfilling.

impossible to require contract adherence

A lot of my complaints about Ruby are about contracts because contracts, and enforcement of contracts, is how you can support large scale teams and projects with grace. Without being able to explicitly say “you cannot call me without filling this stuff in” you end up in a world where people have to fail to explore (either tests failing or production failing). I would much rather lead people to water by not allowing their code to be valid until the contracts are satisfied (compilers are great!). Just because something looks like a duck and quacks like a duck doesn’t make it a duck.

Not being able to adhere to contracts also cripples semantic refactoring via tooling like in an IDE. If the IDE can’t figure out that xyz uses abc then it can’t refactor things for you. This means that refactoring now all has to happen by hand, which de-incentivizes the activity. That then means that doing the organic cleanup that has to happen in a codebase often goes without, and as with all organic things code rots. If you make doing the right thing hard and time consuming, people won’t do it. The easy thing has to be the right thing.

sorbet (generics, tapioca)

I keep coming back to this because I want to believe so badly that sorbet is going to provide the guard rails and safety that I want but I have to accept that it’s a very poor mechanism and frankly not that robust. Unlike typescript which allowed libraries to bundle type definitions out of band sorbet has no way to bundle that data with a library. You end up having to use another tool from Shopify called tapioca to re-compile and scan the type annotations from all your Gems.

This means anytime you update a gem, you have to manually run some tooling. It also forces your repo to house all the generated type information which frankly should be hidden away in a Gem and not versioned in my repository.

On top of that sorbet has no way to validate any kind of generic or higher kinded type, so even trying to do basic signatures like

sig { params(data: T::Array[String]).void }

is mostly useless because all sorbet can guarantee to you is that data is an array, but not an array of String.

Shapes is a nice thing of sorbet, allowing you to structurally type object bags, but it doesn’t allow you to model missing fields and there is no concept of type narrowing (like with typescript). The end result is a lackluster experience, especially having come from the insane powers of typescript.

At this point I have resigned that sorbet is yardoc++ with some runtime checks. I still use it, because a large part of the time it is able to validate types but it really misses the mark on being actually fully fledged.

rails conventions outside of MVC

I’m on board with auto loading, and directory conventions with rails. What I find to be particularly annoying though is that top level folders of rails projects are automatically in the root namespace. What this means is you can have logical data of a module split across mulitple disparate locations. For example, this class

module Foo
 class Bar
 end
end

can live anywhere in the following format /<root folder>/foo/bar.rb.

Why is this an issue? Well imagine you want to co-locate a module’s read/write/entities/logic/etc. The rails folder structure with model, controller, etc is very much tied to an old school MVC idea.

I find this makes it hard to properly organize your code because not everything slots in so neatly in the real world. Code is less organized by function and more by domain. I want to co-locate top level entrypoints like API handlers, but otherwise have things delegated to domain function. For example:

/core
  /foo
   .. all the things foo needs    
  /bar
   .. all the things bar needs
/api
  .. calls things in core
/db
  .. shared db things
/common
  .. stuff everyone uses

I know you can get this with rails but the fact that it forces a convention that doesn’t encourage your own folder modeling is frustrating. So you end up with code in folders that doesn’t quite make sense, and its split across unclear lines. Maybe this is a reflection of my personal experiences, but I haven’t ever dealt with this level of disorganization in other languages.

symbols vs strings inconsistencies

Ruby in the old days didn’t inter strings, so every instance of a hardcoded string was a new allocation. Instead they had the concept of symbols, the funky stuff with a leading colon: :this_is_a_symbol and is meant to basically act as a single instance of that “stringlike” object. Every where you see that same symbol it’s the same memory representation. That’s fine, a nice clear distinction! Use symbols as object keys/etc and strings to be strings.

Well later Ruby decided that this was dumb, and now strings are automatically frozen so that this distinction no longer matters. But what we are left with is inconsistent usages of when things are symbols and when things are strings. You now often have hashes that intermix the two

{
  :symbol => {
    "string" => {}
  },
  "string2" => {}
}

Which makes your object pattern access wildly inconsistent. To work around this people often deep_stringify_keys or deep_symbolize_keys and then sprinkle on .to_s or .to_sym to access keys and get results. This just adds more feed for the fodder about ruby being slow since we’re iterating on objects and converting things to work around shape nuisances.

I’ve seen so many failed instances in production due to string/symbol inconsistencies its amazing this is something a language allows you to do.

I’m sure that the argument is “if you know what you are doing you you can create nice API’s and be consistent” but in the real world most people don’t know what they are doing and need guardrails to make sure they do the right thing.

slow

Ruby is just slow. Time and time again I’m shocked at how slow it is when you start to throw real data at it. If you have to process a few thousand items but loop over them a couple of times (really trivial honestly) the time can start to creep up.

I recently had a situation where we had to process 10k items, which included mapping them once, running each item item through a string template, and then serializing it all to JSON. This process took over 20 seconds. After doing benchmarking I found that all the time was just spent in ruby iteration time, so there was very little I could do to speed this process up.

Usually I don’t care that things are slow, but this was… hilariously slow. Doing the same thing in node, or go, or java, would have taken milliseconds.

procs and lambda gotchas

I don’t know if I’m missing something or if this is due to a side effect of implementation, but requiring to .call a lambda or a proc really makes it annoying to pass around. This forces a distinction between a function, method, or proc/lambda and that nuance isn’t necessary in any other language or ecosystem. I’m not sure what the reasoning there is.

On top of that the various return semantics of procs vs lambdas is convoluted where in one model it returns from the lambda itself (like every other language in the world) and in another it forces a return call of the pattern… which is weird. This variance continues to illustrate the lack of consistency in the Ruby world where there’s a nuanced flavor of everything but no coherence in language design. It’s almost like they did it because they could.

case when pattern matching… sort of

The case statement is pretty cool allowing you to do some basic pattern matching in ruby as I illustrated earlier in the method missing example. But deep down inside this works by abusing a triple equals operator, which makes the case statement both powerful (in that you can customize this behavior) and insane (in that it’s not discoverable at all that this is what drives the behavior). Often I reach for a case statement but find that the triple equals of the thing I’m comparing doesn’t work and I’m forced to do an ugly if tree.

Conclusion

Given the choice would I choose Ruby to build anything on my own? Definitely not. But would I accept a job where they work in Ruby? If the bar was high, the team philosophies matched mine, and the product (and compensation) was stellar I would. But all things being equal I think I’ll stick to typed languages - kotlin, typescript, java, go, c#, etc.

Ruby was an extremely attractive, modern, option in 2015 but now feels like a poorly curated frankenstien and I keep bumping up into these rough edges which takes the joy out of the things that are joyful in Ruby.

So what was I supposed to do? There are a few ways to handle this, one way is to use a lazy dependency initializer and another is to use a pub sub model. In this post I’ll talk about a strongly typed in memory pub sub.

Pub sub

Pub(lish) and sub(scribe) is a pattern to decouple the necessity of interacting with a components directly. Instead of A calling B directly let B subscribe to events that it needs to react on. This can be really nice when you have lifecycle type things - account created, user deleted, stuff like that. When a user is deleted, you might need to delete recurring jobs, delete accounts, notify someone, etc. A direct call pattern might look like

class Users {
  ...

  async userDeleted(id: UserId): Promise<void> {
    await Promise.all([
      accounts.delete(id),
      jobs.unregister(id),
      notifications.notifyUserDeleted(id)
    ])
  }
}

But this has problems in that if accounts, jobs, or notifications needs users then you have a circular problem.

A simple solution is to use an in memory (or swappable) pubsub abstraction

class Users {
  ...

  async userDeleted(id: UserId): Promise<void> {
    await pubsub.notify({ type: 'user-deleted', id: id })
  }
}

And let anyone subscribe to this event. Building a simple version this is trivial, register hooks based on type and invoke based on registration. Then processing events is as easy as

pubsub.register('user-deleted', async event => { ... })

Pubsub happens all the time, its just another form of event processing. In fact using a pattern like this can be quite nice because you can even swap out the internals and allow for over the wire queueing as well. When you have a generic PubSub<T> abstraction creating decoupled interactions is quite nice.

Building on this simple idea what I really want is to

• Have the compiler validate that registered type callbacks are actually correct, and ensure that the handler of the event is typed

• Ensure that we can only notify the right types of events and that event types are properly typed

Lets do some typescript magic.

Subscribe now

Type magic

If we have some sample events

interface A {
  type: 'a';
  data: number;
}

interface B {
  type: 'b';
  other: number;
}

type E = A | B;

Then to get the set of all types “a” and “b” we can create a conditional type

type Types<Obj> = Obj extends { type: infer Key } ? (Key extends string ? Key : never) : never;

Where if we do Types<E> should return to use the set of “a” and “b”

We can even make a reverse lookup, so if we know E and the type of “a” we can resolve the type of A

type ByType<Obj, Key extends string> = Key extends Types<Obj> ? (Obj extends { type: Key } ? Obj : never) : never;

We can now make a class with a signature like this

export class PubSub<T extends { type: Types<T> }, Keys extends string = Types<T>> 

Which requires a shape of

{
   type: "...",
   ...
}

For all events, and gives us a generic called Keys with the values of “a” and “b”

Subscribing

We can build a subscribe method really easily now that allows consumer to register a callback based on the event type

protected callbacks = new Map<Keys, Array<(data: T) => void | Promise<void>>>();

/**
 * Subscribe to an event
 * @param key the type of event to subscribe to
 * @param onEvent  A callback with that type of event
 */
subscribe<Key extends Types<T>>(key: Key, onEvent: (data: ByType<T, Key>) => void) {
  if (!this.callbacks.has(key)) {
    this.callbacks.set(key, []);
  }
  if (this.callbacks.has(key)) {
    this.callbacks.get(key)!.push(onEvent as (data: T) => void);
  }

If we look at an example usage

We can see that by providing the key of “a” the callback is automatically typed to the type of A. Very cool.

Publishing

How about publishing?

/**
   * Waits on all consumers to finish.
   * @param event
   * @param onError if set, proxies errors to this and nothing is thrown
   */
  async publish(event: T, onError?: (e: unknown) => void): Promise<void> {
    await Promise.all(
      this.callbacks.get(event.type as Keys)?.map(async callback => {
        try {
          const result = callback(event);

          if (result instanceof Promise) {
            if (onError) {
              await result.catch(onError);
            } else {
              await result;
            }
          }
        } catch (e) {
          if (onError) {
            onError(e);
          } else {
            throw e;
          }
        }
      }) ?? []
    );
  }

Publishing all we have to do is given the type of the message, look up the callbacks for that type, invoke their handlers, do some exception handling, and await the result.

I don’t have unsubscribe but it can very easily be added by having subscribe return a function that allows you to unsubscribe which would remove that listener from the callbacks set.

Now we have a fully strongly typed pubsub that type checks the publishers and subscribers!

Pitfalls

Pub sub models are highly flexible and can really help in a lot of situations, however I would warn to not abuse this though as understanding the implications of who does what when can get extremely complicated. If you’ve ever built UI forms that are all async knowing who reacts on a “click” can be mind bogglingly complex.

To make that simpler I would suggest that event registrations are all centralized, have one place in code where you register all your listeners so that you can quickly audit who is reacting to what and when. For example, do this

pubsub.subscribe('a', handler1)
pubsub.subscribe('a', handler2)
...
pubsub.subscribe('a', handler20)

And avoid this temptation

class Handler1
  constructor(pubsub: PubSub<E>)
   pubsub.register('a', this.handler) 
  }
}

Because this side effect is not clear from the api contract. You aren’t sure if you are publishing OR subscribing to the pubsub just by the constructor argument list.

In fact, I might even suggest that registration and publication are separated into different interfaces so you can create stronger contracts. This would allow only certain areas to register and only certain areas to publish, which you could use to control access to the eventing pipeline.

As with all things, there’s a lot of options and considerations to be had.

Full source available on my github.

I consider every layer of an application as a distinct set of API’s that layers talk to - database repositories, business logic classes, externally accessible API’s, unit tests, etc. Each layer here has its own boundaries and those boundaries I care a lot about.

Creating strong, consistent, contracts, makes these different lego pieces composable. This is really how you build flexible systems!

Things I consider when crafting an API are

• Do the naming conventions match other layers? If we call something an annotation in on place, are we calling it an attachment in another? That inconsistency makes fitting pieces together annoying, and potentially error prone.
  

• What data we taking into our API? Are we exposing data objects that only take (and return) what we need? Even if what we need at this layer is a subset of what a another layer needs, we shouldn’t conflate the data object layers. For example, if you have a config class for a mapping layer like below, if the v2 engine takes a lot of the same properties we probably still want to create a new unique interface for the v2 engine instead of re-using this config. Why? Well, what does enableV2 even mean once you are already in this v2 engine? It’s kind of a nonsense thing, since it was presumably used for logic to determine if you even enter this engine or not.

  interface MappingConfig {
    // enables the v2 engine
    enableV2: bool
    
    // flags related to mapping
    useDistinctNames: bool
    ...
  }
  
  interface EngineConfig {
    // flags related to the engine
    useDistinctNames: bool
    ...
  }

  This model can probably be simplified by composing the configs, but the point stands which is to isolate and control the data at each layer and not leak context
  

• Consistent access patterns. This goes hand in hand with naming, but we want contracts and API’s within the system to play well together. If one thing works on batches and another thing works on individual units, are they easily composed with mapping? Is it simple to transform and consume data between layers? If not, then the API is clunky and needs to be adjusted. Internally to a codebase you have a lot of flexibility and I often argue that we should ruthlessly refactor our internals to make this easy for us. After all, when things are smooth we can deliver value (product, bugfixes, features) faster! I talk about this at length in my upcoming book “Building A Startup - A primer for the individual contributor” (which should come out sometime in April).
  

• Internal re-use? Are you able to re-use sections of your code often and plug in different strategies, implementations, and components to create new functionality? If not, why not? Most non trivial codebases have a ton of supporting tooling and classes - rate limiters, loggers, feature flag providers, reflection wrappers, etc. When systems are properly componentized they allow you to plug and play building blocks. It should be obvious when this happens! A build block takes no more no less than it needs. If you find its missing something, really evaluate if that’s a generic thing that needs to be exposed or if the interactions between other objects is what needs tweaking.

Conclusion

Building internal tiered API’s is a little bit of a hand-wavy art. It’s impossible to give concrete examples because every system is different. What I can suggest is that you listen to your gut and constantly evaluate if the lego blocks you are using feel like they are fitting right, minimizing repetition, and maximizing re-usablity. It is ok to duplicate things if they act as linear separations between layers. Contracts and distinct lines drawn are important and they let you iterate safely at different locations in your codebase.

I’ve been on call thousands of times in my career with varying degrees of annoyance and I got to thinking about how I view on call, what it is, what’s the responsibility, and how to improve it when it’s painful.

What is it?

The on call person is a first responder. In my mind their only job is to be around off-hours to react when something horribly breaks. Things that go down during business hours the entire team should react to. Engineering is a team sport. Off hours, if paged the on call person should be able to quickly check some dashboards, graphs, logs and determine if something is on fire and whether they need to escalate to others to help out or do some sort of obvious manual intervention to get the system stable.

That’s basically it.

Subscribe now

What it isn’t

Defining on call is easy, but defining what on call isn’t requires a bit more exposition.

First, the on call person isn’t a dumping ground of “keep the lights on” tasks, and they certainly aren’t responsible for fixing the broken thing. Their only job is triage, provide an immediate response (if it’s obvious) and otherwise it’s just to say ACK and raise the alarm.

Some teams like to have the on call person be the point of contact for team questions during the week and thats mostly fine, but those again should be not super invasive. Like with code you can minimize interruptions by DRY - create FAQ’s, document your systems, invest time into self-serve solutions. Being pestered all day is just as bad as being paged all day.

In fact pages should be rare. For about 4 years I was the primary on call at a small startup (by nature of being the founding engineer on a very small team) and I was paged only a handful of times the entire time. This was an ideal on call scenario! It’s not that we ignored pages, its that we prioritized making sure that things don’t page. That means making systems resilient, and being actionable to every page to ensure it doesn’t happen again (unless things are critically on fire).

The on call person should mostly be able to live their lives without even realizing they are on call. It should not be impactful, and certainly people should feel empowered to go to the gym, shop, eat out, during their on call time. Paired with a backup secondary, and a tertiary manager based escalation, the amount of times all 3 people are gone is very rare.

I have been on teams where people are terrified to be away from a laptop for more than a few minutes and their lives grind to a halt for their on call time. I think it’s annoying and a sign of poor team health when people are coordinating minute-to-minute “I will be AFK from 2:15 to 2:32”. This is not healthy, and is in fact extraordinarily toxic because it self selects for people without families, hobbies, caretakers, etc. It doesn’t have to be this way!

Making on call better

What do you do if your on call is a dumpster fire? There’s a handful of ways to improve this process and if you have executive and leadership buy in this is absolutely possible to improve.1

The rule for improving on call long term is to be actionable about a page. An ignored page is one that will come back. There are four actions that should be considered every single time a page occurs

• Tune - If the alert is too sensitive, tune it. Should the alert page you at all? Is it actually important that someone at 2am gets up to look at this? Should you adopt a multi tiered paging model of business hours vs off-hours paging? I discuss topic this in my book “Building a Startup - A primer for the individual contributor” in depth (due to be released this spring).

• Remove - Ask yourself, does this alert even need to exist. If you just ack the alert and it goes away, or you mute it and move on, then delete it. Be ruthless in your decisioning.

• Fix - Regardless if the alert was real or not, is there a way to fix the underlying issue? If a queue is backing up and you are alerting on it, a fix can be to auto scale workers based on queue size. If a 3rd party provider is failing with 429’s (rate limited) can you slow down your request rate programmatically? What other ways can you do to make your system more resilient. Can you detect errors ahead of time and react to them? Can you repair broken data automatically? Can you use twilio to call the support of a bad banking processor partner2?

• Escalate - If the world is actually on fire the on call person is not going to fix this themselves and needs to put the bat-signal up. This is the only acceptable time something should page someone in my mind.

There is no world where a page does result in one of these actions. Teams that don’t take action are doomed to have tough on call rotations, and that breeds resentment and burnout. Teams that ignore pages or do an “ack and wait” model since things “tend to resolve themselves” are basically telling each other “we don’t value your time”.

Alerts that actually do fire should include information about why they were set up in the first place. What are they monitoring, why are they monitoring it?

Runbooks3 are common to include on alerts, but in my mind the runbook should not be “here is how you fix it”, instead the runbook should be “here is how you validate the severity of this and get more details”. If the runbook describes a step by step situation to solve a page can it be programmatically enforced? Obviously some things like “the MySQL instance needs to be restarted because this rare scenario has occurred” is not worth programmatically controlling but this alert should also be extraordinarily rare!

Reacting to pages takes a lot of work, especially if they’ve been neglected for a long time. But it is possible to make improvements, the team just needs to be militant about responding to each and every one. Avoid the temptation to have dedicated time to improve things, that never works. When you fall into the trap of doing “let’s have a sprint where we just squash bugs” you’re telling the team that reacting continuously doesn’t matter and that it’s ok to ignore things when its not part of regular scheduled programming.

To me, off hours pages should be treated like a catastrophe. It’s all-hands-on-deck the next morning if a page happened and the entire world isn’t melting. Every person on the team should be clamoring to make improvements, because it’s could very well happen on their watch next time!

If an incident does happen the team needs to do work to make sure that the cause of that incident can’t ever happen again. I like to frame a lot of my architectural decisions in how much I hate getting paged. I mull on every failure mode and whether the system is resilient enough to handle it. In the beginning it feels a little like whack-a-mole, plugging holes as you find them. But over time, systems become hardened, almost bullet proof, if teams actually follow through on it.

With practice this mindset becomes part of every system design, so you build resilient systems first instead of building reactive systems that need to become resilient later.

Conclusion

Investing time and energy into improving the on call process can massively impact the morale of a team. Not to mention this is a huge selling point in recruitment - if you care about the on call process people will clamor to join your team, and you can encourage the best and brightest to do their best work with fewer distractions.

A large part of the on call process is cultural - what does your team care about and how do they prioritize their work? Teams that don’t iterate and action on the on call process tend to sink under their own weight and it becomes a self fulfilling cycle of failure. Pages get ignored, then real issues get missed, incidents take longer to resolve, leadership gets frustrated, the team feels more pressure, goto 1.

With focus and determination, the on call process can be easy and forgettable!

Recently I was working on building a templating engine1 that allows external stakeholders to create display based templates based on some predicate matching, i.e. if this template matches some values then apply it and use the override result. In this case a template is an ERB block to evaluate. ERB is a templating syntax that comes out of the box in Ruby. For example:

when:
  some_field: <matches some value>
override:
  some_other_field: "This is an erb block <%= value %>.

A simple example using ERB to render this would look like

ERB.new("This is an erb block <%= value %>.").result_with_hash({
  value: 123
})

Which generates

This is an erb block 123.

Cool. But what happens if value is missing? ERB is going to be totally fine with that and just render

This is an erb block  .

Notice the missing ending value.

I wanted to ensure that we could prevent a user from ever allowing a template to process empty strings creating nonsensical ERB resultant data.

To do that I opted to find a way to break down the ERB template string into its component parts, effectively an AST of the ERB syntax. By getting access to an AST we could then render each segment and validate that its result is non empty. AST’s are fun and it’s a great way to think about a problem when you decompose something into its tree based metadata. AST’s let you lean into your favorite LISPism: “code is data”.

Thankfully I didn’t have to write an ERB parser, and was able to find one called Temple. Using Temple we can break down the ERB AST now by simple doing:

template = "This is an erb block <%= value %>."

erb_parser = Temple::ERB::Parser.new

erb_parser.call(template)

Which will give us a loosely structured hash based syntax tree of the form

[
  [0] :multi,
  [1] [
    [0] :static,
    [1] "This is an erb block "
  ],
  [2] [
    [0] :escape,
    [1] false,
    [2] [
      [0] :dynamic,
      [1] " value "
    ]
  ],
  [3] [
    [0] :static,
    [1] "."
  ]
]

Kind of weird, but I can work with this. At this point we just need to find all values where the first array element (I wish this was structured data and not just positional array fields2) is the symbol :dynamic, and in my research I had already stumbled on a nice gist of someone gracious enough to share just such a function saving me a little bit of time. Thank you stranger!

def extract_dynamics(ast)
  dynamics = []
  ast.each do |node|
    next unless node.is_a?(Array)
    if node[0] == :dynamic
      dynamics << node
    else
      dynamics += extract_dynamics(node)
    end
  end
  dynamics
end

Once we have all the dynamic blocks we can then map the values and re-render them against the context:

template = "This is an erb block <%= value %>."

erb_parser = Temple::ERB::Parser.new

ast = erb_parser.call(template)

dynamic_data = extract_dynamics(ast)

all_present =
  dynamic_data.all? do |entry|
    # individually render each component of the erb 
    # template and validate if they are all
    # non blank. This ensures we don't accidentally 
    # have a "nil" or "" value rendered in the final resulting string

    ERB.new("<%= #{entry} %>").result(render_api.bind).present?
  end

From here we can decision on what to do and prevent accidental errors during dynamic template invocation.

Because there is no way to catch this at template write time (given it’s dynamic yml files provided by external parties) we can make the right thing safe by validating it template time.

Anytime I work on some sort of platform, SDK, or library, I like to think about how I can minimize errors of the consumer and maximize feedback back to them. Since we know the the ERB sub-block we are evaluating we can even tell the consumer that the particular block is empty, giving them some nice discoverability instead of wondering why something is missing.

I’ve come around, slowly, to the power that dynamic ruby has. I still much prefer to add sorbet typing to everything as much as possible and to avoid working with untyped hashes, but sometimes dropping into the magic of meta-programming can be really impressive.

Subscribe now

interface DynamicNode {
   type: 'dynamic',
   value: string
}

interface StaticNode {
   type: 'static',
   value: String
}

interface Escaped {
   type: 'escape',
   isEscaped: boolean,
   node: DynamicNode
}

type Node = DynamicNode | StaticNode | Escaped

Why is this a problem? Well it’s because most everything in your application uses data! Data informs the domain model, it determines how classes and data objects are structured, and that then propagates upward into how relationships and code is structured. If the data is poorly managed then often the higher level API and abstractions become cludgy and difficult to use. Direct REST API’s or CRUD from gRPC/Thrift/etc can leak the difficult data model requiring disjoint inputs just to do basic work. These issues all stem from a database that isn’t structured well or wasn’t given enough love and care to be able to properly model the domain it’s storing.

Naming is hard

First and foremost is naming. Names are hard but changing column or field names in a database after the fact can be daunting, especially if the data size is enormous because migrations can be error prone, take a long time, and sometimes require application downtime (which nobody likes to do). Do the names match the domain they are modeling? Are they descriptive? Short cuts and acronyms aren’t worth the effort.

If its an “account_id” then don’t call it “acc_id”. Long term think about who is going to read this content: other developers, data scientists, you (after you’ve forgotten everything). Ensure those names are consistent with other names as well! If you’re using snake case, keep snake case everywhere. Consistency helps minimize cognitive load, people will learn and understand patterns from the consistent naming conventions. Don’t reference “account_id” in one table, and “account” in another table.

When modeling cross domain references I like to use context to prefix a foreign key id. For example, if we have a users table that references an internal and external account id’s (but both fields are foreign keys to the “accounts” table) then we might want to call them “external_account_id” and “internal_account_id”.

users
id, name, external_account_id, internal_account_id

Keeping with a logical convention, however you choose, makes it easy to reason about the data system and to understand what links and relationships data has.

Consider also how you name or your foreign keys. Use a consistent naming pattern. It might be

fk_<source_table>_<field>_<foreign table>_<field>

like “fk_users_external_account_id_accounts_id” which now tells someone

users.external_account_id -> accounts.id

Names aren’t just important in relational storage either, they have just as much, if not more importance, in non-relational data because in a No-SQL world different key elements can have different shape’d data. Shapes are important, and ensuring that the shapes you store are consistent across the semantic type they represent is important. I often say that just because No-SQL is schema-less doesn’t mean you don’t enforce a schema. The opposite in fact! You have to enforce the schema in application code now, and without a clear and unified schema managing your data becomes a nightmare. What data is being represented by a specific key/value pair? Is that data correct? What does changing that data mean when you start to add/remove new fields?

In the No-SQL world I strongly recommend storing data blobs with version and type fields, so that you can disambiguate how data maps to model classes in code. For example, using a JSON format like below can make your key-value storage data easily consumable and discoverable as well as safe from compatibility mismatches

{
   version: 1,
   type: "foo",
   data: {
     ... your fields ..
   }
}

Sources of truth

The next thing I often consider, after agonizing over how to name things, is what the relationship of data is and what to store in a particular table (or data blob). For example, why are we storing? Do we have to it? What does it mean to store this data? The goal of this thought exercise is to make sure that we’re storing exactly what we need, we’re not denormalizing it (or if we are it’s for good reason), and that adding this particular field creates value for us long term. I’ve seen it in in certain places where people store things like

table
id, name, event_payload, first_name, name_title, name2, event_name

It seems contrived, but we’ve probably all seen some travesties of data modeling that look similar. In this example we have 4 different name types, and presumably some raw data in “event_payload” that maybe “event_name” is part of. In this example, I’d argue that we didn’t think clearly about what it means to store a name and we didn’t think clearly about what it means to store parts (or all) of an event. Who is the source of truth here? Not to mention is the name being stored somewhere else as well? Is it cross databases? Is there a centralized way to query and get that data?

Without clear sources of truth and ownership questions tend to arise of how you manage and update this data. What do you need to actually do an update? Is the update valid? Do you have to update multiple places at once? All these questions can arise!

Assuming this is the one and only place this name has to exist, I’d probably recommend (in a relational world) virtual columns to project data from a blob to make querying easier, or just store compound data as a JSON blob in a field. For example, if we created a unified name object:

interface Name {
  given_name: string
  family_name: string
  middle_name?: string
  ..
}

Then we can pair data together as a group, update it as a group, and query it more sanely. Without proper pairing of data we might end up having to query or join multiple tables to materialize a person’s full name. This makes it a difficult and frustrating experience for developers to answer basic questions.

Smell tests

In general, I like to use a litmus test of “how easy is it to do basic CRUD and answer domain related questions”. If your domain has user data split across 4 tables and they are always queried together… are they really actually separate? Should it all be flattened in one? Conversely if you have disparate data mashed into one table should it logically be split out and referenced separately? When CRUD is complex, either by requiring multiple different calls, or complex SQL to be written that’s signal that something is amiss. In my experience non trivial queries tend to be limited in scope, and most things do not require more than a handful of joins.

I also like to make sure that data is never contextual. You should never have a field that means X sometimes and Y other times. Having this kind of implicit codification based on data state creates challenges in reasoning about systems. If you have conditional data, pair it together as a blob or types or add explicit metadata about what you need to be storing. Contextual examples might be something like “sometimes field A is the user id and sometimes it’s the account id”. This happens a lot when designing single table No-SQL schemas where columns get overloaded based on context of what the key is. Woe is the engineer who has to decipher this spaghetti.

Conclusion

Working with data is challenging because data often changes over time. The domain you think you are modeling now might not be the domain you end up with over time. However, data is the most basic layer of abstraction in your system. If the data is hard to work with, all layers above it will be hard to work with. Giving the data model the respect and diligence it deserves pays off dividends over time.

• Proxy

• Decorator

• Adapters

Wrappers usually wrap an object and provide either extra functionality explicity (exposing new methods) or implicity (providing the same contract as the wrapped object). Lets look at some examples. Wrappers come in many names: adapter, decorator, proxy, but they all tend to do the same thing. They wrap something.

Personally I find that this is maybe the most useful pattern out there. It’s so common I hardly ever think about the names of the patterns anymore. I should note that just because you use a pattern doesn’t necessitate the name of the pattern being in the class/interface name.

Proxy

A proxy is a wrapper that takes a thing but gives the contract of the underlying object.

Lets pretend we have a state trait that defines how to get and put something based on a key.

trait State {
  def save[T](key: String, item: T): Unit
  def get[T](key: String): Option[T]
  def delete(key: State)
}

We can have a simple implementation of our state:

class InMemoryState extends State {
  private val map = new mutable.HashMap[String, Any]
  
  override def save[T](key: String, item: T): Unit = {
     map.add(key, item)
  }
  
  override def get[T](key: String): Option[T] = {
     map.get(item).map(_.asInstanceOf[T]))
  }
  
  override def delete(key: String): Unit = {
    map.remove(key)
  }
}

This works great. But what if we want to now add monitoring to our state object. We can obviously put monitoring into our state object:

class InMemoryState extends State {
  private val monitor: Monitoring =  ...
  
  private val map = new mutable.HashMap[String, Any]
  
  override def save[T](key: String, item: T): Unit = {
     monitor.increment("save")
     
     map.add(key, item)
  }
  
  override def get[T](key: String): Option[T] = {
     monitor.increment("get")
  
     map.get(item).map(_.asInstanceOf[T]))
  }
  
  override def remove(key: String): Unit = {
    monitor.increment("remove")
    
    map.remove(key)
  }
}

But the problem here is for every state implementation we create, we need to make sure we consistently add monitoring to it. Especially problematic is if the state implementation is not owned by us. Maybe this InMemoryState comes from a 3rd party library. How do we add monitoring to it? Enter the proxy.

The proxy will take an instance of State, but also implement State and proxy calls to the inner object. For each call it will add monitoring itself:

class MonitoringStateProxy(state: State) extends State {
  private val monitor: Monitoring =  ...
       
  override def save[T](key: String, item: T): Unit = {
     monitor.increment("save")
     
     state.save(key, item)
  }
  
  override def get[T](key: String): Option[T] = {
     monitor.increment("get")
  
     state.get[T](item)
  }
  
  override def remove(key: String): Unit = {
    monitor.increment("remove")
      
    state.remove(key)
  }
}

What we have now is a proxy. Anyone who accepts an instance of State won’t know, or care that the state they got isn’t the one they used to use. We can now easily add monitoring to any state implementation!

This kind of stuff is especially useful for things like

• Extra failure handling or fallback functionality on failure

• Consistent monitoring or logging

• Controlling access to resources (via permissions/etc)

• Adding caching to heavy resources

The big win is the fact that the proxy implements the same trait as the underlying resource. This forces the proxy to handle all methods of the trait so if the trait changes, the proxy has to follow suit.

Decorator

A decorator is like a proxy++. It usually (but not always) implements the source interface but also may implement other interfaces OR just expose extra methods itself.

Lets make a new state object, but this time lets add some extra methods that let us fix keys based on a prefix:

class FixedState(state: State) extends State { 
  override def save[T](key: String, item: T): Unit = {            
     state.save(key, item)
  }
  
  override def get[T](key: String): Option[T] = {    
     state.get[T](item)
  }
  
  override def remove(key: String): Unit = {   
    state.remove(key)
  }
  
  def saveWithPrefix(prefix: String, key: String, item: T): Unit = {
    save[T](prefix + key, item)
  }
}

Notice the new saveWithPrefix method. What we’ve done is taken an object of type State and “decorated” it with new functionality. The scala world does this all the time with Rich methods (also known as the Pimping pattern). If we were to rewrite this with the Rich extension method pattern, we’d do it like:

object StateExtensions {
  implicit class FixedState(state: State) {    
    def saveWithPrefix(prefix: String, key: String, item: T): Unit = {
      state.save[T](prefix + key, item)
    }
  }
}

To use this, we’d do:

// this provides the implicit conversion to a `FixedState` object
import StateExtensions._ 

val state: State = ...

state.saveWithPrefix(prefix = "foo", key = "bar", value = 123)

This effectively acts as a decorator pattern even though it doesn’t actually implement the same interface.

Adapters

Adapters are a way to square peg round hole something. Imagine you have two inconsistent interfaces

trait Dog {
  // returns a byte stream of a bark sound
  def bark(): AudioStream
}

trait Cat {
  // returns a byte stream of a cat sound
  def meow(): AudioStream
}

And you have some code that takes a Dog. But lets say you only have a Cat laying around. Is it possible to transform a cat into a dog in such a way that the code that wants the dog can take a cat? Maybe…

class CatLikeDog(cat: Cat) extends Dog {
  def bark(): AudioStream = {  
    cat.meow()
  }
}

Ok, I will admit this isn’t a great example, a cat isn’t a dog. But… it can pass as a dog (albeit a weirdly meowing one)! The idea behind an adapter is to do provide the adaptation mechanism to allow an object to behave like something it’s not. In a non trivial example the adapter may take several other classes and utilities that help it achieve its goal.

Visitor pattern is one that people often times have a hard time grokking. This entire pattern predicates on the fact that a thing called dynamic dispatch exists. Dynamic dispatch is the fact that overloaded methods are determined at runtime based on the referred to argument type.

Given that, suppose we have the following base and derived classes:

class Base()
class Derived() extends Base

object Runner {
  def method(base: Base) = println("base")
  def method(derived: Derived) = println("derived")
}

If we were to call the overloaded methods with the following types:

val derived: Base = new Derived()
val base: Base = new Base()

Runner.method(base)
Runner.method(derived) 

What do you expect to be printed out? Turns out its

base
base

This is because the dispatcher selects the method based on the referred type, which we explicitly defined as Base. How do we get this to do what we want which is to call derived for runtime types of derived without changing the variable definition?

The answer here is double dispatch. We need the method overloading to see the referred type as Derived and not actually as Base. What if we refactor things a bit to look like:

class Runner {
  def method(base: Base) = {
    println("base")
  }

  def method(derived: Derived) = {
    println("derived")
  }
}

class Base {
  def execute(runner: Runner) = {
    runner.method(this)
  }
}

class Derived extends Base {
  override def execute(runner: Runner) = {
    runner.method(this)
  }
}

And we do now:

val base: Base = new Base
val derived: Base = new Derived
val runner = new Runner

base.execute(runner)
derived.execute(runner)

This now prints out:

base
deried

What’s the difference? The difference is that in the first example the compiler determined the overloaded method to call because we explicity said that the types were of type Base. To the compiler there’s no question about what the type is, it doesn’t care that the runtime value of Base may be Derived. However, in the second example, we forced an instance of Derived to call into the runner so that the overloaded method resolves to the Derived instance and not the Base instance!

In this scenario, we did a double dispatch. We first dispatched on the Base object calling execute, then in execute we dispatched again back to runner which resolves the correct overloaded method.

Why is this useful? Double Dispatch is used to invoke an overloaded method where the parameters vary among an inheritance hierarchy [1]. This is especially handy when traversing things like trees or other hierarchies. Imagine we have a really crappy programming language that consists only of if statements, boolean expressions, and brackets:

sealed trait SyntaxNode
case class If(predicate: SyntaxNode, body: SyntaxNode) extends SyntaxNode
case class Block(statements: List[SyntaxNode]) extends SyntaxNode
case class BooleanExpression() extends SyntaxNode

And we want traverse over a tree of this language starting at a block statement and print out the structure of the language. Later if the structure is OK, we’ll want to actually run the language. Given we want to iterate over the tree in two separate ways, lets make sure the iteration is decoupled from the implementation of the iteration. First lets define a visitor that defines the overloaded methods we want to handle:

trait SyntaxVisitor {
  def visit(ifNode: If)
  def visit(blocK: Block)
  def visit(boolean: BooleanExpression)
}

Lets also make sure that syntax nodes are forced to implement a visitor with another trait:

trait SyntaxVisitable {
  def accept(visitor: SyntaxVisitor)
}

sealed trait SyntaxNode extends SyntaxVisitable

Now all our nodes have to implement an accept method that takes a visitor. The methods will all look the same:

case class If(predicate: SyntaxNode, body: SyntaxNode) extends SyntaxNode {
 override def accept(visitor: SyntaxVisitor) = visitor.visit(this)
}

case class Block(statements: List[SyntaxNode]) extends SyntaxNode {
  override def accept(visitor: SyntaxVisitor) = visitor.visit(this)
}

case class BooleanExpression() extends SyntaxNode {
  override def accept(visitor: SyntaxVisitor) = visitor.visit(this)
}

Now, we can actually implement a visitor and apply it. Remember, the visitors job is to just be able to properly see the runtime type! It’s just a way of forcing the method overloading to work the way you want it to

class SimpleVisitor extends SyntaxVisitor {
  override def visit(ifNode: If): Unit = {
    println("Got an if")
    ifNode.predicate.accept(this)
    ifNode.body.accept(this)
  }

  override def visit(blocK: Block): Unit = {
    println("Got a block")
    blocK.statements.foreach(_.accept(this))
  }

  override def visit(boolean: BooleanExpression): Unit = {
    println("Got a boolean expression")
  }
}

And if we run this:

val language = Block(List(
  If(BooleanExpression(), Block(List(
    BooleanExpression()
  )))
))

new SimpleVisitor().visit(language)

We get the result of:

Got a block
Got an if
Got a boolean expression
Got a block
Got a boolean expression

Ok, neat, but so what. The real power in this pattern is that the tree (or inheritance heirarchy) doesn’t care WHO is iterating over it. Just that someone IS iterating over it. We can make a different visitor that maybe actually runs our language. Or maybe we have one that lints that the syntax is correct. This is actually a really common pattern with language design. You have visitors that parse a syntax tree, you have visitors that compile a syntax tree, and you may have visitors that help you debug and print a syntax tree. This lets you decouple how the tree is walked from the actual tree!

Visitors are difficult to wrap your head around because you have item A calling item B which calls item A again, and we’re tricking the runtime into executing the correct overloaded method due to the fact that overloaded methods are determined at runtime by the type of the argument.

In languages like scala that support pattern matching you don’t really need to do a visitor pattern since you can force pattern matching on exhaustive traits to do the same kind of thing:

def matchLang(node: SyntaxNode): Unit = {
  node match {
    case If(predicate, body) =>
      println("If!")
      matchLang(predicate)
      matchLang(body)
    case Block(statements) =>
      println("Block!")
      statements.foreach(matchLang)
    case BooleanExpression() =>
      println("Boolean!")
  }
}

matchLang(language)

But we’re not really relying on dispatching here, under the hood this is akin to doing instanceOf checks on items. On top of that, if we don’t specifically close the interface tree with a sealed trait AND we don’t enable warnings as errors, then it is possible for us to add new items to the node tree and forget to implement visitor overloads for it. However, with a visitor pattern when we add new types we MUST implement the visit method which is a great compile time safety net.

That said, visitors have other problems. The biggest one is that in order to return a result they tend to mutate themselves. This is because if we wanted to be able to return a result, we have to tell the visitor it can return a result, and all the visito methods must also return that type. This means that we can’t have generic visitors that do anything. That said, you can totally make your visitor return a type if you are sure the type will always be returned.

Subscribe now

[1] Double Dispatch is a Code Smell

The strategy pattern is just a fancy way of saying of providing several implementations to an interface and choosing which implementation to use based on some criteria. To quote wikipedia:

It defines a family of algorithms, encapsulates each algorithm, and makes the algorithms interchangeable within that family.

A great example is retrying. Retries are common in software. You have transient errors all the time: network connectivity issues, file system locking, timeouts, crashes, etc. Building robust software requires generous use of retries.. up to a point. You don’t want to retry forever and depending on what you are retrying you may want to retry a few times really quick and then give up, or maybe a couple times really quickly and then slower until a maximum amount of retries occurs, or maybe fast really quick then slow for a while and then peak out and try forever.

Each of those options is a strategy for how to retry. We can implement a strategy pattern pretty easily and lets do an example with retries.

trait RetryStrategy {
  def execute[T](block: => T): T
}

This defines a strategy that given some block of code block that returns some value of type T we’ll try to do it. If it fails, the strategy will retry until it either succeeds or the strategy gives up and throws an error.

Lets define a family of implementations:

case class ConsistentWaitRetry(
  maxTimes: Int, 
  waitBetweenFailuresMilli: Int
) extends RetryStrategy { ... }

case object NoRetry extends RetryStrategy { ... }

case class ExponentialRetry(
  maxTimes: Int, 
  waitBetweenFailuresMilli: Int,
  maxWaitTimeMilli: Int
) extends RetryStrategy { ... }

You can see we now can choose between 1 of these 3 stratgies whenever we want to do a retry. Anyone who needs to retry can accept the RetryStrategy trait and it shouldn’t care what the strategy actually is, just that its contract says it will retry in some fashion.

To drive the point home lets implement a simple consistent wait retry that retries every N milliseconds up to Y times

case class ConsistentWaitRetry(
  maxTimes: Int, 
  waitBetweenFailuresMilli: Int
) extends RetryStrategy {
  override def execute[T](block: => T): T = {
     var timesToRetry = maxTimes
     while(timesToRetry > 0) {
        Try(block) match {
          case Failure(ex) =>
            timesToRetry -= 1 
            Thread.sleep(waitBetweenFailuresMilli)
          case Success(value) =>
            return value
        } 
     }
     
     throw new RetryFailedException()
  }
}

Now we can use it like:

val strategy: Strategy = ConsistentWaitRetry(
  maxTimes = 10, 
  waitBetweenFailuresMilli = 10
)

strategy.execute {
  throw new RuntimeException("I am always failing!")
}

And this will retry the block 10 times, waiting 10 milliseconds between each retry until it fails with a hard error.

If we want to make other strategies, like ExponentialRetry or LinearRetry we can do that. The strategy part of this pattern is choosing which implementation to use given some criteria. Each implementation is a strategy, so however you decide to choose a strategy they are interchangeable due to adhering to a common contract. You can even do things now like:

class Worker(retryStrategy: RetryStrategy) {
  def doWork(): Unit = {
    retryStrategy.execute {
      // some work
    }
  }
}

class WorkerUser(name: String) {
  def doWork: Unit = {
    val strategy: RetryStrategy = name match {
      case "Anton" => 
        ConsistentWaitRetry(...)
      case "Joel" =>
        ExponentialRetry(...)
      case _ =>
        NoRetryStrategy()
    }
    
    new Worker(strategy).doWork()
  }
}

Assuming we’ve built out the corresponding strategies (and we can even build out our own custom ones!) we can now choose, given the criteria of the worker name, which strategy to use. The worker itself doesn’t care what the strategy is, just that it was given one.

This makes the strategy pattern really flexible in choosing how to execute something at runtime.

I often times ask an interview question for candindates to design pacman. One thing people struggle with is abstracting how the ghosts who chase pacman find pacman. This is a great way to provide strategies. Some examples of how a ghost can find pacman include

• Naively (randomly moving around)

• A* (tracking pacman and trying to find the least distance to them)

• AI (predict pacmans movements based on past movements)

If you abstract how the ghosts look for pacman, you can build a ghost class that is pluggable with how it finds pacman. This means instead of having 10 ghost (sub)classes that each handle how movement occurs, you can have 1 class who is given a strategy. This also is a great example of composition.

Subscribe now

Lets talk about singletons

THERE CAN BE ONLY ONE!!! Ok enough highlander stuff.

Singletons are where there is only ever one instance of the object in your application. Singletons can be static classes, or lazy instantiated classes that are only ever done once.
Good usages for singletons are to encapsulate heavy resources. Some examples may be database abstractions since they tend to have connection pools, or caches because caches are only effective if they’re actually the same instance, etc.

For all the good that singletons give, they get a lot of bad press. This is for a few reasons.

Problem 1: Hidden resource problem

One is that when they are used incorrectly they tend to hide that they are being used. Here’s what I mean:

// a database singleton
object Database {
  def read()
  def write()
}

class User {
   def getName() = {
      Database.read()
   }
}

class Main {
  new User().getName()
}

There’s a few problems here:

1. Looking at just the signature of User you don’t know that it uses a database

2. Even if you did know it used a database, how do you test this without providing an actual database? Or a local in memory database? Or a mock database?

Scala makes this a little simpler for us in that we can pass singletons (static classes, aka companion objects) around as instances, but other languages like Java/C#/etc do not. In those languages you tend to “reach in” to resources directly without having them passed to you.

Another way of providing a singleton is to instantiate only one class ever:

object DatabaseProvider {
  // the only database we'll make
  lazy val database = new Database
}
// a database class
class Database {
  
  def read()
  def write()
}

class User(database: Database) {
   def getName() = {
      database.read()
   }
}

class Main {
  new User(DatabaseProvider.database).getName()
}

In this example we’re passing a database to the user class. The user class doesn’t know or care that the database is effectively a singleton. All it cares is that it got a database! Whether we instantiate the database in the companion object, or we provide it with a dependency injection framework (like spring/guice/whatever), or we create one instance of it in Main and pass it around doesn’t matter anymore. It’s still effectively the same instance floating around.

Making singletons by hand without the functionality of keywords like lazy in scala can be tricky, especially if you are lazily making singletons in a threaded environment. For those interested in reading more about that they should read up on singleton double checked locking

Problem 2: Thread safety

While that fixes the hidden resource problem, singletons still exhibit other potential land-mines. When you have only one instance of an object in your codebase you need to be very aware of the concurrent usages of your class. If you are in a multi-threaded environment then your singleton must be threadsafe. Without thread safety you risk corruption, especially if your singleton contains any kind state (like a cache).

In general, it’s best to avoid singletons for everything but when they are actually needed. In most cases objects are cheap to instantiate and cheap to collect (most objects are very short lived) so don’t be afraid to make new instances of something when you need to.

Subscribe now

Factories make objects. The first thing someone new to factories tends to ask is

why use a factory? isn’t new a factory?

Yes, it is! It’s probably the most “raw” factory you have. It’s a way of creating objects. But sometimes creating objects gets complicated. Or sometimes you want to hide away how something is made and expose a way to get “new” instances of something without having to know all the inner details of how to create it.

Lets build a house:

trait Feature
case object Bed extends Feature
case object Dresser extends Feature
case object Sink extends Feature
case object Stove extends Feature
case object Toilet extends Feature
case object Tv extends Feature
case object Couch extends Feature

case class Dimensions(x: Int, y: Int)

case class Room(
  dimensions: Dimensions,
  features: List[Feature]
)
case class House(  
  rooms: List[Room]
)

We’ve built a pretty generic house structure. A house is some set of rooms where each room has some size and set of features. We can build any house we want! But, it might be nice to provide some easy way to build out common rooms:

object HouseFactory {
  def small = Dimensions(10, 10)
  def large = Dimensions(100, 100)
  
  def bedroom(size: Dimensions = small): Room = {
    Room(size, features = List(
      Bed, Dresser
    ))
  }
  
  def kitchen(size: Dimensions = small): Room = {
    Room(size, features = List(
      Sink, Stove
    ))
  }
  
  def bathroom(size: Dimensions = small): Room = {
    Room(size, features = List(
      Sink, Toilet
    ))
  }
  
  def living(size: Dimensions = small): Room = {
    Room(size, features = List(
      Couch, Tv
    ))
  }
}

Now we have a simple way of creating general rooms in our house. Nothing is stopping us from making a house that is more complicated, but if we want to piece together a 1 bedroom apartament, or a shack in the woods, we can probably do that pretty easily now:

// no bathroom since we have an outhouse
val shack = House(rooms = List(
   HouseFactory.bedroom(),
   HouseFactory.kitchen()
))

val oneBedroom = House(rooms = List(
  Housefactory.bedroom(),
  HouseFactory.kitchen(),
  HouseFactory.bathroom(),
  HouseFactory.living()
))

The power of a factory comes in providing sane and semantic ways of creating complicated objects and being able to pass around that factory as an object itself! Imagine another level of abstraction so that we can have different house factories depending on the criteria we want:

trait HouseFactory {
  def bedroom(): Room
  def bathroom(): Room
  def kithen(): Room
  def living(): Room
}

Lets make a fancy house factory with some extra features:

case object Butler extends Feature

class FancyHouseFactory extends HouseFactory {
  val huge = Dimensions(1000, 1000)
  
  // two beds, two sinks
  def bedroom: Room = {
    Room(size = huge, features = List(
      Bed, Bed, Dresser, Sink, Sink
    ))
  }
   
  def bathroom: Room = {
    Room(size = huge, features = List(
      Sink, Sink, Couch, Tv
    ))
  }
  
  def kithen: Room = {
    Room(size = huge, features = List(
        Sink, Sink, Stove, Stove, Tv
      ))
    }
  }
  
  def living: Room = {
    Room(size = huge, features = List(
        Butler, Tv, Couch, Couch
      ))
    }
  }
}

Oh man, faaaancy. We can now pass around a HouseFactory trait and they’ll all get fancy houses now. If we want to make a CrappyHouseFactory we could do that too. Once we have all these factories, we can apply things like Strategies to choose with factory to use.

In object oriented programming inheritance is a big deal (pun intended). It defines that objects have heirarchical relationships of an “is-a” type. For example a dog “is-a” animal. So you can model dog by having an animal class:

class Animal()

and having Dog subclass Animal:

class Dog() extends Animal

In many cases this makes sense. Having inheritance lets you leverage structural properties like covariance and contravariance and lets you define logical heirarchies.

Where inheritance falls flat is in code re-use. A lot times you will see code in the wild like:

// defines DB queries
class Database()
class Animal() extends Database
class Dog extends Animal()

What you see here is that we’ve defined a dog to be an animal. That makes sense. But an animal is also a… database?

Certainly that doesn’t make sense to be able to pass an animal to code that expects a database. But, often times you do see heirarchies like this because they look to solve code re-use issues. After all you want to DRY (don’t repeat yourself). This is especially abusive in languages that support mix-ins or multiple inheritance.

What we really want here isn’t an is a relationship, we want a has a relationship. To get ourselves out of is a and into has a we can compose objects of other objects. We can refactor this code smell into a heirarchy that makes sense while still allowing us to re-use code:

// defines DB queries
class Database()
class Animal(db: Database) 
class Dog(db: Database) extends Animal(db)

Now its clear that a Dog is a Animal and that Animal has a a database. Advantages to this are

• Well defined object heirarchies that actually model the problem domain

• Easily testable components independently (a database can be tested outside of dog/animal)

• Extensible in you can now add other components and not worry about inheritance conflicts

• Explicit dependency information

This site is an attempt at boiling down some of the Gang of Four nomenclature and to provide simple examples that a reader can extrapolate upon. In my own original readings of Gang of Four I was constantly confused about the subtleties of one pattern over another. While Gang of Four is an extremely thorough in analyzing and defining the patterns, I think that c++/smalltalk plus UML diagrams are not necessarily the best way to grok what are extremely valuable tools in the engineers toolbox.

Some claim that design patterns are a smell, that they are used to cover up warts in your language or tools. I wouldn’t agree. While I do think that many usages of design patterns are wildly overused, just like with any tool you should recognize when is the right time to use it and when to not.

While the examples here are written in Scala I don’t think that the patterns in Gang of Four are mutually exclusive to functional programming. In fact I think they can be used in conjunction with functional paradigms like immutability, recursion, monads, higher kinded types, etc. But in all fairness, some patterns may not translate well to a functional world just like some category theory concepts don’t translate well to object oriented models. This document won’t delve into functional concepts and I’ll leave that for another day.

In either case, it’s important to recognize patterns, whether you agree with them or not, because they are part of our shared professional jargon. Understanding the nuances between a decorator and an adapter means you can communicate more clearly to your peers. That said, I don’t want to encourage anyone to fall into the classical AbstractSingletonProxyFactoryBean joke.

These design patterns tend to be simple and concise. However, no matter what the pattern is the most common thread among all of them is code to the contract. This means working at abstractions where you don’t care what the implementation is. This is the biggest challenge I think that engineers just getting started face and when they overcome this hurdle their code tends to become cleaner, shorter, and less error prone.

In the end, like with all tools, patterns are there to help you go fast, go correct, and go forth. Let’s begin.

If the user id’s are evenly distributed in a mathematical space (say they’re a UUID), you can leverage this to project the distributed ID onto a number line, scale the number to between 0 and 1, and then use that as a flag for if the user gets the feature X or not.

Lets look at an example:

import { createHash } from 'crypto';

/**
 * Hashes the value consistently to a space between 0 and 1. Can be used for determining if certain
 * id's have feature flags enabled automatically (for probabilistic rollouts, etc)
 * @param value The value to convert to a number and convert to a 0 to 1 representation on the number line
 * @param offset The offset on the number line to put the hashed value
 * @param scale Default scale is the entire space of md5 numbers
 */
export function consistentHash(value: string, offset = 0, scale = 1): number {
  const buffer = createHash('md5')
    .update(value)
    .digest();

  const hashNum = buffer.readUInt32BE(0);
  
  const maxMd5 = Math.pow(2, 32)
  
  const numberLineMax = maxMd5 * scale

  // scale the hash by the total number space
  return ((hashNum + offset) % numberLineMax) / numberLineMax;
}

If our user id 6f805e32-592e-46a2-95f3-51826f27e74f we can hash it to a projected value and

consistentHash('6f805e32-592e-46a2-95f3-51826f27e74f')

Gives us: 0.8326200970914215

So user 6f805e32-592e-46a2-95f3-51826f27e74f is always going to get a feature if its configured to be 83% likely. That’s kind of cool cause you don’t need to store the result of this anymore, you can just know.

Buut, now we have a new problem. What happens to poor ol user X who always hashes to 0.01. They get every experiment you throw at them because they’re constantly in the 1% group.

This is where the other paramters of our method come into play like offset. Imagine that we’ve projected the user onto the number line:

0----X-------------2^32

Our 2^32 comes from the fact that we are using md5 to hash the value and the max md5 value is 2^32, which would be the end of our number line.

We could offset our projected value and then loop it around the number line, to create a consistent but moved percentage

X+N

0----(X)-----------(X+N)--2^32

This now gives us the same distribution but allows our poor user to not always be in the 1% group. How do we choose the offset? Well we can hash any string to get a number, so we can use the experiment/feature ID to do that. Imagine your feature is "uses_new_login_flow" you can hash that to a number N and apply that to the number line.

Variants

Now that we can hash users to percentages, what if we could then do experiments with variants. For example, imagine we want to see what happens to 20% of our users if we give them one of 3 choices, where each choice is equally likley. Call the choices variants 1, 2, and a control. This happens all the time that we’d want to experiment on behavior and see what the outcomes are.

Putting a user into an experiment is easy, we just did that above. But how do we put someone into equal probability variants? If a user falls into 33% they’ll fall into the other 33% as well.

Lets walk through an example and see where it leads us.

Going back to our user6f805e32-592e-46a2-95f3-51826f27e74f who we know maps to 0.83. Lets say our experiment is to 50% of our users. Cool, we know this user is going to be in the experiment.

We now want to test to see if this user gets variant 1, 2 or our control (variant 3). What we want to do is deterministically dice roll this user to fall into 1 of 3 buckets consistently.

We know that the variants are equally probably, so we have a number line like this:

 1   2   control
0---|---|---?

What we don’t know is what the end of the number line should be and how to put someone on it.

What if we re-hashed the user id, but scaled it so that instead of the max number being the max of md5, that we scale it to

percentage_experiment * max_md5

ie.

0.5 * 2^32

We know the user falls into the bucket of 50%, so if we constrain the line to now be 50% of the max we can see where they now fall on that line and then test.

If we revisit the method we can now do

consistentHash('6f805e32-592e-46a2-95f3-51826f27e74f', 0, 0.5)

which now gives us 0.665240194182843

This tells us where on the number line this user would fit GIVEN they were already selected for the past experiment.

If we wanted to calculate 3 equally likley probabilities, we can divide the number line into probabilities and say

The first is from 0 to 0.33. The second is from 0.33 to 0.66 The third is from 0.66 to 1

So we can now see that 6f805e32-592e-46a2-95f3-51826f27e74f would fall into our second variant.

Doing this calculation is 100% reproducible, is O(1), and doesn’t require any external storage costs!

Conclusion

Doing consistent hashing (mapping distributed values to a number line) has a lot of fantastic uses, but does require that you have evenly distributed data to start with. If you have evenly distributed values leveraging consistent hashing can save storage and IO costs quite a bit, at least until you have something more complicated :p

JIRA is the beast everyone loves to hate. Rightfully so frankly, it’s slow, it’s tedious, it’s repetitive. But issue and project trackers like JIRA have value. As you become a more seasoned professional you realize that being able to see what you are working, what others are working on, and the state of the team in general is huge. It means you can plan projects in advance, share status with stakeholders, allow people to see what you’re doing and keep you from forgetting the things that are in play.

While other trackers exist (fogbugz, trello, etc) JIRA is by far the most widely used. For the longest time I hated using JIRA, because it didn’t map to my developer workflow. But lately I’ve been using a collection of ruby scripts that wrap the go-jira CLI that makes my life a whole lot easier.

For example, making a new ticket to track a random idea should be easy:

This fires off a ticket into my teams backlog and gives me a link. Now whenever I think to myself “it’d be great to do xyz” I can just pop open the shell and fire that idea off.

But we can take this so much further! I really like tying my commits to JIRA so I can see in the JIRA opened PR’s, merged PR’s, commits etc. To do that JIRA needs to integrate with github and know about your commits via (usually) a regex that looks for the jira ticket number in branch, commit, or PR titles.

Since I’m too lazy to ever remember doing that, I wanted to have my branches auto named with the current JIRA I’m on. If we have the ticket in the branch name, we can use git commit hooks to auto format commit messages that are prefixed with the JIRA.

# file: .git/hooks/prepare-commit-message-std #!/bin/sh COMMIT\_MSG\_FILE=$1 COMMIT\_SOURCE=$2 SHA1=$3 function detect\_branch\_name() { # of the format username-JIRA-3586/description # get the branch, remove the username, # split by / and take first segment, # split by - and take second 2 segments git branch | grep \* | cut -d ' ' -f2 | sed 's/$USER-//' | cut -d '/' -f1 | cut -d '-' -f2,3 } branch\_name=`detect_branch_name` if [["$branch\_name" =~ "TRAFFIC-"]]; then data=`cat $COMMIT_MSG_FILE` echo "$branch\_name - $data" \> $COMMIT\_MSG\_FILE fi

Now if I make a commit like:

~/src (akropp-JIRA-123/title) $ git commit -am "foo"

My git log would look something like

JIRA-123 - foo

What if we could automatically create a JIRA and set it to in progress and add it to my sprint when I make a branch?

Well we can! For example, we can make a new branch that

1. Creates a JIRA

2. Adds it to my sprint

3. Marks it in progress

4. Git checkouts with a standardized branch name scheme that works with my commit hooks

All in a single one liner! On top of that making a new branch can pull down the open JIRA’s I have and prompt me to start working on it if some exist! This way anytime I start a branch it’s automatically tracked. All the random one-offs I do at work now get visibility and I can see at the end of the sprint how much work I really did vs how much work we planned.

On top of that, imagine I have open JIRAs and my branch names have the JIRA name in them. I can then switch between working tickets easily by resuming work on a ticket

My code even works when I have multiple branches for the same JIRA, so if I have multiple logical flows for the same JIRA I can resume different subsections easily (great for large migrations!)

As always source code on my github

Note, a seasoned UI friend of mine told me this is a terrible way to do things and to instead use hooks instead

I decided to give react native a go the last few days, just to try something different. For what it’s worth I haven’t done any real front-end work in quite a bit. The last stuff I did was angular 2/(8/a million?) and it was internal dev tooling stuff that was kind of hacky.

Seems like the javascript world has come quite a distance since back when typescript was still in beta!

That said, I dove into a sample react native app that I scaffolded out using expo. Then I popped openn IntelliJ and started to play with react-native-paper and react-navigation.

The first thing I wanted to see how to do properly was to do DI, since in my mind without clearly having DI you can’t build large scale applications. There’s a lot of ways to do DI but you need to make sure to not couple your code to its dependencies. Seems like React’s answer to that is props, contexts, and redux. I haven’t looked much at redux yet, but my initial reaction to props and context was lackluster. Props smells like a giant property god object to pass around and that never ends well.

However, typescript does make this simpler and I do kind of like the higher order container stuff.

After making a little hello world app and playing with the react router, I wanted to see if we could type the router. For example, in all the examples I can see you use

this.props.navigation.navigate("Home")

To navigate around. This screams bad design to me because I can tell already people will start sprinkling in static strings everywhere, running into typos and other hard to track down problems. What I would rather have is

this.navigation.home()

Which is typed, discoverable, and easy to re-use.

However, it took me quite a bit to figure out how to inject

1. Capture an instance of props

2. Wrap the navigation object into an object we can type

3. Pass that object to a component

Turns out higher order containers were needed and the magic sauce is here. First I can make my routable class that given an instance of the screen router can do the things it wants to do.

type RoutableProps = { navigation: NavigationScreenProp\<any, any\> } class Routable { private navigation: NavigationScreenProp\<any, any\>; constructor(navigation: NavigationScreenProp\<any, any\>) { this.navigation = navigation; } home() { console.log("home") this.navigation.navigate("Home") } back() { console.log("back") this.navigation.goBack() } profile() { console.log("profile") this.navigation.navigate("Profile") } }

Assuming we have this (somehow) we can use it in our nav bar

interface NavProps { router: Routable } export class NavBar extends React.Component\<NavProps\> { render() { const navigate = this.props.router; return ( \<Appbar\> \<Appbar.BackAction onPress={() =\> navigate.back()}/\> \<Appbar.Action icon="archive" onPress={() =\> navigate.home()}/\> \<Appbar.Action icon="mail" onPress={() =\> navigate.profile()}/\> \</Appbar\> ); } }

Now here we can wrap a higher order container to capture the props, inject the setting we want, and return a new decorated component

function withRouter(Component) { // inject the navigation route to the props of the component return withNavigation( class extends React.Component\<RoutableProps\> { render() { const nav = this.props.navigation; // give the component a router return ( \<Component router={new Routable(nav)}/\> ); } } ); } export const Nav = withRouter(NavBar);

Now we can safely use our typed navigation component

export default function Home() { return ( \<PaperProvider theme={theme}\> \<SafeAreaView style={styles.bottom}\> \<Nav/\> \<Title\>Hello\</Title\> \</SafeAreaView\> \</PaperProvider\> ); }

And whamo! Typed navigation.

Full source available at my github https://github.com/devshorts/react-native-samples

In my mind the biggest most important tool of all for an engineer is their IDE. Lots of people have opinions on what flavor editor/IDE they want but given it’s nearly 2020, I think there are a few critical tooling features that must work otherwise they are a non starter.

1. Go to definition. It seems crazy to me that I have to spell this out, but code is data. Any editor should be able to semantically tell you the source of a method/class/function/variable. Grep is not that. Languages are context sensitive, and thats why you need a context sensitive system to help navigate it

2. Find references. Same as above, just like you can find the definition, you need to find who is using it. Again its context sensitive, I don’t want to someone to convince me that vim ctags or grep are good enough. They. Are. Not.

3. Semantic Autocomplete. CTAGS are not semantic autocomplete. CTAGS are the type of completion where an editor indexes the current file(s) and autocompletes based on what’s in the file. I see many developers using this and it pains me to no end. As an engineer my time isn’t worth reading scaladoc or ruby docs to find what arguments are to map. I need to hit dot and have the IDE tell me. And if I’m wrong, the IDE should squiggle it and tell me I’m wrong so I can fix it fast and move on. I’m not getting paid to be a compiler. That’s a compilers job.

4. Semantic Rename and Refactor. Again, semantics are key. Sed is not semantic. You need to be able to do repo wide renames, refactors, moves/etc, with confidence that the AST is still properly intact and semantically rigorous.

5. Integrated debugging and test running. Context switching to the CLI to run tests, and in particular a single test, is insane. While many would argue that test runners and debuggers can be done out of band, and they’re not wrong, but they are missing the fact that if you are doing this task hundreds of times a day (easily) then each moment spent typing or context switching adds up. In an incident any of those things that slow you down matter and have a monetary impact.

What do I mean by “semantic”? Using common CLI tools like sed, grep, is it possible to disambiguate a search for “Foo” in the context of A vs the context of B?:

class A { def Foo() String } 

class B { def Foo() String }

Could you refactor this:

class C(a A, b B){ 
   def Foo() String { 
     a.Foo() 
     b.Foo() 
   } 
}

To be

class C(a A, b B){ 
   def Foo() String { 
     a.Bar() 
     b.Foo() 
   } 
}

You can’t. And yet these situations arise constantly in day to day engineering. Good code is refactored code, and codebases that don’t get constantly improved upon rot and fall apart.

I feel like these requirements are so basic and primitive that I’m constantly shocked when I encounter organizations that run without them!

My thoughts are that anytime systems are being built we need to make sure that the tools continue to work with them. Tooling is important not because I’m too stupid to be a l337 hacker on the CLI, but because I’m smart enough to know that I don’t need to waste my time on that.