I'm working for years with my team on a pretty large Elixir / Phoenix code base, and although we are all very sensitive to unit and automated testing, we have been a bit lax regarding test coverage.

Over the past year, we started refactoring a large part of our UI from React to Phoenix Liveview with a more thorough testing approach (because LiveView testing is 🔥) and have been wondering: how much better is our test harness?

If you can’t measure it, you can’t improve it - Peter Drucker

In this article, I will help you build the appropriate tooling to track and measure your test coverage, and hopefully improve it.

Iterative feedback

Bot chat

As new code is produced and shipped, it’s very legit to question to wonder « is this code tested well enough to be merged? ».

The best place to have this kind of conversation between a contributor and a peer reviewer is a pull request (we use GitHub’s), so we will leverage our existing CI process to let us know how well our new feature is tested.

The screenshot above illustrates what we aim at:

• a PR comment, easy to scan & read 💬
• a short feedback loop (one should not have to wait for hours)  ⚡️
• fully automated (posted by your CI) 🤖

Tweak the CI

The first and foremost thing to do is to compute test coverage, locally. As Elixir developers, we are gifted with a wonderful library: ExCoveralls. It's quite simple to use (locally at least), and I will not give a thorough setup guide as the library itself is well documented.

When the library is set up you can run the following commands:

• mix coveralls.html to get a formatted HTML page displaying coverage information
• mix coveralls.json to get the same data as JSON which could be interesting for automation (beware, there is a plot twist 👀 )

Now, let's have a look at our CI. We have been using Semaphore CI for years and are very satisfied with it (this article is not sponsored whatsoever). But I'm certain you can use the CI tool of your choice, provided you have access to partition & caching features.

The screenshot above is our Semaphore pipeline. The important thing to notice is that our tests have been partitioned to get faster build feedback (please look at this post if you want more details about test partitioning).

Partitioning is very structuring in the way we compute code coverage: we cannot use mix coveralls.json as-is in a new CI task as it would re-run all tests in a lengthy monolithic process, defeating the purpose of test partitions.

What we need is a way to run test coverage, partition-wise, and then collect & merge all data (map-reduce style). To do so, we will switch to another coverage output format that can be easily merged afterward: LCOV.

First, we change the mix test call to mix coveralls.lcov that will both run tests and compute coverage, with partitions and umbrella app support. Test coverage output (the cover/lcov.info file) is stored in SemaphoreCI cache so that data can be retrieved in a subsequent job.

MIX_ENV=test MIX_TEST_PARTITION=$SEMAPHORE_JOB_INDEX mix coveralls.lcov --parallel --umbrella --partitions $SEMAPHORE_JOB_COUNT`
mv cover/lcov.info cover/lcov-$SEMAPHORE_JOB_INDEX.info
cache store coverage-$SEMAPHORE_WORKFLOW_ID-$SEMAPHORE_JOB_INDEX cover

Then we use an after_pipeline semaphore task to collect all coverage files and merge them using the lcov binary.

for i in {1..8}; do cache restore coverage-$SEMAPHORE_WORKFLOW_ID-$i; done
find ./cover -name "lcov-*.info" -exec echo -a {} \; | xargs lcov -o cover/lcov.info

At this moment, you have a single lcov.info file that contains data for the whole project:

~/code/elixir-project ⇒ lcov --summary cover/lcov.info
Reading tracefile coverage/lcov.info
Summary coverage rate:
  lines......: 59.6% (10440 of 17503 lines)
  functions..: no data found
  branches...: no data found

The next step is to post coverage data to GitHub. For that purpose, we're using the following shell script; I won't delve into tedious shell intricacies, but here is what you need to know about this script:

• we use gh CLI to check if a pull request has been opened for the current branch
• we parse the lcov.info output using lcov --summary, grep and cut
• we use the semaphore CI artifact API to store lcov.info for each branch. This way we can compare current branch coverage to master's (we do some basic maths with calc binary)
• we use gh once again to post the pull request comment, and store the comment URL into semaphore cache. This way later builds will be able to update a single PR comment instead of posting a new one each time.

You now have all the elements to get fast and iterative feedback on your application test coverage!

Improve your coverage

Pull request comments are great to get instant feedback regarding the global trend of test coverage, but it doesn't provide any detailed insight whatsoever. If you want to improve your code coverage, you need to get in-depth information about what code is already tested and what code has insufficient coverage.

For this purpose, we chose to rely on our main code editor: VSCode. It has a great plugin ecosystem and searching on the marketplace for lcov allowed us to find two great extensions that fit our needs.

• Koverage to visualize code coverage of your project as a treeview
• Coverage Gutters to visualize line-per-line test coverage of any single file

Since coverage files are quite long to generate (on our project, we need about 15 to 20mn to run our full test suite) we will leverage our CI which is already computing lcov files for every branch. We just need to add the following line to our semaphore pipeline to make the lcov file available as a project artifact:

artifact push project cover/lcov.info --destination coverage/$SEMAPHORE_GIT_BRANCH/lcov.info --force

Then the following shell script can be used to download lcov from CI for the current working branch (given CI has already been run once).

project_id="redacted"
current_branch=$(git rev-parse --abbrev-ref HEAD)
coverage_dir="coverage"
local_path="$coverage_dir/lcov.info"

if [ -n "$SEMAPHORE_TOKEN" ]; then
  url_encoded_path=$(jq -R -r @uri <<< "coverage/$current_branch/lcov.info")
  url="https://org.semaphoreci.com/projects/$project_id/artifacts/$url_encoded_path"
  
  mkdir -p $coverage_dir
  curl -s -f -L -H "Authorization: Token $SEMAPHORE_TOKEN" $url --output $local_path

  if [ $? -eq 0 ]; then
    echo "test coverage has been downloaded at local path $local_path"
  else
    echo "could not download test coverage from $url"
  fi
else
  echo "please set SEMAPHORE_TOKEN in .env"
  echo "grab your token there: https://me.semaphoreci.com/account"
fi

We then run this script with a custom mix alias: mix coverage:  here is a video of what everything looks like.

Final words

I hope this article will help you track and improve your software testing strategy. I'm sure this approach can still be improved and if you have any ideas, feel free to drop a comment below.

Advent Of Code is a yearly event offered by Eric Wastl: every day before Christmas, you can unlock a daily code challenge and crack it with the programming language of your choice.

It's super funny, with a pleasant Xmas/elve/reindeer... background and a soft difficulty curve (anyone with some programming experience can take the first challenges).

This year was my second participation. For the first time I was able to go through all the 25 challenges (it took me a bit more than 25 days) and the only language I used was Elixir.

I will now share with you my Advent Of code takeaways.

1 - You're not an impostor 🥸

Advent Of code is for anyone willing to improve their programming skills.

It doesn't matter if you are:

• a competitive golf programmer used to crack challenges in less than 5 minutes to rank among the first 🤯
• a student or just some folk learning programming
• a seasoned programmer willing to crank up his skills ( 👈 I'm here )
• a seasoned programmer learning a new language (I might do next season's AOC in Rust)

What relieved me the most was how welcoming and helpful the AOC community is. A new Reddit thread is opened every day and people are encouraged to share their solutions or ask for help.

2 - CS101 ... again 👨‍🎓

If like me you graduated computer science quite a while ago, AOC might be the perfect occasion to refresh your academic knowledge.

Advent of Code will make you think about:

• Algorithmic complexity, also known as "WTF why is my code running for 30 minutes and not yielding any result ?!?"
• Data structures. Some challenges are tough without proper data modeling. So I suggest you know the ins and outs of basic data structures (chained lists, trees, and graphs) and related algorithms (depth-first vs breadth-first tree traversal)
• Formal Grammar will help you with some challenge
• Dynamic programming also saved me an evening of hacking.

3 - Coding Kata 🥋

We are no machines and we are not required to know anything by heart: so much knowledge is reachable online, within a few seconds.

Still, knowing a few will help.

AOC helped me to muscle my memory, and I'm now able to code most challenges in a single streak without reaching the documentation or copy-pasting from previous code. During this event, I reinforced my knowledge of some of the most important Elixir APIs (Enum, Stream, String, Regex).

By not abusing alt-tab and staying focused in your code editor you'll be able to reach what most athletes know as flow state: this mental state in which you feel the most enjoyed and productive.

Like athletes, everyday practice will give you reflex, confidence, and improve your judgment.

4 - And a few Elixir lessons 🧪

Reddit hosts the biggest AOC community, but you can also look after smaller AOC sub-communities. I learned much from ad-hoc threads created on Elixir Forum.

Here are a few things I learned, mostly from reading at other's solutions:

• Elixir comprehensions are powerful. They let you iterate, combine, filter and reduce in few line of codes (I recommend you this article)
• Prepending is not appending - Elixir lists are not arrays. Those assumptions sound trivial but helped me make a 60 seconds program run within a few milliseconds. Elixir is immutable and as such if you want to append a single item to a list, Elixir will create a whole new copy of the list (Elixir can't modify the last item to point at your new item - immutable we said ;-). On the opposite, prepending to a list will only create a new item that will point to the previous untouched list. That's why Elixir gives us a powerful prepend operator prepended_list = [item | list]
• Elixir most common APIs hide a few jewels that I never used in 4 years of Elixir Programming. With AOC I discovered Enum.chunk_every, Enum.chunk_while, Enum.zip or Stream.unfold
• You have all Erlang magic at hand. I solved some of the challenges by using raw Erlang APIs such as ets, digraph, or the fairly new atomics module.
• Immutability brings safety. Some challenges were tricky to code in some other languages because if you didn't take care of doing deep copies of your data structure you ran into an inconsistent state. In Elixir, there is no such thing as shared mutable state, so deep copy comes for free 😎
• Immutability is slower. The cost of safety is that any write operation in Elixir will be slower than in any other languages with mutable data structures. ets or atomics Erlang APIs can sometimes be handy.

Final words

If you didn't know Advent Of Code before or just never tried, I hope this post will convince you to give a shot at it. AOC is not timebound and you can follow any event whenever you want at your own pace.

Oh, I forgot to mention that AOC is also a bit addictive and I may have started over from year 2015 🙊 😁 🎄

Metaprogramming is a scary word: it sounds like voodoo for programmers, and in some manners it is. It is used by most popular Elixir libraries (Ecto, Phoenix and Elixir itself) and as I leveled up as an Elixir programmer, I needed to understand how it worked under the hood.

I personally felt intimidated by the topic, and I owned the Metaprogramming Elixir book for more than a year before actually reading it 😱

But for any seasoned Elixir programmer, it is an excellent handbook and very pleasant to read. It is also pragmatic and I concur 100% with Chris McCord’s introductory advice:

Rule 1 : Don't Write Macros - Chris McCord

Be cautious with metaprogramming as it is hardly maintainable and a pain to debug; don't even think about writing your application core features with it. It is more relevant for technical libraries, custom DSL and any other specific subjects.

It's a kind of magic

As many of you, I came from a Ruby on Rails development background and I remember that while learning Rails, I was so impressed by the language expressivity and its magic features. Coming then from a Java background, it was a blessing to me!

irb> 2.years.from_now + 5.days.from_now 
#=> Mon, 28 Nov 2022 13:38:36 UTC

irb> Person.find_by_user_name_and_password(user_name, password)
#=> #<Person id: 10, user_name: "Ryan">

Rails made indeed an intensive usage of Ruby dynamic features to extend the language at runtime: the find_by_user_name_and_password call  just above is an example of dynamically generated class method based on the Person model attributes.

But there are caveats to Ruby magic:

• too much magic may hurt as you lack the understanding of what really happens under the hood. Same logic also applies to Elixir metaprogramming (remember Chris McCord's motto? "Don't Write Macros" 😅)
• Ruby magic is dynamic by nature which means it is performed at runtime, at the expense of performance. Whereas Elixir macros are evaluated at compile time and inserted in your bytecode as if it were manually written code.

What is metaprogramming?

Before going deeper in the subject, it’s worth reminding that metaprogramming is the art of writing code that will generate code. Metaprogramming features are first class members of the Elixir language which provides macro API to rapidly extend the core language.

Did you know that if-then-else or unless block syntax are written in Elixir through its macro mechanism? It provides syntaxic sugar over a binary case pattern matching structure.

defmacro macro_unless(clause, do: expression) do
  quote do
    if(!unquote(clause), do: unquote(expression))
  end
end

You could also be using meta-programmed features everyday without knowing it:

• ExUnit test definitions leverages on macros
• Ecto querying DSL can perfectly mimic SQL language thanks to metaprogramming
• Of course Chris McCord’s Phoenix framework uses macro a lot, like the Routing DSL

By the way, did you know that more than 90% of the Elixir sourcecode is written in Elixir? Meta enough for you? 🤯

Elixir Macros 101

Your first Elixir macros will be a bit tedious to write. I won’t give a full course about Elixir metaprogramming (buy Chris McCord’s book!) but I will instead give you a few keys to understand it.

The main thing you need to understand is that all code you write in Elixir is represented in memory as an Abstract Syntax Tree (AST). The compiler will parse your code and turn it into an AST, then the BEAM will evaluate the AST in order to run it.

Metaprogramming, which we previously defined as « code to generate code » could be achieved by generating raw Elixir code (as strings) but it would be unefficient and hardly maintainable.

That’s why Elixir provides API’s to generate and evaluate AST. Here comes quote: an Elixir primitive which can convert any Elixir code into its AST.

iex(1)> quote do 5 + 8 * 3 end
#=> {:+, [context: Elixir, import: Kernel],
#=>   [5, {:*, 
#=>     [context: Elixir, import: Kernel], 
#=>     [8, 3]
#=>   ]}]
#=> }

This code can be evaluated the same way you would have evaluated code in a string (think eval in Javascript)

iex(1)> Code.eval_string("5 + 8 * 3")
#=> {29, []}
iex(2)> Code.eval_quoted(quote do 5 + 8 * 3 end)
#=> {29, []}

Now what happens if you try to evaluate code with dynamic parts?

iex(1)> a = 5
iex(2)> Code.eval_string("3 * a")
#=> ** (CompileError) nofile:1: undefined function a/0

Of course a is unknown to your evaluated code and a simple way to fix this would be to use text interpolation.

iex(1)> a = 5
iex(2)> Code.eval_string("3 * #{a}")
#=> {15, []}

Well the good news is you can interpolate within your quoted code as well: use the unquote primitive.

iex(1)> a = 5
iex(2)> Code.eval_quoted(quote do 3 * unquote(a) end)
#=> {15, []}

Simple as that! So whenever you think you are lost between your quote / unquote blocks, just look at it as if it were text interpolation.

Then you need to know about defmacro which you probably already encountered in your Phoenix project. It's a special construct able to produce code, from quoted expressions and parameters.

Here is an example that generates a few functions :

defmodule MyMacro do
  defmacro generate_functions(name_and_values) do
    for {name, value} <- name_and_values do
      quote do
        def unquote(name)(), do: unquote(value)
      end
    end
  end
end

defmodule MyModule do
  require MyMacro
  
  MyMacro.generate_functions([{:one, 1}, {:two, 2}, {:three, 3}])
end

iex(1)> MyModule.two + MyModule.three 
#=> 5

There are quite a few other things to know when it comes to Elixir metaprogramming, but quote, unquote and defmacro are the basics and can already help you to build powerful stuff.

Ok, but what for?

I will end this post with some examples of the way we use metaprogramming in our Elixir projects.

Extends Ecto query API

Ecto provides a fully featured querying API that mimics SQL. It relies on Elixir Macros and will be able to trigger compilation errors whenever your query is malformed.

iex(1)> query = from u in User, where: u.age > 0, select: u.name
iex(2)> Repo.all(query)

But every now and then you'll want to use some specific database functions not available in Ecto API. You can then use fragment Ecto function which let you insert raw SQL in your Ecto Query:

query = 
  from u in User,
  where: is_nil(fragment("?->>?", u.metadata, "phone"))

But the proper way is to leverage on macros to extend the Ecto query API:

defmodule MyQueryMacros do
  defmacro jsonb_get(column, key) do
    quote do
      fragment("?->>?", unquote(column), unquote(key))
    end
  end
 end
 
 query = 
   from u in User,
   where: is_nil(jsonb_get(u.metadata, "phone"))

Aliases & Imports

As your Elixir codebase grows, you can see that you repeat over and over the same alias and import blocks at the head of your files. Not really DRY...

Phoenix already offers a solution you can use through the use / __using__ macro. To declare a controller or a view, phoenix suggests a bunch of default aliases and imports declared in my_app_web.ex file. Here is an example of changelog.com *_web.ex file :

thechangelog/changelog.com

Hacker to the 💚. Contribute to thechangelog/changelog.com development by creating an account on GitHub.

GitHubthechangelog

You can then use the macro with the following snippet, which will alias and import a few Phoenix modules and even generate some authorization related functions:

defmodule MyAppWeb.MyController do
  use MyAppWeb, :controller
  
  # ...
end

In our own project, we also brewed our own aliasing system that we can use like this to prevent us from declaring the same aliases over and over.

defmodule MyApp.Contracts.SomeContractService do
  use MyApp,
    aliases: [:campaigns, :contracts, :users],
    private_aliases: [:contracts]
    
  # ...
end   

I may write further about these specific macros in another post 😉

changix

A final example of how we use macros in our main Elixir application is our application changelog. We present new features, bug fixes and improvement to our users with a changelog sidebar (and a blinking badge to draw their attention).

Instead of feeding this changelog from database, we use markdown files with a YAML front matter header.

At first, we used to parse these files at runtime everytime a user triggered the sidebar, then we cached the HTML output in memory in order to improve performance.

But after reading Chris McCord's book, we realized that these markdown files could be considered as source code and be compiled as well (or at least processed at compile-time). Here comes changix: a library relying on macros to provide compile-time changelog features.

cblavier/changix

Compile-time changelog features for Elixir. Contribute to cblavier/changix development by creating an account on GitHub.

GitHubcblavier

Final words

I hope that this post will help you feel more confortable with Elixir metaprogramming and give you enough courage to hack yours; but remember Chris's advice and always be cautious 😅

The more I get experienced as a software engineer and a developer, the more I like to keep things simple. In this article I will explain why I believe that boring - yet pragmatic - choices are often the most powerful.

Did you say "boring"?

I borrowed this term from a talk given by two Doctolib software engineers (Doctolib is a very successful french startup in the health space). In this presentation, Nicolas & Michel explain how Doctolib is dealing with 30k+ simultaneous users with a very simple and straightforward Ruby on Rails architecture: mostly a monolithic web application and a SQL database.

Before going deeper into software architecture, you may want to know that the boring trend has also been mentioned in quite a few other fields:

• LA Times journalist Christopher Hawthorne is praising boring building architecture in his "Boring architecture? Yes please" paper.
• Nick Rollins helps us to remember that a Good User Experience involves a Boring User Interface and urges not be creative when dealing with UI: don't mess with your users and just stick to the - boring - established standards.
• And even Elon Musk named his company The Boring Company ;-) (terrible example, they are actually boring, ie. drilling, tunnels)

Boring architecture

Let's focus again on software architecture: a few years ago I wrote an article about the choice of not using any JS Framework when libraries such as Angular or EmberJS were on the rise, but instead leveraging on a very classic MVC architecture with a thin JS layer. I since had excellent feedbacks on this approach even if I occasionally rely on ReactJS to handle very dynamic pieces of UI.

Unfortunately it seems that my ideas are against the current flow, since average web applications kept getting more and more complex with:

• a lot of different storage solutions (NoSQL)
• GraphQL replacing REST APIs
• The rise of micro-services architectures (M/SOA)
• Containerized (Docker) and orchestrated deployments (Kubernetes)

These architectural solutions are impressive at helping  large scale companies to deal with their storage or operation issues. But what drives me crazy is to see how many, seed or low-scale companies / softwares start using these technologies from the ground-up.

Micro Services architectures

Let's take an example : Micro Services. It looks so neat and clever on your PowerPoint slides, making you think you will divide and conquer on your software by splitting the big business problems you are trying to solve in much smaller pieces of software - easier to manage.

But M/SOA also induces a very important development overhead:

• If you want your micro-services to talk together, you'll have to tie them up with APIs (whereas a simple function call would have been much simpler and faster).
• You can't simply share code between your services. You'll have to build and release libraries shared between your micro-services, even for sharing a simple one-liner function across your services.
• Micro-services also mean multiple units of deployment, which is way more complicated to release and run than a single web application. People usually end up with solutions like Kubernetes.
• Versioning and release-management is really tough.

This overhead will eventually drive you away from delivering actual business value, which is not only a terrible idea for early stage companies, but a handicap for any size of business.

Document databases

Another example is the use of document oriented databases (such as MongoDB or CouchDB). Such databases are designed with very attractive benefits compared to relational databases:

• you can store document (think JSON object) instead of simples rows.
• database is schema-less meaning a single collection (ie. table) can hold documents of different nature (new attributes ...) without any schema migration involved.

Unfortunately, what these databases are not marketed for, is the feature they lack regarding SQL databases:

• you cannot join different collections together. Meaning that if you want to reach best performance, your must really think your schema upfront considering all your use cases and potential queries to store and structure your documents in an optimal way. But your business will change and so will your queries ... leading you eventually to either sub-optimal N+1 queries or de-normalized data structures which are a nightmare to maintain, making you build over-engineered CQRS architectures (overhead!)
• no ACID transaction. You can live without it 90% of the time, but the remaining 10% are a real pain to work-around with software transaction and compensation mechanisms

I used MongoDB for years and learned these lessons the hard way at the expense of days at tweaking for improved performance and using CQRS design pattern to maintain integrity between denormalized data collections.

A few years later, when came the time to build a new product I adopted PostgreSQL and it was such a relief! Postgres even became stronger over years by including new features previously only available in some noSQL system (such as JSON column datatype, partitioning and sharding, or geographic search).

The Majestic Monolith

Monolithic software architectures sound awkward, clumsy - of course boring - and unable to face agile and fast changing requirements. But they are quite the opposite.

Doctolib previous example has already proven me right, but so did Basecamp. In The Majestic Monolith post DHH explains how Basecamp is dealing with million of users with their large and monolithic application.

Run a small team, not a tech behemoth? Embrace the monolith and make it majestic. You Deserve It! - DHH, 2019

Building your software as a monolith doesn't mean that you don't care about architecture and are willing to build a big plate of spaghetti-ware. I sincerely believe that removing the clutter and the boilerplate of non essential technology will help you at focusing on things that really matter such as a well designed business layer and tons of unit tests.

You can totally craft well-thought and battle-tested code in a - majestic - monolithic fashion

Boring organizations

Some might argue that such a simplistic architecture does not add up very well with the size of their team or organization. Please look at it the other way around: maybe that if you started building less complicated software, you might figure out that you need smaller teams?

"Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations"  - Conway, 1967.

50 years ago, John Conway stated about the deep link existing between the structure of software and the structure of organizations which build them. So keep with small teams, hire jack-of-all-trades full-stack developers and build clean, simple and pragmatic software. I'm quite sure this advice should resonate with most companies building software ; not everyone is building Google or Facebook.

The patterns that make sense for organizations orders of magnitude larger than yours, are often the exact opposite ones that’ll make sense for you. It’s the essence of cargo culting. If I dance like these behemoths, surely I too will grow into one. I’m sorry, but that’s just not how the tango goes. - DHH, 2019

Wrapping up

I don't know if building boring and simple softwares with smaller teams will become a new trend in upcoming years, but it is my motto since a few years and it gives strong results until now.

You might wonder "Will I get bored at building boring stuff? . I don't think you will:

• Building simple software does not mean at all being bored: you will deliver more value and joy to your users and it is among the most exciting things and what should fuel your days as a software developer.
• I'm still tech savvy and really interested in new technologies and frameworks, but I'm also getting older and wiser ;-) When considering a new technology I'm always thinking about the tradeoff between the provided value and the induced overhead.

I’m really happy and proud to say that since this story was first posted, I received great feedback and relay throughout the Elixir community! The last advice given in this article has indeed been taken as an inspiration by José Valim to add a test partitioning feature to Elixir 🤗

Some context

I have been using Elixir + Phoenix on a daily basis for 3 years now, and almost since day one, my team and I have set up continuous integration (CI) and continuous delivery (CD) processes.

Since we’re deploying our app on Heroku, we’ve been using:

• SemaphoreCI as our CI tool (because SemaphoreCI is available as a Heroku add-on, simple as that!)
• Heroku native pipelines to perform CD whenever CI is successful on our Git master branch

Although our CD process has always been very reliable, our CI process gradually started to under-perform, becoming both slower and unreliable. Like boiling frogs, we did not really pay attention until very recently when we had to wait almost for 1 hour before being able to deploy our code to staging: each CI run would take 15 minutes, and random test failures forced us to re-run each build a few times before getting the green light. This drove us really crazy, and led us to take action!

In this article, we’ll talk about the actions we took in order to improve an unreliable — randomly failing — 15 minutes long CI process to a solid and steady 3 minutes build.

#1 — Improve worst tests

First action is to find out the culprits: which tests are performing really slowly and which are failing randomly?

Running mix test --trace is very useful as it gives you detailed execution time, per test. But my favorite is mix test --slowest 10 which prints the list of the 10 worst performing tests. I have no general purpose advice to give about fixing these tests, but the slowest are usually quite easy to improve.

Regarding randomly failing tests, two things can really help:

• a simple bash loop to run a single test over and over. If it returns anything else than 0, the test is unreliable.

(for i in {1..10}; do mix test test/lib/my_app/features/my_browser_test.exs:10 > /dev/null; echo $?; done) | paste -s -d+ - | bc

• a wait_until function that helps to fix unreliable browser tests (we're using Hound with chromedriver). Here is the gist.

A last ugly trick 🙈 I am ashamed of and I urge you not to use is to give another chance to your failing tests! Running your tests with mix test || mix test --failed will automatically rerun any failed test and still give you a 0 exit code if it passes on the second time.

Pro tip: I strongly encourage you to read mix help test output: hidden gems in there!

#2 — Cache all things

At this stage, the build became way more reliable (99% success rate) but still really slow (~ 12 min).

My first attempt to improve this was to see if the grass was greener at GitHub’s with their brand new GitHub Actions, and if it would result in an improved build duration. Although promising, I did not manage to run any of our chromedriver tests with their product, and execution time was anyway quite similar.

I then decided to go back to Semaphore and migrate from the classic plan to their new SemaphoreCI 2.0. The new Semaphore (like GitHub Actions btw) requires you to describe your CI pipeline in a YAML file committed along your source code. In this YAML file you can run docker containers, arbitrary shell commands or specific semaphore commands, including the very useful cache commands.

By using cache store and cache restore at strategic locations you can make most of your builds way faster:

• by not re-downloading Elixir dependencies
• not re-compiling all Elixir code, only updated code
• not re-downloading NPM/Yarn dependencies

Pro tip: have a look at fail_fast and auto_cancel Semaphore settings. Real time savers!

#3 —Leverage on parallelism

Caching really helped, and at this step we could have stopped improving the process as our build time already went down to 6 minutes.

But reading SemaphoreCI documentation is insightful, and their concepts of blocks and jobs that can run either sequentially or in parallel suggest there is still room for improvement!

Our build was really straight-forward then:

The first improvement is to run in parallel what can be done in parallel:

• mix credo (the Elixir code analysis tool) and mix test can be run at the same time. With a fail-fast strategy that stops running tests whenever credo fails, it can be a real time saver!
• JS & CSS assets can be compiled at the same time than Elixir code: it can save up to one minute.

The final step is to split our tests in different batches to also run them in parallel. I wrote this custom test runner (which simply wraps mix test shell command) that splits all test files in N batches (by hashing their filename) and only runs a single batch out of N:

# Run all tests in 4 random batches
$> mix n_test 1 4
$> mix n_test 2 4
$> mix n_test 3 4
$> mix n_test 4 4

You can then leverage on SemaphoreCI parallelism to run a single task N times with an incremental index passed as an environment variable.

Our CI finally looks like this, running at a solid and steady 3mn pace!

Pro tip: save time at learning Semaphore by reading our build pipeline setup

Wrapping up

It has been known for years how important Continuous Integration is, for any size of software team. But it should be a source of trust, happiness and increased productivity, never a source of frustration!

So start paying attention, and maybe some of the guidelines in this post will help you at improving your CI build!

NB - I was not paid by SemaphoreCI to write this article, so feel free to experiment with any other CI software, as long as it offers you caching & parallelism.

I’m writing code for over a decade now, consider myself as a geek, and as a geek I’m always very curious and excited about any new fancy technology released every now and then.

These days client-side MVC frameworks are very trendy, and to be honest what I read and learned regarding Ember or Angular (among many others) looked really shiny and well-thought. These are typically frameworks that I would love to love.

But right now, I don’t need them. Do you?

The playground

In addition to my freelancing activity, I’m running Folyo (which by the way, can help you to find great designer talents). It’s as Ruby on Rails website with a UI that could be defined as slick, reactive and responsive (especially thanks to Sacha, my co-founder), but by no means complex.

As a job board, Folyo has quite a common page driven architecture where any action performed on a page will often lead you to another page or refresh current page content. Folyo’s UI does not involve rich interactions like live data charts or dynamic data binding. No, just web pages and it’s perfectly fine!

So what about client-side frameworks?

Why not, since it seems like the way to go these days?

Because such frameworks are complex tools designed to solve complex interaction issues. You must be aware that they also have serious drawbacks regarding productivity, testing, SEO, etc. (read this) and also have a steep learning curve.

Building a website with client-side MVC frameworks will require you to build your Rails server-side application as an API application only, which means depriving you of some of Rails beauty and increasing your development effort (API + UI) significantly.

Still, using a framework also brings some virtues. One of the first being to help you with organizing your code in a more manageable way. While Rails is really directive regarding the way you need to organize server-side code, it dictates only one rule for Javascript:

Put whatever you want in application.js.

Eh. Really?

Time to clean it up

I will now explain the way we decided to organize Javascript code on Folyo, in a much more manageable way than everything in application.js. I believe it’s quite pragmatic and I’m sure it will fit with a lot of other applications.

Required libraries:

• jQuery, obviously.
• Coffeescript, not mandatory but its syntax for class definition is very convenient.
• Turbolinks. It will really improve user experience by making navigation from page to page AJAX only.
• HeadJS, used through headjs-rails gem, will speed up your app by loading your JS asynchronously.

Code hierarchy

To organize the code, I will simply obey to following rules:

• Any page requiring JS code will have it’s own class (a JS view).
• If you need to share piece of code between views, put it in a widget class.
• application.js is just doing the glue between your classes, jQuery, HeadJS and Turbolinks.
• That’s all!

Views

ApplicationView is the default view, inherited by any other view, and instantiated by default when no specific view is provided.

window.Views ||= {}
class Views.ApplicationView

 render: ->
   Widgets.FancyBox.enable()
   Widgets.MarkdownEditor.enable()
   
 cleanup: ->
   Widgets.FancyBox.cleanup()
   Widgets.MarkdownEditor.cleanup()

Since we want FancyBox and our Markdown editor to work on a lot of pages, we put it in the ApplicationView as a default behavior.

Then a typical view would look like that:

window.Views.Newsletters ||= {}
class Views.Newsletters.EditView extends Views.ApplicationView

 render: ->
   super()
   $('a.preview').click (e) ->
     e.preventDefault()
     url = $(e.target).attr('href')
     window.open(url, '_blank', 'width=800,height=800')
     
 cleanup: ->
   super()

Simple as that! But why the cleanup stuff?

Because with Turbolinks, the Javascript environment is not reset between each page. For instance, if you define a timer on a page, it will keep ticking on next pages. So, just remember to stop your timer in the cleanup method (or remove any document-wide event listener)

Widgets

Sorry, no rocket science here.

window.Widgets ||= {}

class Widgets.FancyBox
  @enable:  -> $(".fancybox").fancybox()
  @cleanup: -> $(".fancybox").off()

The glue

application.js is now only the entry point which will listen to Turbolinks events to render proper views and clean them up.#= require everything you need

#= require everything you need

pageLoad = ->
  className = $('body').attr('data-class-name')
  window.applicationView = try
    eval("new #{className}()")
  catch error
    new Views.ApplicationView()
  window.applicationView.render()
  
head ->
  $ ->
    pageLoad()
    $(document).on 'page:load', pageLoad
    
    $(document).on 'page:before-change', ->
      window.applicationView.cleanup()
      true
      
    $(document).on 'page:restore', ->
      window.applicationView.cleanup()
      pageLoad()
      true

You also need to define a specific data attribute on each page, to indicate which JS view needs to be rendered.

In your application_controller.rb, define js_class_name method:

def js_class_name
   action = case action_name
     when 'create' then 'New'
     when 'update' then 'Edit'
    else action_name
   end.camelize
   "Views.#{self.class.name.gsub('::', '.').gsub(/Controller$/, '')}.#{action}View"
 end

And then use it in your layout (as well as HeadJS initialization)

%html
  %head
    = javascript_include_tag 'head.min'
    = headjs_include_tag 'vendor', 'application'
  %body{'data-class-name' => js_class_name}

Final words

I’m seing most and most Rails projects bootstrapped using either Angular, Ember or Backbone as their de facto choice.

Whilst these frameworks are very powerful, I hope that after reading this post you will consider that not using a JS framework is also a valid choice.