Categories
Code Complete Technology

Code Complete Notes, Chapter 3, Section 5: Architecture

Architecture, according to the book (which draws from other sources as well), is:

Software architecture is the high-level part of software design, the frame that holds the more detailed parts of the design.

S. McConnell, Code Complete, second edition. Redmond (Washington): Microsoft Press, 2004.

Now that’s out of the way, let’s talk about how we can make sure that the architecture that’s laid out for our projects is good, since that’s what this chapter is all about. If your architecture is off at this stage, you’ll have to make the changes when you’re in the construction stage… That makes them a lot more difficult to solve, because you’ll likely have to move other things that you’ve already built around as you make the changes.

If you’re working on a system that was architected by someone else, you should be able to find the important components without a bloodhound, a deer-stalker cap, and a magnifying glass.

S. McConnell, Code Complete, second edition. Redmond (Washington): Microsoft Press, 2004.

The important parts of your architecture should be clearly visible. If it’s not clear why a class or component exists, or why it was chosen over other alternatives, then it’s possible it needs to be better defined.

Architectural Components

Photo by Iker Urteaga on Unsplash

I was originally going to outline the different components and give a few details on each, but then I realized I’d mostly just be paraphrasing the book. Since there’s no need to do that, here’s a list of the architectural components, along with some of my associated thoughts.

  • Program Organization
  • Major Classes
  • Data Design
  • Business Rules
  • User Interface Design
  • Resource Management
  • Security
  • Performance
  • Scalability
  • Interoperability
  • Internationalization & Localization
  • Input/Output
  • Error Processing
  • Fault Tolerance
  • Architectural Feasibility
  • Overengineering
  • Buy vs. Build Decisions
  • Reuse Decisions
  • Change Strategy
  • General Architectural Quality

Now, it seems to me that a lot of these, such as the User Interface Design, depend a lot on the type of development that you’re going for. If you’re developing in a waterfall-style approach, then it’s great to have it fully defined upfront. However, I think that most modern software development occurs much more iteratively – so perhaps these should all be thought of in shorter iterations.

If you’re developing using an ‘agile’ approach, it seems like it’d make sense to have these things laid out at the beginning of a sprint. But it all depends on the approach you’re using. I know I’ve never used a strict agile methodology, so you have to apply this where you see fit.

This is covered a bit in the ‘Change Strategy’ point, but I still don’t feel like it fully addresses more iterative development approaches. It’s possible I’m just missing something, though.

Conclusion

Anyway, the point is that for any development, you definitely should have at least a basic architecture in place. Even if you’re developing iteratively, think through the things in the list above so that you can define an overview of the system before you put your fingers to the keyboard & start coding.

It shouldn’t look as if the problem and the architecture have been forced together with duct tape.

S. McConnell, Code Complete, second edition. Redmond (Washington): Microsoft Press, 2004.
Categories
Code Complete Technology

Code Complete Notes, Chapter 3, Sections 2 and 3

Defining the Problem

Section 2 of Chapter 3 is very simple: you should know what problem you’re trying to solve before you try to solve it. It normally shouldn’t be stated in technical terms. Instead, it should be the simplest issue that you’re trying to solve.

For example, “We need developers to be able to parallelize their tests” isn’t a good problem definition, it’s already settled on a solution. “Developers’ tests take way too long to run” would be much better, because it states the problem, not a solution.

Defining Requirements

Section 3 covers devising the requirements for your software project. These should be agreed upon before work begins, and the customer should be the one in charge of validating the requirements.

Crafting well-defined requirements before starting the actual coding is important. It costs a lot more to adjust a design to meet new or updated requirements after code has been written than it does to just write the code to fit the requirements in the first place.

Specifying requirements adequately is a key to project success, perhaps even more important than effective construction techniques.

S. McConnell, Code Complete, second edition. Redmond (Washington): Microsoft Press, 2004.

But of course, requirements are rarely perfectly defined. It’s hard for customers (and developers, too!) to accurately describe requirements initially. As customers come to understand the system better, they’ll be better able to develop accurate requirements. So no matter how much we want things to be perfect the first time, it’s unlikely to ever happen in practice.

… the average project experiences about a 25 percent change in requirements during development.

S. McConnell, Code Complete, second edition. Redmond (Washington): Microsoft Press, 2004.

Dealing with changing requirements can be difficult, but it can be made easier by making it clear to your customer or client that changes cost, often monetary (though that depends on the type of project) and almost always in terms of time. If a customer is too happy-go-lucky with changes, establishing a change-control board to review the proposed changes may also be helpful (though I’d imagine that’s mostly where you’re doing client work).

One other important consideration is keeping the business reason for the project in mind. Oftentimes, you’ll find that features that sound neat or even necessary won’t be so important when you consider the main reason for the project.

Finally, this section concludes with a series of questions to ask yourself about your project’s requirements to ensure they’re solid. Reference the book directly for the list.

Categories
Code Complete Technology

Code Complete Notes, Chapter 3, Section 2

Different types of software require different types of planning. If you’re working on your own blog, for instance, the stakes are a lot lower than if you’re working on, say, an automated flight control system.

If you’re working on one of those high-stakes projects, your planning should be much more thorough & much less iterative. Your development is much more likely to follow a waterfall approach. But on your blog, or even something like an e-commerce site, you can (and arguably should) be a lot more iterative.

In either case, specifying more prerequisites upfront will help save time and effort in the long run, but it’s more important in more sequential projects. But, of course, your project’s balance between sequential and iterative will depend largely on what type of project it is and the risks involved.

One common rule of thumb is to plan to specify about 80 percent of the requirements up front…

S. McConnell, Code Complete, second edition. Redmond (Washington): Microsoft Press, 2004.

Essentially, you’ll need to pick the right balance for your project. If it’s something more stable, where the requirements are well understood and unlikely to change, a more sequential approach is appropriate. But if the requirements may change, or the structure isn’t as well understood, then a more iterative approach will be more beneficial.

Software being what it is, iterative approaches are useful much more often than sequential approaches are.

S. McConnell, Code Complete, second edition. Redmond (Washington): Microsoft Press, 2004.

For more details (as well as an analysis of the potential costs associated with each approach), check out this section of the book.

Categories
Code Complete Technology

Code Complete, Chapter 3, Section 1 Commentary

This is the second post in my series on Code Complete, covering my notes and commentary from Chapter 3, Section 2. The title of this section is Measure Twice, Cut Once: Upstream Prerequisites.

Essentially, this section is talking about the importance of developing prerequisites before beginning work on a project. It reminds me of a phrase that was apparently one of Frank Lloyd Wright’s favorites:

The architect’s two most important tools are: the eraser in the drafting room and the wrecking bar on the site.

Frank Lloyd Wright

In Software, this is equally applicable. The easier that we can catch defects, from design to development to production, the easier they are to fix. Obviously, in Wright’s quote above, using the eraser would be significantly cheaper than using a wrecking bar.

Much of the success or failure of the project has already been determined before construction begins.

S. McConnell, Code Complete, second edition. Redmond (Washington): Microsoft Press, 2004.

The diagram below is one that I created for a presentation given at Etsy – it’s a very generalized diagram, not representative of any actual set of data. However, it gives an idea of the relative cost to fix defects that are introduced during the software development process.

This section in Code Complete features a similar diagram – but backed by more solid data. Essentially, the earlier that defects are detected and corrected, the cheaper they are to fix.

…debugging and associated rework takes about 50 percent of the time spent in a typical software development cycle…

S. McConnell, Code Complete, second edition. Redmond (Washington): Microsoft Press, 2004.

So really, the whole point of this section is that planning reduces the overall cost of software development. Of course, this doesn’t mean that we should do everything in a waterfall approach and attempt to plan everything before we even start coding. We have to strike a balance between the two.

I believe, when using Agile methodologies, we should ensure that we plan carefully for the features that we’re focusing on building. Whether you’re using sprints or not, you can plan ahead for the work that you’re doing to ensure that you’re minimizing the ‘rework’ required.

It’s by striking a balance between planning too much and planning too little that we can be most effective in our software projects.

If you start the process with designs for a Pontiac Aztek, you can test it all you want to, and it will never turn into a Rolls-Royce.

S. McConnell, Code Complete, second edition. Redmond (Washington): Microsoft Press, 2004.
Categories
Code Complete Technology

Code Complete, Chapters 1 and 2

Hello, everyone! I’ve recently started reading Code Complete (the second edition), by Steve McConnell. I haven’t made it very far into it yet, but I figured that I’d share the things that I learn or find interesting here.

I realized that when I read, it helps me to take notes and focus on highlighting so that I can increase my comprehension. So here I am, writing down some notes that I gather as I read Code Complete.

Software Construction

Something that was new to me was the idea of Software Construction. Software Construction encompasses coding, debugging, detailed design (not graphic design, mind you), testing, and integration. In essence, the things that we normally think of when we think of software development.

McConnell focuses Code Complete on Software Construction, rather than architecture, project management, or user interface design . He says that very few books released before the first edition of Code Complete had covered it directly.

It seems to me that a lot of books had touched on the topic of Construction, but it did seem like few had chosen to address it directly. Honestly, it feels like a majority of the books written regarding Software Development cover specific languages or frameworks. Of course, that could just be the fact that I never had a formal Computer Science education and therefore wasn’t exposed to that during my education. So that’s the value in this book – it’s trying to cover all the different aspects of Software Construction in one place.

Anyway, I don’t know why I hadn’t heard of Software Construction before, but I simply hadn’t. Really, though, the idea makes a lot of sense. McConnell also touches on the value of metaphors in understanding Software Construction, and it seems to me that just calling it Construction to begin with is itself a metaphor.

Metaphors

McConnell covers several different metaphors that software construction has been compared to over the years, including writing, farming, oysters (accretion), and construction. Of these, I found construction to be the most applicable.

Software must be architected first, just like buildings. Without a plan, your building/software isn’t going to end up very nice. Remodels are like refactors or adding additional functionality to software. Bigger buildings (or bigger software projects) require more planning than smaller projects.

One example that McConnell gives is that of building a dog house. You don’t really need to plan much ahead to build a dog house. Likewise, a tiny software project probably doesn’t require much in the way of architecture. But if you’re building a skyscraper, that takes an awful lot more work and planning. You wouldn’t just want to go to the hardware store & pick up some random materials for your skyscraper, but that’s feasible in the case of a dog house.

The one area where I think this breaks down some (but not entirely) is in refactoring and adding additional features. It’s still a lot easier to work on software than it is to add on to a house. A lot of software, especially web-based software, can be continually improved. That’s more difficult to do with a building. Not impossible, just more difficult.

Anyway, that covers my thoughts on Chapters 1 and 2 of Code Complete. Keep your eyes peeled for more posts covering my thoughts on the book.

Categories
Quick Tech Tricks

Using `xarg` to pass to `find`

I just found myself needing to run wc -l on all the files in a list of directories – in my case, I had a big old list of directories with a matching name. But I wanted to calculate the total number of files in those directories.

Unfortunately, find is very particular about where its arguments go, so running xargs and passing it to find was resulting in the following:

find: paths must precede expression: tmp/dir/name
Usage: find [-H] [-L] [-P] [-Olevel] [-D help|tree|search|stat|rates|opt|exec] [path...] [expression]

Luckily, the solution is pretty simple. Use the -I flag to xargs to make it replace {} with your argument.

So your full command will look something like this:

cat list_of_directories.txt | xargs -I{} find {} | whatever

The command that I ended up with is this:

find . -name "dir_name" | xargs -I{} find {} -type f | wc -l

I hope that’s helpful for you.

(Reference: http://xion.io/post/code/shell-xargs-into-find.html)

Categories
Automation Configuration Technology

Creating and Applying Diffs with Rsync

At work recently, we had a need to generate diffs between two different directory trees. This is so that we can handle deploys, but it’s after we’ve already generated assets, so we can’t just use git for the diff creation, since git diff doesn’t handle files that aren’t tracked by git itself. We looked into using GNU’s diffutils, but it doesn’t handle binary files.

We tried investigating other methods for deploying our code, but thought it would still be simplest if there was some way to generate just a ‘patch’ of what had changed.

Luckily, one of the Staff Engineers at Etsy happened to know that rsync had just such an option hiding in its very long man page. Because rsync handles transferring files from one place to another, whether it’s local or remote, it has to figure out the diffs between files anyway. It’s really nice that they’ve exposed it so that you can use the diffs themselves. The option that does this is called ‘Batch Mode’, because you can use it to ‘apply’ a diff on many machines after you’ve distributed the diff file.

Creating the Diff

To create the diff itself, you’ll need to first have two directories containing your folder structure – one with the ‘previous’ version and one with the ‘current’ version. In our case, after we run each deploy, we create a copy of the current directory so that we can use that as our previous version to build our next diff.

Your rsync command will look a lot like this:

rsync --write-batch=diff /deploy/current /deploy/previous

Running that command will give you two files, diff and diff.sh. You can just use the .sh file to apply your diff, but you don’t have to. As long as you remember to use the same flags when applying your diff, you’ll be fine. You can also use any filename that you want after the =.

Also, it’s important to note that running this command will update /deploy/previous to the contents of /deploy/current. If you want to keep /deploy/previous as-is so that you can update it later, use --only-write-batch instead of just --write-batch.

Applying the Diff

Next up, you’ll want to distribute your diff to whatever hosts are going to receive it. In our case, we’re uploading it to Google Cloud Storage, where all the hosts can just grab it as necessary.

On each host that’s applying the diff, you’ll want to just run something like the following:

rsync --read-diff=/path/to/diff /deploy/directory

Remember, you need to use the same flags when applying your diff as you did when you created your diff.

In our testing, this worked well for applying a diff to many hosts – updating around 400 hosts in just about 1 minute (including downloading the ~30MB diff file to each host).

Caveats

This will fail if the diff doesn’t apply cleanly. So, essentially, if one of your hosts is a deploy behind, you should make absolutely sure that you know that, and don’t try to update it to the latest version. If you try to anyway, you’ll probably end up with errors in the best case, or a corrupt copy of your code in the worst case. We’re still working on making our scripts handle the potential error cases so that we don’t end up in a corrupt state.

I hope this is helpful to you! If you’ve got any thoughts, questions, or corrections, drop them in the comments below. I’d love to hear them!

Categories
Ruby Technology

Saving Calculated Fields in Ruby on Rails 5

In Ruby on Rails, it’s easy to build custom functions to calculate something and then display the result in your views. While this simplicity is nice, it doesn’t come without its drawbacks.

Recently, when working on a simple app, I came across a situation where loading a page was taking 0.5 seconds. This may not sound like a lot (and wouldn’t be for most sites), but in an app as simple as mine, it’s a sign that something is taking way longer than it should. Luckily, it wasn’t too difficult to determine what it was.

The Problem

Let’s start with an example: say you’re building an application that will contain purchases from a grocery store. You probably want to link the items sold in a purchase with the record from that purchase, right? Well, somewhere you’re going to have to calculate the total. Of course, I’m assuming that you don’t want the customer to calculate the total.

You could calculate the total every time that you need to load the record of the purchase, but first let’s walk through what would be happening when you calculated the total. If there are, say, 30 items in that purchase, you’ll need to load every single one of those items so that you can grab the price (we’re assuming prices don’t change for this example) and add them all together.

As you might imagine, this isn’t a very efficient way to go about things. We’d rather offload some of that computation (that would be happening an awful lot) to the disk, instead. After all, it’s generally easier to store a few bytes than spend valuable CPU time recalculating it every time you need it.

In my case, that’s exactly the sort of thing that was happening. I was working to calculate a field that wouldn’t change often but that involved loading lots of links to other records. On top of that, it was going to be loaded pretty often. It’s much more efficient for me to just store that value than to calculate it for every request.

The Solution

You’ll need to add a new field to your database, which means you’ll need to add a database migration, something like this:

rails generate migration AddTotalToReceipts total:float

After you run your migration (rails db migrate), you’ll have your new field. Now, if you generated all your scaffolding, that’d be showing up in your user interface. That’s not what you want to do, though, since we’re trying to make this easier for your users and calculate it on their behalf.

Thus, we’re going to add something like the following to our model:

before_save :calculate_receipt_total

def calculate_receipt_total
  sum_value = x + y # Whatever you need to do here to calculate
  self.total = sum_value
end

Now that method will run automatically before the record is saved, and place our calculated value into the total value, which means it’ll end up there in the database, as well.

Like I said, just how much benefit (if any) you’ll get out of this depends on your exact circumstances, but in my case it reduced a 500 ms page load to around 100 ms, which is clearly a substantial improvement.

If you’ve got any questions, drop them in the comments, and I’ll do my best to answer them!

Categories
Ruby Technology

Many-to-Many in the UI in Ruby on Rails 5

I know I’m really late to the party, but I’m just finally creating something from scratch in Ruby on Rails. I’ve dealt a tiny bit with Rails before, but it was mostly just in helping my brother with CSS stuff, which obviously isn’t working on the Rails backend.

Anyway, now that I’ve started working with Rails (to build a simple app for my wife), I’ve found myself needing to learn how to do things in Rails 5. The problem is, a lot of things have changed in Rails, but most Google search results for ‘Rails …’ or ‘Ruby on Rails …’ end up with articles that are at least several years old. It’s hard to figure out what actually applies to Rails 5, vs. any other version of Rails. Thus, I’ve decided to write up some of my findings so that they’re hopefully helpful for someone else.

I’ll assume that you already know how to create your initial models, views, and controllers. If you don’t, check out this guide to get started.

Creating Many:Many Associations

Creating links between your records is pretty straightforward. I’m going to be using hypothetical ‘products’ and ‘purchases’ tables, which aren’t necessarily a perfect use-case for this, but they’re good enough. We can use Rails’s simple generator to make a link between our tables:

rails generate migration CreateJoinTableProductsPurchases

All that’s going to do is generate a migration file (if you want those indexes, uncomment the two t.index lines):

class CreateJoinTableProductsPurchases < ActiveRecord::Migration[5.2]
  def change
    create_join_table :products, :purchases do |t|
      # t.index [:product_id, :purchase_id]
      # t.index [:purchase_id, :product_id]
    end
  end
end

After updating our database with the migration (rails db:migrate), we just need to add a has_and_belongs_to_many to each of our models:

class Product < ApplicationRecord
    has_and_belongs_to_many :purchases
end
class Purchase < ApplicationRecord
    has_and_belongs_to_many :product
end

I know that using has_and_belongs_to_many may be going out of favor, but I haven’t had time to look at the alternative just yet, so I’m sticking with HABTM for now.

Creating Many:Many Links in the UI

Now you’ve got a link between your two models, but no convenient way to create any links! That’s what we’ll be focusing on next. Luckily, it’s pretty easy, you just need to know what to do.

Showing Many:Many Forms

First off, in the form for the ‘products’ side of our relationship, we’ll simply add the following:

<%= form.label :products %>
<%= collection_select(:purchase, :product_ids, Product.all, :id, :name, {}, { :multiple => true } )%>

If you’re using Rails’s scaffold generator, you’ll add it to app/views/<model_name>/_form.html.erb. This creates a list of all the ‘products’ (in the example code) in the form for a purchase, and allows you to select multiple products.

If you’d like, you can add similar code to the other side of your relationship. In my case, I’m just adding it on one side. The form will end up looking something like this:

You can use ‘Ctrl/Cmd+click’ to select more than one item in the list, but that’s really all there is to it. It’s a very basic form, but it’ll do the trick.

Of course, you can do something more advanced, but we’re just covering the basics here. Ideally, I’d like a multi-select autocompleting textbox. Hopefully some day in the near future I’ll be able to put a guide up here on how to do that.

Saving Many:Many Relationships

Now, to make our selections actually save, we’ve got to add something to the controller. Since I’m only allowing the creation of links from the ‘Purchase’ UI, I put this code in my app/controllers/purchases_controller.rb file, in both the create and the update methods:

params[:purchase][:product_ids].each do |product_id|
  unless product_id.empty?
  product = Product.find(product_id)
    @boarding.products << product
  end
end

And that’s all there is to it! When you either create or update a ‘purchase’, your selection for its ‘products’ will be saved.

Viewing Many:Many Links in the UI

So now that we’re able to easily create links, it would be nice if we could view those links. So let’s make it possible to see the list of ‘products’ in a ‘purchase’. Using the Rails scaffolding, we’re going to want to expose these in both the ‘list’ view and the ‘single’ view for each side of our link. You can pick if the same applies to you.

In our app/views/purchases/index.html.erb, we’re going to add the following to a new <td> that will display the ‘products’ in each ‘purchase’. Don’t forget to add an associated <th> in the header, as well.

<%= purchase.products.map(&:name).join(', ') %>

Then, upon loading /purchases, you’ll see something a lot like this:

Next up, we need to add some very similar code to our app/views/purchases/show.html.erb file (but note the @ in the name of the purchase variable:

<%= @purchase.products.map(&:name).join(', ') %>

And then you’ll have something like this when you view your ‘purchase’ (at a URL like /purchases/2):

Conclusion

That’s it! As I said, this isn’t a perfect use-case for a many:many relationship, but it’s at least an overview of how to create the UI elements that will allow you to save and view such relationships.

If you want to take a look at the code in its entirety, it’s available on GitHub. And, if you’ve got any tips or comments, drop them in the ‘comments’ section below! That’s what it’s there for!

Categories
Security Technology

Wildcard Let’s Encrypt Certificates with Certbot and GCP

As of last month (March 8th-ish, 2018), Let’s Encrypt supports wildcard certificates! This is great news, because it means that those of us who like using tons of subdomains can now get one cert for all our subdomains, rather than having to get a cert for every single subdomain.

As you may know, Certbot is the tool provided by the EFF that you use to interact with and issue certs from Let’s Encrypt. It used to be called letsencrypt-auto, but when the EFF took it over, it switched names to Certbot.

Now, it’s not quite as easy to get wildcard certs as it is to get normal certs – mainly because there are some prerequisites. The nice thing is, some of these prerequisites make it easier to issue and renew certificates without temporarily disabling your web server.

Prerequisites

The certbot docs aren’t super clear about a lot of this, so you have to do some digging, but essentially it boils down to this:

I’ve decided to just go with Google Cloud DNS, because I’m already doing some  other stuff in Google Cloud Platform, and it’s really quite affordable for what I need. Sure, it’s not as cheap as just using the DNS that your registrar provides, but I know mine doesn’t provide an API, especially not one supported by certbot.

Setting up Google Cloud DNS

To get Google Cloud DNS set up, you’ll obviously need a Google account. If you don’t have one, well, I’ll leave it to you to get one. Then you’ll need to set up Google Cloud Platform – once again, I’ll leave that to you. You’ll also need to set up a project for your DNS records. If you already have a project, you can feel free to use that.

Set up a Service Account

Next up, you’ll need to set up a ‘Service Account’ that will let you access the GCP DNS API with restricted permissions, so that you can safely put the credentials on a box that’ll handle your renewals without having to fully authenticate with GCP yourself. To start out, click the ‘hamburger’ menu on the left, then find ‘IAM & admin’, and finally ‘Service Accounts’:

Access the Service Accounts Screen

Once there, you’re going to want to click the ‘Create Service Account’ button at the top. Currently, the Cloud DNS permissions and whatnot are in beta, so that means that while you could create a custom role that would have exactly the permissions that you need, those are subject to change and there’s a decent chance that you’d need to recreate your service account later. To avoid this, I just made my new service account into a ‘DNS Administrator’. It’s got more permissions than I strictly need, but I’m not super worried about that.

Assign a Role

After you give your service account a name, you’ll want to check the ‘Furnish a new private key’ box so that you can download the credentials file that you’ll need to access the API later:

Create Your Service Account

When you create your account, it should automatically download the JSON file with the credentials. I copied that key to the server I’m using to issue certificates (my good ‘ole Linode server!) so that I can use it later.

Creating your DNS Zone

Naturally, to do anything with DNS, you need to have a domain to do something with. You’ll have to point your registrar to Google/your DNS provider of choice before you can actually issue a wildcard certificate.

I set up my domain in Google Cloud DNS before I switched anything at my domain registrar, so that I wouldn’t have to worry about any downtime where my site was unreachable. To do this, you’ll first want to access the Cloud DNS control panel in GCP:

Access the Cloud DNS Console

Next up, click the ‘Create Zone’ button at the top of the console. Then, you’ll enter the information for your domain:

Create a DNS Zone example

Then you’ll create some records for your domain. At the very least, you’ll probably want one A record pointing to your server, but you can also create subdomains or whatever else you want. If you’re copying your config from somewhere else, put in all the records that you had on your previous provider. I had quite a few records, but even so it didn’t take very long.

Finally, you’ll update the nameservers to point to your DNS provider. I use a few different registrars, but they all make this part pretty easy. In this particular case, I was using Hover – all I had to do was click their ‘Edit Nameservers’ link near the domain I wanted to adjust, and put in Google’s nameserver addresses:

Editing Nameservers

You’ll need to put in whatever your DNS provider wants, but if you’re using Google, the above should work. Once you’ve done that, you might have to wait a few minutes for your DNS to switch over to your new provider. It only took a couple of minutes in my case. I used the DNS lookup tool at MxToolbox to figure out when it switched. It shows your nameserver at the bottom, like this:

Once that’s updated to your new provider, you’re ready to get issuing certificates!

Using Docker to Issue the Certificates

Now, we can do something really nifty here to renew our accounts. Instead of installing certbot-auto on our server, we can just always use the latest up-to-date version in a preconfigured, lightweight Docker container. Sure, this requires you having Docker installed, but who doesn’t these days? If you’d rather, you can do this all manually with your own certbot-auto installation, but I chose to go the Docker route, for simplicity’s sake.

Now it’s actually pretty simple to just run our Docker container. We just need to get the correct arguments, and we’ll be good to go. Note, this does require that you have locations set up where you want to put your certificates, otherwise they’ll just float off into the ether when your container shuts down.

Configuring our Volumes

So first up, let’s create the place we want our certificates to be placed. In my case, I’m going to stick with the default, since my server has already been storing Let’s Encrypt certificates there anyway: /etc/letsencrypt. This is where Let’s Encrypt stores all its configuration and certificates by default. If you’re putting it somewhere else, go ahead and create that directory. Keep note of it for later, naturally. Later, we’re going to map our Docker container to use that as a volume.

Next up, we need to create the place where Let’s Encrypt will store backups. I guess this isn’t strictly necessary (mine is empty anyway), but I figure it can’t hurt. By default it’s in /var/lib/letsencrypt. So we’ll also be mounting that.

Finally, we need to mount the directory where you put your GCP service account’s credentials. I’ve put mine in ~/.secrets/certbot and changed the name to google.json, but you can put it wherever you want and call it whatever you want, really.

So these are going to end up being arguments like this when we run our docker command:

-v "/etc/letsencrypt:/etc/letsencrypt" \
-v "/var/lib/letsencrypt:/var/lib/letsencrypt" \
-v "/home/$(whoami)/.secrets/certbot:/secrets"

Docker & Certbot Arguments

The image that we’re going to be using (assuming you’re sticking with Google DNS) is certbot/dns-google. If you’re using another DNS provider, you can probably figure out pretty easily which image you’ll need.

In the command itself, we’ll also need to use the certonly command, signifying to certbot that we don’t want to have it try to actually install the certificates for us, we only want it to issue them. I personally prefer installing them myself anyway, and that would be very difficult and/or impossible to do from within a Docker container anyway.

We’re also telling certbot to use Google’s DNS with --dns-google, and we’re giving it the path to the credentials file with --dns-google-credentials <file-path>. The last thing we have to do is manually specify the Let’s Encrypt server that we’re using, because right now, wildcard certs are only supported by one server: --server https://acme-v02.api.letsencrypt.org/directory. That should do it for our arguments.

All this means that our full docker command will look like this:

sudo docker run -it --name certbot --rm \
    -v "/etc/letsencrypt:/etc/letsencrypt" \
    -v "/var/lib/letsencrypt:/var/lib/letsencrypt" \
    -v "/home/$(whoami)/.secrets/certbot:/secrets" \
    certbot/dns-google \
    --dns-google \
    --dns-google-credentials /secrets/google.json \
    --server https://acme-v02.api.letsencrypt.org/directory \
    -d '*.russt.me'

Of course, if you’ve made any adjustments in the way you’re doing this, you’ll need to adjust the command itself. The final line is the important one, -d '*.russt.me, is telling certbot to issue a new wildcard certificate for *.russt.me. You’ll want to make sure you change it, because, well, I’d rather you not issue certificates for my domain. Plus, unless you’ve hacked my Google DNS account, you probably don’t have access.

With any luck, you’ll see some output that looks a lot like this:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator dns-google, Installer None
Obtaining a new certificate
/usr/local/lib/python2.7/site-packages/josepy/jwa.py:107: CryptographyDeprecationWarning: signer and verifier have been deprecated. Please use sign and verify instead.
  signer = key.signer(self.padding, self.hash)
Performing the following challenges:
dns-01 challenge for russt.me
Unsafe permissions on credentials configuration file: /secrets/google.json
Waiting 60 seconds for DNS changes to propagate
Waiting for verification...
Cleaning up challenges

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/russt.me-0001/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/russt.me-0001/privkey.pem
   Your cert will expire on <date>. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot
   again. To non-interactively renew *all* of your certificates, run
   "certbot renew"

I’ve cleaned it up a bit, but that’s the gist. I chose to ignore the ‘Unsafe permissions’ warning, since I’m running this in a Docker container anyway and the file on my system actually does have the correct permissions.

Last Thoughts

I’ll leave the configuration and use of these certificates up to you. Hopefully, you have some idea of how to use the certificates on your own server. If not, there are guides all over that should help you out.

I’m extremely thankful to both the EFF and to Let’s Encrypt for enabling us all to issue free SSL certificates, and it’s even better now that they’re letting us issue wildcard certificates. If you’re also feeling thankful, head on over to the EFF’s donation page or Let’s Encrypt’s donation page and drop a donation for them. The web will thank you for it.

If you’ve got any questions or comments, feel free to drop them in the comments below! I’ll do my best to get back to you. Extra props if you correct an error or tell me a better way to do this.