Since being accepted as an intern with the Mirage OS project, I’ve been doing a lot of tinkering with unikernels. In particular, I’ve become convinced that hosting a static site as a unikernel has a lot of benefits over traditional methods. A unikernel is a secure way to self-host your site, which means you have control over your own content. But because unikernels are a relatively new technology and are still largely experimental, building your own with zero prior knowledge is hard. I’m hoping to lower that barrier to entry with this guide. Here’s what we’ll be doing:
- building a custom unikernel to serve our static site using the Mirage OS libraries and build tool
- fetching a TLS certificate for our domain using Let’s Encrypt
- deploying our shiny new unikernel and certificate to Amazon Web Services
In case I made it sound like a cakewalk just now, a word of caution: the unikernel ecosystem is still pretty immature, and there’s a pretty high chance that what worked for me is going to require some small tweaks on your part. If you’re afraid of getting your hands dirty, you might want to wait until the tooling matures a bit before attempting this. This guide will work best for you if you:
- use some type of 64-bit *nix. AFAIK this guide should be easily followable for all linux and BSD users. I am less sure about OSX users. Mirage OS does not have support for Windows.
- want to host a static website. If you are looking to host some other, non-static type of site, or are interested in unikernels for some other reason entirely, you will probably still be able to make use of the later parts of this guide (where we deploy the unikernel to AWS). However, the unikernel code will look significantly different, and for that part you’re on your own.
- already have a custom domain for your site. I will not cover how to purchase and configure a domain name.
Most software guides install all needed dependencies in the first step, but since some of the tools we’ll be using are kind of fragile, we’re going to install them one at a time, verifying that they work as expected at each step. That way, if you need to do some tinkering to get things right at any point, you can continue to follow the guide afterwards. But for the brave/impatient, here’s what we’ll be using:
- Mirage OS, which requires OCaml
- certbot (if you don’t have a TLS cert for your domain but want one)
- An AWS account (free tier will suffice)
- The AWS CLI
- ec2-unikernel, which requires Haskell
Let’s dive in!
1. Install Mirage OS
First, we’re going to build a unikernel that we can run and test locally. We will be using the Mirage OS libraries and build tool to build our unikernels. With Mirage, unikernels are written in OCaml, but as long as you’re okay using my unikernel code, you don’t have to know any OCaml to follow this guide. If you are completely new to unikernels, know that there are other systems that you can use to do this. However, I have no idea if unikernels built using other systems will run on AWS, so I suggest you stick to Mirage for now. Follow the install instructions to install OCaml, OPAM, and Mirage, then run these commands to test your installation:
$ git clone https://github.com/mirage/mirage-skeleton
$ cd mirage-skeleton/tutorial/hello
$ mirage configure -t unix
$ make depend # might take a while
$ make
$ ./hello
2017-06-02 19:47:56 -04:00: INF [application] hello
2017-06-02 19:47:57 -04:00: INF [application] hello
2017-06-02 19:47:58 -04:00: INF [application] hello
2017-06-02 19:47:59 -04:00: INF [application] hello
If you get output similar to above, then your installation works! If not, you can always pop into the #mirage channel on IRC for help troubleshooting your installation.
2. Compile a unikernel for your static site
Now that we have Mirage installed, we can build a unikernel for our static site!
Our unikernel is composed of two OCaml source files, config.ml
and
dispatch.ml
, which can be found
here.
These files are nearly identical to
the ones in mirage-skeleton/applications/static_website_tls/
, except I’ve
tweaked them a bit to handle URLs with a trailing slash.
Let’s say the static site you want to serve
is located at ~/mysite/content
. Save the OCaml files as ~/mysite/config.ml
and ~/mysite/dispatch.ml
, respectively, then edit line 5 of config.ml
to
point to content
. You also need to copy the directory
mirage-skeleton/applications/static_website_tls/tls
to ~/mysite/tls
. Because
our unikernel is configured to automatically redirect HTTP requests to HTTPS, it
needs some type of TLS certificate to build and run correctly. For now we’re
going to use the self-signed ones included with the mirage static site example,
but in the next step we’ll grab a real TLS certificate for our domain using
Let’s Encrypt. If you just want a HTTP-only server, you are free to modify the
code yourself, but you should really be using TLS! With the certificate copied,
now run:
$ cd ~/mysite
$ mirage configure -t unix --net=socket --http=8080 --https=4433
$ make depend && make
$ ./https
2017-06-02 20:04:47 -04:00: INF [tcpip-stack-socket] Manager: connect
2017-06-02 20:04:47 -04:00: INF [tcpip-stack-socket] Manager: configuring
2017-06-02 20:04:47 -04:00: INF [https] listening on 4433/TCP
2017-06-02 20:04:47 -04:00: INF [http] listening on 8080/TCP
These commands should generate a lot of output, the last of which is displayed
above. You should now be able to open http://localhost:8080
in your browser,
which will redirect you to https://localhost:4433
and then block you from
viewing that because the TLS certificate we supplied isn’t trustworthy. Because
we’re trying to verify locally that our site works as expected, this time I
recommend overriding the block so you can view your site (in Firefox click
“Advanced > Add Exception”). Once you’ve verified that your site works as
expected, you’re (almost) ready to upload your site to AWS.
3. Fetch a TLS certificate for your domain using Let’s Encrypt
Because we’re using a self-signed TLS certificate right now, all browsers will
prevent their users from visiting our site as a security precaution. In order to
actually serve our website over HTTPS, we need a real, trustworthy certificate.
This used to mean paying a certificate authority money, but since last year the
Let’s Encrypt project has been giving them away for free. If you were already
self-hosting your site over HTTPS and thus already have a TLS certificate for
your domain, just copy it to ~/mysite/tls
and you can skip this step.
Otherwise, go ahead and follow the instructions for your distro on
the certbot website to download certbot. Select
“None of the above” for the “Software” option. Once you have certbot installed,
there are a couple of ways you can go about actually getting your certificate.
If you are migrating to a unikernel from a self-hosted server, your simplest
option is probably to install certbot on the server and use certbot certonly
--manual --preferred-challenges http
, which will establish your ownership over
your domain by having you enter commands on the server. If, however, you are migrating from
something like wordpress or github pages (my case) where you don’t have control
of the server, you’ll have to run certbot certonly --manual
--preferred-challenges dns
, which establishes your ownership over your domain
by having you set certain DNS records. Note that both methods will record the IP
of the machine you requested the certificate from, which in the dns
case might
be different from the IP you plan to host the unikernel from. If you do not want
the IP of your development machine recorded and associated with the certificate,
one thing you could do (probably, I haven’t tested it) is boot up a linux
instance on AWS, associate it with the amazon elastic IP you plan to use for
your unikernel, point your DNS records at the elastic IP, install a conventional
webserver and certbot on the instance, copy over your static site files and
serve the site, and then use certbot certonly --manual --preferred-challenges
http
to get your cert. Your choice! In any case, make sure you set
--config-dir
, --work-dir
, and --logs-dir
to writeable paths so that you
don’t have to run the script as root (these are given relative to the present
working directory, don’t use absolute paths), and -d example.com
to specify the
domains you want certificates for. See the
certbot manual
if you are unsure or would like more information. So, to summarize, in my case I
ran:
$ cd ~ && certbot certonly -d davidudelson.com -d www.davidudelson.com --manual --preferred-challenges dns \
--config-dir .local/letsencrypt --work-dir .local/letsencrypt/work --logs-dir .local/letsencrypt/logs
to generate my certificate. Once you have yours, it’s time to update our
unikernel’s tls certs to match. It’s a good idea not to move any files in the
directory that certbot placed your certs in, because this might prevent certbot
from updating your certs in the future. Instead, let’s make symlinks to the
live
directory (which always contains the up-to-date certificates)
files:
$ ln -s ~/.local/letsencrypt/live/example.com/privkey.pem ~/mysite/tls/server.key
$ ln -s ~/.local/letsencrypt/live/example.com/cert.pem ~/mysite/tls/server.pem
$ ln -s ~/.local/letsencrypt/live/example.com/chain.pem ~/mysite/tls/ca-roots.crt
Now you have a real TLS certificate to use with your unikernel!
4. Install ec2-unikernel
Next we are going to install
the ec2-unikernel tool,
which automates the process
of uploading and installing unikernels to EC2 as Amazon Machine Images (AMIs).
You need to have Stack
and the guestfish
command-line utility installed in order to build and run the
tool, so install those first if you don’t have them already. Next, run:
$ cd ~ && git clone https://github.com/GaloisInc/ec2-unikernel
$ cd ec2-unikernel
$ stack init
Now normally we would run stack build --install-ghc
to build the tool, but unfortunately at
the time of writing, this package is a little bit broken, and it doesn’t compile
with the default settings. If you’re reading this at least a few weeks after the
time of writing, try a stack build --install-ghc
and see if it works, but if not, I’ll
demonstrate what I had to do to get it working. In essence I needed to use an
earlier version of the amazonka-*
set of packages, which required me to
both choose a resolver for stack that had the
versions of the packages I wanted and rollback the tool to an earlier version:
$ git checkout 4d28b4c
$ stack config set resolver "lts-7.22"
$ stack build --install-ghc
Once the tool builds correctly, the command stack exec ec2-unikernel
should
print out some error message. It fails because we haven’t set up AWS yet, which
is what we’re gonna do next!
5. Setup AWS
There’s a couple things we need to do on AWS to allow ec2-unikernel to upload our unikernels correctly:
- Generate an AWS credential keypair
- Install and configure the AWS CLI
- Make an S3 bucket for our unikernels
- Create the “vmimport” role
- Create a security group for our unikernels
We’ll need to generate the keypair using the AWS web interface, but once we’ve done that, the rest can be done from the command line thanks to the AWS CLI. Of course, all of this assumes that you have an AWS account, so create one first if you don’t already have one. AWS has a free tier that gives access to S3 and the least powerful instances on EC2 for one year. That is, you get a free year of unikernels!
Generate an AWS credential keypair
After your account is set up, log in to the control panel and click on your name in the top-righthand corner followed by “My Security Credentials”. Amazon will prompt you to create an IAM user; I didn’t bother. Click on “Access Keys” > “Create New Access Key” and you’ll be shown a dialogue box with your access and private keys. Do not close this dialogue box yet. There’s no way to see your secret key again once you do. We’ll be giving this keypair to the CLI in just a minute, but if you don’t want to wait you can copy it into a text file or something just to be safe.
Install and configure the AWS CLI
Install the
CLI, then run aws configure
. Enter your AWS access and private keys from the
previous step, and enter “us-west-2” for your region. AWS separates its servers into
regions
and all the configuration we’re doing is only valid for one region.
Unfortunately, due to a bug in ec2-unikernel, the only region that works right
now is Oregon, so that’s the one we have to use.
Make an S3 bucket for our unikernels
AWS S3 is just storage. S3 storage is allocated by “bucket”. The ec2-unikernel tool we installed before will upload our unikernel to an S3 bucket and then import it from the bucket into EC2. To make an S3 bucket, run:
$ aws s3 mb s3://<bucket-name>
When choosing a bucket name, keep in mind that the bucket namespace is shared by your entire region. Make sure to pick a memorable name that’s unique to you. For example, I named my bucket “dudelson-unikernels”.
Create the “vmimport” role
This is something that ec2-unikernel needs to be able to import our unikernels
from S3 into EC2. Follow the instructions
here
to set it up. The json files
described in the article are located in the ec2-unikernel/policies
directory.
Note that you have to edit the parts of role-policy.json
that are red in the
instructions to be the bucket name you chose in the last step.
Create a security group for our unikernels
EC2 instances are controlled by “security groups” which describe what ports are open for inbound and outbound network traffic on the instance. The default security group that is created when you launch an instance opens only port 22 so that the instance can be accessed via SSH. In our case, we cannot access our static site unikernel via SSH and only want it to serve web traffic, so we’re going to create a security group that opens only ports 80 and 443. Use the following command:
$ aws ec2 create-security-group --group-name "<name>" --description "<description>"
$ aws ec2 authorize-security-group-ingress --group-name "<name>" --protocol tcp --port 80 --cidr 0.0.0.0/0
$ aws ec2 authorize-security-group-ingress --group-name "<name>" --protocol tcp --port 443 --cidr 0.0.0.0/0
The group <name>
and <description>
can both be something as simple as
“unikernels”, since you’re not sharing the security group namespace with your
entire region.
6. Build and deploy your unikernel
And now, the main attraction! We’ve put all the pieces in place, and now we can finally deploy our unikernel to AWS. But before we do so, we need to rebuild the unikernel to target the Xen hypervisor, which is what AWS uses. This looks very similar to what we did in step 1:
$ cd ~/mysite
$ mirage configure -t xen --dhcp=true
$ make depend && make
When we build for the xen hypervisor, we need to enable DHCP so that our unikernel can find the default gateway provided by AWS when it’s launched on EC2. Now we can upload it:
$ cd ~/ec2-unikernel
$ stack exec ec2-unikernel -- -o `cat ~/.aws/credentials | grep id | cut -d " " -f 3` -w `cat ~/.aws/credentials \
| grep secret | cut -d " " -f 3` -b <bucket-name> ~/mysite/https.xen
We provide ec2-unikernel with our AWS keypair and the name of the S3 bucket we created, which is all it needs to upload and import our unikernel as an AMI. If the commands to get the AWS keypair don’t work on your system, you can just enter the credentials manually. This takes a while, so now would be a good time to get a drink or something. When that’s done, the last line of output will be the ID of the new AMI. We use that to fire up the instance from the command line:
$ AWS_SEC_ID=`aws ec2 describe-security-groups | grep -C 1 <group_name> | grep GroupId | tr -d '" ,' | cut -d ':' -f 2`
$ aws ec2 run-instances --image-id <image-id> --count 1 --instance-type t1.micro --security-group-ids $AWS_SEC_ID
The <image-id>
should look like “ami-ABCD1234”. <group_name>
is the name of
the security group you created. This should spit out a bunch
of json about the instance. Wait about 60 seconds, then run aws ec2 describe-instances | grep "running"
.
If your unikernel is up,
you should get a line of output back. If not, wait a little longer and try
again. Once you’ve confirmed that your instance is up and running, use aws ec2
describe-instances | grep "PublicDnsName"
to find the URL that you can use to
connect to your website. Pop that in your browser and you should get a security
warning like we did in step 2. This is again because the TLS cert we generated
is for our domain name, not blahblah.compute.amazonaws.com. Put this nonetheless
means that you were able to successfully connect to your unikernel running on
AWS!
7. Maintenance
As I eluded to at the beginning of the guide, one of the really nice properties of unikernels is every time you make a change to your site, you rebuild the entire software stack and upload a fresh image. This means that even if an attacker compromises your system through your website, the damage will be undone with the next image. Here are the steps you’ll perform to update your site by deploying a new unikernel:
- rebuild your static site. I’m using jekyll for this, so for me that’s
bundle exec jekyll build
. - rebuild your unikernel
$ cd ~/mysite
$ mirage configure -t unix --net=socket --http=8080 --https=4433 # test build; OR
$ mirage configure -t xen --dhcp=true # production build
$ make depend && make
- upload your unikernel to EC2
$ cd ~/ec2-unikernel
$ stack exec ec2-unikernel -- -o `cat ~/.aws/credentials | grep id | cut -d " " -f 3` -w `cat ~/.aws/credentials \
| grep secret | cut -d " " -f 3` -b <bucket-name> ~/mysite/https.xen
- launch the unikernel, reallocate the EIP, and terminate the old unikernel
$ aws ec2 describe-instances | grep "InstanceId" # get the old image ID
$ aws ec2 run-instances --image-id <image-id> --count 1 --instance-type \
t1.micro --security-group-ids "<security-group-id>" | grep "InstanceId" # get the new image ID
$ aws ec2 associate-address --instance-id <new-instance> --allocation-id <allocation>
$ aws ec2 terminate-instances --instance-ids <old-instance>
Of course I’ve scripted this entire process from stem to stern.
And that’s it! Go enjoy your new-and-improved unikernel life, tell your friends, and maybe write me a build tool that will make this easier.
8. Assign an elastic IP and configure DNS
AWS elastic IPs allow you to allocate a public IPv4 address that you can assign to a running EC2 instance. As in the previous step when we didn’t have an EIP yet, the instance is given a public IP anyway, but it’s chosen from the pool of available IPs for your region, and once the instance is terminated, its IP is released back into the pool, meaning that the next instance you start will have a different IP. But an EIP, once allocated, belongs to you, even if you terminate one instance and start another. This means you can point the DNS records for your domain at the EIP, and whenever you build and deploy a new unikernel, you can reassign the EIP to point to it, keeping your domain up in a hassle-free way. One warning though: If you allocate an EIP and leave it idle (e.g. not assigned to a running instance), you’re charged hourly for it. So make sure to deploy your new unikernel and reassign the EIP before terminating the old one. With all that in mind, let’s allocate an EIP and assign it to our instance:
$ ALLOC_ID=`aws ec2 allocate-address --domain vpc | grep "AllocationId" | tr -d '" ,' | cut -d ':' -f 2`
$ INST_ID=`aws ec2 describe-instances | grep "InstanceId" | tr -d '" ,' | cut -d ':' -f 2`
$ aws ec2 associate-address --instance-id $INST_ID --allocation-id $ALLOC_ID
After assigning an EIP, update the type A DNS records for your domain to point to the EIP, and your site is live!