Garry Bhattal

What is the DevOps model and what advantages does it bring to your workflow?

2020-07-21T10:00:00-07:00

It’s 2020 and although DevOps is a practice that’s very commonly adopted today, I feel that a not too overly technical explanation of the subject might be beneficial to those individuals, or companies, that are not using it yet but find themselves in a position where it makes sense to use such model. Moreover, if you’re completely or partially new to this subject, this is a great place to start!

So what is this DevOps thing?

DevOps can be defined as a combination between cultural philosophies, practices and tools.

The cultural philosophy aspect is about changing how you develop and deploy software. In the DevOps model people who take care of software development and people who take care of operations (by managing, monitoring and operating infrastructure) work very closely together; sometimes the two roles are combined together. The DevOps term originates exactly from here: Developer Operations.

Next, there are practices. They help you to understand how the DevOps model works, in practice. Don’t worry if you have no idea what these are, more on them in just a moment:

Continuous Integration
Continuous Delivery
Microservices
Infrastructure as Code
Monitoring and Logging
Containerization & Orchestration

Finally, tools. These, along with the aforementioned practices, will help you to concretely adopt the DevOps model. Some of the relevant tools are:

Git
Jenkins
Docker
Kubernetes
Automated testing libraries/frameworks

The DevOps model will improve every aspect of the application lifecycle, from development and testing to deployments and production rollouts. The team relies on automation to speed up manual processes and, thanks to the appropriate tools, applications can grow in a rapid and flexible way.

DevOps is first of all a mentality shift, where you are required to change the way you approach software development and release. While it may be good to have one person to be the leader of this change, you should make sure that your whole team/company is ready to take the leap.

Let’s dive deeper into the practices

DevOps tools

Continuous Integration

It is composed by two main elements: a cultural one and a technical/automated one. The cultural part consists in learning how to make frequent integrations. In other words, developers should not wait an excessive amount of time before merging their commits into the rest of the project. The technical/automated part is represented by the tools that helps automating these frequent integrations. Jenkins, for example, is a popular open source tool widely used for this. Continuous Integration solves the problem of heavy integrations and it helps reducing time spent solving merge conflicts. Statistically, the longer you wait to merge the more chances you will incur in a conflict. Merge often to avoid this!

Continuous Delivery / Deployment

It extends Continous Integration by automating deployments to staging and production environments. These deployments can have pre-requisites such as automated tests; they can even go beyond Unit Tests and require interface tests, load tests, integration tests, API reliability tests and so on. There is a difference between Continuous Delivery and Continuous Deployment. With C. Delivery you will automatically deploy on a staging environment and manual approval will be required if you want to deploy in production. With C. Deployment you go one step further and also automate production rollouts. You can use Jenkins for this or you can search for more specific tools that will hook into the Continuous Integration offered by Jenkins. This practice will help you reduce human errors and time required to complete the deployment process.

Microservices

It is a type of architecture used to build an application or a set of services. The application is based on a smaller set of services that communicates with each other through, for example, HTTP APIs. Each microservice has its own responsibility and the application is broken down into microservices according to the needs of the application domain. Frameworks or programming languages can be used to write microservices and distribute them, either individually or as a group of services. With this approach, our application will have the following benefits:

More easily maintainable. You only touch the microservice you need.
More resilient to errors. If a microservice fails, the rest of the microservices can continue to work since they are independent.
Better testable and more modular. You can use different languages and technologies on each microservice, using the most suitable solution on case by case basis without committing to a single technology for the whole project. Each microservice can then be tested with separate approaches and methodologies.

Microservices have gradually become the standard for building systems that make use of practices such as Continuous Deployment.

Infrastructure as Code

It consists in managing the cloud infrastructure through software development methodologies, such as version control and continuous integration. The cloud can be setup via API, allowing developers and administrators to interact with the infrastructure in a programmatic way and on a very large scale rather than through manual setting and configuration of resources. Since everything can be managed by code, the infrastructure and servers can be deployed as quickly as possible via standardized templates, updated with patches or duplicated iteratively. There are several tools to implement this: Chef, Puppet, Terraform, Ansible, etc. The advantages are remarkable, in addition to those above we have zero downtime (if applied well) and auto-scaling of resources.

Monitoring and Logging

Keeping parameters and logs under check is useful to find out how application and infrastructure performance affects the end-user experience. Data and logs generated by applications and infrastructure are acquired; after categorizing and analyzing them, the possible causes of problems or unforeseen changes can be thoroughly examined. Creating alarms or analyzing data in real time also help in proactively monitoring services. One of the tools to implement this practice is Nagios.

Containerization & Orchestration

These are two techniques used to manage software. Containererization, through tools such as Docker, provides a logical packaging mechanism that allows applications to be abstracted from the underlying environment they run on. Decoupling things like this allows for easy and consistent deployments of container-based applications, whether the target environment is a private data center, the public cloud or a developer’s computer. Compared to virtual machines, containers have several advantages such as reduced disk size and low overhead. Orchestration refers to practices such as automatic deployment, scaling, and management of containerized applications and can be achieved using tools such as Kubernetes.

What are the advantages of the DevOps model?

Speed. Applications developed using a microservice architecture with the addition of continuous integration allows you to release updates more frequently and to better control services.
Rapid delivery. Practices such as continuous integration (CI) and continuous delivery (CD) lead to faster releases and bug fixes.
Reliability. Thanks to CI/CD and monitoring/logging rest assured that releases will meet quality standards without sacrificing the end-user experience.
Scalability. Infrastructure as code allows you to manage development, testing and production environments in an iterative and more efficient way - on any scale.
An evolution of software architecture, development and delivery that is not the same as it was twenty years ago.

A simple pipeline example

Now that we know about DevOps and its advantages, let’s briefly look at a simple pipeline example. There is no correct or definitive structure: you can modify the pipeline to better adapt it to certain situations or as you see fit. This is just an example.

In the first step you version the code, with a tool like Git. Developers then develop and commit the code to a repository.

The second step is not always necessary, it depends on the language and technology used. It consists of building the project, which often means having a working version of the project locally.

The third step is running unit tests on the project. This can also be automated - useful to prepare for the next step.

The fourth step is deployment to development/staging environments. At this point you can view the code running on an environment that can be shared with the client. This point can also be automated.

Subsequently, other tests can be run on the development environment. Other people responsible for testing the project can come into play by running their test tasks, even manually, on a development environment that should be complete and working. Depending on requirements, this step can also be automated.

If all goes well it’s time to go live! Deployment in production happens. Once this is done there is the operational/monitoring phase. Things such as applications, state of the infrastructure, errors, logs and more are kept under control.

Final Thoughts

Hopefully by now you have an idea of what DevOps is, at least on paper. DevOps, as I have repeatedly stated, is first and foremost a cultural change that impacts all the figures involved in software development. In fact, it requires a different, more modern approach to software development and distribution; in return, if implemented well, it provides us with several advantages such as the reduction of different problems and timescales. It also prepares us to better face challenges that previously would have been more problematic to manage.

Should you find any mistake, technical or grammatical, please don’t hesitate to let me know!

How to set up Jekyll with Docker and GitHub Pages

2019-02-18T09:00:00-08:00

Want to set up an easy to maintain and highly scalable blog in little to no time? I personally found that running Jekyll in a Docker container for local development and letting GitHub Pages handle the production stack is the best set up for a developer’s blog. Let’s find out how to achieve this, starting from scratch.

Jekyll and GitHub Pages: a powerful combination!

Installation and configuration of Docker and Docker Compose

I will briefly cover how to install Docker and Docker Compose since they are required for this set up. This tutorial will work with a fresh Ubuntu install, as long as it is the 64-bit version and sports a kernel version of 3.10 or higher. You can easily find how to do this on a different OS if you search around.

We shall first add the GPG key for the official Docker repository:

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

Then, add the Docker repo to your system’s APT sources:

sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

Finally, update the package db and install Docker:

sudo apt-get update
sudo apt-get install -y docker-ce

Now let’s install Docker Compose and apply executable permissions to the binary:

sudo curl -L https://github.com/docker/compose/releases/download/1.24.0/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

Be careful to download the latest version of Docker Composer by changing the 1.24.0 part in the request URI above.

You should now be ready to use Docker and Docker Compose. Let’s see what role they play in our set up.

Creating the container

The easiest way to get started is probably to download an existing theme to a local folder. We will call this folder ~/site/, download a Jekyll theme (for example Pixyll) and store it there.

Let’s now create the configuration file relative to our container and let’s use Docker Compose to easily create and run that container. Inside ~/site/ create a file named docker-compose.yml with the following content:

jekyll:
    image: jekyll/jekyll:pages
    command: jekyll serve --watch --incremental
    ports:
        - 4000:4000
    volumes:
        - .:/srv/jekyll

The content of this file deserves further explanation. With image: jekyll/jekyll:pages we specify that we want to use the official Jekyll image for Docker, in its pages flavor which is suited for GitHub Pages.

With command: jekyll serve --watch --incremental we are executing the container. This will run Jekyll’s built in development server. It’s actually the same as if you would install Jekyll on your machine without a container and tried to make it serve a blog locally. The two additional flags will make sure that Jekyll automatically picks up your edits without having to build the whole site each time. 4000:4000 is there to forward port 4000 of the container to your local port 4000.

The last line will create a mapping between your ~/site/ directory and the directory where the image is configured to look for a Jekyll site, which is /srv/jekyll.

Ok, as if that wasn’t easy enough, it gets even easier. Just run the following command inside ~/site/

docker-compose up

and watch your site go live at http://127.0.0.1:4000. Magic.

Docker is an amazing tool as it allows to create a working dev enironments in no time! Very useful for teams.

Pushing the site live with GitHub Pages

It’s kinda amazing but things are getting even easier, if possible.

Create a new GitHub repository named username.github.io (replace username with your GitHub username). Figure out a way to clone the new repository in ~/site/ while retaining all of your previous blog files.

Now just commit and push the blog files to your new GitHub repo. Done, your site should be live at https://username.github.io .

This is one way to quickly and efficiently set up a fully functional blog at zero cost; I personally find it very convenient! Let me know if you do too.

Should you find any mistake, technical or grammatical, please don’t hesitate to let me know!

Understanding UNIX/Linux File System Permissions (Part 2)

2017-12-21T15:00:00-08:00

In the first part of this couple of posts about UNIX permissions we saw how the system is structured under a users/groups model. We saw how each user belongs to at least one group and how users and groups are unequivocally identified by UIDs and GIDs respectively.

In this part we will see the actual UNIX permission system. We will also explain the concept of ownership.

Ownership and Permissions

Each file in UNIX belongs to a user and to a group. So, for a generic user, we have three main scenarios:

User is the owner of file
A group owns the file and user is part of that group
User does not own the file nor is in a group who owns the file

But what is Unix ownership, to be more precise? It’s actually very simple. The ownership system is used to assign different permission sets to different users or groups of users.

Now that we laid down the basics about users and groups, let’s get straight to the core of the question: permissions.

First of all we need to know how we can actually view the permissions of a file. To do this we execute the following command inside of a directory:

ls -al

This will print a list of all files present in that directory, along with each file’s details. Among these details, as you may have guessed, we also have the file’s permissions.

The output of ls -al

As you can see, a lot of columns will be displayed. Most of them, like the file’s name, size, owner and group are self-explained in the picture above. What’s interesting here is the first column, the Mode one. Each UNIX file is assigned a mode, which contains permissions. Let’s take a closer look at how Mode values should be read.

This the mode for the asset folder in the picture above, properly spaced to allow us to read it better.

d rwx rwx r-x

The first letter indicates whether the entry is a directory (d), a symbolic link (l) or a normal file (-). The following three sets of triads are the actual file’s permission:

The first three characters after the one denoting the file’s type, rwx, is the permission class called User and defines permissions for the owner of the file.
The second three characters after the one denoting the file’s type, again rwx, is the permission class called Group and defines permissions for each user in that group.
The last three characters after the one denoting the file’s type, this time r-x, is the permission class called World (or Other) and defines permissions for anyone else, such as the public.

Read, Write, Execute

Ok but what are these rwx? They stand for Read, Write and Execute. Let’s see each one of them in detail.

Read

In the case of a normal file, read means that the user is allowed to open the file and access its content.
In the case of a directory, read means that the user is allowed to see what’s inside of that directory.

Write

In the case of a normal file, write means that the user is allowed to modify and delete the file’s content.
In the case of a directory, write means that the user is allowed to create new files inside of the directory and delete it. Always be careful with write, since it’s “stronger” than write!

Execute

In the case of a normal file, execute means that the user is allowed to run the file as an executable, provided the user also has read permission.
In the case of a directory, execute means that the user is allowed to traverse the directory, in other words the user can “cd” (the UNIX command to change directory) into it.

Some examples

Let’s make some example to make the permission system more clear.

-rw-rw-rw-

In the case above the file is writable by the user, the group and the world too. Everyone can write the file but no one can execute it.

-rw-------

In the case above the file is writable only by the user; nor the group and the world can read or write the file. Like the previous case, no one can execute.

-rwxr-xr-x

In the case above the file is executable by everyone; the user can also modify or delete it.

drwxr-x---

In the case above the directory is accessible by everyone but the world. The owner can also delete or create new file inside of it.

Now you should have at least a basic understanding about how the UNIX file system permissions work! If you have any question or should you find typos or mistakes in this page please let me know through the comments.

Understanding UNIX/Linux File System Permissions (Part 1)

2016-09-30T16:00:00-07:00

This series of articles is mainly for UNIX/Linux newbies, the topic will be covered in a way that is both simple and easy to understand. One of the things that’s often not clear to users approaching the UNIX world is how the permission system for file and directories works. In fact, UNIX is a multi-user system, which means there are multiple users that can connect to a UNIX system at the same time and do operations with it. This is also true for other UNIX-like system like Linux.

Examples of this? Just think of a machine, running a UNIX system, connected to a network (such as an intranet or the Internet). Multiple users can concurrently establish connections with this machine, via telnet or SSH, and execute programs, get outputs and so on.

So a couple of reasons this permissions system exist is to avoid one user accessing files that are protected by another user and not to allow single users to crash the whole system.

Not every user is human

In UNIX not all users are humans. I may have a user garry in a system, denoting myself, but there may also be a user called www-data which denotes the privileges necessary for a web-server process to run. For the system, users represent an abstraction which defines what they may or may not do with any given entity in the file system.

Each user is unequivocally identified by a UID (User ID), which is a numeric value that allows the system to discriminate between all present users. Login usernames such as garry and www-data are just aliases for human operators, meant to make us more comfortable. For example, in my system, user garry may have the UID 1021. The system does not have problems associating the UID number with me, but the same number gives no valuable information to other human users operating the system. Instead, if another human user sees my username (garry) instead of my UID (1021), it is much easier for them to recognize me.

Users belong to groups

In UNIX every user belongs to at least one group. This means there can’t be UNIX users not associated to a group. But what are these UNIX groups, precisely?

The purpose of groups is basically to assign the same privileges over a determined resource to multiple users, in a quick and efficient way. To make an example, you may need this is in a situation where different users are working on the same project files. Groups, like users, have also their numerical identifiers. They are called GID (Group ID).

A preview from the next article of this series: this is how permissions are generally represented for a given file.

How to find out your UID and GID

If you are on a UNIX or UNIX-like system you can find out UIDs with the following command:

id -u username

Remember to replace username with the desired user. For example, if I want to get the UID of the user www-data in my system, I will have to execute the following:

id -u www-data

The result will be printed in the terminal as a numerical value, which is the UID for the specified user.

You can also get the GID of a user by typing the following at the UNIX prompt:

id -g username

Don’t forget to replace username.

If you want to find out all groups a user belongs to, you will need to enter the following:

id -G username

Note the capital G which is different from the lower-case g in previous command.

To recap, in this post we saw how a UNIX system is structured, under a user/group point of view. Users in UNIX are quite different than users in Windows; in addition, Windows doesn’t have the concept of group as seen in UNIX. These concepts are mandatory to explain what we will see in the next part, the UNIX permission system.

Should you find any mistake, technical or grammatical, please don’t hesitate to let me know!

How to keep Ghost running using pm2

2016-03-10T15:00:00-08:00

So you just set up your brand new blog with Ghost, the node.js-based blogging platform? You will most likely encounter the problem of keeping alive (and/or restarting) the node.js process on which Ghost is running.

In fact, if you want to launch Ghost on production values, you would normally use npm with the following command:

npm start --production

What’s the Problem?

You’ll experience the problem once you close the terminal session where you issued the previous command. Closing the terminal also kills Ghost (i.e. your blog isn’t available anymore) and that’s not what we want! You will also notice that once you restart your machine, Ghost will not automatically start on boot. You will have to start it manually with each physical machine restart.

There are multiple ways to solve the aforementioned issues, we’ll examine one method in particular to achieve the solution. Once we’re done you’ll be able to close the terminal and have Ghost running in the background without problems. This will also help you to have Ghost automatically start when your machine is subject to a system reboot.

I am now using Jekyll but Ghost is a wonderful blogging platform!

Onto the solution

In order to achieve our solution, we will use two npm modules called forever and pm2. They are both easy and relatively straight-forward to install and use.

First of all, we navigate to the directory where Ghost is installed.

cd /path-to-your-Ghost-installation-directory

Then we can proceed with installing and setting up pm2 itself. Execute these commands sequentially, preferably as a non-root user.

sudo npm install -g pm2
cd /path/to/ghost/folder
sudo npm install -g pm2
echo "export NODE_ENV=production" >> ~/.profile
source ~/.profile
pm2 kill
pm2 start index.js --name ghost
pm2 dump

Now there’s only one last command left to execute. This depends on the OS you’re currently on.

pm2 startup your-os

There are good chances that you’re on Ubuntu, Debian or CentOS; if that’s the case you just have to replace “your-os” with the name of your OS. For example, if you’re on CentOS, you’ll have to execute the following command:

pm2 startup centos

If you’re on Ubuntu:

pm2 startup ubuntu

And so on.

Some useful commands

Now that we have everything correctly installed and set up let’s see how to use this module. Following there are a few useful commands to get data and stats with pm2.

Stats
```
  pm2 status
```
Once executed, this command will show you a few useful statistics on your Ghost instance such the status, memory footprint and the process ID.
Stopping and starting

If you want to stop your Ghost instance you should first know what’s the “App Name” of the said instance. This can be found under the “App Name” column when you execute the status command. If you followed this guide, your “App Name” should be “ghost”. You can stop your Ghost blog by issuing the following command:
```
  pm2 stop ghost
```
If you now wish to start Ghost, type and execute the following:
```
  pm2 start index.js --name ghost
```
When using the start command you can assign to the newly created instance whichever name you prefer (with the –name parameter as in the example above); to subsequently stop Ghost you will have to use this same name. Notice that we issued this command in the beginning, along with the pm2 setup. Otherwise you can simply use:
```
  pm2 restart ghost
```
This will restart your Ghost instance with the same previous parameters.
Logs
```
  pm2 logs
```
You can use this command if you want to get live statistics and logs from your Ghost instance.

Echo of Today: a Twitter Bot reminding us what happened today, years ago

2015-09-18T16:00:00-07:00

Twitter bots are everywhere. An article from Quartz dated last year states that up to approximately 8.5% of the accounts considered active are automatically updated. That means 23 million of Twitter’s 271 million monthly active users are bots.

Yes, bots on Twitter are extremely popular so I decided to make one myself. I called it Echo Of Today. Let’s see how I did it and what was the result!

Twitter is becoming increasingly automated

What should my bot do?

When I first started to think about my upcoming Twitter bot, I had to come up with something interesting yet not already done to death. With 20 million bots around, you may guess that a lot of the most simple and obvious concepts are already implemented. I wanted to do something at least partially original, not your usual bot that just follows random people.

This meant that my bot would have to sport a certain level of complexity - let’s say I didn’t want to make something too simple but I also wanted to avoid losing myself in something utterly complex.

Then I had the idea: make a bot that tweets news articles published exactly one year ago, two years ago, three years ago (and so on, until an arbitrary number n of previous years to work on). I’ll explain better this idea with an example. If today is the 21st September, the bot would look up news published on the 21st September of one year ago and tweet them. It will also find and tweet news published on the 21st September of two years ago, of three years ago and so on (the number of years it goes backwards is configurable). In other words, the bot would remind us what was happening exactly one and two, and three and so on years ago.

Sounds like a nice idea right? But how to implement it?

Do you remember what happened today, two years ago?

Getting the pieces together

First of all, I had to decide which language I would use to code the bot. I chose to go with PHP, only because I knew it fairly well. But it can be any language you like, for example, I know many Twitter bots are written in Python or JavaScript. Just be sure that the language you choose can serve your purpose right.

After that, I had to put together the basics: create a new Twitter account for the bot, set up a new app on the newly created account, get all the API keys I needed and start writing the code necessary to create basic features for the bot, such as tweeting. For this purpose, I found Abraham Williams library twitteroauth very useful.

Next I was in front of the biggest challenge I faced when coding this bot: how the get the information I needed (old news articles, in this case). After a bit of brainstorm and research, I found out that the New York Times offers awesome APIs through their Times Developer Network. After studying their APIs, I managed to get the news I needed as a nicely formatted json response, perfect for my needs. This required a bit of coding (not much, really) but what I needed was there in the end.

Next there was the 140 characters challenge. I decided to structure my tweets as follows:

#nYearsAgoToday Article Headline http://article.uri #EchoOfToday

The first hashtag indicates how old the news is, in years (6 years old is the oldest I wanted to go back). Next we have the article’s headline, the article’s URI and the last hashtag is simply the name of the bot.

As you know, each Tweet can be long at most 140 chars and since I wanted to stick in a couple of lengthy hashtags too, not many characters were left for the headline and the URI. Headlines tend to be short enough, though after doing a bit of math I coded an upper limit on how long can I allow a headline to be for a news article to be tweeted. The biggest problem was actually the URI, which can get quite long. The obvious solution was to shorten it, which is what I did. I got my goo.gl API key and with the help of googl-php, a simple PHP class that facilitates the use of goo.gl APIs, I got the job done: now all URIs were much shorter and of the same length.

A final touch and the results

Now that the bot was done I had to decide how much it I wanted it to tweet. I decided to make it tweet 12 times a day, hourly, from around 11:30 CEST to around 22:30 CEST. I chose this time frame since it’s good for Europeans but also fills part of the day for people in the US.

I then divided the tweets into groups of twos: the first two tweets would show news from 6 years ago, the following two from 5 years ago and so on, until the last two tweets, which contain news from exactly one year ago. To achieve this, in addition to standard PHP coding, I used cron jobs on my server. This allowed me to automatically call the bot 12 times a day, just when I need it to tweet.

The Twitter page where the bot is working 12 hours shift - be sure to follow his hard work!

The outcome was nice. It took me about a day to finish everything, but I’m sure a seasoned programmer can achieve this in much less time. If there’s enough interest in this I may open source it in the future, but just to give you an idea the bot consists in just about 200 lines of code. Very simple.

As always, good APIs make work much easier, no surprises here! Actually, there was a surprise… And it was each time I opened Echo Of Today! There’s something joyful, sometimes sad, in reading and remembering what was happening today, years ago.

Thank you for reading! It was very fun to code this bot, I hope you found interesting what’s written here.

If you have any question about this, please do ask in the comments, I’m happy to answer and help. Should you find any mistake, technical or grammatical, please don’t hesitate to let me know!

Difference between URL, URI and URN

2015-08-25T16:00:00-07:00

This is a classic tech debate, and many people will sooner or later ask themselves what’s the difference between URLs, URIs and most possibly even URNs. As of today one can simply refer to URNs and URLs as URIs, but it may be very interesting and useful to know the difference between these three terms since technically a URI can still be further classified as URL, URN or both. If you happen to find these terms somewhere, you should at least know what they mean.

If regular people don’t know the technical difference between these three acronyms and what they represent it shouldn’t be an issue for them, in my opinion. But if you’re a Software Engineer or, more generally, if you work in the IT field then it might be well worth to have this knowledge in your bag.

This post is intended to be a clear explanation of the difference between URLs, URIs and URNs and will help you to quickly understand what you need to know. Are you confused about this topic? Let’s begin!

Origins

These three acronyms are specified by Tim Berners-Lee as an Internet standards track protocol in the document called RFC 3986: Uniform Resource Identifier (URI): Generic Syntax.

Citing the document:

A Uniform Resource Identifier (URI) provides a simple and extensible means for identifying a resource. This specification of URI syntax and semantics is derived from concepts introduced by the World Wide Web global information initiative, whose use of these identifiers dates from 1990 and is described in “Universal Resource Identifiers in WWW”.

This means that the Uniform Resource Identifier was conceptualized back in the 1990s with the purpose of distinguishing between who and where.

Let’s get more technical on this.

Tim Berners-Lee, the man who invented the World Wide Web and actual director of the W3C. Photo by Paul Clarke - Licensed under CC BY-SA 4.0.

The Difference

Let’s start by making one thing clear. URL and URN are both subsets of URI.

In other words, URL and URN are always a URI, but a URI is not always a URL or a URN. To better understand this concept look at the image below.

A great example to understand the difference between URN and URL is the following (taken from Wikipedia): in the case of a person, we can take in consideration his name and his address. The URL is similar to the latter one, the address; it tells you a method to find that object (the person by its street address, in this example). Keep in mind that, by the definition we gave above, this also a URI.

Instead, we can see the name of a person as a URN; the URN is therefore used to unequivocally identify a generic object. To be more precise, in the case of a person’s name this is not entirely true since there may be another person with the same name (and surname), so a more appropriate example may be the ISBN code of a book or a product’s serial number inside of a system; while this doesn’t tell you how or where to find that object, you indeed have enough information to retrieve it by yourself. Formally, URNs also have their syntax. From this document:

All URNs have the following syntax (phrases enclosed in quotes are REQUIRED):
< URN > ::= "urn:" < NID > ":" < NSS >
where NID is the Namespace Identifier, and NSS is the Namespace Specific String.

One example to understand them all

Let’s start by looking at how the concepts above apply to what matters to us: the Internet. The following quotes from Wikipedia give us a bit more technical explanation in comparison to the generic person-address example we used above.

For URL:

A URL is a URI that, in addition to identifying a web resource, specifies the means of acting upon or obtaining the representation, specifying both its primary access mechanism and network location.

For URN:

A URN is a URI that identifies a resource by name in a particular namespace. A URN can be used to talk about a resource without implying its location or how to access it.

Now, if you look around the web you will find a lot of examples and these can sometimes create more confusion than anything else. I will present you only one example, which is very simple and clearly shows the difference between URI, URL and URN in contest of the Internet.

Let’s consider the following fictitious example. This will be our URI:

https://garry.bhatt.al/posts/hello.html#intro

Let’s start by saying that

https://

is the method that defines how to access that resource. Instead,

garry.bhatt.al/posts/hello.html

is the location where the resource resides, while, in this example,

#intro

is the resource.

So we can say that the URL, a subset of the URI that tells us how to access a network location, is the following part of our complete example:

https://garry.bhatt.al/posts/hello.html

The URN, defined as the URI subset that includes a name (within a given space) but does not provide a method to access it, is:

garry.bhatt.al/posts/hello.html#intro

That’s it! Now you should be able to tell the difference between URL and URN.

Should you ever forget the content of this article, you have to options. One is to get back here and read again about the differences. But URIs can be classified as URLs, URNs or both, right? So the other option is to always use the term URI. This way, you will never be wrong.

Should you find any mistakes, technical or grammatical, please don’t hesitate to let me know!