A Journey Exploring DevOps Universe Roadmap

Short Introduction to our topic

Greeting folks! I’m El Muhammad. This would be my first article. And in this blog I would be sharing a lot of Cloud and DevOps related topic such as Docker, Kubernetes, Infrastructure as code, Cloud Providers, etc. However, before we start going down further into those topics, let’s start by planning a roadmap about what we we’ll be learning from now on. And this article would become some sort of Table of Content for our further learning process, I will update this article for every new topic that has been publish.

DevOps Universe Roadmap Exploration

The road to become DevOps Engineer can be overwhelming for most people, especially when we are in the middle of so many tools and practices to learn and adopt. In this this article I have marked the most important tools or practices for you to start learning.

I will include a simple infrographic for us set our priorities straight. The star mark indicated the importance of the tools or practices. And the tools or practices mark in gold are my personal recommendation for you to focus on.

So, let’s start our journey to The DevOps Universe...

DevOps Fundamental

Before we dive further into the actual tools and technologies essential to DevOps, we must fist understand what actually DevOps is. Learn and understand CAMS model on devops principle. Then, we have to understand what are the methodologies that DevOps team usually adept to. Next, we will try to study the DevOps lifescycles and the DevOps team responsibilities.

Linux Fundamental (Learn to live in terminal)

After understanding DevOps and it’s fundamental. Let’s start our journey in the DevOps universe by learning Linux CLI (Command Line Interface) tools. This would be our main weapon for the journey ahead. You should be able to do a lot of things from CLI, such as :

• Process monitoring

  Process monitoring tools is used to monitor and troubleshoot all of the process inside our operating system. Using this tools we can understand what are the process state now, how much resource the process consume, which file are the process attached to, etc. Process monitoring tools include htop, ps, lsof, top, atop, and a lot more.

• Network Tools

  Network tools are used to monitor and troubleshoot the network traffic from and to our operating system. traceroute, mtr, nmap, tcptraceroute, dig, netstat, iptraf, tcpdump iptables, and airmon are some example of network tools to use inside linux.

• System Performance

  System performance tools are used to monitor and understand the system and resource usage of our operating system, this include the virtual memory stat, input / outpot, number of proccess, cpu usage, kernel version, etc. We can use commands like nmon, iostat, vmstat, sar, iotop to check the system performance.

• Text Manipulation

  As the name suggest, the text manipulation tools are used to edit or manipulated the text or stdon and stdout in linux. The example of this tools are awk, sed, grep, sort, cut, uniq, cat, echo, fmt, tr, nl, wc, egrep, fgrep, etc.

Let’s continue our journey, from here on the path would get even harder…

Servers Setup & Configuration

After learning to live in Linux CLI, let’s continue our journey to learn how to setup and configure server and service inside our Operating System. The service and server I mention here are not the only service and server that you could install in your Operating System. However, this are the most common service and server that are used by DevOps.

• Proxy Server

  A proxy server can be simplify as a router or gateway between to party. It can be user to server, or server to server. In generals, proxy server can be classified into forward proxy and reverse proxy. We will learn about it next time.

• Web Server

  Web server are a service or server that deliver a content of a website to it requester for example a web browser. Web server will deliver all of the requested content like images, text, video, etc from server to the web browser. By data, there are two main web server that are mainly used by server which is Nginx and Apache. And some may used IIS in their windows server.

• Database Server

  As the name suggested, Database Server are a service and server that store and deliver a data to the requester. Database Server mainly split into two categories, Relational Database and Non Relational database. And of those two Database Server types the most common language used are SQL and NoSQL.

• Others

  In this category, we will learn another common service and server that are often used by DevOps team for example load balancer, cache, and firewall.

Programming Language

Programming language offer tens of languages and frameworks that you can learn. But for this journey, we will be focusing on languages mostly used by DevOps team in automation or testing process such as Python, Bash/Shell Scripts, Go Lang, Javascript, Rust, etc.

Essential Collaboration Tools

Collaboration tools means and application, service, or tools that we can used to do a collaborative work with our team member including collaborative development, collaborative timeline management, etc.

• Version Control

  This topic will cover a version control tools to control and monitor every changes we as a team would make to our application. Some of them also offer a rollback version if we make some mistake during development process. The most commonly used version control tools are Git, however there are multiple other alternative we can use for our team such as SVN, CVS, Mercurial, etc.

• Code Repository

  Tools in this category usually connect to the version control tools because version control tools is often connect to the code repository where we store our code. You must have heard about Gitlab and Github as they are the giants in this tools category. Regardless, there are multiple alternative for code repository we can use outside those two choice like BitBucket, AWS CodeCommit, TaraVault, and even your local code repository using open-source app.

• Project Management

  We will use this tools to manage our timeline, project status, assign person in charge, etc. The most popular tool in this category are Jira and a lot of it's contender, let say Trello, Asana, ClickUp, etc.

We are still have a long way to go, keep up your spirit

Cloud Provider

In this part, we have to meet the provider of cloud computing solution such as AWS, GCP, Azure, Heroku, Digital Ocean, Alibaba Cloud, Oracle, IBM, etc. We have to understand what solutions and services are they offers. Some of those provider gave us a free trial access for us to try their product. They also offer a complete documentation of their product for us to start learning. Some even gave a free training and hans-on labs for us to practice. We will talk about it later.

Cloud Design Pattern

We will learn design patterns useful for building reliable, scalable, secure applications in the cloud for our application such as Availability, Data Management, Design and Implementation, Management and Monitoring, etc.

Infrastructure as Code (IaC)

This topic is one of the hardest part of learning to become DevOps, because we need to learn to deploy and maintain our infrastructure using code, like a template. For example we need to automated the database and web server installation using infrastructure as code, so that we don't need to install it manually every time we build a new server. This topic includes multiple types of IaC.

• Infrastructure Provisioning

  Provisioning means a process of setting up IT Infrastructure, also can be refer as a process to deploy a new infrastructure to our existing or new environment. Tools in this category are usually used in cloud service for example Terraform, AWS CloudFormation, Pulumi, AWS CDK, Packer, Yocto, etc.

• Configuration Management

  Configuration Management is the process of maintaining systems our a desired state. Configuration Management is also a method of ensuring that systems perform in a manner consistent with expectations over time. In this tools category we have Ansible, Chef, Puppet, Salt, etc.

• Containers

  You can call it the new version of virtualization for short. In which you can run your multiple applications without the need to worry of dependencies crashing. We will dive further into this topic in another article. The example of containers as code are Docker, Podman, rkt, LXC, and etc. You might already familiar with one of them, Docker. It was mention everywhere and pretty popular right now.

• Container Orchestration

  As the name suggested, this tools are use to orchestrate or manage a container such as Docker or Podman. The most popular Container Orchestration right now are Kubernetes, it is an open-source software that are used as base of giant website like Netflix. Other than Kubernetes or k8s for short, we also have Docker Swarm, Mesos, Nomad, etc for alternative.

• Service Mesh

  A service mesh is mainly a tool for adding observability, security, and reliability features to “cloud native” applications by transparently inserting this functionality or tools at the platform layer rather than the application layer. The service mesh is rapidly becoming a standard part of the cloud native stack, especially for Kubernetes adopters. For this category we will learn about Consul, Istio, Linkerd, Envoy, Traefik, etc.

Are you still there? If you are, then congratulation. We have dive super deep into the abyss called DevOps, a universe full of technologies. Let's continued a bit more!

Continues Integration and Continues Deployment (CI/CD)

CI/CD falls under DevOps (the joining of development and operations) and combines the practices of continuous integration and continuous delivery. CI/CD automates much or all of the manual human intervention traditionally needed to get new code from a commit into production such as build, test, and deploy, as well as infrastructure provisioning. With a CI/CD pipeline, developers can make changes to code that are then automatically tested and pushed out for delivery and deployment.

This are what Gitlab says about CI/CD.

• Continues Integration

  This are a practice of integrating the update process of your code as often as possible and simultaneously together with your team member. Using continuous integration, errors and security issues can be identified and fixed more easily and as early as possible. Tools that are fall under this criteria are GitlabCI, TravisCI, CircleCI, Bamboo, etc.

• Continues Delivery/Deployment

  Continuous deployment is a practice to automate the infrastructure provisioning and application release process. Once code has been tested and built by the CI process, continuous deployment will takes over during the final stages to ensure it can be deployed’s packaged with everything it needs to deploy to any environment at any time. Continuous delivery can cover everything from provisioning the infrastructure to deploying the application to the testing or production environment. Tool I used in this topic would be Flux, Jenkins, AgroCD, etc.

Logging and Monitoring

After we setup our server, deploying the apps, now its time to monitor our server and application. For this topic we can classify the logging and monitoring tools into multiple category.

• Infrastructure Monitoring

  We will used this tools to monitor our infrastructure state which includes resource usage, temperature, traffic usage, etc. We can use Prometheus, Datadog, Nagios, Zabbix, Monit, LibreNMS, etc to do all of that.

• Logs Analityc

  This tools are used to analyze logs generates from our infrastructure and application monitoring tools. This can be use to decided a lot of things such as should we upgrade our resource, should we scale our deployment, and a lot more. Tools we can use in this category are Elastic Stalk, Fluentd, Greylog, Splunk, Papertail, etc.

• Application Monitoring

  The same as infrastructure monitoring, this tools are also used for monitoring our deployment which fall under application category. Using this tools we can also monitor the application logs such as access_logs, error_logs, etc. The application monitoring tools for our learning path are Jaeger, New Relic, Istana, AppDynamics, Opentracking, etc.

• Dashboarding

  Some of our monitoring and logging tools do not offer a dashboard or GUI for us. They just generate metrics for us to reads, but we can use a dashboard tools to evaluate the metrics and gave us a visual and more readable logs.

Security

In this topic we will learn to secure our application and infrastructure by applying Zero Trust, SAST, TAST+DAST, and RASP. You will learn further in another article.

Congratulations Folks, we are already finish our current roadmap's journey.

Our Next Journey

Just like the never ending expansion of our dear universe, the advancement of technologies are showing no signs of stopping. And DevOps are The Space Traveler in those universe full of new technologies to explore.

So keep moving forward. #KeepLearningKeepAdvancing