Dmitry Olefirenko works as Techlead DevOps at Provectus. In this interview he shared his career path and spoke in great detail about who DevOps-ers are and what skills such a specialist needs.
A LITTLE HISTORY
The DevOps movement was born in 2007-2008 when the IT operations and software development communities expressed concern about what they saw as a fatal level of dysfunction in the industry.
They argued against the traditional software development model, whereby those who write the code should be organizationally and functionally separate from those who deploy and maintain it.
Developers and IT had separate (and often competing) goals, separate departmental leadership, separate KPIs against which they were measured, and often worked on different floors or even in different buildings.
The result was isolated teams, long hours, failed releases and dissatisfied customers. Of course, things have changed a bit in 16 years, and the development team values the DevOps team's input.
As a product manager, I understand DevOps only in a general sense. So it was very interesting for me to talk about this topic with my former colleague, a high-level specialist, Dmitry Olefirenko.
ON THE EMERGENCE OF THE DEVOPS PROFESSION
Dmitry, please tell us how you started your path to DevOps profession?
First, I'll start with where this profession came from. The development of professions in IT is very much like the development of medicine. When medicine began to develop, to accumulate knowledge, it became clear that one person who can do everything is not suitable. Too much practical experience is needed.
And medicine began to be divided into narrow specializations. The same thing happened in IT. There were technicians who could set up 1C and Windows at the same time and tell an accountant why they couldn't do something in Excel.
How did the division come about? By what?
The DevOps profession is one of the youngest. In a nutshell, it can be called the interaction between the development team and operations. For example, the development team has written a product, but there are 100500 open questions:
- What to do next with this product?
- How do you get this product into production?
- How to keep all the security aspects in place so that there are no holes, so that passwords don't get hacked, so that passwords don't leak out?
- What follows is a huge list of observability issues.
In some companies, for example Google, this is even included in a separate position called Site Reliability Engineering (SRE). These are people who deal specifically with the observability of the product: monitoring, logging, alerting, alerting paths, what happens if an alert drops, and so on.
Writing your own tules for monitoring, for alerting or researching and implementing existing ones. Here, for my part, I can recommend looking at products from Grafana.
So how did you come into this profession?
I came into this profession in a rather interesting way. This is a profession that mostly comes from sysadmins. DevOps is supposed to have a good knowledge of operating systems, in particular various Linux's. Because for the most part, all products run on these operating systems. But I came at it a little differently. I was first a PhP programmer, doing website development. Then I got to work in a near-DevOps profession called Build Release Engineer. And then I smoothly moved into DevOps.
What did you do when you were in the role of Build Release Engineer?
I used to organize builds. When a new version of a product had to be released, I made sure that everything went according to the process. That's partly what I'm doing now.
Can you elaborate on what your duties are now?
There is a huge amount of work for DevOps in a large company. Programming hasn't gone anywhere. I write quite a lot of small crafts, as I call them, that run, for example, inside the Kubernetes cluster: some auxiliary tulles, reporting, and so on.
What language do you use?
Even though it's trendy to use GoLang for this, I, like most DevOps, prefer Python. It's great for all sorts of small developments.
WHAT KIND OF COMPANIES NEED DEVOPS
What types of products do you need DevOPS for? For example, I want to make some small product to sell tickets. Do I need to involve DevOps or not?
It depends on the qualifications of those who will deploy and support this product. We can take a public cloud if it is a simple product. For example, Amazon. We can take a very user-friendly AWS Elastic Beanstalk and an experienced developer familiar with Amazon will deploy it there perfectly. And then there may or may not be questions. For example, what kind of RTO or RPO do we want?1 Will we have disaster recovery?2How many hours are we willing to, how many hours are we willing to wait if this product goes down? In how much time are we willing to lose data? And how easy or how difficult is it to monitor our product?
It turns out that for small products, this role is usually fulfilled by the developers themselves?
You have to look at what a small product is. And it depends on the product itself. What services should it interact with, what kind of RTO and RPO does this product have? For example, if we have an online shop - would it be terrible if it, let's say, lay down for 2 hours? Probably not. If people went to this online shop, made purchases, the base is gone, and the backup was 8 hours ago, and we lost unprocessed orders for 8 hours. Is that critical to the business? For some people yes, for some people no. What if it is, well, let's say, a check-in system for a major airline. What happens if that system goes down for 1 hour? Collapse at airports all over the world where that company operates.
In what case is it necessary to turn to DevOps? Does it depend on the number of services the company uses?
From the number of services, from their reliability, from the danger to the business.
- What happens if this system stops working?
- What happens if attackers get into that system and start trying to break it, start trying to DDoS it3?
In some cases, it makes sense for a company to have a NOC team4This is a system that will sit around the clock, monitor, watch the boards, log incidents, and wake someone up at night if something happens. Hence the question that I have already mentioned, which is observability - how much this system should be monitored, logged, and so on. That is, what requirements does the business have?
And probably the last aspect is what platform it will all be run on. If it is a public cloud, then there are ready-made tools where some service can be simply launched. And if a company has many products, if they are microservice products, then, for example, today it has become fashionable to run it all in Kubernetes.
SERVICES
What is Kubernetes?
Kubernetes is such a Google brainchild that has tremendous capabilities, but it has a very high threshold of entry.
What is the reason for the high threshold of entry?
Google did not design this system to be simple. Google designed this system to be complex, but with a huge number of features.
So to start using it, you need to already have a certain amount of knowledge?
You need to have a baggage of knowledge how to work with this orchestrator. But it provides a tremendous amount of capabilities to start services, to automatically restore services if something went wrong, and to deploy products very quickly to Kubernetes.
But at the same time, Kubernetes is a bare system. When the questions arise, how will we deploy products there, how will we monitor them, how will we collect logs, how will we keep secrets? It turns out that Kubernetes needs about a dozen and a half different Helm charts to make this Kubernetes cluster ready to work with production releases. By the way, this is the topic of my report, which I am ready to share with those who are interested.
Okay, what other services do you use in your work?
For the most part, Kubernetes and all the services that are put into it for monitoring, for logging, for observability, for proper secret handling. Also, a lot of other products that are well known in the world, various Jenkins, GitHub and so on.
WORK PROCESS
What does your working day look like?
I work remotely now. My working hours are irregular. I may have some urgent tasks when something goes wrong. Or someone has an urgent issue, and for the most part these are architectural tasks that can take two or three weeks. So I can start work at different times of the day, even depending on the time of year when I have to take my daughter to school. I can start work at 09:00, finish at 18:00. Some days later, some days earlier.
How do you interact with the product team? What is your place in the product development process?
We try to advise developers on what they need to do so that, for example, their product can be deployed correctly, safely, and without fail in a cluster. Because there are a number of anti-patterns, which products are good to deploy in Kubernetes, and which ones are better not to run there at all. If it's a monolith, it's probably better to deploy it in Amazon using Ansible.
Who sets the tasks for you? How do you interact within the team?
We have a small DevOps team. I am responsible for automation issues, for the deployment of the Kubernetes platform. Communication often happens directly with the customer, with the customer's technical architects. That is, they want something, and then there can be a long process of negotiations on how best to do it. Because not always when the customer wants something, it makes sense to implement it this way. First, it makes sense to convince the customer that a different method should be chosen.
In what language do you usually communicate with the customer?
In English. Of course, you need to know it at a good conversational level. And it is also desirable to understand people from a wide range of countries.
How often is it necessary to work nights in the evening?
I try to spend a lot of time designing this architecture so that I don't have to work at night if possible.
HOW TO BECOME DEVOPS
Interested in learning how to become DevOps? Imagine that I want to become a DevOps. What do I need to do in order to do that? What is the minimum set of qualifications I should have?
Knowledge of Linux operating systems. Some basic knowledge of products like Ansible. It would be good to start learning Docker, some Kubernetes. Also, an understanding of how one of the three most popular public clouds - AWS, Azure or Google Cloud - works would be sufficient to start.
What advice do you have for aspiring DevOps-ers or those who want to become one?
I can see that Kubernetes platform will be really in trend for the next few years, you need to learn it. For those who want to do so, I can advise you to take the Certified Kubernetes Administrator certification, as I did in my time. It helped me a lot. Good training courses can be found, for example, on Udemy.
Course Reference: https://www.udemy.com/course/certified-kubernetes-administrator-with-practice-tests
As far as I remember, certification can be done in Chisinau?
Yes, in Chisinau I took the Amazon certification face-to-face, because there are two companies in Chisinau that are Pearson VUE representatives, and this gives you the opportunity to take almost any exam face-to-face. Pearson VUE is the main intermediary for certification, in particular Amazon.
What questions do DevOps typically ask in a job interview?
Usually, most companies have pre-prepared questions regarding the stack the company uses. It is best to ask situational questions. For example: imagine you have a Kubernetes cluster and one node stops responding. Ask the candidate about their actions. I would recommend having at least one person who has already hit a lot of bumps.
Are there any interview tests for DevOps people?
They happen. I really don't like to do them. To be honest, I might have refused to be interviewed if I had been offered a test interview.
What does it usually consist of?
As an introvert, I can get lost on a test assignment, and I can tell you things like that. I don't like such assignments, when I am asked to just flick around the screen and write something quickly. Yeah, I get lost. Don't get me wrong, I'm not an operating surgeon, I don't need to make a decision instantly. I really don't like this kind of assignment. I'm a big fan of interview questions. For example, we have this and this, how would you implement it? What would you suggest?
DEVOPS-OM
Companies, mostly small in size, want to save money. What happens if you take a junior specialist?
As I've said before, DevOps people often have to deal with the problems of something going down. A junior specialist is unlikely to be able to figure out the causes of the problems, and generally configure everything necessary.
Does it make sense to take a DevOps consultancy in such a case?
If someone has practical experience with services, or on the contrary negative experience with some services, he has already spent many months and earned a lot of grey hairs, figuring out which of these services is good and which is not, then why should not the company just buy this experience?
What advice do you have for employers?
1.It's worth noting that if DevOps tends to work overtime or is looking to get something done quicker, it may not be of the best quality.
2. If you've asked DevOps to design some architecture that will determine the survivability of production, don't rush it. And don't make him simultaneously design something in a two-week sprint, while simultaneously doing caliper and answering five teams' questions about why something isn't working for them? That means you don't have enough DevOps people.
3. Invest in DevOps skills development, motivate them to take certifications, pay them for courses on Udemy. For example, the company I work for motivates them to take certifications at Amazon, pays for the certification and then pays a small bonus.
How do you realize that DevOps brings value to the company?
Remember a very ancient proverb: "If your sysadmin does nothing, it's a good sysadmin". The same goes for DevOps. Also, about the story about Henry Ford and his way of paying locksmiths. When Henry Ford's assembly line was running in the locksmiths' room, a counter was spinning with their wages. When the assembly line stopped, the meter stopped. The wages of the fitters depended on whether the conveyor was running or not. It may have been a fable, but DevOps has a similar story. Well-organized CI/CD processes, a well-built backup system, a well-built monitoring and alerting system saves the company time and money in the long run.
HAPPINESS DEVOPS SPECIALIST
What do you like about your profession? Why do you do it?
I like developing architecture, designing something new. Most of all, I like to get positive results from it all, that what I have designed works and does not fall down.
That is, the happiness of DevOps is.
The happiness of DevOps is that it can sleep well at night. Because the problem with DevOps is that if something happens on the production floor, it's the first thing that comes to mind, because that's where it all starts.
Footnotes:
- RTO (Recovery Time Objective) or RPO (Recovery Point Objective) are key factors in defining backup and disaster recovery scenarios for databases. ↩︎
- Disaster recovery (DR) is the ability of an organisation to restore access and availability of the IT infrastructure after a disaster event, whether natural or caused by human error. ↩︎
- A DDoS attack is a way to block a website from functioning by submitting a large number of requests that exceed the network's bandwidth capacity ↩︎
- The NOC (Network Operating Centre) team is the team that has the ability to set alerts and monitor the health of the infrastructure 24/7. ↩︎
Spelling error report
The following text will be sent to our editors: