Since the inception of the agile manifesto in 2001, many software development methodologies emerged, each trying to improve processes. While the basic agile manifesto still guides workflows, today, significant efforts are extended mainly towards breaking silos across a wide range of fields. This is achieved by unifying siloed departments into collaborative teams.
At first, development and operations teams were unified into DevOps teams. Today, it is becoming increasingly clear that DevOps is not enough.
As more teams build and host applications and infrastructure in the cloud, the term CloudOps has gained traction. IT operations professionals have always been responsible for the monitoring, maintenance and installation of servers, networks and other IT infrastructure. But, third-party cloud vendors like AWS, GCP and Azure are changing the way Ops teams need to operate their services. First came the DevOps movement – but now, many teams are focusing on CloudOps as well.
Security is also a critical aspect that needs to be addressed throughout the process rather than in the end. Hence, the inception of DevSecOps, which adds security to the development cycle.
To ensure database operations and machine learning operations run smoothly, as well, MLOps was created.
The financial challenges derived from the exponential growth of Cloud Computing are being tackled by a newly emerging practice called FinOps (Financial Operations).
This article examines these five primary methodologies — DevOps, CloudOps, DevSecOps, MLOps, and FinOps — providing guiding principles for when and how to use each workflow.
What is DevOps?
The word DevOps is a combination of the terms development and operations, meant to represent a collaborative or shared approach to the tasks performed by a company’s application development and IT operations teams.
In its broadest meaning, DevOps is a philosophy that promotes better communication and collaboration between these teams — and others — in an organization. In its most narrow interpretation, DevOps describes the adoption of iterative software development, automation and programmable infrastructure deployment and maintenance.
The term also covers culture changes, such as building trust and cohesion between developers and systems administrators and aligning technological projects to business requirements. DevOps can change the software delivery chain, services, job roles, IT tools and best practices.
The benefits individuals and groups can obtain through a DevOps culture, and practice includes:
- Rapid development life cycles
- Deployment velocity
- Code quality through the use of testing
While DevOps is not a technology, DevOps environments generally have common methodologies. These include the following:
- continuous integration and continuous delivery or continuous deployment (CI/CD) tools, with an emphasis on task automation;
- products that support DevOps adoption, including real-time monitoring and incident management systems, configuration management and collaboration platforms; and
- cloud computing, microservices and containers implemented concurrently with DevOps methodologies.
A DevOps approach is one of many techniques used to execute IT projects that meet business needs. DevOps can coexist with Agile software development; IT service management frameworks, such as ITIL; project management directives, such as Lean and Six Sigma; and other strategies.
What is CloudOps?
CloudOps is the “formalization of best practices and procedures that allow cloud-based platforms, and applications and data that live there, to function well over a long duration of time.”
CloudOps is essentially a culmination of DevOps and traditional IT operations applied to a cloud-based architecture. Most organizations previous to cloud computing and storage would maintain a network operations center (NOC). The NOC was a physical location where IT professionals could manage and monitor network and server performance.
Now, many teams are building services where they never have to touch the servers hosting their service. But, this doesn’t eliminate the need for operations – it just means alerting and monitoring needs to be cloud-optimized. Software developers, IT operations and security teams still need to collaborate closely when delivering new services or responding to incidents in production.
Therefore, CloudOps isn’t exclusive to DevOps practices. So, let’s go over some of the core tenets of DevOps, define the methodology and see how DevOps and CloudOps can work together.
What is FinOps?
We’re breaking Silos again, and this time it’s between Finance and technical operations.
According to the FinOps Foundation, FinOps (Cloud Financial Management)is “The practice of bringing financial accountability to the variable spend model of the cloud, enabling distributed teams to make business trade-offs between speed, cost, and quality.”
Organizations increase the business value of running in the cloud by bringing together business, engineering, and financial professionals with a new set of disciplines.
The FinOps engineer is the organization’s trustee when it comes to cloud cost optimization. His objective is to collaborate with engineering teams to predict, plan, and purchase cloud infrastructure based on the product’s requirements.
FinOps teams are embedded within the technology group to understand the technical requirements of the product’s lifecycle (starting from the early design stage to large scale). They are trusted to optimize the organization’s cloud infrastructure continuously.
Some are often mistaken that FinOps is all about saving money, but it’s way more than that.
A well-architected cloud budget drives more revenue, enables product enhancements, performance, and feature release speed.
FinOps is the foundation of cross-functional collaboration between financial and technology teams, tying their business’s success hand in hand with continuously optimizing cloud infrastructure.
FinOps is still an emerging practice, but as we’ve seen the DevOps practice reach almost every Ecosystem niche, it is inevitable to have a FinOps practitioner in every organization as they scale and more cloud services are consumed.
What is DevSecOps?
DevSecOps is the combination of DevOps with security teams. It is designed to ensure that responsibility for security is shared across development and operations tasks and to implement the management of “security as code.” Implementing DevSecOps is typically taken on by teams that are already comfortable working with a DevOps strategy.
DevSecOps teams enable security members to bridge the gap between development and deployment operations and security concerns. By breaking down the silos between groups, they can help integrate security practices into existing workflows, reduce friction, and ensure more secure products from the start.
The ways that DevSecOps specifically differs from DevOps is both in tooling and mindset. These implementations shift testing to the left in development processes and focus on teaching security best practices. The idea is to prevent vulnerabilities from entering projects in the first place.
For example, DevSecOps teams often incorporate static application security testing (SAST) tools in integrated development environments (IDEs). This means security audits and testing begin before code is even submitted for traditional testing. Similarly, teams’ focus may include cloud security posture management (CSPM) or compliance auditing tools into environment deployment steps. Again, this helps catch misconfigurations before environments go live.
As opposed to security simply scanning and flagging vulnerabilities after code are delivered, they are part of the process. Thus, development, IT operations, and security are cross-functional and work toward a common goal.
If you think about it, this is a natural evolution of the DevOps philosophy that focuses on collaboration. Integrating all stakeholders into the development pipeline makes the final product better.
In practice, this means including security in the process from the beginning. It also often means adopting security as code and automating things like vulnerability scans.
What is MLOps?
As the field of machine learning has matured in recent years, the need for integrating automatic continuous integration (CI), continuous delivery (CD), and continuous training (CT) to machine learning systems have increased. The application of DevOps philosophy to a machine learning system has been termed MLOps. MLOps aims to fuse machine learning system development (ML) and machine learning system operation (Ops).
In many ways, MLOps also overlaps with DataOps since it also requires the handling, maintenance, and security of datasets and data centers. However, there are some aspects of machine learning workloads that require different focus or implementation. Some of these differences include:
- Team skills — in MLOps, teams need to incorporate ML researchers and data scientists who are often not experienced, software engineers. These members focus on experimentation, model development, and data analysis and may not have the skills needed to perform application development, operations, or security tasks.
- Development — unlike traditional development, which is more linear, ML is often highly experimental. As a result, teams need to be able to manipulate parameters and features and retrain models frequently. This requires more complex feedback loops. Additionally, teams need to be able to track operations for reproducibility without impeding workflow reusability.
- Testing — testing in MLOps requires additional methods on top of what is typically done in DevOps or DevSecOps. For example, MLOps requires tests for data validation, model validation, and testing of model quality.
- Deployment — depending on the type of ML model you are deploying, you may need to set up pipelines for ongoing data handling and training. This requires multi-step pipelines, which can handle the retraining steps and the verification and redeployment processes. Without MLOps this is done manually, but with it, efforts should be automated.
- Production — models in production can face challenges that aren’t faced by standard application deployments, such as issues related to evolving data profiles. This can cause models to decay and reliability to decrease. MLOps implementations need to incorporate continuous monitoring and auditing to confirm that models are both available and accurate. If accuracy decreases, models need to be called back and corrected.
Another vital area that MLOps deviates from DevOps is how continuous integration/continuous development (CI/CD) pipelines are constructed. In MLOps, CI components need to extend to testing and validating data schemas, data, and models. CD components need to support the deployment of the training pipeline and the final model prediction service or application. Additionally, there is another component, continuous testing (CT), that needs to be accounted for to enable automatic model retraining and refinement.
Similarities of MLOps and DevOps
- Continuous integration of codebase amongst developers, data scientists, and data engineers.
- Testing of code and components of the machine learning system code.
- Continuous delivery of the system into production.
Differences between MLOps and DevOps
- In MLOps, in addition to testing the code, you also need to ensure data quality is maintained across the machine learning project life cycle.
- In MLOps, you may not necessarily be deploying just a model artifact. Deployment of a machine learning system can require a machine learning pipeline that involves data extraction, data processing, feature engineering, model training, model registry, and model deployment.
- In MLOps, there is a third concept that does not exist in DevOps, which is Continuous Training (CT). This step is all about automatically identifying scenarios/events that require a model to be re-trained and re-deployed into production due to performance degradation in the currently deployed machine learning model/system.
How to combine DevOps and FinOps
When it comes to having a comprehensive and centralized solution for continuously planning and optimizing cloud infrastructure, nothing beats cross-team collaboration.
Once a new product is added to the roadmap, having DevOps and FinOps work together in synergy with proper planning and embedded processes towards a well-architected cloud smells like team spirit to me.
(Okay, okay, one rock quote too many, I get it)
Implementing financial disciplines into the technical processes can only benefit an organization, drive revenue sky high, and make everyone happy.
Some might argue that the intervention of financial stakeholders in the product lifecycle may cause bureaucratic setbacks and may even impact product performance, and that’s fine. Still, just like every new process, I believe that proper implementation accompanied by trust and leadership will align everyone towards cloud serenity.
DevOps and FinOps don’t need to compete with each other; they need to complete one another.
Looking forward, DevOps and FinOps must work together as a homogenous unit because operating as standalone silos will destine them to clash and eventually fail.
How to combine DevOps and DevSecOps
DevOps heavily relies on automation. The same is true for DevSecOps, which aims at automating every aspect, including security audits.
To stimulate the adoption of DevSecOps, the Cloud Security Alliance company has defined six fundamental aspects or pillars:
- Collective Responsibility: Security is not something ephemeral whose progress and contribution cannot be measured. Each person in the organization has their security responsibility and must be aware of their contribution to its security stance.
- Collaboration and Integration: Security can only be achieved through cooperation, not confrontation.
- Pragmatic Implementation: By using a framework-agnostic “Digital Security and Privacy Model” focused on application development to ensure safety, privacy, and trust in the digital society, organizations will approach security in DevOps in a pragmatic manner.
- Bridging the divide between Compliance and Development: The key to addressing the gap between compliance and development is identifying appropriate controls, translating them to appropriate software measures, and identifying inflection points within the software lifecycle where these controls can be automated and measured.
- Automation: Software quality can be bettered by improving the thoroughness, timeliness, and frequency of testing. Processes that can be automated should be automated, and those that can’t be considered for elimination.
- Measurement, Monitoring, Report, and Action: The qualified people must continuously control the results during software development and post-delivery at the right time for DevSecOps to succeed.
What does CloudOps offer DevOps teams?
Contrary to traditional IT infrastructure and NOC-based org structures, DevOps creates an agile process for the rapid delivery of reliable services. It doesn’t eliminate the need for IT experts and software developers – it just brings them closer. And, with containers, complex microservices and serverless functions becoming more commonplace, teams need to find ways to move faster without compromising stability and security.
This is where CloudOps and cloud computing can offer speed, security and operational efficiency for DevOps teams. Cloud platforms are:
- Distributed, stateless and scalable
Capacity planning and asset management becomes virtualized. You can quickly spin up new servers or remove unnecessary storage. You can set rules to auto-provision servers in order to keep up with demand and maintain uptime. Also, CloudOps helps you monitor key performance metrics and quickly act from anywhere – increasing the flexibility and scalability of the underlying applications and infrastructure.
- Fault and latency tolerant
Because of the flexibility and scalability of cloud services, cloud-based applications and infrastructure are less prone to latency and error. Because cloud-centric applications and services can abstract themselves from the underlying infrastructure, your system becomes more tolerant to error. Basically, cloud-based tools can make your services more performant.
- Data redundant
In CloudOps, your data is much more protected from failures. Cloud-based environments provide tons of opportunity for data storage in multiple physical and logical locations – offering more failover options and protection from data loss. Data is stored in different locations and can be stored in multiple different ways, helping you remove single points of failure – leading to fewer dependencies and a more resilient data pipeline.
- Automated
You can create more robust self-healing systems and implement more automation across every part of the SDLC. By using IaC (Infrastructure-as-code), you can program more automated functions and use your infrastructure to perform commands and tasks based on monitoring thresholds and other key performance metrics.
- Active-active
Active-active cloud networks allow teams to use multiple independent processing nodes where each node can access data from a number of replicated databases for a single application. This allows services to experience less downtime or outages because your application can likely pull the data it needs from a different source in case a server goes down.
Enhance Faddom to leverage your operations engineers
No matter the engineer and his challenges – Faddom can leverage his daily operations to gain operations efficiency.
Faddom enables the engineer to:
- Automatically discover and map complex hybrid environment
- Leave firewalls alone; no configuration needed
- Gain visibility quickly for cyber-security and monitoring
- Reflect a current blueprint of your IT topology
- Plan and migrate your applications to the cloud
- Get an impact analysis for each one of the migration waves
- Automatic sizing of current servers to Azure VM instance types, adjustable with your preferred purchase method
