…Ops: DevOps, MLOps, DevSecOps (Part 2)
Lets talk about …Ops stuff in Amazon Web Service
Sometimes, we are wondering how to accelerate product development journey in Technology Industry. The answer for this era is Agile. What does exactly Agile mean ?
Based on agilealliance, Agile (ag·ile /ˈajəl/) is the ability to create and respond to change. It is a way of dealing with, and ultimately succeeding in, an uncertain and turbulent environment. Agile culture is very important to be adopt when we are working in the large-scale development and giving the best customer journey with our product.
It is very hard to adopt Agile Culture without any concrete methodology, since the agile word is very broad. To help us to understanding the agile culture, in this world community, especially for software/product development, is develop some methodologies which can we follow, there are DevOps, DevSecOps, and MLOps. Lets we discuss one by one and how Amazon Web Service help us to adopt those methodologies
MLDevOps
MLOps, Machine Learning Operations, is a set of practices which is adopted from DevOps concept for Machine Learning Development that aims to deploy and maintain machine learning models in production reliably and efficiently. Normally, Machine learning models are tested and developed in isolated experimental systems. When an algorithm is ready to be launched, MLOps is practiced between Data Scientists, DevOps, and Machine Learning engineers to transition the algorithm to production systems.
MLOps’s concept seeks to increase automation and improve the quality of production models, while also focusing on business and regulatory requirements.
MLOps applies to the entire cycle — from integrating with model generation, orchestration, and deployment, to health, diagnostics, governance, and business metrics.
MLDevOps Architecture Reference in AWS
Realising MLOps concept in AWS, we can use Code Series services with SageMaker. To make it more scalable and convenience , we can use some serverless services to setting up our SageMaker. With those integration, ML Engineer team does not need to touch AWS Console, fully agile.
First of all, to make our MLOps become agile process, We should prepare Infrastructure as a Code to provision Orchestrator. By having IaC, if we have new step or feature, we can change it immediately by adding some lines on code. Component of the Orchestrator are AWS Code Series; for the CI/CD the ML, S3; as the storage, and SageMaker; as the place to do al ML stuff.
After prepare the Orchestrator, we can start to use this MLOps to develop our model with automatic. By just upload the code and data set for developing the ML, we can review and improve the result in continuity.
Good new from AWS
Previously, we will implement the MLOps base on Architecture above. Now AWS already provided new services for us to adopt MLOps culture easier. By using SageMaker MLOps service, we can adopt the MLOps without worry to prepare all pipeline from various services.
By utilise SageMaker Services, we do not have to worry and confuse to select which tools for MLOps flow.
To adopt MLOps, we only need 3 services from SageMaker family; SageMaker Pipeline; to build a repeatable and consistent process for deploying model, SageMaker Projects; to provide end to end traceability CI/CD of ML, and SageMaker Model Registry; to manage all your models by creating model package groups which we tracks all of the models that we train to solve a particular problem.
For tutorial to utilise SageMaker Services, please look on this link: https://aws.amazon.com/getting-started/hands-on/machine-learning-tutorial-mlops-automate-ml-workflows/
The Last part will come out as soon as possible…
If you want to explore other cool blogs of technologies, please stay tune on eCloudture Medium !!!
