Some tips on how to effectively secure your production environment
In this post I just want to talk about approaches on setting ‘Approvals and Checks’ in Azure DevOps. Recently I came to the conclusion that we have been using this in a suboptimal way. But first let me give you some basic context what we’re working with.
Mainly I’m writing CI/CD Pipelines in Yaml in Azure DevOps Pipelines. The purpose often is building and deploying Data Platform Solutions and the resources build on top of those platforms. In example a pipeline for building and deploying an Azure Synapse Workspace and connected resources to DTAP environments. And another pipeline for deploying the artifacts that where build in the Synapse Workspace, like scripts and notebooks.
Pipeline resource protection
The main takeaway from the Microsoft documentation about Pipeline resource protection is that all ‘protected resources’ can be managed on the following elements:
- user permissions; In general, ensure you follow the Principle of least privilege. So only give permissions to users who require them
- pipeline permissions; Ensure to uncheck the option to grant access to “all pipelines” and specify those that are allowed to use the resources
- manual checks an approvals; These allow you to pause the pipeline run until certain conditions are met before the jobs are executed
My mistake; Environments
Taking above options in mind to protect our resources. We configure our environments within Azure Pipelines. We set the user permissions following the principle of least privilege, we make sure to specify our pipelines that are allowed to use this environment and we add a check and approval.
‘Well done! All set.’ you might think.
But let me show you where there is a possibility to skip all previously added protection rules. It is quite simple to bypass, but also easy to fix.
We added checks to environments but these have no direct relation with the Azure Resources. Only the Kubernetes resource and virtual machine resource are currently supported and I’m not using any of these.
See below example of a deploy stage to a production environment. Please note that you need a ‘deployment job‘ to be able to add the Azure DevOps Environment to the job.
See line 8 for the reference to our production environment created in Azure Pipelines that contains all our permissions and checks.
Remove this line of code, make a typo or something similar and there will be a direct deployment to your production resources. No permissions to prevent this action, no manual approval before deployment!
My advice; Service Connections
In my opinion a better approach would be to apply the protection to the Service Connection used to reach your environments, following these recommendations.
Create one Service Connection per environment to minimize the scope. Set permissions and checks on this direct link to your production environment.
In practice this will work exactly the same as the setup where we set the protection on the environment. Only this setup will prevent the occurrence of above example of bypassing the protection by making mistakes in the YAML pipeline.
In the end it is good practice to plan ahead for security and compliance. Review and approve the use of cloud-native tools to ensure setups are compliant and following Architectural Principles.
Re-read the documentation when implementing solutions even when you think you know it all. Microsoft has the tendency to update things 🙂