Ahead of software end of life, Digital Endeavours built the AWS infrastructure for a claims platform. Several companies worked on the wider programme, and Digital Endeavours’ scope ran from application servers to SQL Server databases in every environment, all built from code with automated, tested deployments.
Over more than a year of work, Digital Endeavours wrote the infrastructure as code from scratch in Terraform, extending the client’s GitLab CI pipelines to build, validate, test and deploy it. Custom Packer images are built on the client’s base AMI, each on a build instance sized for its image type to keep builds fast, and rebuilt weekly to stay current. Terraform deploys each server from the matching application or SQL Server image, and automated configuration management completes each server’s build.
Every server can be rebuilt from code at any time. Application servers take 15 to 30 minutes and database servers under an hour, and because both deploy from the same pipeline they can run at the same time. At the busiest point, infrastructure releases were going out several times a day. Digital Endeavours made implementation decisions independently, with architectural consultation on larger technical choices. The platform runs across multiple environments in a multi-account AWS estate, with least-privilege access throughout.
On the application tier, servers are replaced rather than patched, so every deployment starts from the same known image. User data applies each environment’s configuration at boot, and Terraform deploys the SSO metadata certificates, matching the right certificate to each server by hostname and environment. The servers run across multiple Availability Zones and update through rolling deployments. Instance sizes came from load testing rather than estimates, and scheduled scaling matches capacity to business operating hours, cutting cost outside them.
Development and system test environments use managed RDS databases for fast iteration. UAT, end to end and production run SQL Server clusters on Always On availability groups, with a file share witness built to provide quorum. The cluster nodes follow the same image-based approach as the application tier. SQL Server build files are staged in S3, and SSM runs on each instance to pull them down and automate the build. New images roll out node by node, and each replacement joins the cluster automatically.
The same SSM automation applies SQL Server Agent jobs that differ by environment, as a separate step so that deploying jobs doesn’t conflict with building the cluster. UAT, for example, is restored from production data, so its jobs include data masking. The application upgrades its own database schema from the old version to the new, so Digital Endeavours automated a check that each upgrade had completed cleanly. Data restores showed that the application servers needed to stay down while data level work was in progress.
Some credentials already lived in an existing secrets store, and the upgrade needed new ones. Digital Endeavours agreed with the client’s architects to hold both in AWS Secrets Manager, replicating the existing secrets and creating the new ones there. Application lookups stay inside AWS, and access to every secret is controlled through IAM.
Diagnosing application failures during the upgrade meant working through large log files, often with several errors at once and stack traces spanning multiple components. Digital Endeavours used AI-assisted analysis to pinpoint each error, cutting root cause identification from hours of manual searching to minutes. The automation’s scripts each write their own log, so a failed build can be traced to the exact step that broke.
Working alongside the other companies meant judging, task by task, whether a problem fell within Digital Endeavours’ scope, and passing it to the team responsible when it didn’t. Where another workstream needed to reach AWS, such as an API layer hosted in Azure, Digital Endeavours configured the security group rules that let it reach the application.
The hardest part was cultural rather than technical. Ephemeral servers only work when everyone treats them as replaceable, and that understanding grew more slowly than the automation. Digital Endeavours redirected requests for one-off manual changes into code, so every change was tested and would survive the next deployment, and kept building that understanding across teams throughout the engagement.
Handover to the client’s BAU teams included several hours of knowledge transfer sessions on the infrastructure. Rather than a single large document that few people would read, Digital Endeavours documented the platform in Markdown files kept in the repository next to the code each one describes, along with operational procedures and deployment workflows. Everything needed to maintain the platform, and to reuse the deployment approach on other projects, sits alongside the code.
Results
Digital Endeavours’ part of the platform is built entirely from code, and every release goes through automated validation and testing before it deploys. Production backups are configured in code, monitoring is in place, and logs from every environment are exported to CloudWatch. The database cluster has been updated node by node through the automation.
Key Technologies
CI/CD & IaC: GitLab CI, Terraform, Packer, Ansible, AWS Systems Manager
Infrastructure: AWS (multi-account, EC2, RDS, S3, Secrets Manager, IAM, CloudWatch)
Data: SQL Server Always On availability groups
