December 18, 2018

Accountability, stability, and speed: How DevOps at Protenus drives innovation

Brian Reavey, VP of Technical Operations, Protenus

DevOps is a vital part of the production of the Protenus platform. It is pivotal to the functionality and enhanced user experience our customers have come to expect. As a SaaS company dealing with many different regulatory systems, the DevOps team has taken a code-like approach to our infrastructure that involves the engineering team throughout the entire development lifecycle of our analytics platform.

The DevOps team at Protenus has two primary missions. The first ensures that software development best practices, such as version control, peer-review, and testing, are incorporated into the operations workflow. The second creates shared ownership between the development and operations teams. This structure creates a critical foundation for successful expansion and increased functionality of our platform.

Just like the Protenus platform provides accountability, stability, and speed through advanced compliance analytics, our DevOps team is able to do the same for Protenus’ Technical Operations.

Shared consciousness = Focused teams

Our team includes subject matter experts on the development of each component of our platform, including its infrastructure. Every team member uses their area of expertise for code development, testing, and deployment, as well as facilitating production for their specific contributions. By having real ownership of the code in production, engineering is accountable for its success or failure and requires that every team member is on the same page for production deployments.

Immutable infrastructure-as-code

Following the same Agile methodologies as the software engineering teams and utilizing infrastructure-as-code, the production infrastructure is entirely described in code (Terraform) and made up of immutable artifacts that include machine images and Docker containers created with Ansible and Packer. Our deployment pipelines mimic those in our development environments, and every component, including the infrastructure, must pass integration testing and be peer-reviewed before being pushed to production.  

This combination of infrastructure-as-code and immutable artifacts help eliminate many of the issues that plague traditional operations teams. By peer-reviewing infrastructure changes and packaging all application dependencies as part of the deployment, we eliminate unexpected configuration drift, inconsistent deployments, and lengthy, overly complex deployment documents.

Automation is key

The pursuit of innovation and automation takes many forms at Protenus. Deployment pipelines for production rollouts are automated which speed up our deployments and reduce human error. We have also created auto-recovering systems that can repair themselves, allowing our call teams to remain well-rested instead of putting out fires all night long. Automated configuration scans ensure everything in production is working as expected, allowing us to focus on our long-term vision for the platform and enhancing the product for our customers. This automation and predictability is unprecedented and enables us to scale our business faster and utilize  fewer resources than a traditional operations team. As a bonus, it’s more engaging and rewarding!

Leading the industry is great, so where do we go from here?

While our security team is already following the same DevOps principles, we plan to incorporate these best practices throughout Technical Operations across the company.

We currently rely heavily on configuration files in version control for automated configuration changes, so we have plans use more advanced configuration management and service discovery tools (such as etcd or Consul) to further accelerate our code deployments and add much better insight into our platform.

While we’ve got a rock-solid foundation in place, we recognize there is always room for improvement. We are excited about so many of the new opportunities for development on the horizon, and we can’t do it alone. We will continue to add new engineers to our team who are excited to help us continue providing increased scalability, stability, and speed to operations at Protenus. If this sounds like the kind of challenge you’re looking for, get in touch with us. We’d love to hear from you.

Learn more