Blog posts
Events
Cloudgov University
From Software PoC to Production to Operations: The Case for Cost Governance in Cloud Software Engineering
The root cause? Engineering teams are either not asked to pay attention to cloud costs or, worse, aren’t even aware of them. Without real-time alerts and visibility, the cloud promise of pay-as-you-go flexibility turns into ‘financial bleeding’ as you go, causing panic and heartburn at the C-level.
To address this critical issue, we need to evolve the software engineering operating model to make cost considerations a first-class priority from software design to production operations. At Cloudgov.ai, leveraging over 75+ years of combined software engineering experience across our C-level and technology leadership team, we are proposing the following operating model. We practice this at Cloudgov.ai for our own software engineering and product development mechanisms so it’s already been battle tested and we know it works for us and it’s also working for our customers.Â
People most of the time do what’s measured, inspected and not what’s expected. If your application goes down, causing customer outages, latency, or errors, engineers have been trained from early in their careers to view that as a failure, application failure as their personal software engineering skill failure. This is embarrassing for technology teams in-general including tech leadership and creates professional shame and hence software engineering has been optimized from avoiding those issues and public shame that comes with it. Popular software engineering giants like Amazon are known for conducting weekly operational excellence reviews to check system KPIs, using both incentives and disciplinary measures to ensure systems remain stable and reliable. However, from a cost efficiency perspective, little attention is given, which has trained the software engineering community to ‘not worry’ too much about costs because leadership does not care. This implicit messaging has led to a situation where cost spikes are not treated with the same urgency as spikes in latency and error rates. Engineers get paged when these metrics go out of bounds, so why shouldn’t they get paged or alerted on Slack when cost spikes are detected?
Engineers and architects typically pay significant attention to latency, system availability, and uptime but often overlook the importance of cost-optimized architectures early in the design phase. The mindset of “we’ll do a PoC, see if it sticks, and then figure out how to do things right” needs to change. Software engineering teams must evolve their operating model to include ‘cost efficiency’ as one of the critical ‘responsibilities’ alongside reliability, scalability, and performance. Here’s why and how this can be achieved.
The Importance of Early Cost Consideration
Integrating cost governance from the beginning of the software design process ensures that systems are not only functional and reliable but also cost-effective. This proactive approach can prevent costly retrofits and re-architecting later on. According to Dr. Werner Vogels, VP & CTO of Amazon, making cost a non-functional requirement from the start helps balance features, time-to-market, and efficiency.
Key Principles of Frugal Architecture
Dr. Werner Vogels’ seven laws of frugal architecture emphasize the importance of cost-awareness in cloud architecture (The Frugal Architect). These principles highlight the need for continuous monitoring, regular re-evaluation of trade-offs, and incremental cost optimization. Here are some key laws:
- Make Cost a Non-functional Requirement: Considering cost implications early and continuously can help balance features, time-to-market, and efficiency.
- Architecting is a Series of Trade-offs: Regularly re-evaluate technical and business trade-offs, investing in resources aligned with business needs.
- Unobserved Systems Lead to Unknown Costs: Implement robust monitoring to pinpoint wasteful practices and streamline workflows.
- Cost Optimization is Incremental: Continuously monitor systems to understand patterns and trim inefficiencies.
Amazon’s Approach to Operational Excellence
Amazon implements operational excellence by holding weekly Ops review meetings and using the spinning wheel as a random selection method for which team will present. The randomness of the selection ensures that each team comes prepared, as any team can be called upon to present. When presenting, teams must be capable of deep diving into the data, explaining root causes behind notable data changes, and articulating the steps taken or planned to rectify any anomalies. This pushes teams to maintain high-quality operational dashboards that reflect the real-time health and performance of their services (AWS Operational Review Meetings). The spinning wheel concept can be implemented using tools like AWS Ops Wheel.
Cloudgov.ai Customer Case Studies
Case Study 1: Optimizing Existing Infrastructure
One cloudgov.ai customer, a mid-sized e-commerce company, leveraged the platform’s insights and savings opportunities to re-architect their application. By following Cloudgov.ai’s engineering recommendations, they switched from a monolithic architecture to a microservices-based architecture using serverless functions and auto-scaling groups. This change reduced their monthly cloud bill from $150,000 to $90,000, a 40% savings, while also improving their system’s performance and scalability during peak traffic periods.
Case Study 2: Re-evaluating Design with Cloudgov.ai’s FinOps Score & Insights
An early-stage SaaS startup, after reviewing their ‘FinOps’ score on cloudgov.ai for their infrastructure posture and spend, discovered their projected cloud costs were unsustainable. Their initial design relied heavily on expensive on-demand instances and underutilized resources. Armed with Cloudgov.ai’s insights, they paused their development, redesigned their application to use reserved instances and right-sized their resources. This proactive redesign reduced their projected annual cloud expenditure from $1.2 million to $900,000, a 25% savings, enabling them to allocate more budget to product development and marketing.
Monthly FinOps Check-Ins
A unique offering from Cloudgov.ai is the monthly FinOps check-in conducted by their FinOps and cloud experts. These sessions involve detailed reviews of the customer’s current cloud spending, identification of cost-saving opportunities, and strategic advice on optimizing cloud usage. The collaboration during these check-ins allows customers to benefit from the advanced knowledge of Cloudgov.ai’s team, receiving elite FinOps, software engineering, and architecture guidance. This continuous engagement ensures that customers are always on top of their cloud spending and are implementing best practices for cost management.
The Cost of Reactive Approaches
Making infrastructure cost optimization changes after a system is in production can be expensive and time-consuming. For instance, a company that initially neglects cost considerations might later need to re-architect their application to use more cost-effective services like serverless functions or spot instances. This re-architecting can involve significant downtime, increased operational costs, and a delayed return on investment.
Conclusion: The Need for Cloud Governance Platforms
Given the importance of cost governance, it’s essential for companies to adopt platforms like Cloudgov.ai. These platforms provide critical tools for:
- Cost Observability: Gain insights into application-level costs and optimize resource usage.
- Planning and Budgeting: Use detailed cost data to create accurate estimates for future workloads and applications.
- Policy Implementation: Enforce governance policies to ensure compliance and security from the start.
By integrating these practices early in the design phase, companies can build cost-efficient, scalable, and sustainable cloud infrastructures. For more detailed insights on cost-aware architecture and the principles of frugal architecture, explore the Frugal Architect site and Dr. Werner Vogels’ seven laws.