Case study專案案例
Cut infrastructure costs 85% by re-engineering a legacy AML monitoring system重構舊有洗錢防制 (AML) 監控系統,大幅削減 85% 維運成本
Inherited a legacy system running $200K/month in infrastructure costs. Rebuilt the pipeline and renegotiated the SLA, cutting monthly cost to $30K without losing any real operational capability.接手每月基礎架構維運成本高達 20 萬美元的舊有系統。透過重構資料管線與重新議定服務級別協定 (SLA),成功將每月成本壓降至 3 萬美元,且未折損任何實質的營運能力。
Outcome成果
≈85% cost reduction成本縮減約 85%
- Client客戶
- Financial Institution金融機構
- Role角色
- Lead Data Scientist & Full-Stack Developer首席資料科學家暨全端工程師
- Timeframe期間
- 20242024
Situation
At a financial services firm, the monthly cloud infrastructure cost of a legacy AML monitoring platform had reached approximately $200,000. Although the system remained operational, the client considered its cost base unsustainable.
The underlying issue was not a single expensive component, but a mismatch between the platform’s architecture and its actual operational requirements. The original statement of work had specified 99.99% availability without identifying which services were genuinely business-critical or validating the requirement against real usage patterns. As a result, environments were provisioned for maximum redundancy and remained continuously available by default.
The platform also relied on complex legacy SQL logic that was difficult to understand, modify, and optimise. This increased maintenance effort while making it harder to trace workloads and identify the drivers of infrastructure spend.
I was tasked with determining the root causes and reducing costs without compromising the system’s required operational performance.
Strategy
I approached the problem across three connected areas: establishing a transparent cost baseline, redesigning the most inefficient parts of the architecture, and aligning availability requirements with business criticality.
Identifying the underlying cost drivers.
I reviewed the platform’s architecture, infrastructure utilisation, storage patterns, and operating schedules. I also worked through the legacy SQL logic to understand how data moved between components and when each workload genuinely needed to run.
The assessment identified three primary cost drivers:
- Inefficient SQL-scheduled data pipelines
- Environments operating continuously despite intermittent usage
- Storage configurations that were significantly more expensive than their access patterns required
This shifted the discussion from a general concern about cloud costs to a defined set of architectural and operational decisions that could be addressed.
Redesigning the pipeline and infrastructure.
I refactored the legacy SQL code into maintainable Python pipelines and redesigned parts of the data-processing and storage architecture.
A key constraint was the frontend application’s dependency on the primary database, which prevented that database from being shut down even when analytical workloads were inactive. I designed a lightweight database specifically for frontend access, decoupling the user-facing application from the main processing environment.
This allowed the primary database and supporting infrastructure to operate on a schedule rather than continuously. Working with the wider technical team, I also right-sized the remaining resources, aligned storage choices with actual access patterns, and introduced clearer cost monitoring.
Aligning availability requirements with business criticality.
A substantial portion of the over-provisioning originated from the blanket 99.99% availability requirement. Rather than treating this as a purely technical constraint, I worked directly with business stakeholders to revisit the underlying operational need.
We identified which components were genuinely business-critical and established differentiated availability requirements based on their role and usage. This preserved resilience where it created business value while removing unnecessary redundancy from lower-criticality workloads.
Result
Monthly infrastructure expenditure fell from approximately $200,000 to $30,000—an 85% reduction—while the platform continued to meet the client’s operational and availability requirements.
The refactored pipelines were easier to maintain, and the new monitoring provided greater visibility into where infrastructure costs were being generated. The project turned an unsustainable recurring expense into a more efficient platform with a viable long-term operating model.
專案背景
在一家金融服務公司,其舊有洗錢防制 (AML) 監控平台的每月雲端基礎架構成本已高達約 20 萬美元。儘管系統仍能正常運作,但客戶認為此成本結構無法長期負荷。
根本問題並非出自單一昂貴的元件,而是「平台架構」與「實際營運需求」之間存在嚴重的錯置。最初的工作說明書直接規定了 99.99% 的系統可用性 (Availability),卻未去鑑別哪些服務才真正具備業務關鍵性 ,亦未根據實際使用模式來驗證此需求。 這導致系統環境為了追求最大備援 (Redundancy) 而過度配置,並預設為無間斷全時運轉。
此外,該平台高度依賴複雜的舊有 SQL 邏輯,不僅難以理解、修改與最佳化,更增加了維護成本,同時也難以追蹤工作負載並釐清推升基礎架構支出的核心驅動因素。 我的任務是找出根本原因,並在不妥協系統所需營運效能的前提下降低成本。
策略與執行
我從三個相互關聯的面向著手解決此問題:建立透明的成本基準、重新設計架構中最缺乏效率的環節,以及將可用性需求與業務關鍵程度緊密對齊。
辨識核心成本驅動因素 (Cost Drivers)
我全面盤點了平台的系統架構、基礎架構利用率、資料儲存模式以及排程運作狀況;並梳理了舊有的 SQL 邏輯,以釐清資料在各元件間的流動方式,以及各項工作負載真正需要執行的時間點。 評估結果揪出了三個主要的成本痛點:
- 缺乏效率的 SQL 排程資料管線
- 使用頻率屬於間歇性,環境卻持續全時運轉
- 儲存配置的成本,遠高於實際存取模式所需
這項梳理成功將客戶對雲端成本的「廣泛性擔憂」,轉化為一系列可具體解決的「架構與營運決策」。
重構資料管線與基礎架構
我將舊有的 SQL 程式碼重構為易於維護的 Python 資料管線,並重新設計了部分的資料處理與儲存架構。
當時的一大技術限制在於前端應用程式對主資料庫的依賴,導致即便在分析工作負載處於閒置狀態時,主資料庫也無法關閉。為此,我專門為前端存取設計了一個輕量級資料庫,成功將面向使用者的應用程式與核心處理環境進行解耦。
這使得主資料庫與其支援的基礎架構得以依照排程運作,而非毫無彈性地全時執行。透過與技術團隊的跨部門合作,我亦針對剩餘資源進行了規格優化與合理配置,使儲存方案與實際存取模式相符,並導入了更具透明度的成本監控機制。
將可用性需求與業務關鍵程度對齊
很大一部分的資源過度配置源自於全面套用 99.99% 可用性的僵化要求。我沒有單純將其視為技術限制,而是直接與業務利益關係人合作,重新檢視其底層的實際營運需求。
我們精準鑑別出哪些元件才真正攸關業務命脈,並依據其角色與使用場景制定了差異化的可用性標準。此舉不僅在能創造商業價值之處保留了系統韌性,同時也大刀闊斧地消除了低關鍵性工作負載中不必要的備援成本。
成果
每月的基礎架構支出從約 20 萬美元大幅降至 3 萬美元——降幅高達 85%——同時平台仍持續穩定滿足客戶的營運與可用性需求。 重構後的資料管線大幅降低了維護門檻,而全新的監控機制也提供了更高的能見度,讓團隊能精準掌握基礎架構成本的發生源頭。本專案成功將一項難以負荷的經常性支出,轉化為具備可行長期營運模式的敏捷高效平台。