Clean Data, Smart AI: The 70% Rule Every Finance Professional Should Know
Guest article by Asha P Pillai
Why Data Preparation, Not the Algorithm, Decides Whether Your AI Project Succeeds
The Reality Behind AI Success Rates
For all the hype around AI, most implementations stall quietly. Research by MIT and Deloitte consistently shows that roughly 70% of the total time and effort in AI implementation goes into preparing, cleaning, and structuring data.
This often surprises finance professionals. We imagine AI as something futuristic — with advanced models, predictive analytics, and intelligent agents — when in reality, it starts with something much more basic: knowing what your data means and whether it can be trusted.
GCCs, with their scale and process discipline, have the perfect opportunity to get this right. But the foundation isn’t more tools; it’s better data.
Why Finance Data Is Tricky
Finance data looks clean from a distance. It’s numeric, structured, controlled by policy. But dig deeper and you’ll find:
Inconsistent tagging: Journal entries missing clear context (“Marketing” or “Project Spend” aren’t enough).
Unlinked datasets: P&L, headcount, and operational data stored in separate systems.
Uncaptured rationale: No notes on why adjustments were made, leaving AI unable to interpret meaning.
Data aging: Historical entries coded under legacy charts of accounts that no longer match current structures.
AI can’t fix these issues — it amplifies them.
That’s why data readiness is now a finance skill, not just a data science one.
The CRISP-DM Framework: A Finance-Friendly Blueprint
The CRISP-DM (Cross-Industry Standard Process for Data Mining) method has been around since the early 2000s, but it’s finding new life in AI projects. Here’s how it maps beautifully onto finance work:
Business Understanding
Define the finance question before touching data.
Example: “Can we predict margin dips two weeks before month-end?”Data Understanding
Identify what data is needed and assess quality.
Example: Mapping revenue recognition data, campaign spend, and COGS trends.Data Preparation
Clean, tag, and standardize. Remove duplicates. Ensure timestamps align.
Example: Harmonizing region codes across ERP systems.Modeling
Build your AI model or forecasting algorithm.
Example: Using historical sales data and FX to predict revenue shifts.Evaluation
Compare results with actuals and adjust the data model.
Example: Validating if predicted variances matched the real P&L impact.Deployment
Embed the insight into Power BI, Copilot, or other daily tools.
Example: Having AI-generated variance insights appear in your existing dashboard view.
This framework isn’t just for data scientists. It’s how finance teams can structure their own AI readiness journey.
Why 70% of Your CRISP-DM Effort Should Be Data Understanding and Data Cleansing
Finance teams talk a lot about AI models. In practice, models are the easy part. The hard part is making sure the data those models see is complete, consistent, and explainable.
That is exactly what CRISP-DM was designed for — and it is the right way to structure finance AI work.
How GCC Finance Teams Operationalize the 70%
GCCs are built for discipline and scale. Turn that into a data advantage:
At source: Make key attributes mandatory in ERP postings. Use controlled vocabularies and validation rules.
In integration: Run scheduled checks for nulls, duplicates, and code drift in your ETL/ELT jobs (Azure Data Factory, Databricks, Snowflake).
In intelligence: Log every data fix as a rule, not a one-off. If you impute a value this month, encode the logic so it repeats consistently.
In consumption: Expose data-quality badges alongside Power BI visuals. Users should see if a tile is “DQ Green” before acting on it.
A Simple Finance Playbook Aligned to CRISP-DM
Define the business question: “Reduce close cycle by 2 days” or “Flag margin erosion at SKU level mid-month.”
Publish a data catalog: Owner, definition, refresh cadence, lineage for each core table or view.
Create cleansing rules: Null policy, duplicate policy, accepted code lists, hierarchy change protocol, late-post handling.
Version your features: Store the exact transformed fields used by the model so results are reproducible.
Close the loop: Push validated insights into Power BI, Copilot, or alerts, then capture user feedback to refine rules.
Quick Wins That Prove the Point
Collections risk: Add missing customer segment and terms fields, standardize IDs, then model risk. Often lifts precision more than swapping algorithms.
AP duplicates: Normalize vendor names and bank details, dedupe by amount-date-counterparty, then run a simple similarity check. False positives drop sharply.
Variance narration: Require “reason” tags on material journals, align cost center names, then let your narrative tool draft first-pass commentary that actually reads right.
Metrics That Keep You Honest
Track data quality like you track KPIs:
Fill rate on mandatory fields
Duplicate rate per process
Code-list conformity
Hierarchy freshness
Reconciliation success rate
Time from issue detected to rule deployed
Set thresholds and publish them. When quality is visible, quality improves.
The Bottom Line
AI in finance is not a model contest. It is a data discipline contest.
Use CRISP-DM to keep the work structured, and spend about 70% of your effort where it matters most: data understanding and data cleansing.
Do that, and your forecasts, anomaly flags, and narratives will become faster, clearer, and far more trusted.
Clean data is not someone else’s job. It is the foundation of your next decision