Automating Operational Reporting with Apache Airflow and Oracle
Background
Before this automation initiative, our team relied on a highly manual reporting process that demanded over two hours of daily effort. Each day, operational reports were downloaded from Epic — often as password-protected Excel files — and processed one by one. Passwords had to be removed manually, the data cleaned, and the files loaded into Tableau Prep for transformation before finally being uploaded to dashboards for operational review. This workflow created unnecessary friction and slowed down the delivery of time-sensitive insights.
Challenges
Because each step in the process depended on manual effort, auditing and verification were equally time-consuming. Ensuring a report's accuracy meant retracing every step — from downloaded files to Tableau outputs — which made quality assurance a slow and tedious process. Similarly, building new reports often required duplicating ad hoc processes from scratch. While our data was accurate, confirming its lineage and ensuring consistency added a layer of operational drag that limited our team’s capacity to innovate or scale.
Solution
To address these challenges, I led a full-scale modernization of our reporting pipeline. Working closely with backend developers from the Revenue Cycle Systems (RCS) team, we transitioned from an Excel-based workflow to a centralized Oracle database that now serves as the single source of truth for our operational data. This eliminated manual downloads, file handling, and the password-related workarounds.
​
From there, I developed Python scripts to handle transformation logic and orchestrated the workflow using Apache Airflow, enabling hands-free, scheduled execution. We also leveraged Tableau Prep’s scheduling functionality to manage final output processing and dashboard refreshes, completing the transition from a manual to a fully automated system.
Impact
What was once a fragile and resource-intensive process is now a scalable, transparent, and reliable pipeline. We’ve eliminated over two hours of manual work per day, improved consistency, and significantly accelerated the time-to-insight for operational data consumers. Auditing is no longer a chore — it’s built into the system, with clear logs and versioning at every step.
Even more importantly, this project unlocked new analytical possibilities. With the infrastructure in place, we’re now able to report on complex metrics that were previously out of reach. We’ve expanded our operational insight far beyond the original scope — exploring trends, building richer dashboards, and responding to new data needs without overloading the team. This project didn’t just make our workflows faster — it redefined what’s possible for operational analytics at our organization.