A major multinational corporation needed a real-time data warehouse to support customer service and maintenance requirements. Network and customer data was delayed for two days before it was accessible. Data was not being collected and organized in a usable fashion. Existing resources were overwhelmed by the millions of events spread across a variety of systems and message types. Key challenges included:
- High Volume: Average of 750 million discrete events per day, with peaks of 100,000 events per second;
- Real Time Data Recovery: Access required within 1 second of collection;
- Dynamic User Requirements: Increasing with rapid customer base growth;
- Security: Some data collected contained customer-sensitive information;
- 100% Availability: System could not ever be out of service.
David’s team developed an architecture, deployed a prototype system, and established a full production system including these components:
- Grid Computing: Advanced grid computing techniques in each component including the database itself;
- High-Availability Platform: Components selected for full availability;
- Multilevel Buffering: A buffering strategy was implemented to prevent blocking operations, even when different parts of the system were under varying load;
- Pluggable Loaders: The system easily adapts to new forms of data that were not part of the original design.
Corporation is now supported with one of the largest and most sophisticated data warehouse systems in the world. Results of this success:
- Increased Profitability:
- Increased service representative and technician efficiency
- Removed the cost and loss of data caused by outages
- Improved management oversight and business efficiencies
- Increased Customer Satisfaction:
- Reduced customer wait and problem resolution time
- Proactive and rapid troubleshooting