Understanding Etl and Data Warehousing Issues, Challenges and Importance-Real time scenarios-India book

Testing in data warehouse systems is substantial because it is oriented towards the correctness and validation of data/ information supplied for decision making. Keeping in view the idiosyncratic characteristics of data warehouse testing and the complexity of data warehouse projects, this research has reviewed and revised the scope of automated testing in assuring quality data warehouse solutions. Initially a data set generator has been developed to generate synthetic but near to real data; followed by the classification of anomalies in synthesized data with the help of a hand coded Extraction, Transformation and Loading (ETL) routine. To ensure quality data for a data warehouse and to promulgate the importance of Extraction, Transformation and Loading (ETL) routines some test cases of prime importance were identified. Later on automated testing procedures were embedded in hand coded ETL routine to ensure quality data. The statistical analysis revealed major enhancement in data quality with the introduction of automated testing procedures. The various data warehouse architectures have been analyzed to endorse a refined data warehouse architecture named as Data Sharehouse.

What is data warehouse A data warehouse is an organized collection of logically related data. Current and historical data taken from multiple databases for the purpose of reporting and analysis.

What is ETL
ETL stands for extract transform and finally load the data into the target tables.The data stored in the data warehouse is inconsistent because it stored in the multiple databases . To make this inconsistent to consistent from we have introduced a concept called ETL (Extract Transform and Load).

Book Info