The ETL load status update table is one of the must have tabele in DW, the table should maintain staging and target load start and end time with most recent load indicator
here is an example,
This example describe the staging and target load start and end time for customer data. the MOST_REC_LOAD_IND field should be "Y" for the most recent process date and "N" for the history date. This field can be used as a trigger to run the report automatically once the data gets loaded.
here is an example,
This example describe the staging and target load start and end time for customer data. the MOST_REC_LOAD_IND field should be "Y" for the most recent process date and "N" for the history date. This field can be used as a trigger to run the report automatically once the data gets loaded.
What are challenges you faced during etl testing?Give solution to it?
ReplyDeleteHere is the list of common testing methods I have followed in many projects, if you have any specific issue the let me know I can provide more details on it.
ReplyDeleteETL testing requires SQL knowledge for testers to understand the relationship between the tables and how they join by keys, as a tester I would do the following
1) Create a simple row count scripts which will do row count on each source and Target tables and dose the comparison(make sure you have included the business logic withthin the script for target tables if any)
2)Check the join between Fact and Dimension, a surrogate key in fact must exists in Dimesnion table
3)Field level vlidation, this is where things can go wrong as you might have to validate row by row, you can use maual method or any automated tool method
4) Performance testing, run a query by joining ffact and dimension tables and record the timings,do the same level of testing in reporting as well,
5) Field level Key indexing testing
6) Table level Partition testing
Hi. Went thru ur blog. Your blog is very informative. Please update regularly.
ReplyDelete