The Data warehouse testing is one of the different types of testing performed when the project involves the huge volume of data.
Data warehouse Testing:
Huge volumes of data or information stored in the data warehouse. This information is used by the organisation to make analysis and decision. Data is important for business and data warehouse testing technique plays an significant role in validating the data and provides the exact information for business decisions. It minimises the data loss.
Data warehouse is used in Business Intelligence which helps any organisation to make proper decisions. Business Intelligence is a process of changing the raw data into useful information for business analysis and data warehouse is a part or subset of business intelligence. Data warehouse is technique of extracting data from different sources and integrating them together and loading them to the proper sources for data visualisation.
This process is used in business intelligence to achieve some objectives designed as per the organisation. For example when you type texts in ms word or any chatting app you have an option of spelling and grammar correction feature which always provides the proper set of words or checks the grammar for our sentences. Collection of data from various formats and transforming into a particular format as per the business rules is called as data warehousing and the testing of the data is done through a independent group of experts for smooth conversion of the projects.
The data integrity issues can be avoided using the data warehouse testing. The common skills required by the data warehouse testers are sql basics, knowledge of database, etc. The purpose of data warehouse testing are listed below:
- identify and reduce the ETL errors
- data quality improvement
- data integrity
The common data warehouse testing tools used are Informatica, Query-surge etc.
11 Responses
Unlike Database testing, Data warehouse testing involved the testing of a large volume of unnormalized data. For example, some of the important issues with Data Warehouse Testing are:
• Data Warehouse/ETL testing requires SQL programming: This has become a major issue as most of the testers are manual testers and have limited SQL coding skills, thus making data testing very difficult
• Performing Data completeness checks for transformed columns is tricky
• Certain testing strategies used are time consuming
Identify the possible challenges in data warehouse testing.
Some Important data Warehouse Testing challenges are
1. Unavailability of inclusive test bed at times
2. Lack of proper flow of business information.
3. Loss of data might be there during the ETL process
4. Existence of many ambiguous software requirements
5. Existence of apparent trouble acquiring and building test data
6. Production sample data is not a true representation of all possible business processes
7. Data warehouse/ETL testing requires SQL programming. Testers with limited SQL coding skills making data testing very difficult
8. Performing Data completeness checks for transformed columns is tricky
9. Certain testing strategies used are time consuming.
Data warehouse testing involved the testing of a large volume of unnormalized data. For example, some of the important issues with Data Warehouse Testing are:
Data Warehouse/ETL testing requires SQL programming: This has become a major issue as most of the testers are manual testers and have limited SQL coding skills, thus making data testing very difficult
Performing Data completeness checks for transformed columns is tricky.
Certain testing strategies used are time consuming
Data ware house testing is a type of testing where inproject involves huge volume of datea
As we know that Data warehouse testing must deal with a huge volume of data to be tested. On like ordinary testing which identifies the defeats and validates the quality, and correctness of the client’s requirements. Because Data warehouse testing are done by more expects tester, there is a credibility of data testing integrity. However, there many possible challenges which faces the data warehouse testing:
1. Due to the large volume of information to be tested, the possibility of data loss is real
2. Duplication of data is also real
3. Incompatibility of data is common
4. Business decision making can be delayed or become erroneous due incomplete or missing information.
Data warehouse testing has many challenges in its way as we need to test millions of record at a time.
– Lack of exhaustive test data plans is the Major challenge as client may not be comfortable in providing access to production data for testing purpose or sufficient data is not available from various systems for testing.
– Since there is lot of data for testing in various tables and business logic is complex hence complete testing may not be
practically possible due to time and budget constraints whereas client may expect the testing team to use all the available test data for testing.
– Data Warehouse Testing needs to deal with large volumes of data from multiple systems. Lot of data issues may be reported initially & if these are not resolved quickly it may have an impact on project schedule.
– Since data warehouse has complex architecture a small change may have an impact at multiple places.
-If data volume is not considered for arriving at the configuration of the infrastructure needed for the project then there could be delays due to long procurement cycles and/or infrastructure level bottleneck during test execution phase as normal desktop
machines may not be able to handle the magnitude of data.
The challenges in data warehouse testing include;
-loss of data
-incorrect data
-data not working efficiently in the new environment
-data integrity issues with the load /transform process
-duplication of data
Data Warehouse Testing: Here Huge volume of data or information is stored and used by organizations to analyze and take decisions as its important for every business.
The data warehouse is used in Business Intelligence which helps any organization to take prior decisions. The purpose of data warehousing are
=Data quality improvement
=Data integrity
=identify and reduce errors.
Data warehouse testing must deal with a huge volume of data to be tested, identifies the defeats and validates the quality, and correctness of the client’s requirements.
1.Since the large volume of information to be tested, the possibility of data loss is real
2. Duplication of data is possible.