There are many issues regarding data warehouse architecture that occur at every stage of data warehouse implementation i.e., from designing of a data warehouse to its implementation.
Issues in Designing a Data Warehouse
1.A business analysis framework is to be constructed for which very good understanding and analysis of business needs isrequired. The design of data warehouse is done with respect to four different views, the top-down, the data source, the data warehouse and the business query view.
2. Implementation and maintenance of a data warehouse is a complicated task as it requires business, technological and program management skills.
Business skills are necessary in order to understand how data is stored and managed by the systems, how migration software can be built so as to transfer data from databases and legacy systems to data warehouse, how to build customized software meant for keeping the data warehouse updated. Making use of data warehouse requires understanding the importance of the data stored and also converting business requirements into database queries which can be run on a data warehouse.
-
Technology skills require the data analyst to understand the analysis of statistical information and history in order to deduce conclusions and facts and take decisions to improve business processes.
-
Program management skills include technology integration, communication with vendors and end users so that business is time specific and profitable.
3. Design of a data warehouse architecture can be done from either of two approaches or combination of both. The two approaches used are top-down and bottom-up approach. There are a set of pros and cons of each of these approaches. The top-down approach provides a systematic solution and reduces technology integration issues. However, it is expensive, consumes lot of time and doesn't offer flexibility.
The bottom-up approach provides flexibility, cost effectiveness and quick returns investment but also causes problems during integration of various data marts which are in different formats to implement a consistent enterprise of data warehouse. Issues in Selecting a Data Warehouse Architecture Model: There are three data warehouse models,
1. Enterprise warehouse: An enterprise warehouse consists of information which is well organized and summarized. The amount of data is usually in large volume and can range from few Gigabytes to many terabytes. The biggest issue with enterprise warehouse is that it involves extensive business modelling and consume a lot of time in its implementation.
2. Data Mart: A data mart is meant for few people belonging to a specific group. It is a subset of enterprise data which will be of interest to only a particular/selected group of employees. It is implemented using low-cost departmental servers and the time taken is also measured in weeks. But the problem arises when data marts of different types are to be integrated and they are not designed and planned with respect to an enterprise wide data.
3. Virtual Warehouse: A virtual warehouse does not involve implementation of a new data warehouse instead it is implemented with the help of a set of views over existing databases. It is easy to build a virtual warehouse but it increases the processing on existing database servers which in turn requires the database servers to be faster and capable enough to run the views.
It is clear from the above discussion that, construction of a data warehouse is complex and time consuming task, its goal should be specific and clearly defined, the first implementation of a data warehouse should have specific, achievable and measurable goals. These goals in essence are determination of budget and time allocation, part of the enterprise to be modelled, number of departments for which the data warehouse is implemented and the number of databases needed. A data warehouse implementation requires all the responsibilities of a database administrator like performance tuning, data updation, recovery management, security and access control, managing data backups, managing increase in data, etc. Scope management should also be considered wherein the size of the data warehouse, the number of resources, examples, expenditure, etc., should be managed.
Recommended Questions
Useful Files
Users Joined
There are many issues regarding data warehouse architecture that occur at every stage of data warehouse implementation i.e., from designing of a data warehouse to its implementation.
Issues in Designing a Data Warehouse
1.A business analysis framework is to be constructed for which very good understanding and analysis of business needs isrequired. The design of data warehouse is done with respect to four different views, the top-down, the data source, the data warehouse and the business query view.
2. Implementation and maintenance of a data warehouse is a complicated task as it requires business, technological and program management skills.
Business skills are necessary in order to understand how data is stored and managed by the systems, how migration software can be built so as to transfer data from databases and legacy systems to data warehouse, how to build customized software meant for keeping the data warehouse updated. Making use of data warehouse requires understanding the importance of the data stored and also converting business requirements into database queries which can be run on a data warehouse.
-
Technology skills require the data analyst to understand the analysis of statistical information and history in order to deduce conclusions and facts and take decisions to improve business processes.
-
Program management skills include technology integration, communication with vendors and end users so that business is time specific and profitable.
3. Design of a data warehouse architecture can be done from either of two approaches or combination of both. The two approaches used are top-down and bottom-up approach. There are a set of pros and cons of each of these approaches. The top-down approach provides a systematic solution and reduces technology integration issues. However, it is expensive, consumes lot of time and doesn't offer flexibility.
The bottom-up approach provides flexibility, cost effectiveness and quick returns investment but also causes problems during integration of various data marts which are in different formats to implement a consistent enterprise of data warehouse. Issues in Selecting a Data Warehouse Architecture Model: There are three data warehouse models,
1. Enterprise warehouse: An enterprise warehouse consists of information which is well organized and summarized. The amount of data is usually in large volume and can range from few Gigabytes to many terabytes. The biggest issue with enterprise warehouse is that it involves extensive business modelling and consume a lot of time in its implementation.
2. Data Mart: A data mart is meant for few people belonging to a specific group. It is a subset of enterprise data which will be of interest to only a particular/selected group of employees. It is implemented using low-cost departmental servers and the time taken is also measured in weeks. But the problem arises when data marts of different types are to be integrated and they are not designed and planned with respect to an enterprise wide data.
3. Virtual Warehouse: A virtual warehouse does not involve implementation of a new data warehouse instead it is implemented with the help of a set of views over existing databases. It is easy to build a virtual warehouse but it increases the processing on existing database servers which in turn requires the database servers to be faster and capable enough to run the views.
It is clear from the above discussion that, construction of a data warehouse is complex and time consuming task, its goal should be specific and clearly defined, the first implementation of a data warehouse should have specific, achievable and measurable goals. These goals in essence are determination of budget and time allocation, part of the enterprise to be modelled, number of departments for which the data warehouse is implemented and the number of databases needed. A data warehouse implementation requires all the responsibilities of a database administrator like performance tuning, data updation, recovery management, security and access control, managing data backups, managing increase in data, etc. Scope management should also be considered wherein the size of the data warehouse, the number of resources, examples, expenditure, etc., should be managed.