Rolling out a new customer-facing application can be quite a challenge, considering the time it takes to develop and test the code. In addition, much time is taken on data modeling and database design, usually with multiple levels of operating systems, network and database professionals. Providers of database as a service (DBaaS) promise to handle all these database tasks, including hardware acquisition, database management software installation and configuration, database definition, performance tuning, and backup and recovery.
While this service can greatly shorten the time-to-market of a new application, there are certain risks inherent in delegating database management to an outside provider, especially if the data is to be stored off-site or in the cloud. In this article we address the risks, how should they be mitigated and what questions need to be addressed when choosing a DBaaS provider.
Contract Negotiations Begin
Databases don’t function in a vacuum. They require hardware and an operating system to run on, storage media to store data, and network connections to transport data in and out. DBaaS should include the creation and maintenance of this hardware and software environment, and that should be spelled out in the contract. In addition to that, there are considerations as the application and database both grow, including costs for additional storage media, costs for additional CPU power and the inevitable performance monitoring and tuning.
Here are the most common issues to get right from the beginning. These will affect both the fixed and variable costs of your contract with the DBaaS provider.
Top Priority - Critical
Hardware maintenance and upgrades. The DBaaS provider controls the specific hardware on which your database will reside. As your application matures the time may come when the hardware becomes out-of-date, or needs to be upgraded. Upgrade reasons include recent fixes from the vendor, hardware no longer being in service or other upgrades in operating system or DBMS. In addition, competing applications from other companies may be running on faster or newer machines. Lastly, some hardware upgrades are related to security, such as new hardware-based encryption or anti-virus protection.
Operating system maintenance and upgrades. Similarly, as with the hardware, operating system software needs frequent upgrades. These include fixes for software issues, more effective resource scheduling and utilization and new functionality. At times, new operating system functionality may support DBMS software upgrades. These include improved data compression algorithms, new data storage options or support for a wider variety of disk drive arrays.
Database management system (DBMS) maintenance and upgrades. This may be crucial for a new application, especially if your application uses new or complex query options such as cube, on-line analytical processing (OLAP) or resource-intensive calculations. The upgrade process will most likely involve an application outage, and may mean an increase in software license fees.
Disaster recovery planning. Disasters range from small (a disk drive failure) to large (a complete site outage). What are your DBaaS provider’s plans for this range of issues? Smaller/local issues are usually mitigated with extra hardware such as redundant disk arrays and additional network nodes. However, for large issues you need to discuss three things: (1) Will your data be backed up, and how often? (2) In the case of a site-wide disaster, how long will it take for your application to be back on-line (the recovery time objective, or RTO)? (3) When your data is recovered, or when your application comes back on-line, to what point will it have been recovered (the point-in-time objective or PTO)? For example, the provider may promise to back up your data nightly. Does this mean that if their site fails during the working day, when your application comes back up will the data be as of the prior midnight, thus losing all of the day’s transactions? If you insist that your application’s data be recovered to a point immediately prior to the disaster, how long will the recovery take, and how often will the provider need to take backups? Will you be paying for all of this?
Second Priority - Essential
Performance monitoring, notification and tuning. A major part of supporting a database management system is performance tuning. Tuning occurs at multiple levels. For the application using DBaaS there are three main tuning categories. System tuning involves looking at DBMS installation and configuration parameters including memory allocation. Object tuning is something the database administrator does when creating tables and indexes, and includes choosing horizontal and vertical data partitioning, choosing table columns for indexing, and determining how and if tables should be physically sorted (clustered) by certain key columns. Finally, query tuning happens by capturing SQL statements, the corresponding data access paths and the result answer set sizes and resources used. In all cases, performance monitoring and tuning means using one or more software packages, requiring staff familiar with their use. Verify which tools will be used by the DBaaS provider, how the results will be reported to you and what their staff will do to react to performance issues.
Resource performance options. Resource constraints can occur for a variety of reasons. Your application may be quite different from its competitors or may involve innovative features or techniques. This may make it difficult for the DBaaS vendor to accurately estimate the hardware, software, network and data storage required to support it. Once your application goes on-line, performance reporting may raise red flags such as poor transaction elapsed times. The problem may well be due to a lack of one or more resources. In this case, what will the DBaaS provider do, and who will pay for additional resources? Of course, one option is to do a resource constraint analysis, which can point out how some resources can be re-purposed. For example, if database disk storage appears to be slow, perhaps either database reorganizations or additional table indexes could alleviate the problem.
Business rule implementation and maintenance. Many application developers are unaware of this issue until it’s too late. Many data integrity rules exist in a data model. Some of these are:
- Entity integrity -- Entity keys (customer id, part number, order number) must be unique;
- Domain integrity -- Data values must conform to the data type of the attribute; for example, date columns must contain valid dates (e.g. 00/00/0000 is not allowed) and a salary column must contain a non-negative value;
- Referential integrity -- When a column refers to a key in another table, values in that column must exist in the other table; for example, if an order table contains a customer number column, all of the values in that customer number column must be valid customer numbers.
The question now becomes, which software enforces these rules? For many in-house developed applications, most business rules are enforced in the database management system. As one example, the DBMS enforces entity integrity using unique indexes. For your application, you must specify which of the various rules will be enforced in your application, and which by the DBMS. It is important to realize that any changes to your data model or your application must take these rules into account. (It’s no use implementing a feature in your application that allows duplicate order numbers if the DBMS constrains them to be unique.) You and your DBaaS provider must closely coordinate the maintenance of your data model, since many business rules may exist in it and in the application code.
Security. Of course, security is always a concern. In general, database data is only accessible through the query interface. Security in this case is handled by managing query permissions. One question for you is whether or not DBaaS provider staff have permission to query the database. While this may be necessary for performance tuning, you own your data and you should specify who can access it and how. Another issue is the raw database data itself. Your provider will have copies of it in database backups, and the backups (and presumably the original disks) are accessible as files through the basic operating system. Ensure that you know that the database files and backups are secured.
Third Priority - Nice to Have
Old or stale data purge. It is the function of databases to store data so that it can be accessed. What will your provider do with data that is old or stale? For example, if your application involves on-line product sales, how long will completed sales be retained? The longer the data is retained the larger your database and the poorer the performance, generally speaking. On the other hand, retaining historical data allows you to issue analytical queries against the data to predict customer trends or develop product re-stocking schedules. One option is to store old data in a separate data mart for analysis. Discuss these options with the DBaaS provider.
Database and database object reorganizations. Depending upon the specific DBMS, database and object reorganizations are sometimes necessary. In the case where reorganizations are recommended by the vendor, when will they be executed and will they affect application performance? And, as the application matures, sales increase and you expand the length of your on-line day, will reorganizations run longer and longer? Consider coordinating the reorganization schedule with purging of stale data.
Planned outages for maintenance and upgrades. As noted above, the DBaaS provider will need to schedule outages for hardware and software maintenance and upgrades. You will need to coordinate with the provider for your own application changes as well. Immediately after the first application implementation you should schedule several change windows to implement any application or database fixes that arise during application execution. Note that database changes and application changes will affect configuration and size of the database, which affects backup and recovery processes and execution times.
Adding analytics. As the application matures the time comes when you want to add new functions and features or expand product and service offerings. Which ones do you add first, and which customers will they attract the most? This points out the eventual need for executing business analytics queries against your data. There are different methods to configure a system that permits efficient analytics, including storing your data in a hybrid hardware/software environment or copying your data to a separate data mart for analysis. These kind of systems may involve high costs and are usually not considered during initial application implementation. You may wish to consult with your provider regarding how analytics could be included at the beginning or how it could be implemented later.
Working with a database as a service vendor allows you to implement a new application on an accelerated schedule. There are costs associated with each portion of the service and the application developer needs to understand what services are critical and important for application success.