IBM DB2 for z/OS Trends and Directions Embed Size (px)
344 x 292 429 x 357 514 x 422 599 x 487
DESCRIPTION
IBM DB2 for z/OS Trends and Directions. Jeff Josten Distinguished Engineer DB2 for z/OS Development IBM Silicon Valley Lab. Please note. IBM ’ s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM ’ s sole discretion. - PowerPoint PPT Presentation
Text of IBM DB2 for z/OS Trends and Directions 2007 zSummit Initial ReviewJeff Josten Distinguished Engineer Please note IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. IBM IOD 2011 DB2 and the new zEnterprise EC12 machine DB2 11 (next release) Fastest uptake Adoption Driven by: Virtual Storage Constraint relief for more threads Security, RAS improvements As of March 2012 Over two times as many customers –and– over three times as many licenses as what we had with DB2 V8 or DB2 9 at the same point in time. With about 25% of the customers migrating from DB2 V8. From a quick survey with a set of 500 customers on a DB2 10 for z/OS migration call, a number of the respondents said they are targeting to be in production by YE2011 Customers have been enthusiastic about DB2 10 for z/OS performance and scalability. Some customers are able to simplify their structure and many are seeing better productivity. The security and temporal function are seeing strong early acceptance. A safer infrastructure with better audit function can help customers avoid the need for a new security structure. Customers are signing up for the V8 to DB2 10 migration to save time and to get these improvements faster. * © 2012 IBM Corporation Creating the Hybrid Data Server - Netezza and Z Combine DB2 for z/OS with Netezza to provide an industry exclusive Transaction Processing Systems (OLTP) and recoverability Netezza Accelerator Data Mart Data Mart Data Mart Best in Consolidation Best in OLTP and Transactional Analytics Industry recognized leader in mission critical transaction systems Together: Destroying the myth that transactional and decision support workloads have to be on separate platforms Transactional Analytics What’s New? High Performance Storage Saver Store a DB2 table or partition of data solely on the Accelerator. Removes the requirement for the data to be replicated on both DB2 and the Accelerator Incremental Update Enables tables within the Accelerator to be continually updated throughout the day. zEnterprise EC12 Support Version 3 will support the zEnterprise EC12, z196 and z114 System z platforms Query Prioritization Brings System z workload management down to the individual query being routed to the Accelerator High Capacity Support has been extended to include the entire Netezza 1000 line (1.28 PB) UNLOAD Lite * * * * DB2 Analytics Accelerator for DB2 z/OS is the next generation accelerator for System z Tightly integrated with DB2 for z/OS Transparent to DB2 applications Capitalizes on the availability of System z Minimizes data movement Leverages Netezza Technology FPGA for hardware query acceleration (These FPGAs are used for data decompression, data filtering and early SQL projections and restriction) No need for indexes. MQTs, or query plans Disk mirroring and blade failover Box can be remissioned when new technology is introduced Offloads long running queries from System z Created for mixed workload of operational transaction systems, data warehouse, operational data stores, and consolidated data marts The IBM DB2 Analytics Accelerator is: An integration of IBM hardware, software, and storage and advanced technologies focused on business analytics, that combine to give IBM clients the industry's most high performance analytic capability, to extract business insight from information assets, providing the right answers to the right questions at the right time. . It removes the costs of traditional performance tuning such as indices, materialized query tables (MQTs) and query plan tuning that required extensive time and expensive resources. * Customer interaction Multiple infrastructures required Copy ETL / OLTP Using web service calls to score new transactional data as it is coming into the business is currently very network intensive and can be costly when trying to ensure SLAs. An alternative to using a web services call is to score the transactional data after it is moved into the data warehouse/data mart/ODS. Because we are now reliant on the ETL schedule, the score is only as current as the last refresh and scoring of the data. This makes it difficult if not impossible to provide the highest level of business insight at the point of customer interaction Score is static and does not change based on new customer interactions Score is only as current as the ETL/batch copy/model schedule High Networking Traffic As volume increases so do the costs SLA’s = Real-time Scoring needs to provide the same levels of service as the OLTP application, as our performance, availability, scalability requirements grow as do our costs & complexity, with the needs for more hardware and software to support demand. Applications may be built off the System z platform , increasing the cost & complexity of scoring © 2012 IBM Corporation Predictive SPSS Modeler for Linux on System z Version: Modeler v15 Data mining tool used for generating hypotheses and scoring Text analysis for unstructured data to model consumer behavior In-Transaction Scoring with DB2 z/OS: Embeds the Scoring Algorithm Directly within the Transactional Application IBM SPSS Modeler 14 IBM SPSS Statistics 19 Ad-hoc analysis, IBM SPSS Decision Management A decision process framework that uses predictive analytics to optimize the outcomes of the thousands of operational decisions IBM SPSS Collaboration and Deployment Services A platform for the management and deployment of analytical assets. It supports predictive models as they are prepared, validated and integrated with operational systems. © 2012 IBM Corporation Customer Interaction Reduced Networking in-database scoring on the same platform Consolidates Resources Scoring Algorithm Provide the same qualities of service as the business system Improve the accuracy of our decision because we are scoring the most current data Can deliver both in-transaction and in-database scoring on the same platform for greater economies of scale & expertise Potential to reduce ETL requirements Co-locate OLTP, DW and Modeler, reducing the copy/movement Reducing the network band width required for scoring because all functionality is internal to System z Improves the speed and accuracy of building & repeating models With System z’s proven performance we can define new patterns faster & more frequently Capacity to scale to build models off larger data sets Consolidates resources Reduce floor space, heating, cooling required We can further accelerate IBM DB2® queries with in-memory processing, massively parallel architecture, row and columnar store technologies, highly compressed data and compressed data operations © 2012 IBM Corporation Meets most demanding workload Meets stringent SLA requirement Provides best value Most accurate score is calculated in real time Can you really scale to support the volume of transactions a OLTP application processes without impacting performance? Workload runs for 14 minutes At 8 CP, windows server to small to driver sufficient workload HS = Historical Score RT = Real Time Score Workload is single select that gathers input data for the scoring model, executes that model and returns inputs and scoring results. Typical application will include more logic than this. SELECT T0.C0 AS C0, T0.C1 AS C1, T0.C2 AS C2, T0.C3 AS C3, T0.C4 AS C4, T0.C5 AS C5, UNPACK( HUMSPSS.SCORE_COMPONENT('H','ce815fe5b47e83cc548b65dcf5785eb5', FROM ( SELECT T0.SALARY AS C3,T0."TRANS_SPEND_1" AS C4, T0."TRANS_POINTS_1" AS C5 FROM "data_1_by_20" T0) Dramatically different than prior releases We keep innovations coming Existing customers can modernize and consolidate their BA environments and extend their current QMF investment at low cost Workstation and WebSphere environments let you create and deploy visually rich reports, interactive dashboards, and much more TCO is based on an enterprise-wide model. The greater the number of users and databases accessed, the lower the cost IBM DB2 Analytics Accelerator creates significant performance improvements and savings for QMF Faster responses to queries Act faster on results, dig deeper, generate personalized reports and dashboards Make more timely, informed decisions Provides a lower cost option for massive batch operations, one of QMF’s known strengths What’s new? New QMF redbook: Complete Analytics with IBM DB2 Query Management Facility http://www.redbooks.ibm.com/redbooks/pdfs/sg248012.pdf Join our next beta program! Ask your IBM representative to visit the following web page for information: * 75-90% less usage in DB2 10 compared to DB2 9 Some of working storage (stack, xproc storage) stays below 2GB Larger number of threads Improve CPU with storage DB2 10 Thread / Stack/ working Laboratory measurements and early customer experience have shown substantial savings in the primary constrained address space, DBM1. Most measurements have shown 75% to 90% savings for the virtual storage in that address space below the 2 GB bar. Some EDMPOOL and some working storage remains below the bar. This storage relief allows many more threads or concurrent users in a DB2 subsystem, allowing new possibilities for optimization. Some customers will be able to consolidate data sharing members, saving on memory, CPU and administration time. Other customers will be able to use the storage to improve service or to reduce CPU time more. Some common examples are expected to be use of RELEASE(DEALLOCATE) and larger amounts of dynamic statement cache. © 2012 IBM Corporation DB2 10 Performance Most customers can see a 5% - 10% out-of-the-box CPU reduction (transactions and batch) after rebind Some workloads and customer situations can see a CPU reduction of up to 20% Synergistic operation with latest System z hardware Sample: Preliminary Measurements of IBM Relational Warehouse Workload (IRWW) with Data Sharing Base: DB2 9 NFM REBIND with PLANMGMT EXTENDED DB2 9 NFM DB2 10 CM without REBIND showed 1.3% CPU reduction DB2 10 CM REBIND with same access path showed 4.8% CPU reduction DB2 10 NFM brought 5.1% CPU reduction DB2 10 CM or NFM with RELEASE DEALLOCATE 12.6% CPU reduction from DB2 9 CPU Percent reduced from DB2 9 © 2012 IBM Corporation Avoids processing to go inactive and then back to active Enabling High Performance DBAT, e.g. for WebSphere BIND client packages into different collection coll2 with RELEASE(DEALLOCATE) BIND other frequently executed packages with RELEASE(DEALLOCATE) Set -MODIFY DDF PKGREL(BNDOPT) to enable In WAS datasource property definition point to new collection E.g. jdbcCollection=coll2 © 2012 IBM Corporation Easier scaling, simpler memory management Reduce contention, more online processing Reduced need for REORG Index list prefetch enhancements Statement level monitoring DDF thread management enhancements REORG Customers appreciate the improved performance and scalability. Virtual storage management almost disappears. DB2 has a strong focus on making DB2 easier to use by automating tasks and eliminating tasks where possible. Avoiding the manual invocations can also help avoid problems for running the function too often or not often enough. Where the task cannot be eliminated, the frequency and monitoring can be reduced, such as the need to reorganize. The improvements for virtual storage and for availability also help administrator productivity. Some of the improvements come within DB2 for z/OS. Improvements in SQL and XML improve productivity for those who develop new applications and for those who are porting from other platforms. Some of the improvements remove complexity from application tasks. Some of the improvements come with Data Studio for application programming and administration – stronger cross-platform graphical interfaces, better integration with Java, improvements in the ability to develop and debug. Allowing tailored names for DSNHDECP will permit many subsystems to share the SDSNEXIT data set. © 2012 IBM Corporation Problem: For distributed workloads, low priority or poorly behaving client applications may monopolize DB2 resources and prevent high-priority applications from executing. Solution: Increased granularity of monitoring for system level activities Number of connections Number of threads Idle thread timeout Profiles specified in SYSIBM.DSN_PROFILE_TABLE DB2 10 for z/OS adds support for filtering and threshold monitoring of system related activities via new keywords Number of threads Number of connections Idle thread timeout New scope filters Product-specific identifier © 2012 IBM Corporation Page size Hashed DSSIZE Improve availability and productivity Increase maximum size substantially DDL concurrency also improved from removal of DBD01 hash anchor locks Many more table spaces, partition by growth Row level locking, reordered row format CLOB and BLOB columns for long strings Inline for performance Online reorganization and check More automatic: DB2-managed SMS-controlled Allow query of SYSLGRNX * The DB2 catalog and directory are restructured in DB2 10 ENFM to improve productivity and availability. You’ll see these improvements in NFM. The current size limits are increased substantially and contention among process like BIND, dynamic SQL, data definition and utilities is reduced. With more table spaces and more structures, more work is required for some process, such as BIND. The primary techniques are changes in the DB2 catalog to remove links and the special structures for the catalog. These table spaces change from many tables to one table per table space in a partition by growth table space defined as DSSIZE 64 GB and MAXPART 1. Row level locking is used in place of page level locking. The new catalog tables use a partition by growth universal table space structure. Each table space holds a single table, so many more table spaces are needed. Rather than repeating columns with parts of long strings, the catalog will use CLOB and BLOB columns to store the data, expanding maximum sizes. Inline LOBs are used for the performance improvements. The new structure allows more standard processes, so that all catalog tables can be reorganized and checked online. The DB2 catalog changes from using manual definition and extension to DB2 managed data sets under SMS control. The changes improve productivity and availability, but take time to set up. © 2012 IBM Corporation Protect sensitive data from privileged users and improve productivity Separation of duties More granular control New SECADM authority for security administration System level DB admin with/without access control and with/without data access privileges Usability: DBADM for all DB New Explain privilege Audit admin authority usage Support distributed identities introduced in z/OS 1.11 Support client certificate authentication in z/OS 1.10 Support password phrases in z/OS 1.10 Connection level security enforcement using strong authentication Row and column access control Allow masking of value Monitor Tasks Customers are being pressed for a wide range of improved security and compliance. Data retention is a growing need. Protecting sensitive data from the privileged users and administrators is required. Separation of authority for security, access, and some common tasks, like EXPLAIN will help. Auditing for privileged users can also make compliance simpler. Access control is refined in several ways with better granularity for the administrative privileges and with finer grained access control at the row and column level, including the ability to mask access to some fields. Auditing policies: new IFCID for auditing admin authority usage. Dynamic table auditing (all access, stmt id level). Granular admin authorities, separation of duties: sysadm: system level dbadm with/without accessctrl, with/without dataaccess. System level SECADM (separation of dutes, shift from dba to security group, zparm control to remove privileges from sysadm). System level SQLADM. App level explain privileges. Distributed identities: supported in v9 via EIM, but not much used due to complexity. V10 allows for use of new RACF feature to map distributed identities passed in from app server to a RACF id. Ap server passes authenticated distributed end user identify using trusted context. Client certificate: Eliminates need to manage RACF ids on DB2 clients. Used to be supported only via trusted context, but now also outside trusted context. Connection level security enforcement: Zparm TCPLVER value SERVER_ENCRYPT © 2012 IBM Corporation Temporal Based Analysis System-maintained temporal tables DB2 generated history AS OF query User-maintained temporal tables Automatic business time key enforcement. Query over any current, any prior, future point/period in business time. New time range update/delete statements support automatic row splitting, exploited by the merge statements. Bi-temporal, combination of the above two © 2012 IBM Corporation System Time Query SELECT Dept FROM employees SELECT Dept WHERE EmpID=12345 SELECT count(distinct Dept) WHERE EmpID=12345 EmpID Dept System_start System_end 12345 M15 05/31/2000 12/31/9999 EmpID Dept System_start System_end 12345 J13 11/15/1995 01/31/1998 12345 M24 01/31/1998 05/31/2000 67890 K25 11/15/1995 03/31/2000 DB2 Family Row and Column Access Control A table level authorization function fully implemented in the database that provides: Row level access control based on customer-supplied rules A doctor can see rows representing his patients only A manager can see rows representing his employees only Column level access control based on customer-supplied rules This is also called data masking A teller can see only the last 4 digits of the credit card number column Defined and managed by SECADM © 2012 IBM Corporation Prerequisites: migrate from DB2 9 NFM or DB2 V8 NFM z/OS V1.10 SMS-controlled DB2-managed DB2 catalog System z z196, z10, z9, z890, z990, and above (no z800, z900) DB2 Connect 9 FP1, 9.7 FP3a for 10 new function Premigration check DSNTIJPA PM04968 Items deprecated in earlier versions eliminated: more for V8 mig. Private protocol DRDA Plans containing DBRMs packages * Here are a couple of thoughts about what might be required in hardware and software to run DB2 10. Much will depend upon the timing of the deliveries and market acceptance. Moving forward as quickly as possible means that some of the past must be left behind. See the list of deprecated functions from prior versions. The above functions are still included in DB2 9, but are generally deprecated and may be removed from future versions. Note the direction indicated to the right of the arrows, as these are the functions provided to replace the existing function. If you are using any of these functions, you are advised to move to the new function. See the Installation Guide section, “Functions that are deprecated” and the announcement material for more information on these changes. http://www.ibm.com/common/ssi/rep_ca/8/897/ENUS206-098/ENUS206-098.PDF Embedded xQuery support – PM47618 Needed for SPSS in-database analytics Stored procedure monitoring improvements – PM53243 More easily identify a problematic stored procedure or a statement within that stored procedure New zparm for OPTIMIZE FOR 1 ROW to allow sort access plans – PM56845 V10 migration issue for some customers GBP Delete Name enhancement – PM67544 Performance enhancement for sysplexes that span longer distances © 2012 IBM Corporation New DEL_CFSTRUCTS_ON_RESTART ZPARM for auto-delete of CF structures on restart if no active connections exist Useful for DR environments ALTER MAXROWS to set AREO* rather than AREOR PM43597 (V10) Ability to SELECT from SYSLGRNX PM35190 & PM42331 (V10) PM31641 (V10) 5X increase in recommended limit on concurrent RECOVER jobs per DB2 LOB pageset support for RECOVER BACKOUT YES PM45650 (V10) Latest z servers can support TB main memory sizes A single DB2 member can currently support up to 1 TB of BP space DB2 10 has improvements for large BP scalability/performance 1MB page frame support “In Memory” BP attribute Avoid BP scans for data sharing TS/part/index inter-DB2 interest level changes Internal BP hashing and latching improvements VSCR improvements for 5-10x more active threads per DB2 member/subsys Opportunities to tradeoff increased memory usage for reduced CPU DB2 10 new RTS column to indicate SSD usage New zEC12 machine introduces flash memory Future: continue to leverage large main memory and automatic storage hierarchy management for performance advantages © 2012 IBM Corporation Multi core, future slowing growth in single thread performance Higher n-ways, more parallelism bring potential latching bottlenecks,…