Service Overview - HUAWEI CLOUD · Teradata. Applications can be smoothly migrated to DWS with few changes. Interface Applications can connect to DWS through the standard JDBC 4.0

Data Warehouse Service

Service Overview

Issue 04

Date 2020-04-13

HUAWEI TECHNOLOGIES CO., LTD.

Copyright © Huawei Technologies Co., Ltd. 2020. All rights reserved.

No part of this document may be reproduced or transmitted in any form or by any means without priorwritten consent of Huawei Technologies Co., Ltd. Trademarks and Permissions

and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd.All other trademarks and trade names mentioned in this document are the property of their respectiveholders. NoticeThe purchased products, services and features are stipulated by the contract made between Huawei andthe customer. All or part of the products, services and features described in this document may not bewithin the purchase scope or the usage scope. Unless otherwise specified in the contract, all statements,information, and recommendations in this document are provided "AS IS" without warranties, guaranteesor representations of any kind, either express or implied.

The information in this document is subject to change without notice. Every effort has been made in thepreparation of this document to ensure accuracy of the contents, but all statements, information, andrecommendations in this document do not constitute a warranty of any kind, express or implied.

Issue 04 (2020-04-13) Copyright © Huawei Technologies Co., Ltd. i

Contents

1 What Is DWS?...........................................................................................................................1

2 Advantages............................................................................................................................... 3

3 Application Scenarios............................................................................................................. 5

4 Functions................................................................................................................................... 9

5 Concepts.................................................................................................................................. 135.1 DWS Management Concepts ...........................................................................................................................................135.2 DWS Database Concepts.................................................................................................................................................... 13

6 Related Services.....................................................................................................................15

7 DWS Permissions Management......................................................................................... 17

8 DWS Access............................................................................................................................. 22

9 Pricing Details........................................................................................................................ 24

10 Restrictions........................................................................................................................... 27

11 Service Quota....................................................................................................................... 28

12 Version Description.............................................................................................................29

Data Warehouse ServiceService Overview Contents

Issue 04 (2020-04-13) Copyright © Huawei Technologies Co., Ltd. ii

1 What Is DWS?

Data Warehouse Service (DWS) is an online data processing database that usesthe public cloud infrastructure to provide scalable, fully-managed, and out-of-the-box analytic database service that frees you from database management andmonitoring. It is a native cloud service based on the Huawei converged datawarehouse GaussDB, and is fully compatible with the standard ANSI SQL 99 andSQL 2003, as well as the PostgreSQL and Oracle ecosystems. DWS providescompetitive solutions for PB-level big data analysis in various industries.

DWS is widely used in domains such as finance, Internet of Vehicles (IoV),government and enterprise, e-commerce, energy, and telecom. It has been listed inthe Gartner Magic Quadrant for Data Management Solutions for Analytics for twoconsecutive years. Compared with conventional data warehouses, DWS is morecost-effective and has large-scale scalability and enterprise-level reliability.

ArchitectureDWS employs the shared-nothing architecture and massively parallel processing(MPP) engine and consists of numerous independent logical nodes that do notshare system resources, such as CPUs, memory, and storage. In such a systemarchitecture, service data is separately stored on numerous nodes. Data analysistasks are executed at the location nearest the data. Massively parallel dataprocessing enables quick response.

Data Warehouse ServiceService Overview 1 What Is DWS?

Issue 04 (2020-04-13) Copyright © Huawei Technologies Co., Ltd. 1

Figure 1-1 Architecture

● Application layerData loading tools, Extract-Transform-Load (ETL) tools, Business Intelligence(BI) tools, and data mining and analysis tools can be integrated with DWSthrough standard interfaces. DWS is compatible with PostgreSQL. The SQLsyntax has been modified to make it compatible with Oracle, MySQL, andTeradata. Applications can be smoothly migrated to DWS with few changes.

● InterfaceApplications can connect to DWS through the standard JDBC 4.0 and ODBC3.5.

● DWS (MPP cluster)A data warehouse cluster contains nodes with the same flavor in the samesubnet. These nodes jointly provide services. DataNodes (DNs) in a clusterstore data on disks. The coordinator node (CN) receives access requests fromthe client and returns the execution results back. Then, the CN divides eachtask into several smaller ones and assigns them to the DNs for execution.

● Automatic data backupCluster snapshots can be automatically backed up to Object Storage Service(OBS), an EB-level object storage service, which facilitates periodic backup ofthe cluster during off-peak hours, ensuring data recovery after exceptionsoccur.A snapshot is a complete backup that records point-in-time configurationdata and service data of a data warehouse cluster.

● Tool chainDWS provides the parallel data loading tool General Data Service (GDS), SQLsyntax migration tool DSC, and SQL development tool Data Studio. You canuse the management console for cluster O&M and monitoring.

Data Warehouse ServiceService Overview 1 What Is DWS?


2 Advantages

DWS uses the Huawei self-developed GaussDB database kernel and is compatiblewith PostgreSQL 9.2.4. GaussDB 200 is transformed from a single OLTP databaseto an enterprise-class, MPP-based, distributed OLAP database oriented to massivedata analysis.

Compared with conventional data warehouses, DWS excels in hyper-scale dataprocessing and general platform management and delivers the following benefits:

Easy to Use

● Visualized one-stop managementDWS allows you to easily complete the entire process from project concept toproduction deployment. With the DWS management console, you do notneed to install the data warehouse software or deploy the data warehouseserver. On the contrary, you can obtain a high-performance and highlyavailable enterprise-class data warehouse cluster within several minutes.With just a few clicks, you can easily connect applications to the datawarehouse, back up data, restore data, and monitor data warehouse resourcesand performance.

● Seamless integration with big dataYou can use standard SQL statements to query data on HDFS and OBSwithout data migration.

● Heterogeneous database migration toolsDWS provides various migration tools to migrate SQL scripts of Oracle,MySQL, and Teradata to DWS.

High Performance

● Cloud-based distributed architectureDWS adopts the fully parallel MPP database so that service data is separatelystored on numerous nodes. Data analysis tasks are executed at the locationnearest the data. Massively parallel data processing enables quick response.

● Response to querying of trillions of data records within secondsDWS improves data query performance by executing multi-thread operatorsin parallel, running commands in registers in parallel with the vectorizedcomputing engine, and using LLVM to dynamically compile the logicalconditions of redundancy prune.

Data Warehouse ServiceService Overview 2 Advantages


DWS supports hybrid row-column storage, providing users with a better datacompression ratio (column-based storage), higher index performance(column-based storage), and better point update and query (row-basedstorage) performance.

● Fast data loadingDWS provides you with GDS, a high-speed parallel bulk data loading tool.

High Scalability

● On-demand scale-out: The shared-nothing open architecture of DWS allowsyou to add nodes at any time based on service requirements to expand thesystem's data storage capability and query and analysis performance.

● Linear performance improvement after scale-out: The capacity andperformance increase linearly with the cluster scale. The linear ratio is 0.8.

● Service continuity during scale-out: During the scale-out, data can be added,deleted, modified, and queried, and DDL operations can be performed (Drop/Truncate/Alter tables). With the table-level online scale-out technology,service continuity can be better ensured.

High Reliability

● ACIDSupport for distributed transaction Atomicity, Consistency, Isolation, Durability(ACID), ensuring data consistency.

● Comprehensive HA designAll software processes of DWS are in active/standby mode. Logicalcomponents such as the CNs and DNs of each cluster also work in active/standby mode. This ensures data reliability and consistency when any singlepoint of failure (SPOF) occurs.

● High securityDWS supports transparent data encryption and can interconnect with theDatabase Security Service (DBSS) to better protect user privacy and datasecurity with network isolation and security group rule setting options. Inaddition, DWS supports automatic full and incremental backup of data,improving data reliability.

Low Cost

● Pay per use: DWS is billed based on the usage and use duration. You onlyneed to pay for the resources you use.

● Low requirements: You do not need to invest in fixed infrastructure in theearly stage. You can start from a data warehouse instance with lowspecifications and flexibly scale it up and down based on service conditions atany time.

Data Warehouse ServiceService Overview 2 Advantages


3 Application Scenarios

Data Warehouse MigrationThe data warehouse is an important data analysis system for enterprises. As theservice volume grows, performance of self-built data warehouses cannot meet theactual service requirements due to their poor scalability and high costs. As anenterprise-class data warehouse on the cloud, DWS features high performance,low cost, and easy scalability, satisfying requirements in the big data era.

Figure 3-1 Data warehouse migration

Advantages

● Seamless MigrationDWS provides various migration tools to ensure seamless migration ofpopular data analysis systems such as Teradata, Oracle, MySQL, SQL Server,PostgreSQL, Greenplum, and Impala.

● Compatibilities with Traditional Data Warehouses

Data Warehouse ServiceService Overview 3 Application Scenarios


DWS supports the SQL 2003 standard and stored procedures. It is compatiblewith some Oracle syntax and data structures, and can be seamlesslyinterconnected with common BI tools, saving service migration efforts.

● High Security and ReliabilityDWS supports data encryption and connects to DBSS to ensure data securityon the cloud. In addition, DWS supports automatic full and incrementalbackup of data, improving data reliability.

Converged Big Data Analysis

Data has become the most important asset. Enterprises must be able to integratetheir data resources and build big data platforms to mine the full value of theirdata. In predictive analysis use cases, massive volumes of data must be processed.Huawei DWS delivers the needed processing power to handle these intensecompute scenarios.

Figure 3-2 Converged big data analysis

Advantages

● Unified Analysis EntranceThe DWS SQL serves as the unified entry of upper-layer applications, so thatapplication developers can access all data using the SQL.

● Real-Time Interactive AnalysisAnalysis personnel can obtain immediately-actionable information from thebig data platform in real time using broad analysis requests.

● Auto ScalingAdding nodes allows you to easily expand into PB-range capacity whileenhancing query and analysis performance of the system.



Enhanced ETL + Real-Time BI AnalysisThe data warehouse is the pillar of the BI system for collecting, storing, andanalyzing massive volumes of data. It powers business decision analysis for theIoT, finance, education, mobile Internet, and Online to Offline (O2O) industries.

Advantages

● Data MigrationAbility to import data in batches in real time from multiple data sources.

● High PerformanceCost-effective PB-level data storage and response to correlation analysis oftrillions of data records within seconds.

● Real-TimeReal-time consolidation of service data to produce actionable insights inoperational decision-making.

Figure 3-3 Enhanced ETL + real-time BI analysis

Real-Time Data AnalysisIn the mobile Internet and IoT domains, huge volumes of data must be processedand analyzed in real time to extract the full data from data. The quick data importand query capabilities of DWS accelerate data analysis to enable real-timeingestion, processing, and value generation.



Figure 3-4 Real-time data analysis

Advantages

● Real-Time Import of Streaming DataData from IoT and Internet applications can be written into DWS in real timeafter being processed by the stream computing and AI services.

● Real-Time Monitoring and PredictionDevice monitoring, control, optimization, supply, self-diagnosis, and self-healing based on data analysis and prediction.

● Converged AI AnalysisCorrelation analysis can be conducted on results of image and text dataanalysis by AI services and other service data on DWS.



4 Functions

DWS enables you to use this service through various methods, such as the DWSmanagement console, DWS client, and REST APIs. This section describes the mainfunctions of DWS.

Cluster Management

A data warehouse cluster contains nodes with the same flavor in the same subnet.These nodes jointly provide services. DWS provides a professional, efficient, andcentralized management console, allowing you to quickly apply for clusters, easilymanage data warehouses, and focus on data and services.

Main functions of cluster management are described as follows:

● Creating clusters

You can specify the node flavor and node quantity based on servicerequirements to quickly create a cluster You can also purchase a monthly/yearly package to create a cluster.

● Managing snapshots

A snapshot is a complete backup that records point-in-time configurationdata and service data of a data warehouse cluster. A snapshot can be used torestore a cluster at a certain time. You can manually create snapshots for acluster or enable automatic snapshot creation (periodic). Automatic snapshotshave a limited retention period. You can copy automatic snapshots togenerate manual snapshots for long-term retention.

When you restore a cluster from a snapshot, the system creates a new clusterwith the same flavor and node quantity as the original one, and imports thesnapshot data.

You can delete snapshots that are no longer needed to release the storagespace.

● Scaling out clusters

As the service volume increases, the current scale of a cluster may not meetservice requirements. In this case, you can scale out the cluster by addingcompute nodes to it. Services are not interrupted during the scale-out.

● Restarting clusters

Data Warehouse ServiceService Overview 4 Functions


Restarting a cluster may cause data loss in running services. If you have torestart a cluster, ensure that there is no running service and all data has beensaved.

● Deleting clusters

You can delete a cluster when you do not need it. Deleting a cluster is riskyand may cause data loss. Therefore, exercise caution when performing thisoperation.

DWS allows you to manage clusters and snapshots in the following ways:

● Management console

Use the management console to access data warehouse clusters. After youhave registered a cloud account, log in to the management console andchoose Data Warehouse Service.

For more information about cluster management, see Managing Clusters.

● REST APIs

Use REST APIs provided by DWS to manage clusters. In addition, if you needto integrate DWS into a third-party system for secondary development, useAPIs to access the service.

For details, see the Data Warehouse Service API Reference.

Enterprise-Class Data Warehouses and Compatibility with Standard SQL

After a data warehouse cluster is created, you can use the SQL client to connect tothe cluster and perform operations such as creating a database, managing thedatabase, importing and exporting data, and querying data.

DWS provides petabyte-level (PB-level) high-performance databases with thefollowing features:

● MPP computing framework, hybrid row-column storage, and vectorizedexecution, enabling response to billion-level data correlation analysis withinseconds

● Optimized in-memory computing based on Hash Join of Bloom Filter,improving the performance by 2 to 10 times

● Optimized communication between large-scale clusters based ontelecommunication technologies, improving data transmission efficiencybetween compute nodes

● Cost-based intelligent optimizers, helping generate the optimal plan based onthe cluster scale and data volume to improve execution efficiency

DWS has comprehensive SQL capabilities:

● Supports SQL 92 and SQL 2003 standards, stored procedures, GBK and UTF-8character sets, and SQL standard functions and OLAP analysis functions.

● Compatible with the PostgreSQL ecosystem and supports interconnection withmainstream database ETL and BI tools provided by third-party vendors.

For details about the SQL syntax and database operation guidance, see the DataWarehouse Service Database Developer Guide.



https://support.huaweicloud.com/en-us/mgtg-dws/dws_01_0018.html

https://support.huaweicloud.com/en-us/api-dws/dws_02_0033.html

https://support.huaweicloud.com/en-us/devg-dws/preface.html

https://support.huaweicloud.com/en-us/devg-dws/preface.html

Diverse Data Import Modes

DWS supports efficient data import from multiple data sources. The following liststypical data import modes. For details, see Import Modes in the Data WarehouseService Database Developer Guide.

● Concurrently Importing Data from OBS● Using GDS to Import Data from a Remote Server● Running the INSERT Statement to Insert Data● Running the COPY FROM STDIN Statement to Import Data● Using a gsql Meta-Command to Import Data● Importing Data from MRS to a Data Warehouse Cluster● Importing Stream Data from DIS to DWS● Using CDM to Migrate Data to DWS● Using Database Schema Convertor (DSC) to Migrate SQL Scripts

In addition, DWS supports data import using mainstream third-party ETL tools.

Application Programming Interfaces

You can call standard interfaces, such as Java Database Connectivity (JDBC), OpenDatabase Connectivity (ODBC), Python, and third-party psycopg2 to accessdatabases in clusters.

For details, see Using a JDBC Driver to Connect to the Database, Using anODBC Driver to Connect to the Database, and Using the Third-Party FunctionLibrary psycopg2 of Python to Connect to a Cluster.

High Reliability● Supports instance and data redundancy, ensuring zero single points of failure

(SPOF) in the entire system.● Supports multiple data backups, and all data can be manually backed up to

OBS.● Automatically isolates the faulty node, uses the backup to restore data, and

replaces the faulty node when necessary.● Combines automatic snapshot creation and OBS storage, implementing cross-

AZ disaster recovery (DR).

Security Management● Isolates tenants and controls access permissions to protect the privacy and

data security of systems and users based on the network isolation andsecurity group rules, as well as security hardening measures.

● Supports SSL network connections, user permission management, andpassword management, ensuring data security at the network, management,application, and system layers.For details, see Configuring SSL Connection and Separating Rights ofRoles.



https://support.huaweicloud.com/en-us/devg-dws/dws_04_0909.html


https://support.huaweicloud.com/en-us/mgtg-dws/en-us_topic_0056218063.html







Monitoring and Auditing● Monitoring Clusters

DWS integrates with Cloud Eye, allowing you to monitor compute nodes anddatabases in the cluster in real time. For details, see Monitoring a Cluster.

● Notifying Events

DWS interconnects with Simple Message Notification (SMN) so that you cansubscribe to events and view events that are triggered. For details, seeNotifying Events.

● Audit Logs

– DWS integrates with Cloud Trace Service (CTS), allowing you to auditoperations performed on the management console and API invocationoperations. For details, see Viewing Audit Logs of Key Operations onthe Management Console.

– DWS records all SQL operations, including connection attempts, queryattempts, and database changes. For details, see Configuring theDatabase Audit Log.

Multiple Database Tools

DWS provides the following self-developed tools. You can download the toolpackages on the DWS management console. For details about how to use thetools, see the Data Warehouse Service Tool Guide.

● gsql tool

gsql is a command line SQL client tool running on the Linux operating system.It helps connect to, operate, and maintain the database in a data warehousecluster.

● Data Studio tool

Data Studio is a Graphical User Interface (GUI) SQL client tool running on theWindows operating system. It is used to connect to the database in a datawarehouse cluster, manage the database and database objects, edit, run, anddebug SQL scripts, and view the execution plans.

● GDS tool

GDS is a data service tool provided by DWS. It works with the foreign tablemechanism to implement high-speed data import and export.

The GDS tool package needs to be installed on the server where the datasource file is located. This server is called the data server or the GDS server.

● DSC SQL syntax migration tool

The DSC is a command-line tool running on the Linux or Windows OS. It isdedicated to providing customers with simple, fast, reliable application SQLscript migration services. It parses SQL scripts of source database applicationsby using the built-in syntax migration logic, and migrates them to beapplicable to DWS databases.

The DSC can migrate SQL scripts of Teradata, Oracle, Netezza, MySQL, andDB2 databases.









https://support.huaweicloud.com/en-us/tg-dws/dws_07_0001.html

5 Concepts

5.1 DWS Management Concepts

Cluster

A cluster is a server group that consists of multiple nodes. DWS is organized usingclusters. A data warehouse cluster contains nodes with the same flavor in thesame subnet. These nodes work together to provide services.

Node

Each data warehouse cluster contains at least three nodes, all of which supportdata storage and analysis.

Flavor

You need to specify the node flavors when you create a data warehouse cluster.CPU, memory, and storage resources vary depending on node flavors.

Snapshot

You can create a snapshot to back up data warehouse cluster data so as to restorethe cluster data from the snapshot. A snapshot is retained until you delete it onthe management console. Automatic snapshots cannot be manually deleted.Snapshots will occupy your OBS quotas.

5.2 DWS Database Concepts

Database

A data warehouse cluster is an analysis-oriented relational database platform thatsupports online analysis.

Data Warehouse ServiceService Overview 5 Concepts


OLAPOLAP is a major function of data warehouse clusters. It supports complex analysis,provides decision-making support tailored to analysis results, and delivers intuitivequery results.

MPPOn each node in the data warehouse cluster, memory computing and disk storagesystems are independent from each other. With MPP, DWS distributes service datato different nodes based on the database model and application characteristics.Nodes are connected through the network and collaboratively process computingtasks as a cluster and provide database services that meet service needs.

Shared-Nothing ArchitectureThe shared-nothing architecture is a distributed computing architecture. Eachnode is independent so that nodes do not compete for resources, which improveswork efficiency.

Database VersionEach data warehouse cluster has a specific database version. You can check theversion when creating a data warehouse cluster.

Database ConnectionYou can use a client to connect to a data warehouse cluster on the public cloudand over the Internet.

Database UserYou can add and control users who can access the database of a data warehousecluster by assigning specific permissions to them. The database administratorgenerated when you create a cluster is the default database user.

Data Warehouse ServiceService Overview 5 Concepts


6 Related Services

IAM

DWS uses Identity and Access Management (IAM) for authentication andauthorization.

Users who have the DWS Administrator permissions can fully utilize DWS. Toobtain the permissions, contact a user with the Security Administratorpermissions or directly create a user with the DWS Administrator permissions.Users granted the DWS Database Access permissions can generate temporarydatabase user credentials based on IAM users to connect to databases in the datawarehouse clusters.

ECS

DWS uses an Elastic Cloud Server (ECS) as a node in the cluster.

VPC

DWS uses the Virtual Private Cloud (VPC) to provide a network topology forclusters to isolate clusters and control access.

OBS

DWS uses OBS to convert cluster data and external data, satisfying therequirements for secure, reliable, and cost-effective storage.

MRS

DWS uses MRS to migrate data from MRS to data warehouse clusters so that DWScan analyze and query the data after the data is processed by Hadoop.

CDM

DWS uses CDM to migrate data from multiple sources to DWS.

Data Warehouse ServiceService Overview 6 Related Services


DISYou can use the Data Ingestion Service (DIS) to ingest stream data to DWS in realtime.

Cloud EyeDWS uses Cloud Eye to monitor cluster performance metrics, delivering statusinformation in a concise and efficient manner. Cloud Eye supports alarmcustomization so that you are notified of the exception instantly.

CTSDWS uses CTS to audit your non-query operations on the management console toensure that no invalid or unauthorized operations are performed, enhancingservice security management.

SMNDWS uses SMN to actively push notification messages according to your eventsubscription requirements, so that you can immediately receive a notificationwhen an event occurs (for example, a key cluster operation).

TMSWith TMS, DWS can provide cross-region and cross-service centralized tagmanagement and resource classification functions, allowing you to customize tagsfor resources and implement resource classification, facilitating search forresources using tags.

DNSDWS uses the Domain Name Service (DNS) to provide data warehouse cluster IPaddresses mapped from domain names.

Data Warehouse ServiceService Overview 6 Related Services


7 DWS Permissions Management

If you need to assign different permissions to employees in your enterprise toaccess your DWS resources, IAM is a good choice for fine-grained permissionsmanagement. IAM provides identity authentication, permissions management,and access control, helping you secure access to your HUAWEI CLOUD resources.

With IAM, you can use your HUAWEI CLOUD account to create IAM users for youremployees, and assign permissions to the users to control their access to specificresource types For example, some software developers in your enterprise need touse DWS resources but must not delete them or perform any high-risk operations.To achieve this result, you can create IAM users for the software developers andgrant them only the permissions required for using DWS resources.

If your HUAWEI CLOUD account does not need individual IAM users forpermissions management, you may skip over this chapter.

IAM can be used free of charge. You pay only for the resources in your account.For more information about IAM, see IAM Service Overview.

Supported System Policies

By default, new IAM users do not have any permissions assigned. You need to adda user to one or more groups, and assign permissions policies or roles to thesegroups. The user then inherits permissions from the groups it is a member of. Thisprocess is called authorization. After authorization, the user can perform specifiedoperations on DWS based on the permissions.

DWS is a project-level service deployed and accessed in specific physical regions.To assign DWS permissions to a user group, specify the scope as region-specificprojects and select projects (such as cn-north-1 in CN North-Beijing1) for thepermissions to take effect. If All projects is selected, the permissions will takeeffect for the user group in all region-specific projects. When accessing DWS, theusers need to switch to a region where they have been authorized to use DWS.

● Role: IAM initially provides a coarse-grained authorization mechanism todefine permissions based on users' job responsibilities. This mechanismprovides only a limited number of service-level roles for authorization. Whenusing roles to grant permissions, you need to also assign other roles on whichthe permissions depend to take effect. However, roles are not an ideal choicefor fine-grained authorization and secure access control.

Data Warehouse ServiceService Overview 7 DWS Permissions Management


https://support.huaweicloud.com/en-us/productdesc-iam/iam_01_0026.html

● Policies: A type of fine-grained authorization mechanism that definespermissions required to perform operations on specific cloud resources undercertain conditions. This mechanism allows for more flexible policy-basedauthorization, meeting requirements for secure access control. For example,you can grant DWS users only the permissions for managing a certain type ofDWS resources.Most policies define permissions based on APIs. For the API actions supportedby DWS, see Permissions Policies and Supported Actions.For details about how to create a fine-grained policy, see Creating a DWSCustom Policy.

Table 7-1 lists all the system-defined roles and policies supported by DWS.

Table 7-1 DWS system permissions

Role/PolicyName

Description Category Dependencies

DWSReadOnlyAccess

Read-only permissions for DWS. Usersgranted these permissions can onlyview DWS data.

System-definedpolicy

N/A

DWSFullAccess

Administrator permissions for DWS.Users granted these permissions canperform all operations on DWS.

System-definedpolicy

N/A

DWSAdministrator

Administrator permissions for DWS.Users granted these permissions canoperate all DWS resources.● Users granted permissions of the

VPC Administrator policy cancreate VPCs and subnets.

● Users granted permissions of theCloud Eye Administrator policycan view monitoring information ofdata warehouse clusters.

System-definedrole

Dependent on theTenantGuestandTenantAdministratorpolicies,whichmust beassignedin thesameproject asthe DWSAdministratorpolicy.



https://support.huaweicloud.com/en-us/api-dws/dws_02_0056.html



Role/PolicyName

Description Category Dependencies

DWS DatabaseAccess

Database access permissions for DWS.Users granted these permissions cangenerate temporary database usercredentials based on IAM users toconnect to databases in the datawarehouse clusters.

System-definedrole

Dependent on theDWSAdministratorpolicy,whichmust beassignedin thesameproject asthe DWSDatabase Accesspolicy.

Table 7-2 lists the common operations supported by each system permission ofDWS. Please choose proper system policies according to this table.

Table 7-2 Common operations supported by each system-defined policy or role ofDWS

Operation DWSFullAccess

DWSReadOnlyAccess

DWSAdministrator

DWS DatabaseAccess

Creating/Restoringclusters

√ x √ x

Obtainingthe clusterlist

√ √ √ √

Obtainingthe detailsof a cluster

√ √ √ √

Settingautomatedsnapshotpolicy

√ x √ x

Settingsecurityparameters/parametergroups

√ x √ x




DWSReadOnlyAccess

DWSAdministrator

DWS DatabaseAccess

Restartingclusters

√ x √ x

Resizing acluster

√ x √ x

Resettingpasswords

√ x √ x

Applyingparametertemplatesto clusters

√ x √ x

Deletingclusters

√ x √ x

Configuringmaintenance windows

√ x √ x

Binding EIPs √ x √ x

UnbindingEIPs

√ x √ x

CreatingDNSdomainnames

√ x √ x

ReleasingDNSdomainnames

√ x √ x

ModifyingDNSdomainnames

√ x √ x

CreatingMRSconnections

√ x √ x

UpdatingMRSconnections

√ x √ x

DeletingMRSconnections

√ x √ x




DWSReadOnlyAccess

DWSAdministrator

DWS DatabaseAccess

Adding/Deletingtags

√ x √ x

Editing tags √ x √ x

Creatingsnapshots

√ x √ x

Obtainingthesnapshotlist

√ √ √ √

Deletingsnapshots

√ x √ x

Copyingsnapshots

√ x √ x

Creatingparametertemplates

√ x √ x

Deletingparametertemplates

√ x √ x

Modifyingparametertemplates

√ x √ x

Creatingextendedapplications

√ x √ x

Deletingextendedapplications

√ x √ x

Helpful Links● IAM Service Overview● Creating a User and Granting DWS Permissions● Permissions Policies and Supported Actions



https://support.huaweicloud.com/en-us/productdesc-iam/iam_01_0026.html



8 DWS Access

The following figure shows the process of using DWS.

Figure 8-1 Process of using DWS

Accessing a ClusterDWS provides a web-based management console and HTTPS-compliant APIs foryou to manage data warehouse clusters.

Accessing the Database in a ClusterDWS supports database access using the following methods:● DWS clients

Data Warehouse ServiceService Overview 8 DWS Access


Use the DWS client to access the database in a cluster. For details, see Usingthe gsql Client to Connect to a Cluster and Using the Data Studio GUIClient to Connect to a Cluster.

● Open-source PostgreSQL clientsThe following lists compatible open-source clients:– PostgreSQL psql 9.2.4 or later

For more information, see https://www.postgresql.org/.– pgAdmin

For more information, see https://www.pgadmin.org/.– dbeaver

For more information, see https://dbeaver.jkiss.org/download/.● Calling methods of interfaces such as JDBC and ODBC

You can call standard interfaces, such as Java Database Connectivity (JDBC),Open Database Connectivity (ODBC), Python, and third-party psycopg2 toaccess databases in clusters.For details, see Using a JDBC Driver to Connect to the Database, Using anODBC Driver to Connect to the Database, and Using the Third-PartyFunction Library psycopg2 of Python to Connect to a Cluster.

End-to-End Data Analysis ProcessDWS has been seamlessly integrated with other services on the public cloud,helping you rapidly deploy end-to-end data analysis solutions.

The following figure shows the end-to-end data analysis process. Services in useduring each process are also displayed.

Figure 8-2 End-to-end data analysis process

Data Warehouse ServiceService Overview 8 DWS Access






https://www.postgresql.org/

https://www.pgadmin.org/

https://dbeaver.jkiss.org/download/






9 Pricing Details

Billing ItemsTo use HUAWEI CLOUD DWS, you only need to pay for the data warehouse nodeflavor you choose and corresponding resources you use. For more DWS pricingdetails, see the DWS Pricing Details. You can use the price calculator of DWS toquickly obtain an estimate price of a cluster with the flavor you choose.

Table 9-1 DWS billing items

Billing Item Description

Datawarehousenode

You pay for the node flavor you choose. For the pricing details,see DWS Pricing Details. Two billing modes are available fornode flavors:● Pay per use (hourly)● Yearly/Monthly packageFor details about the preceding billing modes, see BillingModes.

Snapshotstorage space

DWS provides some free-of-charge storage space for you tostore the snapshot data. However, if you use more space thanthe free-of-charge storage space, the excess part is chargedbased on the OBS billing rule. For details, see OBS PricingDetails.The free-of-charge space is the same as the size of the totalstorage space of your cluster. (Free-of-charge space = Storagespace of a single node x Number of nodes)

Volume ofdata scannedby the Expresscluster

To use the Express function, you will be billed by the numberof bytes scanned by Express on a pay-per-use basis. You donot need to pay for the DDL statements (such as CREATE/ALTER/DROP TABLE), partition management statements, andfailed queries. To learn more details, see the DWS PricingDetails.

Data Warehouse ServiceService Overview 9 Pricing Details


https://www.huaweicloud.com/en-us/pricing.html?tab=detail#/dws


https://www.huaweicloud.com/en-us/pricing.html?tab=detail#/obs

https://www.huaweicloud.com/en-us/pricing.html?tab=detail#/obs



Billing Item Description

(Optional) EIPand bandwidth

● You can bind an elastic IP address (EIP) to a datawarehouse cluster, and must pay for the fee based on theEIP billing rule of the Virtual Private Cloud (VPC) service.For more pricing details, see Elastic IP Pricing Details.

● You are not billed for the traffic generated by the DWScluster on the HUAWEI CLOUD network.

(Optional)Databaseencryption key(DEK)

If the Encrypt DataStore function is enabled when you createa data warehouse cluster, a Data Encryption Workshop (DEW)key is required to encrypt and decrypt the database. The key isbilled based on the billing rules of DEW keys. For details, seeDEW Pricing Details.

Billing ModesDWS provides two billing modes, the pay per use (hourly) and yearly/monthlypackage. The longer you use, the less you pay.

● Pay per use (hourly): In this billing mode, you can enable or disable DWS asyou like. You are billed by your use duration on an hourly basis (the time isaccurate to minutes) and the system generates a bill every natural hour anddeducts fees from your account balance. This mode is applicable to customerswho need to perform preliminary operation tests and Proofs-of-Conceptverifications, and short-term users.

● Yearly/Monthly package: You can choose this mode and make a one-offpayment when purchasing DWS if you plan to use this service for a long time.This mode offers you significant discounts over pay-per-use mode and savesyou 35% to 45% of costs. This mode is applicable to customers who plan touse DWS for a long time after the tests and verifications.The yearly/monthly package is a type of billing mode. You can regard it as acoupon. Purchasing a yearly/monthly package will not have the system tocreate a cluster. When you are using a cluster, the billing system automaticallycompares the number of valid yearly/monthly nodes you purchase with thenumber of nodes in the same region and flavor you are using. The nodeswithin the yearly/monthly node quantity will not be charged and the extranodes are charged on a pay-per-use basis. After the yearly/monthly packageyou purchase expires, you will be billed in pay-per-use mode if you continueto use the service.Nodes in a yearly/monthly package are not bound to clusters. For details, seethe following descriptions:

a. After you purchase a yearly/monthly package, the system will not createa cluster automatically. If you have not run any clusters, go to the DataWarehouse Service console to create a cluster with the correspondingregion, node flavor, and node quantity.

b. If your cluster is billed in pay-per-use mode, the billing systemautomatically identifies yearly/monthly nodes and gives you discountswhen generating the bill after you purchase yearly/monthly nodes withthe corresponding region, flavor, and quantity.



https://www.huaweicloud.com/en-us/pricing.html?tab=detail#/eip

https://www.huaweicloud.com/en-us/pricing.html?tab=detail#/dew

https://console.huaweicloud.com/dws/?locale=en-us

https://console.huaweicloud.com/dws/?locale=en-us

c. If you want to scale out your cluster and you have valid yearly/monthlynodes in the region, the billing system automatically identifies yearly/monthly nodes and gives you discounts when generating the bill. If youdo not have any available yearly/monthly nodes, the newly added nodeswill be billed on a pay-per-use basis.

d. If you delete an existing cluster and create a new one, your yearly/monthly nodes can still be used in the new cluster with the same regionand flavor.

NO TICE

Precautions of purchasing yearly/monthly packages are as follows:● Yearly/Monthly nodes are bound to a specific region. The purchased yearly/

monthly nodes can be used in the bound region only.● Yearly/Monthly nodes are bound to a specific node flavor. Nodes with

different flavors have different package prices.● A yearly/monthly package takes effect upon purchase and will

automatically expire when the usage duration is reached. Then, you will bebilled in pay-per-use mode if you continue to use the service. For example,if you purchase a package with three nodes on January 1 at a year and thepackage duration is one year, then the package automatically ends at theend of December 31 in the year. Note that if you do not run any clusterwithin the validity period, the validity period will not be extended and thefee cannot be refunded. Therefore, to save your costs, you are advised tocreate a cluster immediately after purchasing a yearly/monthly package, orpurchase a yearly/monthly package after creating a cluster.



10 Restrictions

● You can manage clusters but cannot directly access nodes in a cluster. You canuse a cluster's IP address and port to access the database in the cluster.

● You cannot change the flavor of an existing cluster. If you need nodes with ahigher flavor, create a new one.

● If you use a client to connect to a cluster, its VPC subnet must be the same asthat of the cluster.

● If you copy commands from the document to the execution environment, thetext wraps automatically, causing command execution failures. To solve theproblem, delete the line break.

Data Warehouse ServiceService Overview 10 Restrictions


11 Service Quota

Quotas are enforced for service resources on the platform to prevent unforeseenspikes in resource usage. Quotas can limit the number or amount of resourcesavailable to users.

Table 11-1 shows the default user quota of DWS. For more information abouthow to view and increase quotas, see Quotas.

Table 11-1 User quotas

Resource Type Total Quota

Nodes 32

Data Warehouse ServiceService Overview 11 Service Quota


https://support.huaweicloud.com/en-us/usermanual-iaas/en-us_topic_0040259342.html

12 Version Description

This section describes the cluster version update records of DWS.

V1.5.200Release Date: August 8, 2019

● Optimized Functions

a. Increased the backup frequency of automated snapshots. By default,automated snapshots are token three times each day, starting from00:00:00 (UTC). You can set the snapshot execution period when creatinga cluster or modify the automated snapshot policy for an existing cluster.

b. In the cluster list, you can click More in the Operation column of acluster to select the Create Snapshot operation.

Data Warehouse ServiceService Overview 12 Version Description


Documents

Service Overview - HUAWEI CLOUD · Teradata. Applications can be smoothly migrated to DWS with few changes. Interface Applications can connect to DWS through the standard JDBC 4.0