Using Oracle NoSQL Database Cloud Service · 2020-06-01 · This section provides an overview of Oracle NoSQL Database Cloud Service. Oracle NoSQL Database Cloud Service is a fully

Oracle® CloudUsing Oracle NoSQL Database Cloud Service

Latest Cloud ReleaseE90090-16October 2020

Oracle Cloud Using Oracle NoSQL Database Cloud Service, Latest Cloud Release

E90090-16

Copyright © 2018, 2020, Oracle and/or its affiliates.

Primary Author: Vandanadevi Rajamani

This software and related documentation are provided under a license agreement containing restrictions onuse and disclosure and are protected by intellectual property laws. Except as expressly permitted in yourlicense agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license,transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverseengineering, disassembly, or decompilation of this software, unless required by law for interoperability, isprohibited.

The information contained herein is subject to change without notice and is not warranted to be error-free. Ifyou find any errors, please report them to us in writing.

If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it onbehalf of the U.S. Government, then the following notice is applicable:

U.S. GOVERNMENT END USERS: Oracle programs (including any operating system, integrated software,any programs embedded, installed or activated on delivered hardware, and modifications of such programs)and Oracle computer documentation or other Oracle data delivered to or accessed by U.S. Governmentend users are "commercial computer software" or "commercial computer software documentation" pursuantto the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such,the use, reproduction, duplication, release, display, disclosure, modification, preparation of derivative works,and/or adaptation of i) Oracle programs (including any operating system, integrated software, any programsembedded, installed or activated on delivered hardware, and modifications of such programs), ii) Oraclecomputer documentation and/or iii) other Oracle data, is subject to the rights and limitations specified in thelicense contained in the applicable contract. The terms governing the U.S. Government’s use of Oracle cloudservices are defined by the applicable contract for such services. No other rights are granted to the U.S.Government.

This software or hardware is developed for general use in a variety of information management applications.It is not developed or intended for use in any inherently dangerous applications, including applications thatmay create a risk of personal injury. If you use this software or hardware in dangerous applications, then youshall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure itssafe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of thissoftware or hardware in dangerous applications.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks oftheir respective owners.

Intel and Intel Inside are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks areused under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Epyc,and the AMD logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registeredtrademark of The Open Group.

This software or hardware and documentation may provide access to or information about content, products,and services from third parties. Oracle Corporation and its affiliates are not responsible for and expresslydisclaim all warranties of any kind with respect to third-party content, products, and services unless otherwiseset forth in an applicable agreement between you and Oracle. Oracle Corporation and its affiliates will notbe responsible for any loss, costs, or damages incurred due to your access to or use of third-party content,products, or services, except as set forth in an applicable agreement between you and Oracle.

Contents

Preface

Audience vii

Documentation Accessibility vii

Related Resources vii

Conventions vii

1 Getting Started

About the Service 1-1

Region Availability 1-1

Service Limits 1-2

Service Quotas 1-2

Service Events 1-3

Service Metrics 1-5

Before You Begin 1-6

Typical Workflow 1-6

Key Features 1-7

Cloud Concepts 1-8

Quick Start Tutorials 1-10

2 Connecting your Application

Acquiring Credentials 2-1

Connecting your Application using Java 2-3

Connecting your Application using Python 2-4

Connecting your Application using Node.js 2-4

Connecting your Application using Go 2-4

Data Regions and Associated Service Endpoints 2-4

3 Developing in Oracle Cloud

Table Design 3-1

Supported Data Types 3-2

iii

Table Fields 3-3

Primary Keys and Shard Keys 3-5

Time to Live 3-6

Table States and Life Cycles 3-7

Table Management 3-8

Data Definition Language Reference 3-8

Using Tables in Java 3-10

About the Oracle NoSQL Database Java SDK 3-10

About Compartments 3-10

Obtaining a NoSQL Handle 3-11

Creating Tables and Indexes 3-12

Adding Data 3-14

Adding JSON Data 3-15

Reading Data 3-16

Using Queries 3-17

Deleting Data 3-19

Modifying Tables 3-19

Dropping Tables and Indexes 3-21

Handling Errors 3-21

Using Tables in Python 3-22

Using Tables in Node.js 3-22

Using Tables in Go 3-22

Query Language Reference 3-22

Oracle NoSQL Database Cloud Service Limits 3-24

Estimating Capacity 3-25

Handling Capacity 3-29

4 Developing in Oracle NoSQL Database Cloud Simulator

Downloading the Oracle NoSQL Database Cloud Simulator 4-1

Oracle NoSQL Database Cloud Simulator Compared With Oracle NoSQL DatabaseCloud Service 4-2

5 Using Plugins for Development

About IntelliJ Plugin 5-1

Setting Up IntelliJ Plug-in 5-1

Creating a NoSQL Project in IntelliJ 5-2

Connecting to Oracle NoSQL Database Cloud Simulator from IntelliJ 5-2

Connecting to Oracle NoSQL Database Cloud Service from IntelliJ 5-3

Managing Tables Using the IntelliJ Plugin 5-5

iv

About Eclipse Plugin 5-6

6 Using the Console to Manage Tables

Accessing the Service from the Infrastructure Console 6-1

Managing Table Data 6-1

Inserting Data Into Tables 6-2

Inserting Data Into Tables: Simple Input Mode 6-2

Inserting Data Into Tables: Advanced JSON Input Mode 6-2

Viewing Table Data 6-3

Updating Table Data 6-3

Deleting Table Data 6-4

Downloading Table Data 6-5

Managing Tables and Indexes 6-5

Creating Tables 6-5

Creating Table: Simple Input Mode 6-6

Creating Table: Advanced DDL Input Mode 6-8

Creating Indexes 6-10

Editing Tables 6-10

Altering Tables 6-11

Adding Table Columns: Simple Input Mode 6-11

Adding Table Columns: Advanced DDL Input Mode 6-13

Deleting Table Columns 6-13

Moving Tables 6-14

Deleting Tables 6-14

Deleting Indexes 6-15

Monitoring Tables and Indexes 6-15

Viewing Tables 6-15

Viewing Indexes 6-16

Viewing Table Details 6-16

Viewing Table DDL 6-16

Viewing Table Metrics 6-17

7 Managing Subscriptions

Setting up Your Service 7-1

Estimating Your Monthly Cost 7-2

Creating a Compartment 7-2

8 Managing Table Access and Security

About Oracle NoSQL Database Cloud Service Security Model 8-1

v

Typical Process to Manage Security for Oracle NoSQL Database Cloud Service 8-3

Setting Up Users, Groups, and Policies 8-3

Policy Reference 8-4

Resource-Types 8-4

Supported Variables 8-5

Details for Verb + Resource-Type Combinations 8-6

Permission Required for Each NoSQL Cloud Driver Request 8-7

Permission Required for Each REST API Operation 8-8

Typical Policy Statements to Manage Tables 8-9

Giving Another User Permission to Manage NoSQL Tables 8-10

9 Using Oracle NoSQL Migrator

Overview 9-1

Terminology used with NoSQL Data Migrator 9-2

Using Oracle NoSQL Data Migrator 9-5

Sources and Sinks 9-7

Supported Sources and Sinks 9-7

Source Configuration Templates 9-7

JSON File 9-8

MongoDB-Formatted JSON File 9-9

Oracle NoSQL Database 9-11

Oracle NoSQL Database Cloud Service 9-13

Sink Configuration Templates 9-17

JSON File 9-17

Oracle NoSQL Database 9-20

Oracle NoSQL Database Cloud Service 9-27

Transformation Configuration Templates 9-37

ignoreFields 9-38

renameFields 9-41

aggregateFields 9-44

Use Case Demonstrations 9-46

Migrate from Oracle NoSQL Database Cloud Service to a JSON file 9-46

Migrate from Oracle NoSQL Database On-Premise to Oracle NoSQL DatabaseCloud Service 9-51

Migrate from MongoDB-Formatted JSON file to an Oracle NoSQL DatabaseCloud Service 9-53

Troubleshooting the NoSQL Data Migrator 9-55

Oracle NoSQL Data Migrator Vs. Import/Export Utility 9-58

Transitioning from Import/Export to NoSQL Data Migrator 9-59

vi

Preface

This document describes how to use Oracle NoSQL Database Cloud Service andprovides references to related documentation.

AudienceThis document is intended for developers who want to create tables and query datain Oracle NoSQL Database Cloud Service. This document is also intended for OracleCloud administrators who want to manage users and their privileges in Oracle NoSQLDatabase Cloud Service.

Documentation AccessibilityFor information about Oracle's commitment to accessibility, visit theOracle Accessibility Program website at http://www.oracle.com/pls/topic/lookup?ctx=acc&id=docacc.

Access to Oracle Support

Oracle customers that have purchased support have access to electronic supportthrough My Oracle Support. For information, visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=info or visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trsif you are hearing impaired.

Related ResourcesFor more information, see About Oracle Cloud.

ConventionsThe following text conventions are used in this document:

Convention Meaning

boldface Boldface type indicates graphical user interface elements associated with anaction, or terms defined in text or the glossary.

italic Italic type indicates book titles, emphasis, or placeholder variables for which yousupply particular values.

monospace Monospace type indicates commands within a paragraph, URLs, code inexamples, text that appears on the screen, or text that you enter.

vii

http://www.oracle.com/pls/topic/lookup?ctx=acc&id=docacc

http://www.oracle.com/pls/topic/lookup?ctx=acc&id=docacc

http://www.oracle.com/pls/topic/lookup?ctx=acc&id=info

http://www.oracle.com/pls/topic/lookup?ctx=acc&id=info

http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs

1Getting Started

This chapter gives an overview of Oracle NoSQL Database Cloud Service andprovides hands-on tutorials to get started with the service.

Topics

• About the Service

• Key Features

• Cloud Concepts

• Quick Start Tutorials

About the ServiceThis section provides an overview of Oracle NoSQL Database Cloud Service.

Oracle NoSQL Database Cloud Service is a fully managed database cloud servicethat is designed for database operations that require predictable, single digitmillisecond latency responses to simple queries. NoSQL Database Cloud Serviceallows developers to focus on application development rather than setting up clusterservers, or performing system monitoring, tuning, diagnosing, and scaling. NoSQLDatabase Cloud Service is suitable for applications such as Internet of Things, userexperience personalization, instant fraud detection, and online display advertising.

Once you are authenticated against your Oracle Cloud account, you can create aNoSQL table, and specify throughput and storage requirements for the table. Oraclereserves and manages the resources to meet your requirements, and provisionscapacity for you. Capacity is specified using read and write units for throughput andGB for storage units. See Estimating Capacity.

Service Access Options

As a developer, you can connect to the Oracle NoSQL Database Cloud Service andwork with NoSQL tables using the NoSQL SDKs available in multiple languages. Tolearn how to download and use the NoSQL SDKs, see Developing in Oracle Cloud.

In addition to the NoSQL SDKs, Oracle NoSQL Database Cloud Service also providesOracle Cloud Infrastructure Console for you to access, manage, and use the NoSQLDatabase Cloud Service. The Oracle Cloud Infrastructure Console lets you create andmanage NoSQL tables and indexes declaratively. You can also manipulate the data inyour NoSQL tables and monitor your Oracle NoSQL Database Cloud Service from theconsole. See Using the Console to Manage Tables.

Region AvailabilityFor the latest information on the regions where Oracle NoSQL Database CloudService is available, see Data Regions for Platform and Infrastructure Services inOracle Cloud Infrastructure Documentation.

1-1

Tip:

To find the service endpoint for a specific data region, see Data Regions andAssociated Service Endpoints.

Service LimitsOracle NoSQL Database Cloud Service has various default limits. Whenever youcreate an Oracle NoSQL Database Cloud Service table, the system ensures that yourrequests are within the bounds of the specified limit.

This table lists the Oracle NoSQL Database Cloud Service limits that you canreference. For detailed list of service limits, see Oracle NoSQL Database CloudService Limits.

Resource Monthly Universal Credits Pay-as-You-Go or Promo

Read Units 100,000 units 100,000 units

Write Units 40,000 units 40,000 units

Table Size 5,000 GB 5,000 GB

You can increase your service limits by submitting a request either from Limits,Quotas, and Usage page in Oracle Cloud Infrastructure Console or using theTableRequest API.

See About Service Limits and Usage in Oracle Cloud Infrastructure Documentation.

Service QuotasYou can use quotas to determine how other users allocate Oracle NoSQL DatabaseCloud Service resources across compartments in Oracle Cloud Infrastructure. Acompartment is a collection of related resources (such as instances, virtual cloudnetworks, block volumes) that can be accessed only by certain groups that havebeen given permission by an administrator. Whenever you create an Oracle NoSQLDatabase Cloud Service table or scale up the provisioned throughput or storage,the system ensures that your requests are within the bounds of the quota for thatcompartment.

This table lists the Oracle NoSQL Database Cloud Service quotas that you canreference.

Name Scope Description

read-unit-count Regional Read Unit Count

write-unit-count Regional Write Unit Count

table-size-gb Regional Table Size (GB)

You can set quotas using the Console or API. You can execute quota statements fromthe Quota Policies page under the Governance option in Oracle Cloud InfrastructureConsole.

Chapter 1About the Service

1-2

Example Quota Statements for Oracle NoSQL Database Cloud Service

• Limit the number of Oracle NoSQL Database Cloud Service read units thatusers can allocate to tables they create in my_compartment to 20,000.

set nosql quota read-unit-count to 20000 in compartment my_compartment

• Limit the number of Oracle NoSQL Database Cloud Service write units thatusers can allocate to tables they create in my_compartment to 5,000.

set nosql quota write-unit-count to 5000 in compartment my_compartment

• Limit the maximum storage space of Oracle NoSQL Database Cloud Servicethat users can allocate to tables they create in my_compartment to 1,000 GB.

set nosql quota table-size-gb to 1000 in compartment my_compartment

See About Compartment Quotas in Oracle Cloud Infrastructure Documentation.

Service EventsActions that you perform on Oracle NoSQL Database Cloud Service tables emitevents.

You can define rules that trigger a specific action when an event occurs. For example,you might define a rule that sends a notification to administrators when someonedrops a table. See Overview of Events and Get Started with Events in Oracle CloudInfrastructure Documentation.

This table lists the Oracle NoSQL Database Cloud Service events that you canreference.

Friendly Name Event Type

Alter Table Begincom.oraclecloud.nosql.altertable.begin

Alter Table Endcom.oraclecloud.nosql.altertable.end

Change Table Compartment Begincom.oraclecloud.nosql.changecompartment.begin

Change Table Compartment Endcom.oraclecloud.nosql.changecompartment.end


1-3

Friendly Name Event Type

Create Index Begincom.oraclecloud.nosql.createindex.begin

Create Index Endcom.oraclecloud.nosql.createindex.end

Create Table Begincom.oraclecloud.nosql.createtable.begin

Create Table Endcom.oraclecloud.nosql.createtable.end

Drop Index Begincom.oraclecloud.nosql.dropindex.begin

Drop Index Endcom.oraclecloud.nosql.dropindex.end

Drop Table Begincom.oraclecloud.nosql.droptable.begin

Drop Table Endcom.oraclecloud.nosql.droptable.end

Example

This example shows information associated with the event Create Table Begin:

{ "cloudEventsVersion": "0.1", "contentType": "application/json", "source": "nosql", "eventID": "<unique_ID>", "eventType": "com.oraclecloud.nosql.createtable.begin", "eventTypeVersion": "<version>", "eventTime": "2019-12-30T00:52:01.343Z", "data": { "additionalDetails": {}, "availabilityDomain": "<availability_domain>", "compartmentId": "ocid1.compartment.oc1..<unique_ID>", "compartmentName": "my_compartment", "freeformTags": { "key":"value" }, "resourceId": "ocid1.nosqltable.oc1..<unique_ID>", "resourceName": "my_nosql_table"


1-4

}, "extensions": { "compartmentId": "ocid1.compartment.oc1..<unique_ID>" }}

Service MetricsLearn about the metrics emitted by the metric namespace oci_nosql (Oracle NoSQLDatabase Cloud Service).

Metrics for Oracle NoSQL Database Cloud Service include the following dimensions:

• RESOURCEIDThe OCID of the NoSQL Table in the Oracle NoSQL Database Cloud Service.

Note:

OCID is an Oracle-assigned unique ID that is included as part of theresource's information in both the console and API.

• TABLENAMEThe name of the NoSQL table in the Oracle NoSQL Database Cloud Service.

Oracle NoSQL Database Cloud Service sends metrics to the Oracle CloudInfrastructure Monitoring Service. You can view or create alarms on these metricsusing the Oracle Cloud Infrastructure Console SDKs or CLI. See OCI SDKs and CLI inOracle Cloud Infrastructure Documentation.

Available Metrics

Metric Metric DisplayName

Unit Description Dimensions

ReadUnits Read Units Units The number ofread unitsconsumed duringthis period.

resourceIdtableName

WriteUnits Write Units Units The number ofwrite unitsconsumed duringthis period.

resourceIdtableName

StorageGB Storage Size GB The maximumamount ofstorageconsumed by thetable. As thisinformation isgenerated hourly,you may seevalues that areout of date inbetween therefresh points.

resourceIdtableName


1-5

Metric Metric DisplayName

Unit Description Dimensions

ReadThrottleCount

Read Throttle Count The number ofread throttlingexceptions on thistable in the timeperiod.

resourceIdtableName

WriteThrottleCount

Write Throttle Count The number ofwrite throttlingexceptions on thistable in the timeperiod.

resourceIdtableName

StorageThrottleCount

Storage Throttle Count The number ofstorage throttlingexceptions on thistable in the timeperiod.

resourceIdtableName

Before You BeginWhen you order Oracle NoSQL Database Cloud Service through Universal Credits,you automatically get access to other required services, including Oracle CloudInfrastructure.

Here's some information about how Oracle NoSQL Database Cloud Service usesother services and what you need to do if you're setting up Oracle NoSQL DatabaseCloud Service for the first time.

Service What is it for? Do I need to do anything?

Oracle Cloud Infrastructure Identityand Access Management

• You use Oracle CloudInfrastructure Identity andAccess Management to createuser accounts for the people inyour organization who will useOracle NoSQL Database CloudService.

• You can give users permissionto inspect, read, use, or manageOracle NoSQL Database CloudService tables. See Setting UpUsers, Groups, and Policies

• You use compartments withinyour tenancy to organize tableson Oracle Cloud Infrastructure.

Yes.Before you create your first OracleNoSQL Database Cloud Servicetable, Oracle recommends that youset up one or more compartmentsin which you deploy and secure yourtable.

To know more, see the followingresources in Oracle CloudInfrastructure Documentation:• Setting Up Your Tenancy• Managing Compartments

Typical WorkflowTypical sequence of tasks to work with Oracle NoSQL Database Cloud Service.

If you're developing applications using Oracle NoSQL Database Cloud Service for thefirst time, follow these tasks as a guide.


1-6

Task Description More Information

Connect your application Connect your application to use Oracle NoSQLDatabase Cloud Service tables.

Connecting yourApplication

Develop your application Develop your application after connecting to OracleNoSQL Database Cloud Service tables.

Developing in Oracle CloudDeveloping in OracleNoSQL Database CloudSimulator

If you're setting up Oracle NoSQL Database Cloud Service for the first time, seeSetting up Your Service.

Key FeaturesLearn the key features of Oracle NoSQL Database Cloud Service.

• Fully Managed with Zero Administration: Developers do not need to administerdata servers or the underlying infrastructure and security. Oracle maintains thehardware and software which allows developers to focus on building applications.

• Faster Development Life Cycle: After purchasing access to the service,developers write their applications, and then connect to the service using theircredentials. Reading and writing data can begin immediately. Oracle performsDatabase Management, Storage Management, High Availability, and Scalabilitywhich helps developers concentrate on delivering high-performance applications.

• High Performance and Predictability: Oracle NoSQL Database Cloud Servicetakes advantage of the latest component technologies in the Oracle CloudInfrastructure by providing high performance at scale. Developers know that theirapplications return data with predictable latencies, even as their throughput andstorage requirements increase.

• On-Demand Throughput and Storage Provisioning: Oracle NoSQL DatabaseCloud Service scales to meet application throughput performance requirementswith low and predictable latency. As workloads increase with periodic businessfluctuations, applications can increase their provisioned throughput to maintain aconsistent user experience. As workloads decrease, the same applications canreduce their provisioned throughput, resulting in lower operating expenses. Thesame holds true for storage requirements. Those can be adjusted based onbusiness fluctuations. You can increase or decrease the storage using the OracleCloud Infrastructure Console or the TableRequest API.

• Simple APIs: Oracle NoSQL Database Cloud Service provides easy-to-useCRUD (Create Read Update Delete) APIs that allow developers to easily createtables and maintain data in them.

• Data Modeling: Oracle NoSQL Database Cloud Service supports both schema-based and schema-less (JSON) modeling.

• Data Safety in Redundancy: The Oracle NoSQL Database Cloud Service storesdata across multiple Availability Domains (ADs) or Fault Domains (FDs) in singleAD regions. If an AD or FD becomes unavailable, user data is still accessible fromanother AD or FD.

• Data Security: Data is encrypted at rest (on disk) with Advanced EncryptionStandard (AES 256). Data is encrypted in motion (transferring data between theapplication and Oracle NoSQL Database Cloud Service) with HTTPS.

Chapter 1Key Features

1-7

• ACID-Compliant Transactions: ACID (Atomicity, Consistency, Isolation,Durability) transactions are fully supported for the data you store in Oracle NoSQLDatabase Cloud Service. If required, consistency can be relaxed in favor of lowerlatency.

• JSON Data Support: Oracle NoSQL Database Cloud Service allows developersto query schema-less JSON data by using the familiar SQL syntax.

• Partial JSON Updates: Oracle NoSQL Database Cloud Service allowsdevelopers to update (change, add, and remove) parts of a JSON document.Because these updates occur on the server, the need for a read-modify-write cycleis eliminated, which would consume throughput capacity.

• Time-To-Live: Oracle NoSQL Database Cloud Service lets developers set a timeframe on table rows, after which the rows expire automatically, and are no longeravailable. This feature is a critical requirement when capturing sensor data forInternet Of Things (IoT) services.

• SQL Queries: Oracle NoSQL Database Cloud Service lets developers accessdata with SQL queries.

• Secondary Indexes: Secondary indexes allow a developer to create an index onany field of a supported data type, thus improving performance over multiple pathsfor queries using the index.

Cloud ConceptsLearn the Oracle NoSQL Database Cloud Service concepts.

• Table: A Table is a collection of rows where each row holds a data record fromyour application.

Each table row consists of key and data fields which are defined when a tableis created. In addition, a table has a specified storage, can support a definedmaximum read and write throughput, and has a maximum size. The storagecapacity is specified at table creation time and can be changed later.

– High-Level Data Types: Oracle NoSQL Database Cloud Service supports allthree types of Big Data. You can create NoSQL tables to store structured,unstructured, or semi-structured data.

* Structured: This type of data can be organized and stored in tables witha predefined structure or schema. For example, the data stored in regularrelational database tables come under this category. They adhere to afixed schema and are simple to manage and analyze. Data generatedfrom credit card transactions and e-commerce transactions are a fewexamples of structured data.

* Semi-Structured: The data that can not fit into a relational database butcan be organized into rows and columns after a certain level of processingis called semi-structured data. Oracle NoSQL Database Cloud Servicecan store and process semi-structured data by storing key-value pairs inNoSQL tables. XML data is an example of semi-structured data.

* Unstructured: The data that can not be organized or stored in tableswith a fixed schema or structure are called Unstructured data. Videos,images, and media are a few examples of unstructured data. OracleNoSQL Database Cloud Service lets you define tables with rows of JSONdata type to store unstructured data.

Chapter 1Cloud Concepts

1-8

– Data Types: A table is created using DDL (Data Definition Language) whichdefines the data types and primary keys used for the table.

Oracle NoSQL Database Cloud Service supports several data types, includingseveral numeric types, string, binary, timestamp, maps, arrays, records, anda special JSON data type which can hold any valid JSON data. Applicationscan use unstructured tables where a row uses the JSON data type to store thedata, or use structured tables where all row types are defined and enforced.See Supported Data Types to view the list of data types supported in OracleNoSQL Database Cloud Service.

Unstructured tables are flexible. But typed data is safer from an enforcementand storage efficiency point of view. Table schema can be modified , but thetable structure is less flexible to change.

– Indexes: Applications can create an index on any data field which has adata type that permits indexing, including JSON data fields. JSON indexes arecreated using a path expression into the JSON data.

– Capacity: When you create a table, you also specify throughput and storageresources available for the table. Read and write operations to the tableare limited by the read and write throughput capacity that you define. Theamount of space that the table can use is limited by the storage capacity.See Estimating Capacity to learn how to estimate capacity for your applicationworkload.

• Distribution and Sharding: Although not visible to the user, Oracle NoSQLDatabase Cloud Service tables are sharded and replicated for availability andperformance. Therefore, you should consider this during schema design.

– Primary and Shard keys: An important consideration for a table is thedesignation of the primary key, and the shard key. When you create a table inOracle NoSQL Database Cloud Service, the data in the table is automaticallysharded based on a portion of the table primary key, called the shard key.See Primary Keys and Shard Keys for considerations on how to designate theprimary and shard keys.

– Read Consistency: Read consistency specifies different levels of flexibilityin terms of which copy of the data is used to fulfill a read operation. OracleNoSQL Database Cloud Service provides two levels of consistency, EVENTUAL,and ABSOLUTE.

Applications can specify ABSOLUTE consistency, which guarantees that allread operations return the most recently written value for a designated key.Or, applications capable of tolerating inconsistent data can specify EVENTUALconsistency, allowing the database to return a value more quickly even if it isnot up-to-date.

ABSOLUTE consistency results in a higher cost, consuming twice the number ofread units for the same data relative to EVENTUAL consistency, and should onlybe used when required. Consistency can be set for a NoSQL handle, or as anoptional argument for all read operations.

• Identity Access and Management: Oracle NoSQL Database Cloud Serviceuses the Oracle Cloud Infrastructure Identity and Access Management to providesecure access to Oracle Cloud. Oracle Cloud Infrastructure Identity and AccessManagement enables you to create user accounts and give users permission toinspect, read, use, or manage Oracle NoSQL Database Cloud Service tables.See Overview of Oracle Cloud Infrastructure Identity and Access Management inOracle Cloud Infrastructure Documentation.

Chapter 1Cloud Concepts

1-9

Quick Start TutorialsAccess the quick start tutorials and get started with the service.

1. Get access to the service.

Sign up for a Cloud promotion or purchase an Oracle Cloud subscription. Activateyour order, create users (optional). See Setting up Your Service.

2. Acquire credentials to connect your application. See Acquiring Credentials.

3. Create a table in Oracle NoSQL Database Cloud Service.

• To start in Java, see Quick Start: Create a Table in Oracle NoSQLDatabase Cloud Service.

Chapter 1Quick Start Tutorials

1-10

http://apexapps.oracle.com/pls/apex/f?p=44785:112:0::::P112_CONTENT_ID:24605

http://apexapps.oracle.com/pls/apex/f?p=44785:112:0::::P112_CONTENT_ID:24605

2Connecting your Application

Your application must acquire and provide credentials to establish a secure connectionwith the Oracle NoSQL Database Cloud Service.

Topics

• To connect your application, see:

– Acquiring Credentials

– Connecting your Application using Java

– Connecting your Application using Python

– Connecting your Application using Node.js

– Connecting your Application using Go

• To learn about data regions supported for Oracle NoSQL Database Cloud Service,see Data Regions and Associated Service Endpoints.

Acquiring CredentialsLearn how to acquire credentials to connect your application to Oracle NoSQLDatabase Cloud Service.

You need an Oracle Cloud account to connect your application to the Oracle NoSQLDatabase Cloud Service. If you do not already have an Oracle Cloud account, you canstart with Oracle Cloud. The credentials that are used for connecting your applicationare associated with a specific user. If you want to create a user for your application,see Setting Up Users, Groups, and Policies.

Information Comprising the Credentials:

Table 2-1 Credentials

What is it? Where to find it?

Tenancy ID, an OCID See Where to Get the Tenancy's OCID andUser's OCID in Oracle Cloud InfrastructureDocumentation.

User ID, an OCID

API Signing Key For the application user, an API signing keymust be generated and uploaded. If this hasalready been done this step can be skipped.

To know more, see the following resources inOracle Cloud Infrastructure Documentation:• How to Generate an API Signing Key to

generate the signing key with an optionalpass phrase.

• How to Upload the Public Key to uploadthe API signing key to the user's account.

2-1

Table 2-1 (Cont.) Credentials

What is it? Where to find it?

Fingerprint for the Signing Key The fingerprint and pass phrase of thesigning key are created while generating anduploading the API Signing Key. See How toGet the Key's Fingerprint in Oracle CloudInfrastructure Documentation.

(Optional) Pass Phrase for the Signing Key

Tip:

Make a note of the location of your private key, optional pass phrase, andfingerprint of the public key while generating and uploading the API SigningKey.

After performing the tasks discussed above, collect the credentials information andprovide them to your application.

Providing the Credentials to your Application:

The Oracle NoSQL Database SDKs allow you to provide the credentials to anapplication in multiple ways. The SDKs support a configuration file as well asone or more interfaces that allow direct specification of the information. See thedocumentation for the programming language driver that you are using to know aboutthe specific credentials interfaces.

If you are using a configuration file, the default location is ~/.oci/config. The SDKsallow the file to reside in alternative locations. It's content looks like this:

[DEFAULT]user=ocid1.user.oc1..aaaaaaaas...7apfingerprint=d1:b2:32:53:d3:5f:cf:68:2d:6f:8b:5f:77:8f:07:13key_file=~/.oci/oci_api_key_private.pemtenancy=ocid1.tenancy.oc1..aaaaaaaap...keqpass_phrase=mysecretphrase

The [DEFAULT] line indicates that the lines that follow specify the DEFAULT profile.A configuration file can include multiple profiles, prefixed with [PROFILE_NAME]. Forexample:

[DEFAULT]user=ocid1.user.oc1..aaaaaaaas...7usfingerprint=d1:b2:32:53:d3:5f:cf:68:2d:6f:8b:5f:77:8f:07:15key_file=~/.oci/oci_api_key_private.pemtenancy=ocid1.tenancy.oc1..aaaaabbap...keqpass_phrase=mysecretphrase

[MYPROFILE]user=ocid1.user.oc1..aaaaaaaas...7apfingerprint=d1:b2:32:53:d3:5f:cf:68:2d:6f:8b:5f:77:8f:07:13key_file=~/.oci/oci_api_key_private.pem

Chapter 2Acquiring Credentials

2-2

tenancy=ocid1.tenancy.oc1..aaaaaaaap...keqpass_phrase=mysecretphrase

Connecting your Application using JavaLearn how to connect your Java application to Oracle NoSQL Database CloudService.

Your application connects to Oracle NoSQL Database Cloud Service by specifyingcredentials and a target region for the connection. See Acquiring Credentials for therequired credentials information.

You provide credentials in your application using either:

• An API that allows you to directly provide the credentials, or

• A configuration file

Connecting Using API

/* Use the SignatureProvider to supply your credentials to NoSQL Database. * By default, the SignatureProvider will read your OCI configuration file * from the default location, ~/.oci/config. See SignatureProvider for * additional options for reading configurations in other ways.*/SignatureProvider sp = new SignatureProvider( tenantId, // a string, OCID userId, // a string, OCID fingerprint , // a string privateKey, // a string, content of private key passPhrase // optional, char[]);//Create an handle to access the cloud service in the us-ashburn-1 region.NoSQLHandleConfig config = new NoSQLHandleConfig(Region.US_ASHBURN_1);config.setAuthorizationProvider(sp);NoSQLHandle handle = NoSQLHandleFactory.createNoSQLHandle(config);

//At this point, your handle is set up to perform data operations.

Connecting Using a Configuration File

/* Use the SignatureProvider to supply your credentials to NoSQL Database. * By default, the SignatureProvider will read your OCI configuration file * from the default location, ~/.oci/config. See SignatureProvider for * additional options for reading configurations in other ways. */SignatureProvider sp = new SignatureProvider();

//Create an handle to access the cloud service in the us-ashburn-1 region.NoSQLHandleConfig config = new NoSQLHandleConfig(Region.US_ASHBURN_1);config.setAuthorizationProvider(sp);

Chapter 2Connecting your Application using Java

2-3

NoSQLHandle handle = NoSQLHandleFactory.createNoSQLHandle(config);

//At this point, your handle is set up to perform data operations.

Connecting your Application using PythonLearn how to connect your Python application to Oracle NoSQL Database CloudService.

Before connecting your application to cloud, see the Prerequisites section in NoSQLDatabase Python SDK to learn about the credentials required to create the connection,and how to access them.

To connect your application using Python, see the Configure for the Cloud Servicesection in NoSQL Database Python SDK.

Connecting your Application using Node.jsLearn how to connect your Node.js application to Oracle NoSQL Database CloudService.

Before connecting your application to cloud, see the Prerequisites section in NoSQLDatabase Node.js SDK to learn about the credentials required to create theconnection, and how to access them.

To connect your application using Node.js, see the Connecting an Application toOracle NoSQL Database Cloud Service tutorial in NoSQL Database Node.js SDK.

Connecting your Application using GoLearn how to connect your Go application to Oracle NoSQL Database Cloud Service.

Before connecting your application to cloud, see the Prerequisites section in NoSQLDatabase Go SDK to learn about the credentials required to create the connection,and how to access them.

To connect your application using Go, see the Connecting an Application to the CloudService section in NoSQL Database Go SDK.

Data Regions and Associated Service EndpointsLearn about the data regions supported for Oracle NoSQL Database Cloud Serviceand access region-specific service endpoints.

Data Regions

To start with Oracle NoSQL Database Cloud Service, you must create an account(either for free trial or to purchase provisioning). Along with other details, the accountapplication requires you to choose the default data region.

Service Endpoints Associated with Data Regions

A service endpoint is the regional network access point to the Oracle NoSQLDatabase Cloud Service. The general format of a region endpoint is https://nosql.

Chapter 2Connecting your Application using Python

2-4

{region}.oci.oraclecloud.com. For example, the service endpoint for the AshburnOracle NoSQL Database Cloud Service region identifier in North America regionis https://nosql.us-ashburn-1.oci.oraclecloud.com. Different data regions havedifferent {region} components of their URLs.

This table lists the service endpoints for all the data regions which are or will besupported by Oracle NoSQL Database Cloud Service. See Service Availability forthe latest information about the regions that support Oracle NoSQL Database CloudService.

Data Region Region Identifier Service Endpoint

North America us-ashburn-1 https://nosql.us-ashburn-1.oci.oraclecloud.com

North America us-phoenix-1 https://nosql.us-phoenix-1.oci.oraclecloud.com

North America ca-toronto-1 https://nosql.ca-toronto-1.oci.oraclecloud.com

North America us-sanjose-1 https://nosql.us-sanjose-1.oci.oraclecloud.com

North America ca-montreal-1 https://nosql.ca-montreal-1.oci.oraclecloud.com

EMEA eu-frankfurt-1 https://nosql.eu-frankfurt-1.oci.oraclecloud.com

EMEA uk-london-1 https://nosql.uk-london-1.oci.oraclecloud.com

EMEA eu-zurich-1 https://nosql.eu-zurich-1.oci.oraclecloud.com

EMEA eu-amsterdam-1 https://nosql.eu-amsterdam-1.oci.oraclecloud.com

EMEA me-jeddah-1 https://nosql.me-jeddah-1.oci.oraclecloud.com

EMEA me-dubai-1 https://nosql.me-dubai-1.oci.oraclecloud.com

APAC ap-seoul-1 https://nosql.ap-seoul-1.oci.oraclecloud.com

APAC ap-tokyo-1 https://nosql.ap-tokyo-1.oci.oraclecloud.com

APAC ap-mumbai-1 https://nosql.ap-mumbai-1.oci.oraclecloud.com

APAC ap-sydney-1 https://nosql.ap-sydney-1.oci.oraclecloud.com

APAC ap-hyderabad-1 https://nosql.ap-hyderabad-1.oci.oraclecloud.com

APAC ap-osaka-1 https://nosql.ap-osaka-1.oci.oraclecloud.com

APAC ap-melbourne-1 https://nosql.ap-melbourne-1.oci.oraclecloud.com

APAC ap-chuncheon-1 https://nosql.ap-chuncheon-1.oci.oraclecloud.com

LAD sa-saopaulo-1 https://nosql.sa-saopaulo-1.oci.oraclecloud.com

Chapter 2Data Regions and Associated Service Endpoints

2-5

3Developing in Oracle Cloud

Learn table design concepts, and develop your application using your driver.

When you are ready to work with Oracle Cloud, you create a NoSQL application. YourNoSQL application uses tables to organize data.

Topics

• To learn how to design and configure tables in Oracle NoSQL Database CloudService, see Table Design.

• To learn how to create and manage tables to suit your application, see TableManagement.

• To learn about the APIs that your application uses to insert, read, and delete thedata contained within your tables, see:

– Using Tables in Java

– Using Tables in Python

– Using Tables in Node.js

– Using Tables in Go

• To learn how to use the SQL query language on tables, see Query LanguageReference.

• To learn about limits that exist for elements of Oracle NoSQL Database CloudService, see Oracle NoSQL Database Cloud Service Limits.

• To learn how to estimate and handle capacity, see Estimating Capacity andHandling Capacity.

Table DesignLearn how to design and configure tables in Oracle NoSQL Database Cloud Service.

Table

A table is a collection of rows, where each row holds a data record. Each tablerow consists of key and data fields, which are defined when a table is created. Inaddition, a table has a specified storage, can support a defined maximum read andwrite throughput, and has a maximum size.

Learn about the following concepts before designing a table in Oracle NoSQLDatabase Cloud Service:

• Table Fields: Tables are created using DDL (Data Definition Language) thatdefines the data types and primary keys used for the table. Oracle NoSQLDatabase Cloud Service supports multiple data types, including several numerictypes, string, binary, timestamp, maps, arrays, records, and a special JSON typewhich can hold valid JSON data. An application can choose data types for the

3-1

table fields based on the data model. See Supported Data Types for a completelist of data types supported in Oracle NoSQL Database Cloud Service.

See Table Fields to learn more about table fields.

• Primary and Shard Keys: Every table must have one or more fields designatedas the primary key. This designation occurs at the time of table creation, andcannot be changed after the table is created. A primary key uniquely identifiesevery row in the table. In the simplest case, a primary key is used to retrieve aspecific row to examine or modify.

Shard keys identify which primary key fields are meaningful in terms of shardstorage. That is, rows which contain the same values for all the shard keys areguaranteed to be stored on the same shard. This shard storage matters for someoperations that promise atomicity of the results.

See Primary Keys and Shard Keys to learn more.

• Indexes: Indexes represent an alternative way of retrieving table rows. Normallyyou retrieve table rows using the primary key. By creating an index, you canmore efficiently retrieve rows based on fields that are not part of the primarykey. Indexes provide more querying capability, but consume both storage andthroughput resources.

See Creating Tables and Indexes to learn how to create an index.

• Capacity: When you create a table, you also specify throughput and storageresources available for the table. Read and write operations to the table are limitedby the read and write throughput capacity that you define. The amount of spacethat the table can use is limited by the storage capacity.

See Estimating Capacity for information on how to estimate the capacity that youshould specify for your table.

See Handling Capacity to learn how to handle the capacity of your table.

• Time to Live (TTL): Time to Live allows you to automatically expire table rows.TTL is expressed as the time data is allowed to live in the table. Data which hasreached the expiration timeout value can no longer be retrieved, and does notappear in any read operations.

See Time to Live to learn more.

• Identity Columns: Identity columns are a special type of columns that gettheir values automatically assigned by Oracle NoSQL Database Cloud Service.These values are generated from a sequence generator. Identity columns are notsupported from the NoSQL Database Cloud Service console. For more detailsabout identity columns, see Identity Column from SQL Reference for OracleNoSQL Database . To learn how to access identity columns from an OracleNoSQL Database Cloud Service application, see Inserting IDENTITY ValuesProgrammatically from Getting Started with NoSQL Database Table Java Driver.

• Table Life cycles: When tables are created, modified, or deleted, they transitionthrough different table life cycle states.

See Table States and Life Cycles to learn about the life cycle of a table.

Supported Data TypesOracle NoSQL Database Cloud Service supports many common data types.

Chapter 3Table Design

3-2

Data Type Description

BINARY A sequence of zero or more bytes.

FIXED_BINARY A fixed-size byte array.

BOOLEAN A data type with one of two possible values: TRUE or FALSE.

DOUBLE A 64 bit (8 bytes) long floating-point number.

LONG A 64-bit long integer number.

INTEGER A 32 bit (4 bytes) long integer number.

STRING A sequence of Unicode characters.

NUMBER An arbitrary-precision signed decimal number.

TIMESTAMP A point in time with a precision. The precision affects the storage size and usage.Timestamp is stored and managed in UTC (Coordinated Universal Time).

ENUM An enumeration, represented as an array of strings. ENUM values are symbolicidentifiers (tokens) and are stored as a small integer value representing an orderedposition in the enumeration.

ARRAY An ordered collection of zero of more typed items. Arrays that are not defined asJSON cannot contain NULL values.

Arrays declared as JSON can contain any valid JSON, including the special value,null, which is relevant to JSON.

MAP An unordered collection of zero or more key-item pairs, where all keys are stringsand all items are the same type. All keys must be unique. The key-item pairs arecalled fields, the keys are field names, and the associated items are field values.Field values can have different types, but maps cannot contain NULL field values.

RECORD A fixed collection of one or more key-item pairs, where all keys are strings. All keys ina record must be unique.

JSON Any valid JSON data.

Table FieldsLearn how to design and configure data using table fields.

An application may choose to use schemaless tables, where a row consists of keyfields and a single JSON data field. A schemaless table offers flexibility in what can bestored in a row.

Alternatively, the application can choose to use fixed schema tables, where all of thetable fields are defined as specific types.

Fixed schema tables with typed data are safer to use from an enforcement andstorage efficiency standpoint. Even though the schema of fixed schema tables canbe modified, their table structure cannot easily be changed. A schemaless table isflexible and the table structure can be easily modified.

Finally, an application can also use a hybrid data model approach where a table canhave typed data and JSON data fields.

The following examples demonstrate how to design and configure data for all threeapproaches.


3-3

Example 1: Designing a Schemaless Table

You have multiple options to store information about browsing patterns in your table.One option is to define a table that uses a cookie ID as a key and keeps audiencesegmentation data as a single JSON field.

// schema less, data is stored in a JSON fieldCREATE TABLE audience_info ( cookie_id LONG, audience_data JSON, PRIMARY KEY(cookie_id))

In this case, the audience_info table can hold a JSON object such as:

{ "cookie_id": "", "audience_data": { "ipaddr" : "10.0.00.xxx", "audience_segment: { "sports_lover" : "2018-11-30", "book_reader" : "2018-12-01" } }}

Your application will have a key field and a data field for this table. You have flexibilityin what you chose to store as information in your audience_data field. Therefore, youcan easily change the types of information available.

Example 2: Designing a Fixed Schema Table

You can store information about browsing patterns by creating your table with moreexplicitly declared fields:

// fixed schema, data is stored in typed fields.CREATE TABLE audience_info( cookie_id LONG, ipaddr STRING, audience_segment RECORD(sports_lover TIMESTAMP(9), book_reader TIMESTAMP(9)), PRIMARY KEY(cookie_id));

In this example, your table has a key field and two data fields. Your data is morecompact, and you are able to ensure that all data fields are accurate.

Example 3: Designing a Hybrid Table

You can store information about browsing patterns by using both typed and JSON datafields in your table.

// mixed, data is stored in both typed and JSON fields.CREATE TABLE audience_info ( cookie_id LONG,


3-4

ipaddr STRING, audience_segment JSON, PRIMARY KEY(cookie_id));

Primary Keys and Shard KeysLearn the purpose of primary keys and shard keys while designing your application.

Primary keys and shard keys are important elements in your schema and help youaccess and distribute data efficiently. You create primary keys and shard keys onlywhen you create a table. They remain in place for the life of the table, and cannot bechanged or dropped.

Primary Keys

You must designate one or more primary key columns when you create your table. Aprimary key uniquely identifies every row in the table. For simple CRUD operations,Oracle NoSQL Database Cloud Service uses the primary key to retrieve a specific rowto read or modify. For example, consider a table has the following fields:

• productName

• productType

• productLine

From experience, you know that the product name is important as well as unique toeach row, so you set the productName as the primary key. Then, you retrieve rows ofinterest based on the productName. In such a case, use a statement like this, to definethe table.

/* Create a new table called users. */CREATE TABLE if not exists myProducts ( productName STRING, productType STRING, productLine INTEGER, PRIMARY KEY (productName))";

Shard Keys

The main purpose of shard keys is to distribute data across the Oracle NoSQLDatabase Cloud Service cluster for increased efficiency, and to position records thatshare the shard key locally for easy reference and access. Records that share theshard key are stored in the same physical location and can be accessed atomicallyand efficiently.

Your Primary and shard key design has implications on scaling and achievingprovisioned throughput. For example, when records share shard keys, you can deletemultiple table rows in an atomic operation, or retrieve a subset of rows in your table ina single atomic operation. In addition to enabling scalability, well-designed shard keyscan improve performance by requiring fewer cycles to put data to, or get data from, asingle shard.


3-5

For example, suppose that you designate three primary key fields:

PRIMARY KEY (productName, productType, productLine)

Because you know that your application frequently makes queries using theproductName and productType columns, specifying those fields as shard keys isappropriate. The shard key designation guarantees that all rows for these two columnsare stored on the same shard. If these two fields are not shard keys, the mostfrequently queried columns could be stored on any shard. Then, locating all rows forboth fields requires scanning all data storage, rather than one shard.

Shard keys designate storage on the same shard to facilitate efficient queries for keyvalues. However, because you want your data be distributed across the shards forbest performance, you must avoid shard keys that have few unique values.

Note:

If you do not designate shard keys when creating a table, Oracle NoSQLDatabase Cloud Service uses the primary keys for shard organization.

Important factors to consider when choosing a shard key

• Cardinality: Low cardinality fields, such as a user home country, group recordstogether on a few shards. In turn, those shards require frequent data rebalancing,increasing the likelihood of hot shard issues. Instead, each shard key shouldhave high cardinality, where the shard key can express an even slice of recordsin the data set. For example, identity numbers such as customerID, userID, orproductID are good candidates for a shard key.

• Atomicity: Only objects that share the shard key can participate in a transaction.If you require ACID transactions that span multiple records, choose a shard keythat lets you meet that requirement.

What are the best practices to follow?

• Uniform distribution of shard keys: When shard keys are uniformly distributed,no single shard limits the capacity of the system.

• Query Isolation: Queries should be targeted to a specific shard to maximizeefficiency and performance. If queries are not isolated to a single shard, the queryis applied to all shards which is less efficient and increases query latency.

See Creating Tables and Indexes to learn how to assign primary and shard keys usingthe TableRequest object.

Time to LiveLearn how to specify expiration times for tables and rows using the Time-to-Live (TTL)feature.

Many applications handle data that has a limited useful lifetime. Time-to-Live (TTL)is a mechanism that allows you to set a time frame on table rows, after which therows expire automatically, and are no longer available. It is the amount of time datais allowed to remain in the Oracle NoSQL Database Cloud Service. Data that reaches


3-6

expiration time can no longer be retrieved, and does not appear in any storagestatistics.

By default, every table that you create has a TTL value of zero, indicating no expirationtime. You can declare a TTL value when you create a table, specifying the TTL witha number, followed by either HOURS or DAYS. Table rows inherit the TTL value of thetable in which they reside, unless you explicitly set a TTL value for table rows. Settinga row's TTL value overrides the table's TTL value. If you change the table's TTL valueafter the row has a TTL value, the row's TTL value persists.

You can update the TTL value for a table row at any time before the row reachesthe expiration time. Expired data can no longer be accessed. Therefore, using TTLvalues is more efficient than manually deleting rows, because the overhead of writinga database log entry for the data deletion is avoided. Expired data is purged from thedisk after expiration date.

Table States and Life CyclesLearn about the different table states and their significance (table life cycle process).

Each table passes through a series of different states from table creation to deletion(drop). For example, a table in the DROPPING state cannot proceed to the ACTIVE state,while a table in the ACTIVE state can change to the UPDATING state. You can trackthe different table states by monitoring the table life cycle. This section describes thevarious table states.

Table State Description

CREATING The table is in the process of being created. It is not ready to use.

UPDATING Updating the table is in process. Further table modifications are not possiblewhile the table is in this state.

A table is in the UPDATING state when:

• The table limits are being changed• The table schema is evolving• Adding or dropping a table index

ACTIVE The table can be used in the current state. The table may have been recentlycreated, or modified, but the table state is now stable.

DROPPING The table is being dropped and cannot be accessed for any purpose.


3-7

Table State Description

DROPPED The table has been dropped and no longer exists for read, write, or queryactivities.

Note:

Once dropped, a table with the same name canbe created again.

Table ManagementLearn how to design tables and use Data Definition Language to work with tables.

You can create, modify, and delete NoSQL tables from your application. You can alsoadd indexes to optimize your application common access paths. The table DDL iswhat you use to specify these actions.

• To view the range of options that you have for designing and configuring yourtables and indexes, see Query Language Reference.

• To view the complete description of the table DDL, see Data Definition LanguageReference.

The language driver of your choice executes your DDL statement. See Using Tables inJava for examples of how to issue your DDL statements.

Data Definition Language ReferenceLearn how to use DDL in Oracle NoSQL Database Cloud Service.

Use Oracle NoSQL Database Cloud Service DDL to create, alter, and drop tables andindexes.

For information on the syntax of the DDL language, see Table Data DefinitionLanguage Guide. This guide documents the DDL language as supported by theon-premises Oracle NoSQL Database product. The Oracle NoSQL Database CloudService supports a subset of this functionality and the differences are documented inthe DDL Differences in the Cloud section.

Also, each NoSQL <language> driver provides an API to execute a DDL statement. Towrite your application, see Using Tables in Java.

Typical DDL Statements

Few samples of common DDL statements are as follows:

Create Table

CREATE TABLE [IF NOT EXISTS] ( field-definition, field-definition-2 ..., PRIMARY KEY (field-name, field-name-2...),) [USING TTL ttl]

Chapter 3Table Management

3-8

For example:

CREATE TABLE IF NOT EXISTS audience_info ( cookie_id LONG, ipaddr STRING, audience_segment JSON, PRIMARY KEY(cookie_id));

Alter Table

ALTER TABLE table-name (ADD field-definition)ALTER TABLE table-name (DROP field-name)ALTER TABLE table-name USING TTL ttl

For example:

ALTER TABLE audience_info USING TTL 7 days;

Create Index

CREATE INDEX [IF NOT EXISTS] index-name ON table-name (path_list)

For example:

CREATE INDEX segmentIdx ON audience_info (audience_segment.sports_lover AS STRING)

Drop Table

DROP TABLE [IF EXISTS] table-name

For example:

DROP TABLE audience_info;

See the reference guides for a complete list:

• Table Data Definition Language guide

• SQL Reference for Oracle NoSQL Database

DDL Differences in the Cloud

The cloud service DDL language differs from what is described in the reference guidein the following way:

Table Names

• Limited to 256 characters, and are restricted to alphanumeric characters andunderscore

• Must start with a letter

Chapter 3Table Management

3-9

http://www.oracle.com/pls/topic/lookup?ctx=en/cloud/paas/nosql-cloud&id=tbl_def

• Cannot include special characters

• Child tables are not supported

Unsupported Concepts

• DESCRIBE and SHOW TABLE statements.

• Full text indexes

• User and role management

• On-premise regions

Using Tables in JavaLearn how to create, update, and delete tables from your Java application.

Topics

• About the Oracle NoSQL Database Java SDK

• About Compartments

• Obtaining a NoSQL Handle

• Creating Tables and Indexes

• Adding Data

• Reading Data

• Using Queries

• Deleting Data

• Modifying Tables

• Dropping Tables and Indexes

• Handling Errors

About the Oracle NoSQL Database Java SDKLearn about the Oracle NoSQL Database Java SDK.

The Oracle NoSQL Database Java Driver contains the jar files that enable anapplication to communicate with the on-premises Oracle NoSQL Database or theOracle NoSQL Database Cloud Service or the Oracle NoSQL Database CloudSimulator.

Download and unpack the Java SDK from Oracle Technology Network. Extracting theOracle NoSQL Database Java SDK provides you with all the Java classes, methods,interfaces, examples, and documentation. See Java API Reference Guide.

About CompartmentsLearn how to specify the compartment while creating and working with Oracle NoSQLDatabase Cloud Service tables using the Oracle NoSQL Database Java Driver.

Oracle NoSQL Database Cloud Service tables are created in a compartment andare scoped to that compartment. When authenticated as a specific user, your tables

Chapter 3Using Tables in Java

3-10

are managed in the root compartment of your tenancy unless otherwise specified.Organizing tables into different compartments will help with respect to organization andsecurity.If you have been authenticated using an instance principal (accessing the service froman OCI compute instance), you must specify a compartment using its id (OCID), asthere is no default in this case. See Calling Service From an Instance in Oracle CloudInfrastructure Documentation.

There are several ways to specify a compartment in your application code:

1. Use a default compartment in NoSQLHandleConfig so that it applies to all theoperations using the handle. See Obtaining a NoSQL Handle for an example.

2. Use the compartment name or id (OCID) in each request in addition to the tablename. This overrides any default compartment.For example:

GetRequest getReq = new GetRequest().setTableName("mytable") .setCompartment("mycompartment");

3. Use the compartment name as a prefix on the table name. This overrides anydefault compartment as well as a compartment specified using API.For example:

GetRequest getReq = new GetRequest().setTableName("mycompartment:mytable");

When using a named compartment, the name can be the simple name of a top-levelcompartment or a path to a nested compartment. In the latter case, the path is a "."(dot) separated path.

Note:

While specifying the path to a nested compartment, do not include the top-level compartment's name in the path as that is inferred from the tenancy.

Obtaining a NoSQL HandleLearn how to access tables using the Oracle NoSQL Database Java Driver.

Start developing your application by creating a NoSQL Handle. Use the NoSQLHandleto access the tables and execute all operations. To create a connection represented bya NoSQLHandle, obtain a handle using the NoSQLHandleFactory.createNoSQLHandlemethod and the NoSQLHandleConfig class. The NoSQLHandleConfig class allows anapplication to specify the handle configuration. See the Java API Reference guide tolearn more.

Obtain a NoSQL Handle

Use the following code to obtain a NoSQL handle:

/* Configure a handle for the desired Region and AuthorizationProvider. * By default this SignatureProvider constructor reads authorization


3-11

* information from ~/.oci/config and uses the default user profile and * private key for request signing. Additional SignatureProvider * constructors are available if a config file is not available or * desirable. */AuthorizationProvider ap = new SignatureProvider();

/* Use the us-ashburn-1 region */NoSQLHandleConfig config = new NoSQLHandleConfig(Region.US - ASHBURN - 1, ap);config.setAuthorizationProvider(ap);

/* Sets a default compartment for all requests from this handle. This * may be overridden in individual requests or by using a * compartment-name prefixed table name. */config.setDefaultCompartment("mycompartment");

// Open the handleNoSQLHandle handle = NoSQLHandleFactory.createNoSQLHandle(config);

// Use the handle to execute operations

A handle has memory and network resources associated with it. Use theNoSQLHandle.close method to free up the resources when your application is doneusing the handle.

To minimize network activity and resource allocation and deallocation overheads, it'sbest to avoid creating and closing handles repeatedly. For example, creating andclosing a handle around each operation would result in poor application performance.A handle permits concurrent operations, so a single handle is sufficient to accesstables in a multi-threaded application. The creation of multiple handles incursadditional resource overheads without providing any performance benefit.

Creating Tables and IndexesLearn how to create tables and indexes.

Creating a table is the first step of developing your application. You use theTableRequest class and methods to execute all DDL statements, such as,creating, modifying, and dropping tables. You also set table limits using theTableRequest.setTableLimits method.Before creating a table, see:

• Supported Data Types, and

• Oracle NoSQL Database Cloud Service Limits

The TableRequest class lets you pass a DDL statement to theTableRequest.setStatement method. Examples of DDL statements are:

/* Create a new table called users */CREATE IF NOT EXISTS users(id INTEGER, name STRING, PRIMARY KEY(id));


3-12

/* Create a new table called users and set the TTL value to 4 days */CREATE IF NOT EXISTS users(id INTEGER, name STRING, PRIMARY KEY(id))USING TTL 4 days;

/* Create a new index called nameIdx on the name field in the users table */CREATE INDEX IF NOT EXISTS nameIdx ON users(name);

Note:

The following example considers that the default compartment is specifiedin NoSQLHandleConfig while obtaining the NoSQL handle. See Obtaining aNoSQL Handle. To explore other options of specifying a compartment for theNoSQL tables, see About Compartments.

To create a table and index using the TableRequest and its methods:

/* Create a simple table with an integer key and a single json data * field and set your desired table capacity. * Set the table TTL value to 3 days. */String createTableDDL = "CREATE TABLE IF NOT EXISTS users " + "(id INTEGER, name STRING, " + "PRIMARY KEY(id)) USING TTL 3 days";

TableLimits limits = new TableLimits(200, 100, 5);TableRequest treq = new TableRequest().setStatement(createTableDDL) .setTableLimits(limits);

// start the asynchronous operationTableResult tres = handle.tableRequest(treq);

// wait for completion of the operationtres.waitForCompletion(handle, 60000, // wait for 60 sec 1000); // delay in ms for poll

// Create an index called nameIdx on the name field in the users table.treq = new TableRequest().setStatement("CREATE INDEX IF NOT EXISTS nameIdx ON users(name) ");

// start the asynchronous operation handle.tableRequest(treq);

// wait for completion of the operation tres.waitForCompletion(handle,


3-13

60000, // wait for 60 sec 1000); // delay in ms for poll

Related Topics

• About Time to Live

Adding DataAdd rows to your table.

When you store data in table rows, your application can easily retrieve, add to, ordelete information from a table.The PutRequest class represents the input to aNoSQLHandle.put(oracle.nosql.driver.ops.PutRequest) operation. This requestcan be used to perform unconditional and conditional puts to:

• Overwrite any existing row. Overwrite is the default functionality.

• Succeed only if the row does not exist. Use the PutRequest.Option.IfAbsentmethod in this case.

• Succeed only if the row exists. Use the PutRequest.Option.IfPresent method inthis case.

• Succeed only if the row exists and the version matches a specificversion. Use the PutRequest.Option.IfVersion method for this case and thesetMatchVersion(oracle.nosql.driver.Version) method to specify the versionto match.

Note:

First, connect your client driver to Oracle NoSQL Database Cloud Service toget a handle and then complete other steps. This topic omits the steps forconnecting your client driver and creating a table. If you do not yet have atable, see Creating Tables and Indexes.

The following example assumes that the default compartment is specified inNoSQLHandleConfig while obtaining the NoSQL handle. See Obtaining a NoSQLHandle. To explore other options of specifying a compartment for the NoSQL tables,see About Compartments.To add rows to your table:

/* use the MapValue class and input the contents of a new row */MapValue value = new MapValue().put("id", 1).put("name", "myname");

/* create the PutRequest, setting the required value and table name */PutRequest putRequest = new PutRequest().setValue(value) .setTableName("users");

/* use the handle to execute the PUT request * on success, PutResult.getVersion() returns a non-null value */PutResult putRes = handle.put(putRequest);


3-14

https://docs.oracle.com/en/cloud/paas/nosql-cloud/csnjv/oracle/nosql/driver/TimeToLive.html

if (putRes.getVersion() != null) { // success} else { // failure}

You can perform a sequence of PutRequest operations on a table that share theshard key using the WriteMultipleRequest class. If the operation is successful, theWriteMultipleResult.getSuccess() method returns true.See the Java API Reference Guide for more information about the APIs.

You can also add JSON data to your table. You can either convert JSON data into arecord for a fixed schema table or you can insert JSON data into a column whose datatype is of type JSON. See Adding JSON Data.

Adding JSON DataLearn how to add JSON data to a fixed schema table.

Note:


Table rows are added to the table by using APIs which let you individually specify eachtable field value. For example, you use the MapValue.put() method to fill in each fieldvalue for a row, before inserting the entire row into the table.

The PutRequest class also provides the setValueFromJson method which takes aJSON string and uses that to populate a row to insert into the table. The JSON stringshould specify field names that correspond to the table field names.

Note:

The following example assumes that the default compartment is specifiedin NoSQLHandleConfig while obtaining the NoSQL handle. See Obtaining aNoSQL Handle. To explore other options of specifying a compartment for theNoSQL tables, see About Compartments.

To add JSON data to your table:

/* Construct a simple row, specifying the values for each * field. The value for the row is this: * * { * "cookie_id": 123, * "audience_data": { * "ipaddr": "10.0.00.xxx", * "audience_segment": {


3-15

* "sports_lover": "2018-11-30", * "book_reader": "2018-12-01" * } * } * } */MapValue segments = new MapValue() .put("sports_lover", new TimestampValue("2018-11-30")) .put("book_reader", new TimestampValue("2018-12-01"));MapValue value = new MapValue() .put("cookie_id", 123) // fill in cookie_id field .put("ipaddr", "10.0.00.xxx") .put("audience_segment", segments); PutRequest putRequest = new PutRequest() .setValue(value) .setTableName(tableName);PutResult putRes = handle.put(putRequest);

The same row can be inserted into the table as a JSON string:

/* Construct a simple row in JSON */ String jsonString = "{\"cookie_id\":123,\"ipaddr\":\"10.0.00.xxx\", \"audience_segment\":{\"sports_lover\":\"2018-11-30\", \"book_reader\":\"2018-12-01\"}}"; PutRequest putRequest = new PutRequest() .setValueFromJson(jsonString, null) // no options .setTableName(tableName); PutResult putRes = handle.put(putRequest);

Reading DataLearn how to read data from your table.

You can read data from your application by using the NoSQLHandle.get() method.This method allows you to retrieve a record based on a single primary key value, or byusing queries. The GetRequest class provides a simple and powerful way to read data,while queries can be used for more complex read requests.

Note:


To read data from a table, specify the target table and target key using the GetRequestclass and use NoSQLHandle.get() to execute your request. The result of the operationis available in GetResult.

The following example considers that the default compartment is specified inNoSQLHandleConfig while obtaining the NoSQL handle. See Obtaining a NoSQL


3-16

Handle. To explore other options of specifying a compartment for the NoSQL tables,see About Compartments.To read data from your table:

/* GET the row, first create the row key */MapValue key = new MapValue().put("id", 1);GetRequest getRequest = new GetRequest().setKey(key) .setTableName("users");GetResult getRes = handle.get(getRequest);

/* on success, GetResult.getValue() returns a non-null value */if (getRes.getValue() != null) { // success} else { // failure}

Note:

By default, all read operations are eventually consistent. You can changethe default Consistency for a NoSQLHandle instance by using theNoSQLHandleConfig.setConsistency(oracle.nosql.driver.Consistency)and GetRequest.setConsistency() methods.

See the Java API Reference Guide for more information about the GET (read) APIs.

Using QueriesLearn about some aspects of using queries to your application in Oracle NoSQLDatabase Cloud Service.

Oracle NoSQL Database Cloud Service provides a rich query language to read andupdate data. See SQL Reference for NoSQL Database for a full description of thequery language.

To execute your query, you use the NoSQLHandle.query() API. See the Java APIReference Guide for more information about this API.

Note:

The following examples consider that the default compartment is specifiedin NoSQLHandleConfig while obtaining the NoSQL handle. See Obtaining aNoSQL Handle. To explore other options of specifying a compartment for theNoSQL tables, see About Compartments.

To execute a SELECT query to read data from your table:

/* QUERY a table named "users", using the primary key field "name". * The table name is inferred from the query statement. */


3-17

QueryRequest queryRequest = new QueryRequest().setStatement("SELECT * FROM users WHERE name = \"Taylor\"");

/* Queries can return partial results. It is necessary to loop, * reissuing the request until it is "done" */

do { QueryResult queryResult = handle.query(queryRequest);

/* process current set of results */ List<MapValue> results = queryResult.getResults(); for (MapValue qval : results) { //handle result }} while (!queryRequest.isDone());

When using queries, be aware of the following considerations:

• You can use prepared queries when you want to run the same query multipletimes. When you use prepared queries, the execution is more efficient thanstarting with a query string every time. The query language and API support queryvariables to assist with the reuse. See NoSQLHandle.prepare in the Java APIReference Guide for more information.

• You can set important query attributes, such as the usable amount of resources, orthe read consistency used by the read operations, using the QueryRequest class.See QueryRequest in the Java API Reference Guide for more information.

For example, to execute a SELECT query to read data from your table using a preparedstatement:

/* Perform the same query using a prepared statement. This is more * efficient if the query is executed repeatedly and required if * the query contains any bind variables. */String query = "DECLARE $name STRING; " + "SELECT * from users WHERE name = $name";

PrepareRequest prepReq = new PrepareRequest().setStatement(query);

/* prepare the statement */PrepareResult prepRes = handle.prepare(prepReq);

/* set the bind variable and set the statement in the QueryRequest */prepRes.getPreparedStatement() .setVariable("$name", new StringValue("Taylor"));QueryRequest queryRequest = new QueryRequest().setPreparedStatement(prepRes);

/* perform the query in a loop until done */do { QueryResult queryResult = handle.query(queryRequest); /* handle result */} while (!queryRequest.isDone());


3-18

Deleting DataLearn how to delete rows from your table.

After you insert or load data into a table, you can delete the table rows when they areno longer required.

Note:


The following example considers that the default compartment is specified inNoSQLHandleConfig while obtaining the NoSQL handle. See Obtaining a NoSQLHandle. To explore other options of specifying a compartment for the NoSQL tables,see About Compartments.

To delete a row from a table:

/* identify the row to delete */MapValue delKey = new MapValue().put("id", 2);

/* construct the DeleteRequest */DeleteRequest delRequest = new DeleteRequest().setKey(delKey) .setTableName("users");

/* Use the NoSQL handle to execute the delete request */DeleteResult del = handle.delete(delRequest);

/* on success DeleteResult.getSuccess() returns true */if (del.getSuccess()) { // success, row was deleted} else { // failure, row either did not exist or conditional delete failed}

You can perform a sequence of DeleteRequest operations on a table using theMultiDeleteRequest class.See the Java API Reference Guide for more information about the APIs.

Modifying TablesLearn how to modify tables.

You modify a table to:

• Add new fields to an existing table

• Delete currently existing fields in a table

• To change the default TTL value


3-19

• Modify table limits

Note:


Examples of DDL statements are:

/* Add a new field to the table */ALTER TABLE users (ADD age INTEGER);

/* Drop an existing field from the table */ALTER TABLE users (DROP age);

/* Modify the default TTL value*/ALTER TABLE users USING TTL 4 days;

Note:


When altering a table, you may also use the TableRequests.setTableLimits methodto modify table limits. For example:

/* Alter the users table to modify the TTL value to 4 days. * When modifying the table schema or other table state you cannot also * modify the table limits. These must be independent operations. */String alterTableDDL = "ALTER TABLE users " + "USING TTL 4 days";TableRequest treq = new TableRequest().setStatement(alterTableDDL);

/* start the operation, it is asynchronous */TableResult tres = handle.tableRequest(treq);

/* wait for completion of the operation */tres.waitForCompletion(handle, 60000, /* wait for 60 sec */ 1000); /* delay in ms for poll */


3-20

Dropping Tables and IndexesLearn how to delete a table or index that you have created in Oracle NoSQL DatabaseCloud Service.

To drop a table in Oracle NoSQL Database Cloud Service, you must have theNOSQL_TABLE_DROP permission. See Details for Verb + Resource-Type Combinationsto learn about different permissions.

To drop a table or index, use the DROP TABLE or DROP INDEX DDL statements. Forexample:

/* Drop the table named users */DROP TABLE users;

/* Drop the index called nameIndex on the table users */DROP INDEX IF EXISTS nameIndex ON users;

Note:


To drop a table using the TableRequests.setStatement method:

/* create the TableRequest to drop the users table */TableRequest tableRequest = new TableRequest().setStatement("drop table users");

/* start the operation, it is asynchronous */TableResult tres = handle.tableRequest(tableRequest);

/* wait for completion of the operation */tres.waitForCompletion(handle, 60000, /* wait for 60 sec */ 1000); /* delay in ms for poll */

Handling ErrorsLearn how to handle errors and exceptions.

Java errors are thrown as exceptions when you build or run your application.The NoSQlException class is the base for most exceptions thrown by thedriver. However, the driver throws exceptions directly for some classes, such asIllegalArgumentException and NullPointerException.

In general, NoSQL exception instances are split into two broad categories:

• Exceptions that may be retried with the expectation that they may succeed onretry.


3-21

These exceptions are instances of the RetryableException class. Theseexceptions usually indicate resource consumption violations such asThrottlingException.

• Exceptions that will fail even after retry.

Examples of exceptions that should not be retried are IllegalArgumentException,TableNotFoundException, and any other exception indicating a syntactic orsemantic error.

See the Java API Reference guide to learn more about these exceptions and how tohandle them.

Using Tables in PythonLearn how to create, update, and delete tables from your Python application.

Oracle NoSQL Database Cloud Service provides a Python SDK that enables yourPython application to create, update, and drop tables as well as add, read, anddelete data in the tables. See Working with Tables in NoSQL Database Python SDKdocumentation.

Using Tables in Node.jsLearn how to create, update, and delete tables from your Node.js application.

Oracle NoSQL Database Cloud Service provides a Node.js SDK that enables yourNode.js application to create, update, and drop tables as well as add, read, anddelete data in the tables. See Working with Tables in NoSQL Database Node.js SDKdocumentation.

Using Tables in GoLearn how to create, update, and delete tables from your Go application.

Oracle NoSQL Database Cloud Service provides a Go SDK that enables your Goapplication to create, update, and drop tables as well as add, read, and delete datain the tables. See a simple example that demonstrates working with tables in NoSQLDatabase Go SDK documentation.

Query Language ReferenceLearn how to use SQL statements to update and query data in Oracle NoSQLDatabase Cloud Service.

The Oracle NoSQL Database uses the SQL query language to update and query datain NoSQL tables. See SQL Reference for Oracle NoSQL Database to learn the querylanguage syntax.

Chapter 3Using Tables in Python

3-22

Note:

The SQL Reference for Oracle NoSQL Database documents the SQL querylanguage as supported by the on-premise Oracle NoSQL Database product.Oracle NoSQL Database Cloud Service supports only a subset of thisfunctionality.

Typical Queries

SELECT <expression>FROM <table name>[WHERE <expression>][GROUP BY <expression>][ORDER BY <expression> [<sort order>]][OFFSET <number>][LIMIT <number>];

For example:SELECT * FROM Users;SELECT id, firstname, lastname FROM Users WHERE firstname = "Taylor";

UPDATE <table_name> [AS <table_alias>] <update_clause>[, <update_clause>]*WHERE <expr>[<returning_clause>];

For example:UPDATE JSONPersons $j SET TTL 1 DAYS WHERE id = 6 RETURNING remaining_days($j) AS Expires;

Query Language Differences in the Cloud

The cloud service query support differs from what is described in the query languagereference guide in the following way:

Exclusions

Joins are not available since child tables are not supported in Oracle NoSQL DatabaseCloud Service.

Restrictions on Expressions Used in the SELECT Clause

Oracle NoSQL Database Cloud Service supports grouping expressions or arithmeticexpressions among aggregate functions. No other kinds of expressions are allowed inthe SELECT clause. For example, CASE expressions are not allowed in the SELECTclause.

Each NoSQL Database driver provides an API to execute a query statement. To writeyour application and use query language, see Using Tables in Java.

Chapter 3Query Language Reference

3-23

Oracle NoSQL Database Cloud Service LimitsCurrent limits that apply to Oracle NoSQL Database Cloud Service.

Scope Description Value

Table Maximum total storage size per tenant. Thetotal space used for one or more tablescannot exceed this value.

5 TB

Table Names Maximum number of characters, allowedcharacters, and initial character.

Table names can have a maximum of 256characters. All names must begin with aletter (a–z, A–Z). Subsequent characterscan be letters (a–z, A–Z), digits (0–9), orunderscore.

Table Maximum read and write throughput. 40,000 read units and 20,000 write units pertable.

Table Maximum number of tables 30

Table Maximum number columns 50

Table Maximum number of table schema updates 100

Table Maximum number of indexes 5

Table Maximum number of changes for throughputand storage limits

Oracle allows:

• Unlimited number of throughput andstorage increases per day

• Up to four throughput or storagedecreases per 24-hour period.

Index Names Maximum number of characters, allowedcharacters, and initial character.

Index names can have a maximum of 64characters. All names must begin with aletter (a–z, A–Z). Subsequent characterscan be letters (a–z, A–Z), digits (0–9), orunderscore.

Request Maximum number of individual operationsper WriteMultiple request.

50

Request Maximum data size for WriteMultiplerequest.

25 MB

Field Names Maximum number of characters, allowedcharacters, and initial character.

Field names can have a maximum of 64characters. All names must begin with aletter (a–z, A–Z). Subsequent characterscan be letters (a–z, A–Z), digits (0–9), orunderscore.

Field Maximum index key size. 64 bytes

Field Maximum primary key size. 64 bytes

Row Maximum row size. 512 KB

Query Maximum query string length. 10 KB

Tenant Maximum supported rate of DDL operations. 4 per minute

Chapter 3Oracle NoSQL Database Cloud Service Limits

3-24

Scope Description Value

Tenant Maximum values for throughput and datastorage resources.

Per tenant, Oracle allows:

• A maximum of 100,000 read units• A maximum of 40,000 write unitsOracle allows a maximum storage size of 5-TB per-tenant. The tenant can have a singletable with storage size of 5 TB, in whichcase the tenant is not able to create anothertable. Or have multiple tables and ensurethat the data within all these tables is withinthe maximum storage size of 5 TB.

Estimating CapacityLearn how to estimate throughput and storage capacities for your Oracle NoSQLDatabase Cloud Service.

Basics Behind the Calculation

Before you learn how to estimate throughput and storage for the service, let's reviewthe throughput and storage unit definitions.

• Write Unit (WU): One Write Unit is defined as throughput for up to 1 kilobyte(KB) of data per second. A write operation is any Oracle NoSQL Database CloudService API call that results in insertion, update, or deletion of a record. A NoSQLtable has a write limit value which specifies the number of write units that may beused each second. Index updates also consume write units.

For example, a record size of less than 1 KB requires one WU for a writeoperation. A record size of 1.5 KB requires two WUs for the write operation.

• Read Unit (RU): One Read Unit is defined as throughput for up to 1 KB of dataper second for an eventually consistent read operation. Your NoSQL table has aread limit value which specifies the number of read units that may be used eachsecond.

For example, a record size of less than 1 KB requires one RU for an eventuallyconsistent read operation. A record size of 1.5 KB requires two RUs for aneventually consistent read operation and four RUs for an absolutely consistentread operation.

• Storage Capacity: One storage unit is a single gigabyte (GB) of data storage.

• Absolute Consistency: The data returned is expected to be the most recentlywritten data to the database.

• Eventual Consistency: The data returned may not be the most recently writtendata to the database; if no new updates are made to the data, eventually allaccesses to that data return the latest updated value.

Factors that Impact the Capacity Unit

Before you provision the capacity units, it is important to consider the following factorsthat impact the read, write, and storage capacities.

• Record size: As the record size increases, the number of capacity unitsconsumed to write or read data also increases.

Chapter 3Estimating Capacity

3-25

• Data consistency: Absolute consistency reads are twice the cost of eventualconsistency reads.

• Secondary Indexes: In a table, when an existing record is modified (added,updated, or deleted), updating secondary indexes consume Write Units. Thetotal provisioned throughput cost for a write operation is the sum of write unitsconsumed by writing to the table and updating the local secondary indexes.

• Data modeling choice: With schema-less JSON, each document is self-describing which adds metadata overhead to the overall size of the record. Withfixed schema tables, the overhead for each record is exactly 1 byte.

• Query pattern: The cost of a query operation depends on the number of rowsretrieved, number of predicates, the size of the source data, projections, and thepresence of indexes. The least expensive queries specify a shard key or indexkey (with an associated index) to allow the system to take advantage of primaryand secondary indexes. An application can try different queries and examine theconsumed throughput to help tune the operations.

Real World Example: How to Estimate your Application Workload

Consider an E-commerce application example to learn how to estimate reads andwrites per second. In this example, Oracle NoSQL Database Cloud Service is used tostore the product catalog information of the application.

1. Identify the data model (JSON or fixed-table), record size, and key size for theapplication.

Assume that the E-commerce application follows the JSON data model and thedeveloper has created a simple table with two columns. A record identifier (primarykey) and a JSON document for the product features and attributes. The JSONdocument, which is under 1 KB is as follows:

{ "additionalFeatures": "Front Facing 1.3MP Camera", "os": "Macintosh OS X 10.7", "battery": { "type": "Lithium Ion (Li-Ion) (7000 mAH)", "standbytime" : "24 hours" }, "camera": { "features": ["Flash","Video"], "primary": "5.0 megapixels" }, "connectivity": { "bluetooth": "Bluetooth 2.1", "cell": "T-mobile HSPA+ @ 2100/1900/AWS/850 MHz", "gps": true, "infrared": false, "wifi": "802.11 b/g" }, "description": "Apple iBook is the best in class computer for your professional and personal work.", "display": { "screenResolution": "WVGA (1280 x 968)", "screenSize": "13.0 inches" }, "hardware": { "accelerometer": true, "audioJack": "3.5mm", "cpu": "Intel i7 2.5 GHz", "fmRadio": false,


3-26

"physicalKeyboard": false, "usb": "USB 3.0" }, "id": "appleproduct_1", "images": ["img/apple-laptop.jpg"], "name": "Myshop.com : Apple iBook", "sizeAndWeight": { "dimensions": [ "300 mm (w)", "300 mm (h)", "12.4 mm (d)" ], "weight": "1250.0 grams" }, "storage": { "hdd": "750GB", "ram": "8GB" }}

Assume that the application has 100,000 such records and the primary key isabout 20 bytes in size. Also, assume that there are queries that would readrecords via secondary index. For example, to find all the records that have screensize of 13 inches. So, an index is created on the screenSize field.

The information in summarized as follows:

Tables Rows perTable

Columns perTable

Key Size inBytes

Value Size inBytes (sumof allcolumns)

Indexes Index KeySize in Bytes

1 100000 2 20 2 KB 1 20

2. Identify the list of operations (typically CRUD operations and Index reads) on thetable and at what rate (per second) they are expected.

Operation Number ofOperations (persecond)

Example

Create Records. 3 To create a product.

Read records using the primary key. 200 To read product details using the product ID.

Read records using the secondary index. 1 To fetch all products that have a screen sizeof 13 inches.

Update or add an attribute to a record. 5 To update the product description of acamera

or

To add information about the weight of acamera.

Delete record. 5 To delete an existing product.

3. Identify the read and write consumption in KB.


3-27

Operation Assumptions (Ifany)

Formula Read Consumptionin KB

WriteConsumption inKB

Create Records. Assume that therecords arecreated withoutperforming anycondition checks(if they exist).

Record size (roundedto next KB) + 1KB(index) * (numberof indexes)

0 1 KB + 1 KB (1 )= 2 KB

Read records usingthe primary key.

Minimum Read (1 KB) +Record size round upto KB + Overhead of 2KB

Record size = 1 KB 0

Read records usingthe secondary index.

Assume that 100records arereturned.

[1 KB(index) +record_size] *number_of_records_matched

100 * (1 KB + 1 KB) =200 KB

200 KB + 10 KB = 210KB

Where 10 KBaccounts for variableoverhead that mayoccur depending onthe number of batchesreturned and the sizelimit set for the query.

0

Update existingrecords

Assume that theupdated recordsize is the sameas the old recordsize (1 KB).

Read consumption =[1 KB(index) * 2] +[record_size * 2]

Write consumption =original_record_size+ new_record_size + 1KB (index) * (numberof writes)

2 KB + 2 KB = 4 KB 1 KB + 1 KB + 1KB = 3 KB

Delete record Read consumption = 1KB (index) * 2

Write consumption= record_size +1KB (index) *(number_of_indexes)

1 KB + 2 KB = 2 KB 1 KB + 1 KB (1index) = 2 KB

Using steps 2 and 3, let us determine read and write units for our applicationworkload.

Operations Rate of Operations Reads per Second Writes per Second

Create records 3 0 6

Read records using the primary Key 300 300 0

Read records using the secondary index 10 2100 0

Update existing record 5 20 15

Delete record 1 2 2

Total Read Units: 2422

Total Write Units: 23


3-28

Therefore, the E-commerce application is estimated to have a workload of 2422reads per second and 23 writes per second. Download the Capacity Estimatortool available on Oracle Technology Network to input these values and estimatethroughput and storage of your application.

Note:

The preceding calculations assume eventually consistent read requests. Foran absolute consistency read request, the operation consumes double thecapacity units. Therefore, the read capacity units would be 4844 Read Units.

Handling CapacityLearn how to handle throughput and storage consumed by your table.

Before you run your application in the Oracle NoSQL Database Cloud Service,estimate throughput and storage capacity of the table as described in EstimatingCapacity.

Setting Table Limits

When you create a table, you use the setTableLimits method within theTableRequest class to specify throughput and capacity that the table consumes. Youcan also change the table limits of an existing table by using the same class. See theJava API Reference guide for more information.

To change limits after a table exists:

/* Create a new TableLimits object, setting values for read, write, and storage units */TableLimits newLimits = new TableLimits(30, 10, 10);

/* create the TableRequest object. Set the table limits and the table name.*/TableRequest treq = new TableRequest().setTableLimits(newLimits). setTableName(tableName);TableResult tres = handle.tableRequest(treq);System.out.println("Altering table limits");

/* Wait for the operation to complete */tres.waitForCompletion(handle, 20000, 1000);

To read the new table limits, use the GetTableRequest class:

GetTableRequest gtr = new GetTableRequest().setTableName(tableName);tres = handle.getTable(gtr);System.out.println("New table limits: " + "read units=" + tres.getTableLimits().getReadUnits() + ", write units=" + tres.getTableLimits().getWriteUnits() + ", table size=" + tres.getTableLimits().getStorageGB());

Chapter 3Handling Capacity

3-29

See the NoSQLHandle.getTableUsage API in the Java API Reference guide to getinformation about your table usage characteristics.

Chapter 3Handling Capacity

3-30

4Developing in Oracle NoSQL DatabaseCloud Simulator

Get familiar with the Cloud APIs by using the Oracle NoSQL Database CloudSimulator.

The Oracle NoSQL Database Cloud Simulator simulates the cloud service and letsyou write and test applications locally without accessing Oracle NoSQL DatabaseCloud Service. The Oracle NoSQL Database Java SDK contains a few examples forthe developer to get started with.

You can start developing your application in the Oracle NoSQL Database CloudSimulator, using and understanding the basic examples, before you get started withOracle NoSQL Database Cloud Service.

Extract Oracle NoSQL Database Java SDK and the Oracle NoSQL Database CloudSimulator bundles. Create your application using the Cloud APIs. After building,debugging, and testing your application with the Oracle NoSQL Database CloudSimulator, move your application to Oracle NoSQL Database Cloud Service.

Topics

1. Downloading the Oracle NoSQL Database Cloud Simulator

2. Oracle NoSQL Database Cloud Simulator Compared With Oracle NoSQLDatabase Cloud Service

Downloading the Oracle NoSQL Database Cloud SimulatorLearn how to download and extract the Oracle NoSQL Database Cloud Simulator.

The Oracle NoSQL Database Cloud Simulator is available for download from theOracle Cloud website. To install the Oracle NoSQL Database Cloud Simulator, firstdownload and extract the file.

Note:

Your local system should meet the following requirements to run the OracleNoSQL Database Cloud Simulator:

• Java JDK version 10 or higher installed in your machine.

• A minimum of 5-GB available disk space where you plan to install theOracle NoSQL Database Cloud Simulator.

Perform the following steps:

1. Download the Oracle NoSQL Database Java SDK file from the Oracle NoSQLCloud SDK download page.

4-1

2. Gunzip and untar the .tar.gz package (or extract the files if you have downloadeda .zip package).

The SDK package oracle-nosql-cloud-simulator-1.2.0 is used in thisexample. The actual package name and the .jar file names can change,depending on the release version that you download.

$ tar xvfz oracle-nosql-cloud-simulator-1.2.0.tar.gz

The output displays all directories and files that are part of the package. All theOracle NoSQL Database Cloud Simulator related .jar files are placed in thecloudsim/lib directory.

After extracting the package, read the oracle-nosql-cloud-simulator-1.2.0/README.txt file for instructions on how to start and stop the simulator.

In order to use the Oracle NoSQL Database Cloud Simulator you must download oneof the supported Oracle NoSQL language SDKs. The SDKs have instructions andexample code to connect to either the Oracle NoSQL Database Cloud Simulator or theOracle NoSQL Database Cloud Service.

Oracle NoSQL Database Cloud Simulator Compared WithOracle NoSQL Database Cloud Service

Learn the difference between Oracle NoSQL Database Cloud Simulator and OracleNoSQL Database Cloud Service. The differences helps determine important designconsiderations that you should make before using your application in a productionenvironment.

Oracle NoSQL Database Cloud Simulator is a local version of the Oracle NoSQLDatabase Cloud Service. The server instance that you create in Oracle NoSQLDatabase Cloud Simulator supports relatively limited aggregate throughput whencompared to the Oracle NoSQL Database Cloud Service. Also, the performance ofNoSQL operations on the Oracle NoSQL Database Cloud Simulator is based on thespeed and capability of the machine on which it is deployed.

By comparison, Oracle NoSQL Database Cloud Service is suitable for production usebecause of features such as scalability, availability, and durability.

Oracle NoSQL Database Cloud Simulator has the following limitations when comparedto Oracle NoSQL Database Cloud Service:

• The Oracle NoSQL Database Cloud Simulator can be used for only developmentand testing purposes. Do not use Oracle NoSQL Database Cloud Simulator forperformance measurements or in a production environment.

• At least 5 GB of disk drive space must be available to run the Oracle NoSQLDatabase Cloud Simulator.

• A single instance of the Oracle NoSQL Database Cloud Simulator should bestarted in a root directory (directory where the Oracle NoSQL Database CloudSimulator data is located). Oracle NoSQL Database Cloud Simulator assumesexclusive control over the data storage directory.

• The Oracle NoSQL Database Cloud Simulator does not support or requiresecurity-relevant configurations.

Chapter 4Oracle NoSQL Database Cloud Simulator Compared With Oracle NoSQL Database Cloud Service

4-2

• No hard limit is enforced on the number of tables, size of tables, number ofindexes, or maximum throughput specified for tables (except for the amount ofstorage on the local disk drive).

• Data Definition Language (DDL) operations, such as creating or dropping a table,and creating or dropping an index, are not throttled.

• Operational history is not maintained.

Chapter 4Oracle NoSQL Database Cloud Simulator Compared With Oracle NoSQL Database Cloud Service

4-3

5Using Plugins for Development

Get familiar with the plugins available for developing NoSQL applications in the OracleNoSQL Database Cloud Service from external integrated development environmentsor IDEs.

Topics

1. About IntelliJ Plugin

2. About Eclipse Plugin

About IntelliJ PluginBrowse tables and execute queries on your Oracle NoSQL Database Cloud Serviceinstance or simulator from IntelliJ.

The Oracle NoSQL Database Cloud Service IntelliJ plugin connects to a runninginstance of Oracle NoSQL Database Cloud Service or simulator and allows you to:

• Quickly get started with Oracle NoSQL Database Cloud Service by using theexamples available with the plugin.

• View the tables in your cloud account or simulator.

• Retrieve columns, indexes, primary keys, and shard keys for each table.

• Build and test your SQL queries on a table and obtain results in a tabular format.

• View the data in each column in the JSON format.

Topics:

• Setting Up IntelliJ Plug-in

• Creating a NoSQL Project in IntelliJ

• Connecting to Oracle NoSQL Database Cloud Simulator from IntelliJ

• Connecting to Oracle NoSQL Database Cloud Service from IntelliJ

• Managing Tables Using the IntelliJ Plugin

Setting Up IntelliJ Plug-inLearn how to set up the IntelliJ plug-in for Oracle NoSQL Database Cloud Serviceinstance or simulator.


1. Download and start Oracle NoSQL Database Cloud Simulator. See Downloadingthe Oracle NoSQL Database Cloud Simulator.

2. Download and extract Oracle NoSQL Database Java SDK. See About the OracleNoSQL Database Java SDK.

5-1

3. Install the IntelliJ plugin, and restart the IDE.

You have two options to install the plugin:

• Search the Oracle NoSQL Database Connector in the JetBrains plug-inrepository, and install it, or

• Download the IntelliJ plugin from Oracle Technology Network, and install theplugin from disk.

Tip:

Don't extract the downloaded plugin zip file. Select the plugin in the zipformat while installing it from disk.

After you successfully set up your IntelliJ plugin, create a NoSQL project, and connectit to your Oracle NoSQL Database Cloud Service instance or simulator.

Creating a NoSQL Project in IntelliJLearn how to create a NoSQL project in IntelliJ.


1. Open IntelliJ IDEA. Click File > New > Project.

2. Select Oracle NoSQL examples from the explorer window, and click Next.

3. Browse to the location where you extracted Oracle NoSQL Database Java SDK onyour hard-disk, and click OK.

For example, if you extracted the Oracle NoSQL Database Java SDK in your D:\drive, the path looks like D:\oracle-nosql-java-sdk-5.2.11

4. Click Next.

5. Enter a value for Project Name and Project Location, and click Finish.

6. Once your NoSQL project is created, you can browse the example java files fromthe Project Explorer window.

After you successfully create a NoSQL project in IntelliJ, connect your project to yourOracle NoSQL Database Cloud Service or simulator.

Connecting to Oracle NoSQL Database Cloud Simulator from IntelliJLearn how to connect your NoSQL project to Oracle NoSQL Database CloudSimulator using the IntelliJ plugin.


1. Open your NoSQL project in IntelliJ.

2. Click the icon in the Schema Explorer window to open the Settings dialog forthe plugin.

3. Expand Tools > Oracle NoSQL in the Settings Explorer, and click Connections.

4. Select Cloudsim from the drop-down menu for the connection type.

5. Enter values for the following connection parameters, and click OK.

Chapter 5About IntelliJ Plugin

5-2

Table 5-1 Connection Parameters

Parameter Description Sample Value

Service URL The IP address and porton which the Oracle NoSQLDatabase Cloud Simulator isrunning.

The default value is http://localhost:8080

Tenant Identifier Unique identifier to identifythe tenant.

The default value isexampleId. Retain this valueif you want to test theexamples.

SDK Path Complete path to thedirectory where youextracted the Oracle NoSQLDatabase Java SDK.

D:\oracle-nosql-java-sdk-5.2.11

6. The Intellij plugin connects your project to the Oracle NoSQL Database CloudSimulator and displays its schema in the Schema Explorer window.

Note:

Before connecting your project to Oracle NoSQL Database CloudSimulator, it must be started and running. Otherwise, your connectionrequest will fail in IntelliJ.

After you successfully connect your project to your Oracle NoSQL Database CloudSimulator, you can manage the tables and data in your schema.

Connecting to Oracle NoSQL Database Cloud Service from IntelliJLearn how to connect your NoSQL project to Oracle NoSQL Database Cloud Serviceusing the IntelliJ plugin


1. Open your NoSQL project in IntelliJ.

2. Click the icon in the Schema Explorer window to open the Settings dialog forthe plugin.

3. Expand Tools > Oracle NoSQL in the Settings Explorer, and click Connections.

4. Select Cloud from the drop-down menu for the connection type.

5. Enter values for the following connection parameters, and click OK.


5-3

Table 5-2 Connection Parameters

Parameter Description Sample Value

Endpoint Regional network accesspoint to the Oracle NoSQLDatabase Cloud Service.

https://nosql.us-ashburn-1.oci.oraclecloud.com (for the AshburnOracle NoSQL DatabaseCloud Service regionidentifier in the NorthAmerica region. See DataRegions and AssociatedService Endpoints for a list ofservice endpoints.

SDK Path Complete path to thedirectory where youextracted the Oracle NoSQLDatabase Java SDK.

D:\oracle-nosql-java-sdk-5.2.11

Tenant ID Tenancy's OCID for yourOracle NoSQL DatabaseCloud Service.

See Where to getthe Tenancy's OCIDand User's OCID inOracle Cloud InfrastructureDocumentation.

User ID User's OCID for your OracleNoSQL Database CloudService.

Fingerprint The fingerprint andpassphrase of the signingkey created while generatingand uploading the APISigning Key.

See the followingresources in OracleCloud InfrastructureDocumentation:• How to Generate an API

Signing Key to generatethe signing key with anoptional passphrase.

• See How to Get theKey's Fingerprint to getthe key's fingerprint

Passphrase(Optional)

Private Key The private key generated forthe user.

For the application user,an API signing key mustbe generated and uploaded.See How to Generate an APISigning Key to generate thesigning key with an optionalpassphrase.

Compartment (Optional) The compartment ID for yourNoSQL database schema.

If no value is specified,it defaults to the Rootcompartment.

6. The Intellij plugin connects your project to the Oracle NoSQL Database CloudService and displays its schema in the Schema Explorer window.

7. If required, you can change your service endpoint or compartment from the

Schema Explorer window itself. To do this, click the icon in the SchemaExplorer window.

A dialog window appears where you can provide the new values for Endpoint andCompartment. Enter the values that you want to modify, and click OK.You can provide values for:

• Both Endpoint and Compartment, or


5-4

• Endpoint alone. In this case, the Compartment defaults to the Rootcompartment in that region.

After you successfully connect your project to your Oracle NoSQL Database CloudService, you can manage the tables and data in your schema.

Managing Tables Using the IntelliJ PluginLearn how to create tables and view table data in Oracle NoSQL Database CloudService or Oracle NoSQL Database Cloud Simulator from IntelliJ.

After connecting to the Oracle NoSQL Database Cloud Simulator or Oracle NoSQLDatabase Cloud Service, you can execute the examples downloaded with OracleNoSQL Database Java SDK to create a sample table. With the help of the IntelliJPlugin, you can view the tables and their data in the Schema Explorer window.

To execute an example program:

1. Open the NoSQL project connected to your Oracle NoSQL Database CloudService or simulator.

2. Locate and click BasicTableExample in the Project Explorer window. By looking atthe code, you can notice that this program creates a table called audienceData,puts two rows into this table, queries the inserted rows, deletes the inserted rows,and finally drops the audienceData table.

3. To pass the required arguments, click Run > Edit Configurations. Depending onthe connection type, enter the following program arguments, and click OK.

Table 5-3 Program Arguments

Connection Type Program Arguments More Information

Cloudsim http://localhost:8080 If you started the OracleNoSQL Database CloudSimulator on a different port,you must replace 8080 withthat port number.

Cloud us-ashburn-1-configFileD:\OCI_PROP\config

The first argument indicatesthe data region of yourOracle NoSQL DatabaseCloud Service. Thesecond argument passesa configuration file thatcontains the credentialsto connect to the OracleNoSQL Database CloudService.

4. To execute this program, click Run > Run 'BasicExampleTable' or press Shift +10.

5. Verify the logs in the terminal to confirm that the code executed successfully. Youcan see the display messages that indicate table creation, rows insertion, and soon.


5-5

Tip:

As the BasicExampleTable deletes the inserted rows and drops theaudienceData table, you can't view this table in the Schema Explorer.If you want to see the table in the Schema Explorer, comment thecode that deletes the inserted rows and drops the table, and rerun theprogram.

6. To view the tables and their data:

a. Locate the Schema Explorer, and click the icon to reload the schema.

b. Locate the audienceData table under your tenant identifier, and expand it toview its columns, primary key, and shard key details.

c. Double-click the table name to view its data. Alternatively, you can right-clickthe table and select Browse Table.

d. A record viewer window appears in the main editor. Click Execute to run thequery and display table data.

Note:

As of the current release, only SELECT queries are supported on theNoSQL tables from the Schema Explorer.

e. To view individual cell data separately, double-click the cell.

About Eclipse PluginBuild and run your Oracle NoSQL Database Cloud Service application quickly from theEclipse IDE.

To enhance your experience of building an Oracle NoSQL Database Cloud Serviceapplication, a plugin is available in Eclipse. This plugin connects to a running instanceof Oracle NoSQL Database Cloud Service or Cloud Simulator and allows you to:

• Quickly get started with Oracle NoSQL Database Cloud Service by using theexamples available with the plugin.

• Explore development/test data from tables in your Oracle NoSQL Database CloudService or Cloud Simulator.

• Build and test your queries.

• Retrieve columns, indexes, primary keys, and shard keys for each table.

• Build and test your SQL queries on a table and obtain results in a tabular format.

To use the Eclipse plugin:

1. Download the Eclipse plugin from Oracle Technology Network.

2. Follow the instructions given in the README file and install the plugin.

3. After installing the Eclipse plugin, you can connect to your Oracle NoSQLDatabase Cloud Service or Oracle NoSQL Database Cloud Simulator and execute

Chapter 5About Eclipse Plugin

5-6

the code to read/write the tables. For more details, you can access the helpcontent embedded with Eclipse. To access the help content:

a. Click Help Contents from the Help menu.

b. Locate and expand the Oracle NoSQL Plugin Help Contents section.

c. This lists all the help topics available for Oracle NoSQL Plugin.

d. Refer the help topic as per your requirement.

Chapter 5About Eclipse Plugin

5-7

6Using the Console to Manage Tables

Learn how to manage Oracle NoSQL Database Cloud Service tables using theconsole.

Topics:

• Learn how to read, update, and delete data from Oracle NoSQL Database CloudService tables from the console. See Managing Table Data.

• Learn how to create and manage Oracle NoSQL Database Cloud Service tablesand indexes from the console. See Managing Tables and Indexes.

• Learn how to monitor Oracle NoSQL Database Cloud Service tables from theconsole. See Monitoring Tables and Indexes.

Accessing the Service from the Infrastructure ConsoleLearn how to access the Oracle NoSQL Database Cloud Service service from theInfrastructure Console.

1. Locate the service URL from the welcome email, and then sign in to your OracleNoSQL Database Cloud Service.

2. In Console, click the icon in the top left corner.

3. Under Database, select NoSQL Database.

4. Select the compartment created for your tables by the service administrator.

To view help for the current page, click the help icon at the top of the page.

Managing Table DataLearn how to manage Oracle NoSQL Database Cloud Service table data for yourorganization.

Topics:

• Inserting Data Into Tables

• Viewing Table Data

• Updating Table Data

• Deleting Table Data

• Downloading Table Data

6-1

Inserting Data Into TablesLearn how to insert data into Oracle NoSQL Database Cloud Service tables from theNoSQL console.

The NoSQL console lets you insert new rows into the Oracle NoSQL Database CloudService tables in two modes:

1. Simple Input Mode: You can use this mode to provide the values for the newrows declaratively.

2. Advanced JSON Input Mode: You can use this mode to provide the values forthe new rows in JSON format.

Inserting Data Into Tables: Simple Input ModeLearn how to insert data into Oracle NoSQL Database Cloud Service tables by usingthe Simple Input insertion mode.

To insert data into tables:

1. Access the NoSQL console from the Infrastructure Console. See Accessing theService from the Infrastructure Console.

2. The NoSQL console lists all the tables in the tenancy. To view table details, doeither of the following:

• Click the table name, or

• Click the action menu corresponding to the table name and select ViewDetails.

The Table Details page opens up.

3. Click Insert Row.

4. In the Insert Row window, select Simple Input for Entry Mode.

5. All the columns in the table are listed. Input the data for the columns of the table.For some column types, such as Binary, you upload the data.

Note:

Entering a value is mandatory for all non-nullable columns of the table.


The record is inserted into the table.

To view help for the current page, click the help link at the top of the page.

Inserting Data Into Tables: Advanced JSON Input ModeLearn how to insert data into Oracle NoSQL Database Cloud Service tables by usingthe Advanced JSON input mode.

To insert data into tables:

Chapter 6Managing Table Data

6-2







4. In the Insert Record window, select Advanced JSON Input for Entry Mode.

5. Paste or upload the Record Definition in JSON format.


The record is inserted into the table.


Viewing Table DataLearn how to view data in Oracle NoSQL Database Cloud Service tables from theNoSQL console.

To view table data:






3. In the Table Details page, select the Table Rows tab under Resources.

4. By default, the query text is populated with a SQL query that will retrieve all therecords from the table. You can modify this query with any valid SQL for OracleNoSQL statement. See SQL Reference for Oracle NoSQL Database.

5. Click Run Query.

The table data is displayed in the Records section.

Updating Table DataLearn how to update data in Oracle NoSQL Database Cloud Service tables from theNoSQL console.

To update table data:



6-3







5. Click the action menu corresponding to the row you wish to update, and selectUpdate Row.

6. Modify the values in Simple Input or Advanced JSON Input Updation Mode.

7. Click Update Row.


Deleting Table DataLearn how to delete data in Oracle NoSQL Database Cloud Service tables from theNoSQL console.

To delete table data:








5. Click the action menu corresponding to the row you wish to delete, and selectDelete.

The Delete Row confirmation dialog opens.

6. Click Delete.

The row is deleted.


6-4

Downloading Table DataLearn how to download data in Oracle NoSQL Database Cloud Service tables from theNoSQL console.

To download table data:








5. Click the action menu corresponding to the row you wish to download, and selectDownload JSON.

The row downloads in JSON format.

Managing Tables and IndexesLearn how to create and manage Oracle NoSQL Database Cloud Service tables andindexes for your organization.

Topics:

• Creating Tables

• Creating Indexes

• Editing Tables

• Moving Tables

• Altering Tables

• Deleting Tables

• Deleting Indexes

Creating TablesYou can create new Oracle NoSQL Database Cloud Service table from the NoSQLconsole.

The NoSQL console lets you create the Oracle NoSQL Database Cloud Service tablesin two modes:

Chapter 6Managing Tables and Indexes

6-5

1. Simple Input Mode: You can use this mode to create the NoSQL Database CloudService table declaratively, that is, without writing a DDL statement.

2. Advanced DDL Input Mode: You can use this mode to create the NoSQLDatabase Cloud Service table using a DDL statement.

Creating Table: Simple Input ModeLearn how to create a table from the NoSQL console by using the Simple Input tablecreation mode.

To create a table:


2. Click Create Table.

3. In the Create Table window, select Simple Input for Table Creation Mode.

4. In the Name field, enter a table name that is unique within your tenancy.

Table names must conform to Oracle NoSQL Database Cloud Service namingconventions. See Oracle NoSQL Database Cloud Service Limits.

5.

In the Primary Key Columns section, enter primary key details:

• Column Name: Enter a column name for the primary key in your table. SeeOracle NoSQL Database Cloud Service Limits to learn about column namingrequirements.

• Type: Select the data type for your primary key column.

• Precision: This is applicable for TIMESTAMP typed columns only. Specifyprecision for timestamp values ranging from zero to nine. 0 is the minimumprecision, and 9 is the maximum precision. When declaring a timestamp field,the precision is required.

• Set as Shard Key: Click this option to set this primary key column as shardkey.


6-6

• + Another Primary Key Column: Click this button to add more columns whilecreating a composite (multi-column) primary key.

• Use the up and down arrows to change the sequence of columns whilecreating a composite primary key.

6.

In the Columns section, enter column details:

• Column Name: Enter the column name. Ensure that you conform to columnnaming requirements described in Oracle NoSQL Database Cloud ServiceLimits.

• Type: Select the data type for your column.


• Size: This is applicable for BINARY typed columns only. Specify the size inbytes to make the binary a fixed binary.

• Default Value: (optional) Supply a default value for the column.

Note:

Default values can not be specified for binary and JSON data typecolumns.

• Value is Not Null: Click this option to specify that a column must always havea value.

• + Another Column: Click this button to add more columns.

• Click the delete icon to delete a column.

7. In the Reserved Capacity section, enter the appropriate capacity values for thetable.

• Read Capacity (ReadUnits): Enter the number of read units. See EstimatingCapacity to learn about read units.

• Write Capacity (WriteUnits): Enter the number of write units. See EstimatingCapacity to learn about write units.

• Disk Storage (GB): Specify the disk space in gigabytes (GB) to be used bythe table. See Estimating Capacity to learn about storage capacity.


6-7

8.

(Optional) To specify advanced options, click Show Advanced Options and enteradvanced details:

• Table Time to Live (Days): (optional) Specify expiration time for the rowsin the table. After this time, the rows expire automatically, and are no longeravailable. The default value is zero, indicating no expiration time.

Note:

Updating Table Time to Live (TTL) will not change the TTL value ofany existing data in the table. The new TTL value will apply only tothose records that are added to the table after this value is modified.

In the Tags section, enter:

• Tag Namespace: Select a tag namespace from the select list. A tagnamespace is like a container for your tag keys. It is case insensitive andmust be unique across the tenancy.

• Tag Key: Enter the name to use to refer to the tag. A tag key is caseinsensitive and must be unique within a namespace.

• Value: Enter the value to give your tag.

• + Additional Tag: Click to add more tags.


The table is created and listed in the NoSQL console.To view help for the current page, click the help link at the top of the page.

Creating Table: Advanced DDL Input ModeLearn how to create a table from the NoSQL console by using the Advanced DDLInput table creation mode.

To create a table:


6-8



3. In the Create Table window, select Advanced DDL Input for Table CreationMode.

4. In the DDL Input section, enter the create table DDL statement for Query. SeeSQL Reference for Oracle NoSQL Database.

5. In the Reserved Capacity section, enter the appropriate capacity values for thetable.



• Disk Storage (GB): Specify the disk space in gigabyte (GB) to be used by thetable. See Estimating Capacity to learn about storage capacity.

6.

(Optional) To specify advanced options, click Show Advanced Options and enteradvanced details:

• Tag Namespace: Select a tag namespace from the select list. A tagnamespace is like a container for your tag keys. It is case insensitive andmust be unique across the tenancy.

• Tag Key: Enter the name to use to refer to the tag. A tag key is caseinsensitive and must be unique within a namespace.

• Value: Enter the value to give your tag.

• + Additional Tag: Click to add more tags.


The table is created and listed in the NoSQL console.To view help for the current page, click the help link at the top of the page.


6-9

Creating IndexesLearn how to create indexes in Oracle NoSQL Database Cloud Service tables from theNoSQL console.

To create indexes:






3. In the Table Details page, select the Indexes tab under Resources.

You will see a list of all the indexes added to the table.

4. Click Add Index.

5. In the Create Index window, enter a name for the index that is unique withinthe table. See Oracle NoSQL Database Cloud Service Limits to learn about thenaming restrictions for indexes.

6. In the Index Columns section, enter index details:

• Index Column Name: Select the column that you would like included in theindex.

• + Another Index Column: Click this button to include another column in theindex.

• Use the up and down arrow to change the sequence of the columns in theindex being created.

• Click the delete icon next to any column to remove it from the index beingcreated.

7. Click Create Index.

The index is created.


Editing TablesYou can update reserved capacity and Time to Live (TTL) values for your OracleNoSQL Database Cloud Service tables from the NoSQL console.

To edit tables:





6-10



3. In the Table Details page, click Edit.

4. In the Edit Table window, modify the following values for the table:

Updating Reserved Capacity



• Disk Storage (GB): Specify the disk space in gigabytes (GB) to be used bythe table. See Estimating Capacity to learn about storage capacity.

• Click Update Limits.

Updating Time To Live

• Table Time to Live (Days): (optional) Specify expiration time for the rowsin the table. After this time, the rows expire automatically, and are no longeravailable. The default value is zero, indicating no expiration time.

Note:

Updating Table Time to Live (TTL) will not change the TTL value ofany existing data in the table. The new TTL value will apply only tothose records that are added to the table after this value is modified.

• Click Update TTL.

5. (Optional) To dismiss the changes, click Close.


Altering TablesLearn how to alter Oracle NoSQL Database Cloud Service tables by adding in simpleor advanced mode, or deleting columns using the NoSQL console.

The NoSQL console lets you alter the Oracle NoSQL Database Cloud Service tablesin two modes:

1. Simple Input Mode: You can use this mode to alter the NoSQL Database CloudService table declaratively, that is, without writing a DDL statement.

2. Advanced DDL Input Mode: You can use this mode to alter the NoSQL DatabaseCloud Service table using a DDL statement.

Adding Table Columns: Simple Input ModeLearn how to add table columns to an Oracle NoSQL Database Cloud Service table byusing the Simple Input table column update mode.

To add table columns:


6-11






3. In the Table Details page, select the Columns tab under Resources.

You will see a list of all the columns added to the table.

4. Click Add Columns.

5. In the Add Columns window, select Simple Input for Table Column UpdateMode.

6.

In the Columns section, enter column details:

• Column Name: Enter the column name. Ensure that you conform to columnnaming requirements described in Oracle NoSQL Database Cloud ServiceLimits.

• Type: Select the data type for your column.


• Size: This is applicable for BINARY typed columns only. Specify the size inbytes to make the binary a fixed binary.

• Default Value: (optional) Supply a default value for the column.

Note:

Default values can not be specified for binary and JSON data typecolumns.

• Value is Not Null: Click this option to specify that a column must always havea value.


6-12

• + Another Column: Click this button to add more columns.

• Click the delete icon to delete a column.


The new columns are added to the table.


Adding Table Columns: Advanced DDL Input ModeLearn how to add table columns to an Oracle NoSQL Database Cloud Service table byusing the Advanced DDL table column update mode.

To add table columns:









5. In the Add Columns window, select Advanced DDL Input for Table ColumnUpdate Mode.

6. Enter the update table DDL statement. For an example, See Alter Statement inSQL Reference for Oracle NoSQL Database .


The new columns are added to the table.

Deleting Table ColumnsLearn how to delete table columns from an Oracle NoSQL Database Cloud Servicetable by using the NoSQL console.

To delete columns from a table:







6-13



4. Click the action menu corresponding to the column you wish to delete and selectDelete.

5. In the Delete Column confirmation dialog, click Delete to confirm delete.

The column is deleted from the table.

Moving TablesLearn how to move Oracle NoSQL Database Cloud Service table to a differentcompartment from the NoSQL console.

To move a table:






3. In the Table Details page, click Move Table.

4. In the Move Resource to a Different Compartment window, modify the followingvalues for the table:

• Choose New Compartment: Select the new compartment from the select list.

5. Click Move table.

6. (Optional) To dismiss the changes, click the Cancel link on the top right corner.


Deleting TablesLearn how to delete Oracle NoSQL Database Cloud Service tables from the NoSQLconsole.

To delete tables:


2. The NoSQL console lists all the tables in the tenancy. To delete the table, do eitherof the following:

• Click the table name. In the Table Details page, click the Delete button, or

• Click the action menu corresponding to the table name you wish to delete andselect Delete.

The Delete Table confirmation dialog opens.

3. Click Delete.


6-14

The table is deleted.

Deleting IndexesLearn how to delete Oracle NoSQL Database Cloud Service indexes from the NoSQLconsole.

To delete indexes:








4. Click the action menu corresponding to the index you wish to delete, and selectDelete.

The Delete Index confirmation dialog opens.

5. Click Delete.

The index is deleted.

Monitoring Tables and IndexesLearn how to monitor the Oracle NoSQL Database Cloud Service tables and theirindexes.

Topics:

• Viewing Tables

• Viewing Indexes

• Viewing Table Details

• Viewing Table Metrics

Viewing TablesYou can view Oracle NoSQL Database Cloud Service tables from the NoSQL console.

To view tables:


2. You can view all the tables in your tenancy from the NoSQL console.

Chapter 6Monitoring Tables and Indexes

6-15

Viewing IndexesYou can view Oracle NoSQL Database Cloud Service the list of indexes created for aNoSQL table from the NoSQL console.

To view indexes:








Viewing Table DetailsLearn how to view Oracle NoSQL Database Cloud Service table details from theNoSQL console.

To view table details:






3. From the Table Details page, you can view all table columns, indexes, rows, andmetrics.

Viewing Table DDLYou can view the DDL statement used to create a table for the Table Details page.

To view table DDL:

1. In the Table Details page, click View Table DDL.

The View Table DDL window displays the table DDL statement.

2. Now, you can select and copy the table DDL statement from the window. Click OKto close the window.


6-16

Viewing Table MetricsLearn how to view Oracle NoSQL Database Cloud Service table metrics from theNoSQL console.

To view table metrics:






3. In the Table Details page, select the Metrics tab under Resources.

Table metrics such as Read Units, Write Units, Storage GB, Read Throttle Count,Write Throttle Count, and Storage Throttle Count show up. You can filter themetrics by date, change interval, and statistic value.

4. For each of the metrics displayed on this page, you can perform the followingactions:

• View Query in Metrics Explorer: This page lets you write and edit queries inMonitoring Query Language (MQL), using metrics from either your applicationor an Oracle Cloud Infrastructure service. If you're not familiar with MQL, seeMonitoring Query Language (MQL) Reference. To learn more about this page,see Metrics Explorer.

• Copy Chart URL: Click this option to copy the default metrics chart URL forany future reference.

• Copy Query (MQL): Click this option to copy the MQL query used to createthe default metrics chart. If you're not familiar with MQL, see Monitoring QueryLanguage (MQL) Reference.

• Create an Alarm on this Query: Click this option to create alarms to monitoryour cloud resources. To learn about alarms, see Managing Alarms.


6-17

7Managing Subscriptions

Learn how to manage Oracle NoSQL Database Cloud Service subscriptions and it'susers.

Topics

• To get an overview of setting up Oracle NoSQL Database Cloud Service, seeSetting up Your Service.

• To learn how to estimate the monthly cost of your Oracle NoSQL Database CloudService, see Estimating Your Monthly Cost.

• To learn about creating compartments in your Oracle NoSQL Database CloudService, see Creating a Compartment.

• To learn how to set up users, groups, and policies, see Setting Up Users, Groups,and Policies.

Setting up Your ServiceIf you're setting up Oracle NoSQL Database Cloud Service for the first time, followthese tasks as a guide.

Task Reference Related Information

Place an order for Oracle NoSQLDatabase Cloud Service or sign up forthe Oracle Free Trial.

Signing Up for Oracle CloudInfrastructure in Oracle CloudInfrastructure Documentation.

To learn how to estimate themonthly cost of your OracleNoSQL Database Cloud Servicesubscription, see Estimating YourMonthly Cost.

To upgrade your free accountor to change your paymentmethod, see Changing YourPayment Method in Oracle CloudInfrastructure Documentation.

Activate your Oracle Cloud accountand sign in for the first time.

Signing In to the Console in OracleCloud Infrastructure Documentation.

To familiarize yourself with OracleCloud Infrastructure Console,see Using the Console inOracle Cloud InfrastructureDocumentation.

(Recommended) Create acompartment for your service.

Creating a Compartment If you're not familiarwith compartments, seeUnderstanding Compartmentsin Oracle Cloud InfrastructureDocumentation.

Manage security for your service. Typical Process to Manage Securityfor Oracle NoSQL Database CloudService

To familiarize yourself withNoSQL Database Cloud Servicesecurity model, see About OracleNoSQL Database Cloud ServiceSecurity Model.

7-1

Estimating Your Monthly CostLearn how to estimate the monthly cost of your Oracle Cloud subscription.

When you are ready to order your Oracle Cloud service, Oracle provides you witha cost estimator to figure out your monthly usage and costs before you commit to asubscription model or an amount.

The Cost Estimator automatically calculates your monthly cost based on your input ofread units, write units, and storage. But for you to understand how to calculate theread and write units for your application, follow these steps:

1. Step 1: Navigate to the Estimating Capacity topic. Estimate your applicationworkload by using the example and formulas described in this topic.

Step 2: Download and use the Capacity Estimator from Oracle TechnologyNetwork to estimate write units, read units, and the storage capacity for yourapplication based on the application workload and database operations criteria.

2. Step 2: Access the Cost Estimator on the Oracle Cloud website. Selectthe Data Management check box. Scroll through to locate Oracle NoSQLDatabase Cloud, and click Add to add an entry for Oracle NoSQL DatabaseCloud under the Configuration Options. Expand NoSQL Database to find thedifferent Utilization and configuration options. Input values for the Utilizationand Configuration parameters to estimate the cost for Oracle NoSQL DatabaseCloud Service usage from your Oracle Cloud Pay-As-You-Go and MonthlyFlex subscriptions.

Creating a CompartmentWhen you sign up for Oracle Cloud Infrastructure, Oracle creates your tenancy witha root compartment that holds all your cloud resources. You then create additionalcompartments within the tenancy (root compartment) and corresponding policies tocontrol access to the resources in each compartment. Before you create an OracleNoSQL Database Cloud Service table, Oracle recommends that you set up thecompartment where you want the table to belong.

You create compartments in Oracle Cloud Infrastructure Identity and AccessManagement (IAM). See Setting Up Your Tenancy and Managing Compartments inOracle Cloud Infrastructure Documentation.

Chapter 7Estimating Your Monthly Cost

7-2

8Managing Table Access and Security

As administrator, you manage access to your Oracle NoSQL Database Cloud Servicetables using security features in Oracle Cloud Infrastructure Identity and AccessManagement (IAM).

Topics

• To get an overview of Oracle NoSQL Database Cloud Service security model, seeAbout Oracle NoSQL Database Cloud Service Security Model

• To learn about the tasks involved in implementing security in Oracle NoSQLDatabase Cloud Service, see Typical Process to Manage Security for OracleNoSQL Database Cloud Service

• To learn about the writing policies that control access to the Oracle NoSQLDatabase Cloud Service, see Policy Reference

• To view typical policy statements that you might use to authorize access to OracleNoSQL Database Cloud Service tables, see Typical Policy Statements to ManageTables

• To learn how to manage user permissions, see Giving Another User Permission toManage NoSQL Tables

About Oracle NoSQL Database Cloud Service SecurityModel

Learn about the security model for Oracle NoSQL Database Cloud Service.

Policies

Oracle NoSQL Database Cloud Service uses the Oracle Cloud Infrastructure Identityand Access Management security model that is built on the policies. A policy is adocument that specifies who can access which Oracle Cloud Infrastructure resources,including NoSQL tables that your company has, and how they can access theseresources. A policy allows a group to work in certain ways with specific types ofresources such as NoSQL Tables in a particular compartment.

To govern the control of your tables, your company will have at least one policy. Eachpolicy consists of one or more policy statements that follow this basic syntax:

Allow group <group_name> to <verb> <resource-type> in compartment <compartment_name>

To learn how policies work, see Overview of Policies in Oracle Cloud InfrastructureDocumentation.

8-1

Groups

In Oracle Cloud Infrastructure Identity and Access Management, you organize Userswithin groups that usually share the same type of access to a particular set of NoSQLtables or compartments.

You can grant access to the NoSQL Tables at the group and compartment level,by writing a policy that gives a group a specific type of access within a particularcompartment, or to the tenancy itself. If you give a group access to the tenancy, thegroup automatically gets the same type of access to all the compartments inside thetenancy. For example, after you create a table in the compartment ProjectA, you mustwrite a policy to grant access to the group(s) you want them to manage or use thetables. Otherwise, the tables are not even visible to the groups that don't have access.For example, to allow the Developer group to manage all the NoSQL resources, youcan create the following policy:

allow group Developers to manage nosql-family in compartment ProjectA

Verbs

A verb specifies the type of access being granted by the policy. For example, inspectnosql-tables lets you list the NoSQL tables. Inspect, read, use, and manage arethe verbs supported by Oracle NoSQL Database Cloud Service. See Verbs in OracleCloud Infrastructure Documentation.

Resource-types

Resources are the cloud objects that your company's employees create and use wheninteracting with the Oracle Cloud Infrastructure (OCI). Oracle defines resource-typesyou can use in policies. nosql-tables, nosql-rows, and nosql-indexes are threeindividual resource-types supported by NoSQL Database Cloud Service.

By specifying a resource-type in a policy, you give access permissions against thatresource type alone. For example, to grant read permissions on the rows of all NoSQLtables in the tenancy, to the viewers group, you can create a policy as:

allow group viewers to read nosql-rows in tenancy

To simplify writing policies, NoSQL Database Cloud Service also provides anaggregate resource-type called nosql-family. nosql-family includes nosql-tables,nosql-indexes, and nosql-rows that are often managed together. For example, togrant full access to NoSQL Tables in the tenancy, to the viewers group, you can write apolicy as:

allow group viewers to manage nosql-family in tenancy

Compartments

A compartment is the fundamental component of Oracle Cloud Infrastructure. You canorganize the Oracle NoSQL Database Cloud Service resources within compartments.Compartments are used to separate tables for measuring usage and billing, definingaccess, and isolating the resources between different projects or business units.

Chapter 8About Oracle NoSQL Database Cloud Service Security Model

8-2

Note:

Tenancy is the root compartment that contains all of your organization'sOracle Cloud Infrastructure resources.

All the Oracle Cloud Infrastructure Identity and Access Management resources, users,groups, compartments and policies are global and available across all regions, but themaster set of definitions reside in a single region, the home region. All the changesto your IAM resources must be made in your home region. To learn more about theIAM components, see Overview of Oracle Cloud Infrastructure Identity and AccessManagement in Oracle Cloud Infrastructure Documentation.

Typical Process to Manage Security for Oracle NoSQLDatabase Cloud Service

Learn about the typical sequence of tasks that a Oracle NoSQL Database CloudService administrator performs to implement a security model in an organization.

As a Oracle NoSQL Database Cloud Service administrator, you must execute thefollowing tasks to implement a security model in your organization:

• Define users, groups, and one or more compartments to hold the NoSQL tablesfor your organization. See Setting Up Users, Groups, and Policies

• Create one or more policies, each written in the policy language. See PolicyReference

• Place users into the appropriate groups depending on the compartments andresources they need to access. See Setting Up Users, Groups, and Policies andTypical Policy Statements to Manage Tables

• Provide a one-time password to the users to access the Console and work withthe compartments.

After you complete the above steps, the users can access the Console, change theirone-time password, and work with specific cloud resources, as stated in the policies.

Setting Up Users, Groups, and PoliciesOracle NoSQL Database Cloud Service uses Oracle Cloud Infrastructure Identityand Access Management to provide secure access to Oracle Cloud. Oracle CloudInfrastructure Identity and Access Management enables you to create user accountsand give users permission to inspect, read, use, or manage tables.

1. Sign-in to your Cloud Account as Cloud Account Administrator.

2. In Oracle Cloud Infrastructure Console, add one or more users.

a. Under Governance and Administration, click Identity, then Users.

b. Click Create User.

c. Enter details about the user, and click Create.

3. In Oracle Cloud Infrastructure Console, create an OCI group.

a. Under Governance and Administration, click Identity, then Groups.

Chapter 8Typical Process to Manage Security for Oracle NoSQL Database Cloud Service

8-3

b. Click Create Group.

c. Enter details about the group.

For example, if you're creating a policy that gives users permissions to fullymanage Oracle NoSQL Database Cloud Service tables you might name thegroup nosql_service_admin (or similar) and include a short description suchas "Users with permissions to set up and manage Oracle NoSQL DatabaseCloud Service tables on Oracle Cloud Infrastructure" (or similar).

4. Create a policy that gives users belonging to an OCI group, specific accesspermissions to Oracle NoSQL Database Cloud Service tables or compartments.

a. Under Governance and Administration, click Identity, then Policies.

b. Select a compartment, and click Create Policy.

For details and examples, see Policy Reference and Typical Policy Statementsto Manage Tables.

If you're unfamiliar about how policies work, see How Policies Work.

5. To manage and use NoSQL tables via Oracle NoSQL Database Cloud ServiceSDKs, the user must set up the API keys. See Acquiring Credentials.

Note:

Federated users can also manage and use Oracle NoSQL DatabaseCloud Service tables. This requires the service administrator to setup the federation in Oracle Cloud Infrastructure Identity and AccessManagement. See Federating with Identity Providers.

Users belonging to any groups mentioned in the policy statement get their newpermission when they next sign in to the Console.

Policy ReferenceThis topic covers details for writing policies to control access to the Oracle NoSQLDatabase Cloud Service.

A policy is a document that specifies who can access which Oracle CloudInfrastructure resources that your company has, and how. See Overview of Policiesto learn basics of policies.

The overall syntax of a policy statement is as follows:

Allow <subject> to <verb> <resource-type> in <location> where <conditions>

For detailed explanation of this syntax, see Policy Syntax in Oracle CloudInfrastructure Documentation.

Resource-TypesLearn about the resource types supported by Oracle NoSQL Database Cloud Service.

Chapter 8Policy Reference

8-4

Resources are the cloud objects that your company's employees create and use wheninteracting with the Oracle Cloud Infrastructure (OCI).

Individual Resource-Types

An Individual Resource-Type represents a specific type of resource. As a user, youcan create NoSQL tables, build indexes and populate rows in those tables in OracleNoSQL Database Cloud Service. Accordingly, three individual resource-types areidentified for Oracle NoSQL Database Cloud Service, as:

• nosql-tables

• nosql-rows

• nosql-indexes

Aggregate Resource-Type

Multiple individual resource-types that are often managed together are collectivelyidentified as Aggregate Resource-Types. There is only one aggregate resource-type inOracle NoSQL Database Cloud Service, as:

• nosql-family

Note:

A policy that uses <verb> nosql-family is equivalent to writing one witha separate <verb> <individual resource-type> statement for each of theindividual resource-types.

See the table in Details for Verb + Resource-Type Combinations for a detailedbreakout of the API operations covered by each verb, for each individual resource-typeincluded in nosql-family.

Supported VariablesLearn about the variables supported by Oracle NoSQL Database Cloud Service.

Oracle NoSQL Database Cloud Service supports all the general variables. SeeGeneral Variables for All Requests. All three NoSQL resource types can use thefollowing variables, except for ListTables and CreateTable.

Table 8-1 Supported Variables

Variable Variable Type Comments

target.nosql-table.id OCID Use this variable to controlaccess to specific NoSQLtable by OCID.

target.nosql-table.name String Use this variable to controlaccess to specific NoSQLtable by name.


8-5

Details for Verb + Resource-Type CombinationsLearn about the permissions and API operations covered by each verb.

The level of access is cumulative as you go from inspect > read > use > manage. Aplus sign (+) in a table cell indicates incremental access compared to the cell directlyabove it, whereas no extra indicates no incremental access.

For example, the read verb for the nosql-tables resource-type includes the samepermissions and API operations as the inspect verb, plus the NOSQL_TABLE_READpermission and the GetTable API operation. In the case of the nosql-tables resource-type, the use verb covers UpdateTable API operations compared to read. Lastly,manage covers more permissions and operations compared to use.

nosql-tables

Table 8-2 nosql-tables

Verb Permissions REST APIs FullyCovered

NoSQL Cloud DriverRequest Covered

INSPECT NOSQL_TABLE_INSPECT

ListTables ListTableRequest

READ INSPECT +NOSQL_TABLE_READ

GetTable GetTableRequest

ListWorkRequests

GetWorkRequest

ListWorkRequestErrors

ListWorkRequestLogs

None

ListTableUsage TableUsageRequest

USE READ +NOSQL_TABLE_ALTER

UpdateTable

DeleteWorkRequest

TableRequest• change

TableLimits• ALTER TABLE

MANAGE USE +NOSQL_TABLE_CREATE

CreateTable TableRequest(CREATE TABLE)

NOSQL_TABLE_DROP

DeleteTable TableRequest (DROPTABLE)

NOSQL_TABLE_MOVE

ChangeTableCompartment

Not supported

nosql-rows

Table 8-3 nosql-rows



INSPECT None None None


8-6

Table 8-3 (Cont.) nosql-rows



READ NOSQL_ROWS_READ

GetRow

Query (SELECT)

PrepareStatement

SummarizeStatement

• GetRequest• PrepareRequest• QueryRequest

(SELECT)

USE READ +NOSQL_ROWS_INSERT

UpdateRow

Query (INSERT/UPSERT, UPDATE)

• PutRequest• WriteMultipleReq

uest(Put)• QueryRequest(IN

SERT/UPSERT,UPDATE)

MANAGE USE +NOSQL_ROWS_DELETE

DeleteRow

Query (DELETE)

• DeleteRequest• MultiDeleteReque

st• WriteMultipleReq

uest(Delete)• QueryRequest(D

ELETE)

nosql-indexes

Table 8-4 nosql-indexes



INSPECT None None None

READ NOSQL_INDEX_READ

ListIndexes

GetIndex

GetIndexesRequest +indexName

GetIndexesRequest

USE READ + NONE ListIndexes

GetIndex

GetIndexesRequest +indexName

GetIndexesRequest

MANAGE READ +NOSQL_INDEX_CREATE

CreateIndex TableRequest(CREATE INDEX)

NOSQL_INDEX_DROP

DeleteIndex TableRequest(DROPINDEX)

Permission Required for Each NoSQL Cloud Driver RequestLearn about the required permissions for each NoSQL Cloud Driver Request.

The table below lists the API operations in a logical order, grouped by resource type.For information about permissions, see Permissions in Oracle Cloud InfrastructureDocumentation.


8-7

Table 8-5 Permissions

Request Permissions Operation Id(request.operation)

DeleteRequest NOSQL_ROWS_DELETE DeleteRow

GetIndexesRequest NOSQL_INDEX_READ GetIndex

GetRequest NOSQL_ROWS_READ GetRow

GetTableRequest NOSQL_TABLE_READ GetTable

ListTablesRequest NOSQL_TABLE_INSPECT ListTables

MultiDeleteRequest NOSQL_ROWS_DELETE DeleteRow

PrepareRequest NOSQL_ROWS_READ GetRow

PutRequest NOSQL_ROWS_INSERT UpdateRow

QueryRequest (SELECT) NOSQL_ROWS_READ GetRow

QueryRequest (INSERT,UPSERT, UPDATE)

NOSQL_ROWS_INSERT UpdateRow

QueryRequest (DELETE) NOSQL_ROWS_DELETE DeleteRow

TableRequest (CREATETABLE)

NOSQL_TABLE_CREATE CreateTable

TableRequest (ALTER TABLE) NOSQL_TABLE_ALTER UpdateTable

TableRequest (DROP TABLE) NOSQL_TABLE_DROP DeleteTable

TableUsageRequest NOSQL_TABLE_READ GetTable

WriteMultipleRequest has PutRequest:NOSQL_ROWS_INSERT

has DeleteRequest:NOSQL_ROWS_DELETE

UpdateRow

DeleteTable

Permission Required for Each REST API OperationLearn about the required permissions for each REST API operation request.

The table below lists the REST API operations in a logical order, grouped byresource type. For information about permissions, see Permissions in Oracle CloudInfrastructure Documentation.

Table 8-6 Permissions

Request Permissions

ListTables NOSQL_TABLE_INSPECT

CreateTable NOSQL_TABLE_CREATE

GetTable NOSQL_TABLE_READ

UpdateTable NOSQL_TABLE_ALTER

DeleteTable NOSQL_TABLE_DROP

ListIndexes NOSQL_INDEX_READ

CreateIndex NOSQL_INDEX_CREATE

GetIndex NOSQL_INDEX_READ

DeleteIndex NOSQL_INDEX_DROP

GetRow NOSQL_ROWS_READ


8-8

Table 8-6 (Cont.) Permissions

Request Permissions

UpdateRow NOSQL_ROWS_INSERT

DeleteRow NOSQL_ROWS_DELETE

ListTableUsage NOSQL_TABLE_READ

ChangeTableCompartment NOSQL_TABLE_ALTER

Query (SELECT) NOSQL_ROWS_READ

Query (INSERT, UPSERT, UPDATE) NOSQL_ROWS_INSERT

Query (DELETE) NOSQL_ROWS_DELETE

PrepareStatement NOSQL_TABLE_READ

SummarizeStatement NOSQL_TABLE_READ

ListWorkRequests NOSQL_TABLE_READ

GetWorkRequest NOSQL_TABLE_READ

DeleteWorkRequest NOSQL_TABLE_ALTER

ListWorkRequestErrors NOSQL_TABLE_READ

ListWorkRequestLogs NOSQL_TABLE_READ

When you write a policy with request.operation, use the name of API operations. ForQuery operations, use the mapping operation of statement in the query. For example:

SELECT => GetRow INSERT, UPSERT or UPDATE => UpdateRow DELETE=> DeleteRow

Typical Policy Statements to Manage TablesHere are typical policy statements that you might use to authorize access to OracleNoSQL Database Cloud Service tables.

When you create a policy for your tenancy, you grant users access to allcompartments by way of policy inheritance. Alternatively, you can restrict access toindividual Oracle NoSQL Database Cloud Service tables or compartments.

Example 8-1 To allow group Admins to fully manage any Oracle NoSQLDatabase Cloud Service table

allow group Administrators to manage nosql-tables in tenancyallow group Administrators to manage nosql-rows in tenancyallow group Administrators to manage nosql-indexes in tenancy

Example 8-2 To allow group Admins to do any operations against NoSQLTables in compartment Dev, use the family resource type.

allow group Admins to manage nosql-family in compartment Dev

Chapter 8Typical Policy Statements to Manage Tables

8-9

Example 8-3 To allow group Analytics to do read-only operations againstNoSQL Tables in compartment Dev

allow group Analytics to read nosql-rows in compartment Dev

Example 8-4 To only allow Joe in Developer to create, get and drop indexes ofNoSQL tables in compartment Dev

allow group Developer to manage nosql-indexes in compartment Dev where request.user.id = '<OCID of Joe>'

Example 8-5 To allow group Admins to create, drop and move NoSQL Tablesonly but not alter in compartment Dev.

allow group Admins to manage nosql-tables in compartment Dev where any {request.permission = 'NOSQL_TABLE_CREATE', request.permission = 'NOSQL_TABLE_DROP', request.permission = 'NOSQL_TABLE_MOVE'}

Example 8-6 To allow group Developer to read, update and delete rows of table"customer" in compartment Dev but not others.

allow group Developer to manage nosql-rows in compartment Dev where target.nosql-table.name = 'customer'

Giving Another User Permission to Manage NoSQL TablesWhen you activate your order for Oracle NoSQL Database Cloud Service, you (thefirst user) are in the Administrators group by default. Being in the Administratorsgroup gives you full administration privileges in Oracle Cloud Infrastructure so youcan manage Oracle NoSQL Database Cloud Service tables and much more. There'sno need to delegate this responsibility but, if you want to, you can give someoneelse privileges to create and manage Oracle NoSQL Database Cloud Service tablesthrough the manage nosql-tables permission.

In Oracle Cloud Infrastructure you use IAM security policies to grant permissions. First,you must add the user to a group, and then you create a security policy that grants thegroup the manage nosql-tables permission on a specific compartment or the tenancy(any compartment in the tenancy). For example, you might create a policy statementthat looks like one of these:

allow group MyAdminGroup to manage nosql-tables in tenancy

allow group MyAdminGroup to manage nosql-tables in compartment MyOracleNoSQL

To find out how to create security policy statements specifically for Oracle NoSQLDatabase Cloud Service, see Setting Up Users, Groups, and Policies.

Chapter 8Giving Another User Permission to Manage NoSQL Tables

8-10

9Using Oracle NoSQL Migrator

Learn about Oracle NoSQL Data Migrator and how to use it for data migration.

Oracle NoSQL Data Migrator is a tool that supports the movement of OracleNoSQL tables from one data source to another. This tool can operate on tables inOracle NoSQL Database Cloud Service, Oracle NoSQL Database on-premise, andhandle JSON and MongoDB-formatted JSON input files. This utility supports multiplemigration options, such as:

• Oracle NoSQL Database on-premise to Oracle NoSQL Database Cloud Serviceand vice-versa

• Between two Oracle NoSQL on-premise Databases

• Between two Oracle NoSQL Database Cloud Service Tables

• JSON file to Oracle NoSQL Database on-premise and vice-versa

• JSON file to Oracle NoSQL Database Cloud Service and vice-versa

• MongoDB-formatted JSON file to an Oracle NoSQL Database table on-premise orcloud

Topics:

• Overview

• Using Oracle NoSQL Data Migrator

• Supported Sources and Sinks

• Use Case Demonstrations

• Troubleshooting the NoSQL Data Migrator

• Oracle NoSQL Data Migrator Vs. Import/Export Utility

• Transitioning from Import/Export to NoSQL Data Migrator

OverviewOracle NoSQL Data Migrator lets you move Oracle NoSQL tables from one datasource to another, such as Oracle NoSQL Database on-premise or cloud or even asimple JSON file.

There can be many situations that require you to migrate NoSQL tables from or toan Oracle NoSQL Database. For instance, a team of developers enhancing a NoSQLDatabase application may want to test their updated code in the local Oracle NoSQLDatabase Cloud Service (NDCS) instance using cloudsim. To verify all the possibletest cases, they must set up the test data similar to the actual data. To do this, theymust copy the NoSQL tables from the production environment to their local NDCSinstance, the cloudsim environment. In another situation, NoSQL developers mayneed to move their application data from on-premise to the cloud and vice-versa,either for development or testing.

9-1

In all such cases and many more, you can use Oracle NoSQL Data Migrator to moveyour NoSQL tables from one data source to another, such as Oracle NoSQL Databaseon-premise or cloud or even a simple JSON file. You can also copy NoSQL tablesfrom a MongoDB-formatted JSON input file into your NoSQL Database on-premise orcloud.

Oracle NoSQL Data Migrator is created to replace and enhance the existing on-premise-only import/export utility. To know how the NoSQL Data Migrator is differentfrom the existing import/export utility, see Oracle NoSQL Data Migrator Vs. Import/Export Utility .

As depicted in the following figure, the NoSQL Data Migrator utility acts as a connectoror pipe between the data source and the target (referred to as the sink). In essence,this utility exports data from the selected source and imports that data into the sink.This tool is table-oriented, that is, you can move the data only at the table level. Asingle migration task operates on a single table and supports the following options:

• JSON file to Oracle NoSQL Database on-premise and vice versa

• JSON file to Oracle NoSQL Database Cloud Service and vice versa

• Oracle NoSQL Database on-premise to Oracle NoSQL Database Cloud Serviceand vice versa

• MongoDB-formatted JSON file to an Oracle NoSQL Database table

• MongoDB-formatted JSON file to an Oracle NoSQL Database Cloud Service table

• One Oracle NoSQL Database on-premise to another

• One Oracle NoSQL Database Cloud Service to another

Oracle NoSQL Data Migrator is designed such that it can support additional sourcesand sinks in the future. For a list of sources and sinks supported by Oracle NoSQLData Migrator as of the current release, see Supported Sources and Sinks.

Migration PipeSource Sink

NoSQLTable Data

NoSQLTable Data

Transformations

Terminology used with NoSQL Data MigratorLearn about the different terms used in the above diagram, in detail.

• Source: An entity from where the NoSQL tables are exported for migration. Someexamples of sources are Oracle NoSQL Database on-premise or cloud, JSON file,and MongoDB-formatted JSON file.

• Sink: An entity that imports the NoSQL tables from NoSQL Data Migrator. Someexamples for sinks are Oracle NoSQL Database on-premise or cloud and JSONfile.

• Migration Pipe: The data from a source will be transferred to the sink by NoSQLData Migrator. This can be visualized as a Migration Pipe.

• Transformations: You can add rules to modify the NoSQL table data in themigration pipe. These rules are called Transformations. Oracle NoSQL Data

Chapter 9Overview

9-2

Migrator allows data transformations at the top-level fields or columns only. It doesnot let you transform the data in the nested fields. Some examples of permittedtransformations are:

– Drop or ignore one or more columns,

– Rename one or more columns, or

– Aggregate several columns into a single field, typically a JSON field.

• Configuration File : A configuration file is a JSON file where you define allthe parameters required for the migration activity. Later, you pass this JSONfile as a single parameter to the runMigrator command from the CLI. A typicalconfiguration file format looks like as shown below.

{ "source": { "type" : <source type>, //source-configuration for type. See Source Configuration Templates . }, "sink": { "type" : <sink type>, //sink-configuration for type. See Sink Configuration Templates . }, "transforms" : { //transforms configuration. See Transformation Configuration Templates . }, "migratorVersion" : "1.0.0", "abortOnError" : <true|false>}

Group Parameters Mandatory(Y/N)

Purpose SupportedValues

source type Y Represents thesource fromwhich to migratethe data. Thesource providesdata andmetadata (if any)for migration.

To know thetype value foreach source,see SupportedSources andSinks.

source-configuration fortype

Y Defines theconfiguration forthe source.Theseconfigurationparameters arespecific to thetype of sourceselected above.

See SourceConfigurationTemplates forthe complete listof configurationparameters foreach sourcetype.

sink type Y Represents thesink to which tomigrate the data.The sink is thetarget ordestination forthe migration.

To know thetype value foreach source,see SupportedSources andSinks.

Chapter 9Overview

9-3

Group Parameters Mandatory(Y/N)

Purpose SupportedValues

sink-configuration fortype

Y Defines theconfiguration forthe sink. Theseconfigurationparameters arespecific to thetype of sinkselected above.

See SinkConfigurationTemplates forthe complete listof configurationparameters foreach sink type.

transforms transformsconfiguration

N Defines thetransformationsto be applied tothe data in themigration pipe.

See TransformationConfigurationTemplates forthe complete listoftransformationssupported by theNoSQL DataMigrator.

- migratorVersion

N Version of theNoSQL DataMigrator

-

- abortOnError N Specifieswhether to stopthe migrationactivity in caseof any error ornot.

The defaultvalue is trueindicating thatthe migrationstops wheneverit encounters amigration error.

If you set thisvalue to false,the migrationcontinues evenin case of failedrecords or othermigration errors.The failedrecords andmigration errorswill be logged asWARNINGs onthe CLI terminal.

true, false

Note:

As JSON is case-sensitive, all the parameters defined in theconfiguration file are case-sensitive unless specified otherwise.

Chapter 9Overview

9-4

Using Oracle NoSQL Data MigratorLearn about the various steps involved in using the Oracle NoSQL Data Migrator utilityfor migrating your NoSQL data.

The high level flow of tasks involved in using NoSQL Data Migrator is depicted in thebelow figure.

BEGIN

Download the NoSQL Migrator Utility

Identify Source & Sink for Migration

OR

OR

Generate the Configuration JSON File

using runMigrator

Create a Configuration JSON

File Manually

Proceed to Migration with the Generated

Configuration JSON File

Save the Configuration JSON File for a Future

Migration

Run runMigrator by passing the Configuration JSON File as a Parameter

END

You can reuse the Config JSON File multiple times.



Download the NoSQL Data Migrator Utility

The Oracle NoSQL Data Migrator utility is available for download from the OracleNoSQL Downloads page. Once you download and unzip it on your machine, you canaccess the runMigrator command from the command line interface.

Identify the Source and Sink

Before using the migrator, you must identify the data source and sink. For instance,if you want to migrate a NoSQL table from Oracle NoSQL Database on-premise toa JSON formatted file, your source will be Oracle NoSQL Database and sink will be

Chapter 9Using Oracle NoSQL Data Migrator

9-5

JSON file. Ensure that the identified source and sink are supported by the OracleNoSQL Data Migrator by referring to Supported Sources and Sinks. This is also anappropriate phase to decide the schema for your NoSQL table in the target or sink,and create them.

• Identify Sink Table Schema: If the sink is Oracle NoSQL Database on-premise orcloud, you must identify the schema for the sink table and ensure that the sourcedata matches with the target schema. If required, use transformations to map thesource data to the sink table.

• Create Sink Table: Once you identify the sink table schema, create the sinktable either through the Admin CLI or using the schemaInfo attribute of the sinkconfiguration file. See Sink Configuration Templates .

Run the runMigrator command

The runMigrator executable file is available in the extracted NoSQL Data Migratorfiles. You must install Java 8 or higher version and bash on your system tosuccessfully run the runMigrator command.

You can run the runMigrator command in two ways:

1. By creating the JSON configuration file using the runtime options of therunMigrator command as shown below.

[~]$ ./runMigratorconfiguration file is not provided. Do you want to generate configuration? (y/n) [n]: y......

• When you invoke the runMigrator utility, it provides a series of runtimeoptions and creates the configuration JSON file based on your choices foreach option.

• After the utility creates the configuration JSON file, you have a choice to eitherproceed with the migration activity in the same run or save the configurationfile for a future migration.

• Irrespective of your decision to proceed or defer the migration activity withthe generated configuration JSON file, the file will be available for edits orcustomization to meet your future requirements. You can use the customizedconfiguration JSON file for migration later.

2. By passing a manually created JSON configuration file as a runtime parameterusing the -c or --config option. You must create the configuration JSON filemanually before running the runMigrator command with the -c or --configoption. For any help with the source and sink configuration parameters, seeSources and Sinks.

[~]$ ./runMigrator -c </path/to/the/configuration/json/file>

Chapter 9Using Oracle NoSQL Data Migrator

9-6

Sources and SinksLearn about the different sources and sinks supported by the Oracle NoSQL DataMigrator utility and their configuration templates.

Topics:

• Supported Sources and Sinks

• Source Configuration Templates

• Sink Configuration Templates

• Transformation Configuration Templates

Supported Sources and SinksThis topic provides the list of the sources and sinks supported by the Oracle NoSQLData Migrator.

You can use any combination of a valid source and sink from this table for themigration activity. However, you must ensure that at least one of the ends, that is,source or sink must be an Oracle NoSQL product. You can not use the NoSQL DataMigrator to move the NoSQL table data from one file to another.

Entity type Value Valid Source Valid Sink

JSON file file Y Y

MongoDB-formattedJSON file

file Y N

Oracle NoSQLDatabase

nosqldb Y Y

Oracle NoSQLDatabase CloudService

nosqldb_cloud Y Y

Source Configuration TemplatesLearn about the configuration file formats for each valid source and the purpose ofeach configuration parameter.

Topics

• JSON File

• MongoDB-Formatted JSON File

• Oracle NoSQL Database

• Oracle NoSQL Database Cloud Service

Chapter 9Sources and Sinks

9-7

JSON FileThe configuration file format for JSON File as a source of NoSQL Data Migrator isshown below.

Configuration Template

"source" : { "type" : "file", "format" : "json", "dataPath": "</path/to/a/json/file>"}

Explanation

Table 9-1 Source Parameters

Parameter Purpose Data Type Mandatory (Y/N) Example

type Identifies thesource type.

string Y "type" :"file"

format Specifies thesource format.

string Y "format" :"json"

dataPath Specifies theabsolute path to afile or directorycontaining theJSON data formigration.

You must ensurethat this datamatches with theNoSQL tableschema definedat the sink.

If you specify adirectory, theNoSQL DataMigrator identifiesall the files withthe .jsonextension in thatdirectory for themigration. Sub-directories arenot supported.

string Y • Specifying aJSON file

"dataPath": "/home/user/sample.json"

• Specifying adirectory

"dataPath": "/home/user"


9-8

MongoDB-Formatted JSON FileThe configuration file format for MongoDB-formatted JSON File as a source of NoSQLData Migrator is shown below.


"source" : { "type" : "file", "format" : "mongodb_json", "dataPath": "</path/to/a/json/file>"}

Explanation





format Specifies thesource format.

string Y "format" :"mongodb_json"


9-9

Table 9-2 (Cont.) Source Parameters


dataPath Specifies theabsolute path to afile or directorycontaining theMongoDBexported JSONdata formigration.

You must havegenerated thesefiles using themongoexporttool. See mongoexport formore information.

You can supplythe MongoDB-formatted JSONfile that isgenerated usingthe mongoexporttool in eithercanonical orrelaxed mode.Both the modesare supported bythe NoSQL DataMigrator formigration.

If you specify adirectory, theNoSQL DataMigrator identifiesall the files withthe .jsonextension in thatdirectory for themigration. Sub-directories arenot supported.

You must ensurethat this datamatches with theNoSQL tableschema definedat the sink.

string Y • Specifying aMongoDBformattedJSON file

"dataPath": "/home/user/sample.json"

• Specifying adirectory

"dataPath": "/home/user"


9-10

Oracle NoSQL DatabaseThe configuration file format for Oracle NoSQL Database as a source of NoSQL DataMigrator is shown below.


"source" : { "type": "nosqldb", "table" : "<fully qualified table name>", "storeName" : "<store name>", "helperHosts" : ["hostname1:port1","hostname2:port2,..."], "security" : "</path/to/store/security/file>", "requestTimeoutMs" : 5000}

Explanation




string Y "type" :"nosqldb"

table Fully qualifiedtable name fromwhich to migratethe data.

Format:

[namespace_name:]<table_name>

If the table is inthe DEFAULTnamespace, youcan omit thenamespace_name. The table mustexist in the store.

string Y • With theDEFAULTnamespace

"table" :"mytable"

• With a non-defaultnamespace

"table" :"mynamespace:mytable"

storeName Name of theOracle NoSQLDatabase store.

string Y "storeName" :"kvstore"

helperHosts A list of host andregistry port pairsin thehostname:portformat. Delimiteach item in thelist using acomma. You mustspecify at leastone helper host.

array of strings Y "helperHosts":["localhost:5000","localhost:6000"]


9-11



security If your store is asecure store,provide theabsolute path tothe security loginfile that containsyour storecredentials. See ConfiguringSecurity withRemote Accessin Administrator'sGuide to knowmore about thesecurity login file.You can useeither passwordfile basedauthentication orwallet basedauthentication.However, thewallet basedauthentication issupported only inthe EnterpriseEdition (EE) ofOracle NoSQLDatabase.The CommunityEdition(CE)edition supportspassword filebasedauthenticationonly.To authenticateusing a wallet,you needadditional jar filesthat are part ofthe EEinstallation.Without these jarfiles, you will getthe followingexception.java.lang.NoClassDefFoundError: oracle/security/pki/OracleSecretStoreExceptionTo prevent theexception shown

string Y for a securestore.

"security" :"/home/user/client.credentials"

Example securityfile content forpassword filebasedauthentication:

oracle.kv.password.noPrompt=trueoracle.kv.auth.username=adminoracle.kv.auth.pwdfile.file=/home/nosql/login.passwdoracle.kv.transport=ssloracle.kv.ssl.trustStore=/home/nosql/client.trustoracle.kv.ssl.protocols=TLSv1.2,TLSv1.1,TLSv1oracle.kv.ssl.hostnameVerifier=dnmatch(CN\=NoSQL)

Example securityfile content forwallet basedauthentication:

oracle.kv.password.noPrompt=trueoracle.kv.auth.username=admin


9-12



above, you mustcopy the belowjar files from yourEE serverpackage to thenosql-migrator-1.0.0/lib directory:• oraclepki.

jar• osdt_core.

jar• osdt_cert.

jar

oracle.kv.auth.wallet.dir=/home/nosql/login.walletoracle.kv.transport=ssloracle.kv.ssl.trustStore=/home/nosql/client.trustoracle.kv.ssl.protocols=TLSv1.2,TLSv1.1,TLSv1oracle.kv.ssl.hostnameVerifier=dnmatch(CN\=NoSQL)

requestTimeoutMs

Specifies the timeto wait for eachread operationfrom the store tocomplete. This isprovided inmilliseconds. Thedefault value is5000. The valuecan be anypositive integer.

integer N "requestTimeoutMs" : 5000

Oracle NoSQL Database Cloud ServiceThe configuration file format for Oracle NoSQL Database Cloud Service as a source ofNoSQL Data Migrator is shown below.


"source" : { "type" : "nosqldb_cloud", "endpoint" : "<Oracle NoSQL Cloud Service Endpoint. You can either specify the complete URL or the Region ID alone.>", "table" : "<table name>", "compartment" : "<OCI compartment name or id>", "credentials" : "</path/to/oci/credential/file>", "credentialsProfile" : "<oci credentials profile name>",


9-13

"readUnitsPercent" : <table readunits percent>, "requestTimeoutMs" : <timeout in milli seconds>}

Explanation




string Y "type" :"nosqldb_cloud"

endpoint Specifies theService Endpointof the OracleNoSQL DatabaseCloud service.

You can eitherspecify thecomplete URL orthe Region IDalone. See DataRegions andAssociatedService URLs inUsing OracleNoSQL DatabaseCloud Service forthe list of dataregionssupported forOracle NoSQLDatabase CloudService.

string Y • Region ID

"endpoint": "us-ashburn-1"

• URL format

"endpoint":"https://nosql.us-ashburn-1.oci.oraclecloud.com/"

table Name of the tablefrom which tomigrate the data.

string Y "table" :"myTable"


9-14



compartment Specifies thename or OCID ofthe compartmentin which the tableresides.

If you do notprovide anyvalue, it defaultsto the rootcompartment.

You can find yourcompartment'sOCID from theCompartmentExplorer windowunderGovernance inthe OCI CloudConsole.

string Y if the table isnot in the rootcompartment ofthe tenancy.

• Compartment name

"compartment" :"mycompartment"

• Compartment namequalified withits parentcompartment

"compartment" :"parent.childcompartment"

• No valueprovided.Defaults tothe rootcompartment.

"compartment": ""

• Compartment OCID

"compartment" :"ocid1.tenancy.oc1...4ksd"

credentials Absolute path toa file containingOCI credentials.

If not specified, itdefaultsto $HOME/.oci/config

See ExampleConfiguration foran example of thecredentials file.

string N 1. "credentials" : "/home/user/.oci/config"

2. "credentials" : "/home/user/security/config"


9-15



credentialsProfile

Name of theprofile in thecredentials filethat must used toconnect to theOracle NoSQLDatabase CloudService.

If not specified, itdefaults toDEFAULT.

string N 1. "credentialsProfile":"DEFAULT"

2. "credentialsProfile":"ADMIN_USER"

readUnitsPercent

Percentage oftable read units tobe used whilemigrating theNoSQL table.

The default valueis 90. The validrange is anyinteger between 1to 100.

Please note thatamount of timerequired tomigrate data isdirectlyproportional tothis attribute.

It's better toincrease the readthroughput of thetable for themigration activity.

You can reducethe readthroughput afterthe migrationprocesscompletes.

To learn the dailylimits onthroughputchanges, see Cloud Limits inUsing OracleNoSQL DatabaseCloud Service.

integer N "readUnitsPercent" : 90


9-16



requestTimeoutMs

Specifies the timeto wait for eachread operationfrom the sourceto complete.

This is providedin milliseconds.The default valueis 5000.

The value can beany positiveinteger.


Sink Configuration TemplatesLearn about the configuration file formats for each valid sink and the purpose of eachconfiguration parameter.

Topics

• JSON File

• Oracle NoSQL Database

• Oracle NoSQL Database Cloud Service

JSON FileThe configuration file format for JSON File as a sink of NoSQL Data Migrator is shownbelow.


"sink" : { "type" : "file", "format" : "json", "dataPath": "</path/to/a/file>", "schemaPath" : "<path/to/a/file>"}

Explanation

Table 9-5 Sink Parameters


type Identifies the sinktype.


format Specifies the sinkformat

string Y "format" :"json"


9-17

Table 9-5 (Cont.) Sink Parameters


dataPath Specifies theabsolute path to afile where thesource data willbe copied in theJSON format.

If the file does notexist in thespecified datapath, the NoSQLData Migratorcreates it.

If it exists already,the NoSQL DataMigrator willoverwrite itscontents with thesource data.

You must ensurethat the parentdirectory for thefile specified inthe data path isvalid.

string Y "dataPath" :"/home/user/sample.json"


9-18



schemaPath Specifies theabsolute path towrite schemainformationprovided by thesource.

If this value is notdefined, thesource schemainformation willnot be migratedto the sink.

If this value isspecified, themigrator utilitywrites theschema of thesource table intothe file specifiedhere.

The schemainformation iswritten as oneDDL commandper line in thisfile.

If the file does notexist in thespecified datapath, NoSQLData Migratorcreates it.

If it exists already,NoSQL DataMigrator willoverwrite itscontents with thesource data.

You must ensurethat the parentdirectory for thefile specified inthe data path isvalid.

string N "schemaPath": "/home/user/schema_file"


9-19

Oracle NoSQL DatabaseThe configuration file format for Oracle NoSQL Database as a sink of NoSQL DataMigrator is shown below.


"sink" : { "type": "nosqldb", "table" : "<fully qualified table name>", "schemaInfo" : { "schemaPath" : "</path/to/a/schema/file>" or "defaultSchema" : true } "storeName" : "<store name>", "helperHosts" : ["hostname1:port1","hostname2:port2,..."], "security" : "</path/to/store/credentials/file>", "requestTimeoutMs" : <timeout in milli seconds>}

Explanation




string Y "type" :"nosqldb"


9-20



table Fully qualifiedtable name fromwhich to migratethe data.

Format:

[namespace_name:]<table_name>

If the table is inthe DEFAULTnamespace, youcan omit thenamespace_name. The table mustexist in the storeduring themigration, and itsschema mustmatch with thesource data.

If the table is notavailable in thesink, you can usethe schemaInfoparameter toinstruct theNoSQL DataMigrator to createthe table also inthe sink.

string Y • With theDEFAULTnamespace

"table" :"mytable"

• With a non-defaultnamespace

"table" :"mynamespace:mytable"

schemaInfo Specifies theschema for thedata beingmigrated. If this isnot specified, theNoSQL DataMigrator assumesthat the tablealready exists inthe sink's store.

Object N • With DefaultSchema:

"schemaInfo" : { "defaultSchema" : true, "readUnits" : 100, "writeUnits" : 60, "storageSize" : 1 }


9-21



schemaInfo.schemaPath

Specifies theabsolute path to afile containingDDL statementsfor the NoSQLtable.

The NoSQL DataMigrator executesthe DDLcommands listedin this file beforemigrating thedata.

The NoSQL DataMigrator does notsupport morethan one DDLstatement per linein theschemaPath file.

• With a pre-definedschema:

"schemaInfo" : { "schemaPath" : "<complete/path/to/the/schema/definition/file>", "readUnits" : 100, "writeUnits" : 100, "storageSize" : 1 }

string Y, only when theschemaInfo.defaultSchemaparameter is setto No.


9-22



schemaInfo.defaultSchema

Setting thisparameter to Yesinstructs theNoSQL DataMigrator to createa table withdefault schema.The defaultschema isdefined by themigrator itself.

If the source is aMongoDB-formatted JSONfile, the defaultschema for thetable will be asfollows:

CREATE TABLEIF NOT EXISTS<tablename>(ID STRING,DOCUMENTJSON,PRIMARYKEY(SHARD(ID));

Where:• tablename

= valueprovided forthe tableattribute intheconfiguration.

• ID = _idvalue fromeachdocument ofthemongoDBexportedJSON sourcefile.

• DOCUMENT =For eachdocument inthemongoDBexported file,the contentsexcluding the_id field isaggregated

boolean Y, only when theschemaInfo.schemaPathparameter is setto No.

N

o

t

e

:

defaultSchema and schemaPath are mutually exclusive.


9-23



into theDOCUMENTcolumn.

N

o

t

e

:

Ifthe_id value is not provided as a string in the MongoDB-formatted JSON file, NoSQL Data Migrator converts it into string before inserting into the default schema.

For all the othersources, thedefault schemawill be as follows:

CREATE TABLEIF NOT EXISTS<tablename>(ID LONGGENERATEDALWAYS ASIDENTITY,DOCUMENTJSON, PRIMARYKEY(ID))

Where:• tablename


• ID = Anauto-generatedLONG value.

• DOCUMENT =The JSON


9-24



recordprovided bythe source isaggregatedinto theDOCUMENTcolumn.

storeName Name of theOracle NoSQLDatabase store.

string Y "storeName" :"kvstore"

helperHosts A list of host andregistry port pairsin thehostname:portformat. Delimiteach item in thelist using acomma. You mustspecify at leastone helper host.

array of strings Y "helperHosts":["localhost:5000","localhost:6000"]


9-25



security If your store is asecure store,provide theabsolute path tothe security loginfile that containsyour storecredentials. See ConfiguringSecurity withRemote Accessin Administrator'sGuide to knowmore about thesecurity login file.You can useeither passwordfile basedauthentication orwallet basedauthentication.However, thewallet basedauthentication issupported only inthe EnterpriseEdition (EE) ofOracle NoSQLDatabase.The CommunityEdition(CE)edition supportspassword filebasedauthenticationonly.To authenticateusing a wallet,you needadditional jar filesthat are part ofthe EEinstallation.Without these jarfiles, you will getthe followingexception.java.lang.NoClassDefFoundError: oracle/security/pki/OracleSecretStoreExceptionTo prevent theexception shown

string Y for a securestore.

"security" :"/home/user/client.credentials"

Example securityfile content forpassword filebasedauthentication:

oracle.kv.password.noPrompt=trueoracle.kv.auth.username=adminoracle.kv.auth.pwdfile.file=/home/nosql/login.passwdoracle.kv.transport=ssloracle.kv.ssl.trustStore=/home/nosql/client.trustoracle.kv.ssl.protocols=TLSv1.2,TLSv1.1,TLSv1oracle.kv.ssl.hostnameVerifier=dnmatch(CN\=NoSQL)

Example securityfile content forwallet basedauthentication:

oracle.kv.password.noPrompt=trueoracle.kv.auth.username=admin


9-26



above, you mustcopy the belowjar files from yourEE serverpackage to thenosql-migrator-1.0.0/lib directory:• oraclepki.

jar• osdt_core.

jar• osdt_cert.

jar

oracle.kv.auth.wallet.dir=/home/nosql/login.walletoracle.kv.transport=ssloracle.kv.ssl.trustStore=/home/nosql/client.trustoracle.kv.ssl.protocols=TLSv1.2,TLSv1.1,TLSv1oracle.kv.ssl.hostnameVerifier=dnmatch(CN\=NoSQL)

requestTimeoutMs

Specifies the timeto wait for eachwrite operation inthe sink tocomplete. This isprovided inmilliseconds. Thedefault value is5000. The valuecan be anypositive integer.


Oracle NoSQL Database Cloud ServiceThe configuration file format for Oracle NoSQL Database Cloud Service as a sink ofNoSQL Data Migrator is shown below.


"sink" : { "type" : "nosqldb_cloud", "endpoint" : "<Oracle NoSQL Cloud Service Endpoint>", "table" : "<table name>", "compartment" : "<OCI compartment name or id>", "schemaInfo" : { "schemaPath" : "</path/to/a/schema/file>" or "defaultSchema" : true


9-27

"readUnits" : <table read units>, "writeUnits" : <table write units>, "storageSize" : <storage size in GB> }, "credentials" : "</path/to/oci/credential/file>", "credentialsProfile" : "<oci credentials profile name>", "writeUnitsPercent" : <table writeunits percent>, "requestTimeoutMs" : <timeout in milli seconds>}

Explanation




string Y "type" :"nosqldb_cloud"

endpoint Specifies theService Endpointof the OracleNoSQL DatabaseCloud Service.

You can eitherspecify thecomplete URL orthe Region IDalone. See DataRegions andAssociatedService URLs inUsing OracleNoSQL DatabaseCloud Service forthe list of dataregionssupported forOracle NoSQLDatabase CloudService.

string Y • Region ID

"endpoint": "us-ashburn-1"

• URL format

"endpoint":"https://nosql.us-ashburn-1.oci.oraclecloud.com/"


9-28



table Name of the tableto which tomigrate the data.

You must ensurethat this tableexists in yourOracle NoSQLDatabase CloudService.Otherwise, youhave to use theschemaInfoobject in the sinkconfiguration toinstruct theNoSQL DataMigrator to createthe table.

The schema ofthis table mustmatch the sourcedata.

string Y "table" :"myTable"


9-29



compartment Specifies thename or OCID ofthe compartmentin which the tableresides.

If you do notprovide anyvalue, it defaultsto the rootcompartment.

You can find yourcompartment'sOCID from theCompartmentExplorer windowunderGovernance inthe OCI CloudConsole.

string Y if the table isnot in the rootcompartment ofthe tenancy.

• Compartment name

"compartment" :"mycompartment"

• Compartment namequalified withits parentcompartment

"compartment" :"parent.childcompartment"

• No valueprovided.Defaults tothe rootcompartment.

"compartment": ""

• Compartment OCID

"compartment" :"ocid1.tenancy.oc1...4ksd"

schemaInfo Specifies theschema for thedata beingmigrated.

If you do notspecify thisparameter, theNoSQL DataMigrator assumesthat the tablealready exists inyour OracleNoSQL DatabaseCloud Service.

If this parameteris not specifiedand the tabledoes not exist inthe sink, themigration fails.

Object N • WithschemaPath

"schemaInfo" : { "schemaPath" : "</path/to/a/schema/file>", "readUnits" : 500, "writeUnits" : 1000,


9-30



schemaInfo.schemaPath

"storageSize" : 5 }

• WithdefaultSchema

"schemaInfo" : { "defaultSchema" :Yes, "readUnits" : 500, "writeUnits" : 1000, "storageSize" : 5 }

Specifies theabsolute path to afile containingDDL statementsfor the NoSQLtable.

The NoSQL DataMigrator executesthe DDLcommands listedin this file beforemigrating thedata.

The NoSQL DataMigrator does notsupport morethan one DDLstatement per linein theschemaPath file.

string Y, only whenschemaInfo.defaultSchema isset to No.


9-31



schemaInfo.defaultSchema

Setting thisparameter to Yesinstructs theNoSQL DataMigrator to createa table withdefault schema.The defaultschema isdefined by themigrator itself.

If the source is aMongoDB-formatted JSONfile, the defaultschema for thetable will be asfollows:

CREATE TABLEIF NOT EXISTS<tablename>(ID STRING,DOCUMENTJSON,PRIMARYKEY(SHARD(ID));

Where:• tablename


• ID = _idvalue fromeachdocument ofthemongoDBexportedJSON sourcefile.

• DOCUMENT =For eachdocument inthemongoDBexported file,the contentsexcluding the_id field isaggregated

boolean Y, only whenschemaInfo.schemaPath is setto No.

N

o

t

e

:

defaultSchema and schemaPath are mutually exclusive.


9-32



into theDOCUMENTcolumn.

N

o

t

e

:

Ifthe_id value is not provided as a string in the MongoDB-formatted JSON file, NoSQL Data Migrator converts it into string before inserting into the default schema.

For all the othersources, thedefault schemawill be as follows:

CREATE TABLEIF NOT EXISTS<tablename>(ID LONGGENERATEDALWAYS ASIDENTITY,DOCUMENTJSON, PRIMARYKEY(ID))

Where:• tablename


• ID = Anauto-generatedLONG value.

• DOCUMENT =The JSON


9-33



recordprovided bythe source isaggregatedinto theDOCUMENTcolumn.

schemaInfo.readUnits

Specifies theread throughputof the new table.

integer Y

schemaInfo.writeUnits

Specifies thewrite throughputof the new table.

integer Y

schemaInfo.storageSize

Specifies thestorage size ofthe new table inGB.

integer Y

credentials Absolute path toa file containingOCI credentials.

If not specified, itdefaultsto $HOME/.oci/config

See ExampleConfiguration foran example of thecredentials file.

string N "credentials": "/home/user/security/config"

credentialsProfile

Name of theconfigurationprofile to be usedto connect to theOracle NoSQLDatabase CloudService.

If you do notspecify this value,it defaults to theDEFAULT profile.

string N "credentialsProfile":"ADMIN_USER"


9-34



writeUnitsPercent

Specifies thePercentage oftable write unitsto be used duringthe migrationactivity.

The default valueis 90. The validrange is anyinteger between 1to 100.

N

o

t

e

:

Thetimerequiredforthedatamigrat

integer N "readUnitsPercent" : 90


9-35



ionisdirectlyproportionaltothewriteUnitsPercent value.

See Troubleshootingthe NoSQL DataMigrator to learnhow to use this


9-36



attribute toimprove the datamigration speed.

requestTimeoutMs

Specifies the timeto wait for eachwrite operation inthe sink tocomplete.

This is providedin milliseconds.The default valueis 5000.

The value can beany positiveinteger.


Transformation Configuration TemplatesThis topic explains the configuration parameters for the different transformationssupported by the Oracle NoSQL Data Migrator.

Oracle NoSQL Data Migrator lets you modify the data, that is, add datatransformations as part of the migration activity. You can define multipletransformations in a single migration. In such a case, the order of transformationsis vital because the source data undergoes each transformation in the given order.The output of one transformation becomes the input to the next one in the migratorpipeline.

The different transformations supported by the NoSQL Data Migrator are:

Table 9-8 Transformations

Transformation Config Attribute You can use this transformation to ...

ignoreFields Ignore the identified columns from the sourcerow before writing to the sink.

renameFields Rename the identified columns from thesource row before writing to the sink.

aggregateFields Aggregate multiple columns from the sourceinto a single column in the sink. As partof this transformation, you can also identifythe columns that you want to exclude in theaggregation. Those fields will be skipped fromthe aggregated column.

You can find the configuration template for each supported transformation below.

• ignoreFields

• renameFields

• aggregateFields


9-37

ignoreFieldsThe configuration file format for the ignoreFields transformation is shown below.


"transforms" : { "ignoreFields" : ["<field1>","<field2>",...]}


9-38

Explanation

Table 9-9 Transformation Parameters


ignoreFields An array of thecolumn names tobe ignored fromthe sourcerecords.

N

o

t

e

:

Youcansupplyonlytop-levelfields.Tra

array of strings Y To ignore thecolumns named"name" and"address" fromthe sourcerecord:

"ignoreFields" :["name","address"]


9-39

Table 9-9 (Cont.) Transformation Parameters


nsformationscannotbeappliedonthedatainthenestedfield


9-40



s.

renameFieldsThe configuration file format for the renameFields transformation is shown below.


"transforms" : { "renameFields" : { "<old_name>" : "<new_name>", "<old_name>" : "<new_name>," ..... }}


9-41

Explanation



renameFields Key-Value pairsof the old andnew names of thecolumns to berenamed.

N

o

t

e

:

Youcansupplyonlytop-levelfields.Tra

JSON object Y To rename thecolumn named"residence" to"address" and thecolumn named"_id" to "id":

"renameFields" :{ "residence": "address","_id" :"id" }


9-42



nsformationscannotbeappliedonthedatainthenestedfield


9-43



s.

aggregateFieldsThe configuration file format for the aggregateFields transformation is shown below.


"transforms" : { "aggregateFields" : { "fieldName" : "name of the new aggregate field", "skipFields" : ["<field1>","<field2">,...] }}


9-44

Explanation



fieldName Name of theaggregated fieldin the sink.

string Y If the givenrecord is:

{ "id" : 100, "name" : "john", "address" : "USA", "age" : 20}

If the aggregatetransformation is:

"aggregateFields" : { "fieldName" : "document", "skipFields" : "id"}

The aggregatedcolumn in thesink looks like:

{ "id": 100, "document": { "name": "john", "address": "USA", "age": 20 }}


9-45

Use Case DemonstrationsLearn how to perform data migration using the Oracle NoSQL Data Migrator forspecific use cases. You can find detailed systematic instructions with code examplesto perform migration in each of the use cases listed below.

Topics:

• Migrate from Oracle NoSQL Database Cloud Service to a JSON file

• Migrate from Oracle NoSQL Database On-Premise to Oracle NoSQL DatabaseCloud Service

• Migrate from MongoDB-Formatted JSON file to an Oracle NoSQL DatabaseCloud Service

Migrate from Oracle NoSQL Database Cloud Service to a JSON fileThis example shows how to use the Oracle NoSQL Data Migrator to copy data andthe schema definition of a NoSQL table from Oracle NoSQL Database Cloud Service(NDCS) to a JSON file.

Use Case

An organization decides to train a model using the Oracle NoSQL DatabaseCloud Service (NDCS) data to predict future behaviors and provide personalizedrecommendations. They can take a periodic copy of the NDCS tables' data to a JSONfile and apply it to the analytic engine to analyze and train the model. Doing this helpsthem separate the analytical queries from the low-latency critical paths.

Example

For the demonstration, let us look at how to migrate the data and schema definition ofa NoSQL table called myTable from NDCS to a JSON file.

Prerequisites

• Identify the source and sink for the migration.

– Source: Oracle NoSQL Database Cloud Service

– Sink: JSON file

• Identify your OCI cloud credentials and capture them in the OCI config file. Savethe config file in /home/.oci/config. See Acquiring Credentials in Using OracleNoSQL Database Cloud Service.

[DEFAULT]tenancy=ocid1.tenancy.oc1....user=ocid1.user.oc1....fingerprint= 43:d1:....key_file=</fully/qualified/path/to/the/private/key/>pass_phrase=<passphrase>

• Identify the region endpoint and compartment name for your Oracle NoSQLDatabase Cloud Service.

– endpoint: us-phoenix-1

Chapter 9Use Case Demonstrations

9-46

– compartment: developers

ProcedureTo migrate the data and schema definition of myTable from Oracle NoSQL DatabaseCloud Service to a JSON file:

1. Open the command prompt and navigate to the directory where you extracted theNoSQL Data Migrator utility.

2. To generate the configuration JSON file using the NoSQL Data Migrator, run therunMigrator command without any runtime parameters.

[~/nosqlMigrator/nosql-migrator-1.0.0]$./runMigrator

3. As you did not provide the configuration file as a runtime parameter, the utilityprompts if you want to generate the configuration now. Type y.

configuration file is not provided. Do you want to generate configuration? (y/n) [n]: y This command provides a walkthrough of creating a valid config forOracle NoSQL data migrator. The following link explain where to find the information required by thisscript: <link to doc>

4. Based on the prompts from the utility, choose your options for the Sourceconfiguration.

Enter a location for your config [./migrator-config.json]: /home/apothula/nosqlMigrator/NDCS2JSONSelect the source:1) nosqldb2) nosqldb_cloud3) file#? 2Configuration for source type=nosqldb_cloudEnter endpoint URL or region of the Oracle NoSQL Database Cloud: us-phoenix-1Enter table name: myTableEnter compartment name or id of the source table []: developersEnter path to the file containing OCI credentials [/home/apothula/.oci/config]:Enter the profile name in OCI credentials file [DEFAULT]:Enter percentage of table read units to be used for migration operation. (1-100) [90]:Enter store operation timeout in milliseconds. (1-30000) [5000]:

5. Based on the prompts from the utility, choose your options for the Sinkconfiguration.

Select the sink:1) nosqldb


9-47

2) nosqldb_cloud3) file#? 3Configuration for sink type=fileEnter path to a file to store JSON data: /home/apothula/nosqlMigrator/myTableJSONWould you like to store JSON in pretty format? (y/n) [n]: yWould you like to migrate the table schema also? (y/n) [y]: yEnter path to a file to store table schema: /home/apothula/nosqlMigrator/myTableSchema

6. Based on the prompts from the utility, choose your options for the source datatransformations. The default value is n.

Would you like to add transformations to source data? (y/n) [n]:

7. Enter your choice to determine whether to proceed with the migration in case anyrecord fails to migrate.

Would you like to continue migration in case of any record/row is failed to migrate?: (y/n) [n]:

8. The utility displays the generated configuration on the screen.

generated configuration is:{ "source": { "type": "nosqldb_cloud", "endpoint": "us-phoenix-1", "table": "myTable", "compartment": "developers", "credentials": "/home/apothula/.oci/config", "credentialsProfile": "DEFAULT", "readUnitsPercent": 90, "requestTimeoutMs": 5000 }, "sink": { "type": "file", "format": "json", "schemaPath": "/home/apothula/nosqlMigrator/myTableSchema", "pretty": true, "dataPath": "/home/apothula/nosqlMigrator/myTableJSON" }, "abortOnError": true, "migratorVersion": "1.0.0"}

9. Finally, the utility prompts for your choice to decide whether to proceed with themigration with the generated configuration file or not. The default option is y.


9-48

Note:

If you select n, you can use the generated configuration file to run themigration using the ./runMigrator -c or the ./runMigrator --configoption.

would you like to run the migration with above configuration?If you select no, you can use the generated configuration file to run the migration using./runMigrator --config /home/apothula/nosqlMigrator/NDCS2JSON(y/n) [y]:

10. The NoSQL Data Migrator migrates your data and schema from NDCS to theJSON file.

Records provided by source=10,Records written to sink=10,Records failed=0.Elapsed time: 0min 1sec 277msMigration completed.

ValidationTo validate the migration, you can open the JSON Sink files and view the schema anddata.

-- Exported myTable Data [~/nosqlMigrator]$cat myTableJSON{ "id" : 10, "document" : { "course" : "Computer Science", "name" : "Neena", "studentid" : 105 }}{ "id" : 3, "document" : { "course" : "Computer Science", "name" : "John", "studentid" : 107 }}{ "id" : 4, "document" : { "course" : "Computer Science", "name" : "Ruby", "studentid" : 100 }}{


9-49

"id" : 6, "document" : { "course" : "Bio-Technology", "name" : "Rekha", "studentid" : 104 }}{ "id" : 7, "document" : { "course" : "Computer Science", "name" : "Ruby", "studentid" : 100 }}{ "id" : 5, "document" : { "course" : "Journalism", "name" : "Rani", "studentid" : 106 }}{ "id" : 8, "document" : { "course" : "Computer Science", "name" : "Tom", "studentid" : 103 }}{ "id" : 9, "document" : { "course" : "Computer Science", "name" : "Peter", "studentid" : 109 }}{ "id" : 1, "document" : { "course" : "Journalism", "name" : "Tracy", "studentid" : 110 }}{ "id" : 2, "document" : { "course" : "Bio-Technology", "name" : "Raja", "studentid" : 108


9-50

}}

-- Exported myTable Schema [~/nosqlMigrator]$cat myTableSchemaCREATE TABLE IF NOT EXISTS myTable (id INTEGER, document JSON, PRIMARY KEY(SHARD(id)))

Migrate from Oracle NoSQL Database On-Premise to Oracle NoSQLDatabase Cloud Service

This example shows how to use the Oracle NoSQL Data Migrator to copy data and theschema definition of a NoSQL table from Oracle NoSQL Database to Oracle NoSQLDatabase Cloud Service (NDCS).

Use Case

As a developer, you are exploring options to avoid the overhead of managingthe resources, clusters, and garbage collection for your existing NoSQL DatabaseKVStore workloads. As a solution, you decide to migrate your existing on-premiseKVStore workloads to Oracle NoSQL Database Cloud Service because NDCSmanages them automatically.

Example

For the demonstration, let us look at how to migrate the data and schema definitionof a NoSQL table called myTable from the NoSQL Database KVStore to NDCS. Wewill also use this use case to show how to run the runMigrator utility by passing apre-created configuration JSON file.

Prerequisites


– Source: Oracle NoSQL Database

– Sink: Oracle NoSQL Database Cloud Service







9-51

• Identify the following details for the on-premise KVStore:

– storeName: kvstore

– helperHosts: <hostname>:5000

– table: myTable

ProcedureTo migrate the data and schema definition of myTable from NoSQL Database KVStoreto NDCS:

1. Prepare the configuration JSON file with the identified Source and Sink details.See Source Configuration Templates and Sink Configuration Templates .

{ "source" : { "type" : "nosqldb", "storeName" : "kvstore", "helperHosts" : ["<hostname>:5000"], "table" : "myTable", "requestTimeoutMs" : 5000 }, "sink" : { "type" : "nosqldb_cloud", "endpoint" : "us-phoenix-1", "table" : "myTable", "compartment" : "developers", "schemaInfo" : { "schemaPath" : "<complete/path/to/the/JSON/file/with/DDL/commands/for/the/schema/definition>", "readUnits" : 100, "writeUnits" : 100, "storageSize" : 1 }, "credentials" : "<complete/path/to/oci/config/file>", "credentialsProfile" : "DEFAULT", "writeUnitsPercent" : 90, "requestTimeoutMs" : 5000 }, "abortOnError" : true, "migratorVersion" : "1.0.0"}


3. Run the runMigrator command by passing the configuration JSON file using the--config or -c option.

[~/nosqlMigrator/nosql-migrator-1.0.0]$./runMigrator --config <complete/path/to/the/JSON/config/file>

4. The utility proceeds with the data migration, as shown below.

Records provided by source=10, Records written to sink=10, Records failed=0.


9-52

Elapsed time: 0min 10sec 426msMigration completed.

ValidationTo validate the migration, you can login to your NDCS console and verify that myTableis created with the source data.

Migrate from MongoDB-Formatted JSON file to an Oracle NoSQLDatabase Cloud Service

This example shows how to use the Oracle NoSQL Data Migrator to copy Mongo-DBFormatted Data to the Oracle NoSQL Database Cloud Service (NDCS).

Use Case

After evaluating multiple options, an organization finalizes Oracle NoSQL DatabaseCloud Service as its NoSQL Database platform. As its NoSQL tables and data arein MongoDB, they are looking for a way to migrate those tables and data to OracleNDCS.

Example

For the demonstration, let us look at how to migrate a MongoDB-formatted JSON fileto NDCS. We will use a manually created configuration JSON file for this example.

Prerequisites


– Source: MongoDB-Formatted JSON File

– Sink: Oracle NoSQL Database Cloud Service

• Extract the data from Mongo DB using the mongoexport utility. See mongoexportfor more information.

• Create a NoSQL table in the sink with a table schema that matches the data inthe Mongo-DB-formatted JSON file. As an alternative, you can instruct the NoSQLData Migratorto create a table with the default schema structure by setting thedefaultSchema attribute to true.


9-53

Note:

For a MongoDB-Formatted JSON source, the default schema for thetable will be as:

CREATE TABLE IF NOT EXISTS <tablename>(ID STRING, DOCUMENT JSON,PRIMARY KEY(SHARD(ID));

Where:

– tablename = value of the table config.

– ID = _id value from the mongoDB exported JSON source file.

– DOCUMENT = The entire contents of the mongoDB exported JSONsource file is aggregated into the DOCUMENT column excluding the _idfield.






ProcedureTo migrate the MongoDB-formatted JSON data to the Oracle NoSQL Database CloudService:

1. Prepare the configuration JSON file with the identified Source and Sink details.See Source Configuration Templates and Sink Configuration Templates .

{ "source" : { "type" : "file", "format" : "mongodb_json", "dataPath" : "<complete/path/to/the/MongoDB/Formatted/JSON/file>" }, "sink" : { "type" : "nosqldb_cloud", "endpoint" : "us-phoenix-1", "table" : "mongoImport", "compartment" : "developers",


9-54

"schemaInfo" : { "defaultSchema" : true, "readUnits" : 100, "writeUnits" : 60, "storageSize" : 1 }, "credentials" : "<complete/path/to/the/oci/config/file>", "credentialsProfile" : "DEFAULT", "writeUnitsPercent" : 90, "requestTimeoutMs" : 5000 }, "abortOnError" : true, "migratorVersion" : "1.0.0"}


3. Run the runMigrator command by passing the configuration JSON file using the--config or -c option.

[~/nosqlMigrator/nosql-migrator-1.0.0]$./runMigrator --config <complete/path/to/the/JSON/config/file>

4. The utility proceeds with the data migration, as shown below.

Records provided by source=29,353, Records written to sink=29,353, Records failed=0.Elapsed time: 9min 9sec 630msMigration completed.

ValidationTo validate the migration, you can login to your NDCS console and verify that myTableis created with the source data.

Troubleshooting the NoSQL Data MigratorLearn about the general challenges that you may face while using the , and how toresolve them.

Migration has failed. How can I resolve this?

A failure of the data migration can be because of multiple underlying reasons. Theimportant causes are listed below:

Chapter 9Troubleshooting the NoSQL Data Migrator

9-55

Table 9-12 Migration Failure Causes

Error Message Meaning Resolution

Failed to connect toOracle NoSQL Database

The migrator could notestablish a connection with theNoSQL Database.

• Check if the valuesof the storeName andhelperHosts attributesin the configuration JSONfile are valid and that thehosts are reachable.

• For a secured store, verifyif the security file is validwith correct user nameand password values.

Failed to connect toOracle NoSQL DatabaseCloud Service

The migrator could notestablish a connection withthe Oracle NoSQL DatabaseCloud Service.

• Verify if the endpoint URLor region name specifiedin the configuration JSONfile is correct.

• Check if the OCIcredentials file is availablein the path specified in theconfiguration JSON file.

• Ensure that the OCIcredentials provided in theOCI credentials are valid.

Table not found The table identified for themigration could not be locatedby the NoSQL Data Migrator.

For the Source:• Verify if the table is

present in the sourcedatabase.

• Ensure that the tableis qualified with itsnamespace in theconfiguration JSON file, ifthe table is created in anon-default namespace.

• Verify if you havethe required read/writeauthorization to accessthe table.

• If the source is OracleNoSQL Database CloudService, verify if thevalid compartment nameis specified in theconfiguration JSON file,and ensure that you havethe required authorizationto access the table.

For the Sink:• Verify if the table is

present in the Sink. Ifit does not exist, youmust either create thetable manually or usethe schemaInfo configto create it through themigration.


9-56

Table 9-12 (Cont.) Migration Failure Causes

Error Message Meaning Resolution

DDL Execution failed The DDL commands providedin the input schema definitionfile is invalid.

• Check the syntax of theDDL commands in theschemaPath file.

• Ensure that there is onlyone DDL statement perline in the schemaPathfile.

failed to write recordto the sink table withjava.lang.IllegalArgumentException

The input record is notmatching with the tableschema of the sink.

• Check if the datatypes and column namesspecified in the target sinktable are matching withsink table schema.

• If you applied anytransformation, check ifthe transformed recordsare matching with the sinktable schema.

Request timeout The source or sink's operationdid not complete within theexpected time.

• Verify the networkconnection.

• Check if the NoSQLDatabase is up andrunning.

• Try to increaserequestTimeout valuein the configuration JSONfile.

What should I consider before restarting a failed migration?

When a data migration task fails, the sink will be at an intermediate state containingthe imported data until the point of failure. You can identify the error and failure detailsfrom the logs and restart the migration after diagnosing and correcting the error. Arestarted migration starts over, processing all data from the beginning. There is no wayto checkpoint and restart the migration from the point of failure. Therefore, NoSQLData Migrator overwrites any record that was migrated to the sink already.

Migration is too slow. How can I speed it up?

The time taken for the data migration depends on multiple factors such as volume ofdata being migrated, network speed, current load on the database. In case of a cloudservice, the speed of migration also depends on the read throughput and the writethroughput provisioned. So, to improve the migration speed, you can:

• Try to reduce the current workload on your Oracle NoSQL Database whilemigrating the data.

• Ensure that the machine that is running the migration, source, and sink all arelocated in the same data center and the network latencies are minimal.

• In case of Oracle NoSQL Database Cloud Service, provision high read/writethroughput and verify if the storage allocated for table is sufficient or not. Ifthe NoSQL Data Migrator is not creating the table, you can increase the writethroughput. If the migrator is creating the table, consider specifying a higher value


9-57

for the schemaInfo.writeUnits parameter in the sink configuration. Once thedata migration completes, you can lower this value. Be aware of daily limits onthroughput changes. see Cloud Limits and Sink Configuration Templates .

I have a long running migration involving huge datasets. How can I track theprogress of the migration?

You can enable additional logging to track the progress of a long-running migration. Tocontrol the logging behavior of Oracle NoSQL Data Migrator, you must set the desiredlevel of logging in the logging.properties file. This file is provided with the NoSQLData Migrator package and available in the directory where the Oracle NoSQL DataMigrator was unpacked. The different levels of logging are OFF, SEVERE, WARNING,INFO, FINE, and ALL in the order of increasing verbosity. Setting the log level to OFFturns off all the logging information, whereas setting the log level to ALL provides thefull log information. The default log level is WARNING. All the logging output is configuredto go to the console by default. You can see comments in the logging.properties fileto know about each log level.

Oracle NoSQL Data Migrator Vs. Import/Export UtilityThis topic explains how the Oracle NoSQL Data Migrator utility is different from theexisting Oracle NoSQL import/export utility.

The Oracle NoSQL Data Migrator is created to replace and enhance the existingon-premise-only import/export utility. It moves the NoSQL table data and schemadefinition between a source and a sink or target. It supports multiple sources and sinksas listed in Supported Sources and Sinks. However, the import/export utility lets youimport into or export from Oracle NoSQL Database (on-premise) only. That is, usingthe import/export utility, you can either import data into the Oracle NoSQL Databaseor export data from Oracle NoSQL Database. When you export, the source type isalways Oracle NoSQL Database (where you extract data from) and the sink is therecipient of that data. When you import, the source type is currently limited to a file andthe sink is always Oracle NoSQL Database. See Export and Import Functionality.

Apart from this fundamental difference, both the utilities are different in many otherways. You can see them in the comparison table below.

Table 9-13 Comparison Table

Oracle NoSQL Data Migrator Oracle NoSQL Import/Export

Sources and Sinks Supports multiple sourcesand sinks such asOracle NoSQL Database(on-premise), Oracle NoSQLDatabase Cloud Service,JSON file, and MongoDB-formatted JSON file. SeeSupported Sources and Sinks.

ImportSource: JSON, binary, orMongoDB-formatted JSON

Sink: Oracle NoSQL Database(on-premise) only.

ExportSource: Oracle NoSQLDatabase (on-premise) only.

Sink: Local or networkmounted filesystem.

Chapter 9Oracle NoSQL Data Migrator Vs. Import/Export Utility

9-58

Table 9-13 (Cont.) Comparison Table

Oracle NoSQL Data Migrator Oracle NoSQL Import/Export

Supported Platforms Both the Oracle NoSQLDatabase (on-premise) andthe Oracle NoSQL DatabaseCloud Service (NDCS).

Only Oracle NoSQL Database(on-premise).

Migration Level The migration is supportedonly at the table level. You canmigrate only one table at atime.

You can perform the import/export operations at a tablelevel, store level, or annamespace level. You canimport/export multiple tables ina single operation.

Data Formats Supports only JSON formatfor the data. The schemadefinition is represented as afile with one DDL commandper line.

Supports both the Binary andJSON formats.

Unit of Migration The Oracle NoSQL DataMigrator migrates the data andschema definition from theSource to the Sink withoutcreating any intermediatepackages.

When you export data fromOracle NoSQL Database, theimport/export utility creates anexport package (a directorystructure) that containsschema metadata and tabledata. When you import datainto Oracle NoSQL Database,you can run the utility againstthis export package, or anydirectory that contains fileswith data formats supportedby the import/export utility.

Key-Value and Large ObjectData

Not supported. Supported by the exportcommand.

NoSQL Database on-premise to on-premiseMigration

Supported. Not supported.

Transitioning from Import/Export to NoSQL Data MigratorLearn how to transition from the import/export utility to the Oracle NoSQL DataMigrator.

If you have been using the import/export utility for data migration you would be awarethat the kvtool.jar that is part of the Oracle NoSQL Database (on-premise) handlesexport and import functionality with the help of two commands, export and import.You can achieve the same export and import operations using the NoSQL DataMigrator, as explained below.

Equivalent of the export command in NoSQL Data Migrator

The export command is used to export the NoSQL Database (on-premise) data toa local or network mounted file system. You can achieve the equivalent operation bydefining the following attributes in the Configuration JSON file as follows:

Chapter 9Transitioning from Import/Export to NoSQL Data Migrator

9-59

• "source"="nosqldb"

• "sink"="file" with "format"="JSON"

The other configuration attributes can be defined as per the requirement. As theNoSQL Data Migrator supports migrating only one table at a time, you must run theNoSQL Data Migrator separately for exporting multiple tables.

Equivalent of the import command in NoSQL Data Migrator

The import command is used to import a package created by the export command,an external JSON file, or a MongoDB-formatted JSON file. You can achieve theequivalent operation by defining the following attributes in the Configuration JSONfile as follows:

• "source"="file" with "format"="json" or "format"="mongodb_json" dependingon the source file type.

• "sink"="nosqldb"

Note:The NoSQL Data Migrator treats an external JSON file or a package that wasgenerated using the existing import/export utility similarly.

Chapter 9Transitioning from Import/Export to NoSQL Data Migrator

9-60

Documents

Using Oracle NoSQL Database Cloud Service · 2020-06-01 · This section provides an overview of Oracle NoSQL Database Cloud Service. Oracle NoSQL Database Cloud Service is a fully