Release Notes - ClouderaSee Charting Time-Series Data What's new in DAS There are no new features in DAS in this release of Cloudera Runtime. What's New in Apache HBase There are no

Release NotesDate of publish:

https://docs.cloudera.com/


Legal Notice

© Cloudera Inc. 2020. All rights reserved.

The documentation is and contains Cloudera proprietary information protected by copyright and other intellectualproperty rights. No license under copyright or any other intellectual property right is granted herein.

Copyright information for Cloudera software may be found within the documentation accompanying eachcomponent in a particular release.

Cloudera software includes software from various open source or other third party projects, and may bereleased under the Apache Software License 2.0 (“ASLv2”), the Affero General Public License version 3(AGPLv3), or other license terms. Other software included may be released under the terms of alternative opensource licenses. Please review the license and notice files accompanying the software for additional licensinginformation.

Please visit the Cloudera software product page for more information on Cloudera software. For moreinformation on Cloudera support services, please visit either the Support or Sales page. Feel free to contact usdirectly to discuss your specific needs.

Cloudera reserves the right to change any products at any time, and without notice. Cloudera assumes noresponsibility nor liability arising from the use of products, except as expressly agreed to in writing by Cloudera.

Cloudera, Cloudera Altus, HUE, Impala, Cloudera Impala, and other Cloudera marks are registered orunregistered trademarks in the United States and other countries. All other trademarks are the property of theirrespective owners.

Disclaimer: EXCEPT AS EXPRESSLY PROVIDED IN A WRITTEN AGREEMENT WITH CLOUDERA,CLOUDERA DOES NOT MAKE NOR GIVE ANY REPRESENTATION, WARRANTY, NOR COVENANT OFANY KIND, WHETHER EXPRESS OR IMPLIED, IN CONNECTION WITH CLOUDERA TECHNOLOGY ORRELATED SUPPORT PROVIDED IN CONNECTION THEREWITH. CLOUDERA DOES NOT WARRANT THATCLOUDERA PRODUCTS NOR SOFTWARE WILL OPERATE UNINTERRUPTED NOR THAT IT WILL BEFREE FROM DEFECTS NOR ERRORS, THAT IT WILL PROTECT YOUR DATA FROM LOSS, CORRUPTIONNOR UNAVAILABILITY, NOR THAT IT WILL MEET ALL OF CUSTOMER’S BUSINESS REQUIREMENTS.WITHOUT LIMITING THE FOREGOING, AND TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLELAW, CLOUDERA EXPRESSLY DISCLAIMS ANY AND ALL IMPLIED WARRANTIES, INCLUDING, BUT NOTLIMITED TO IMPLIED WARRANTIES OF MERCHANTABILITY, QUALITY, NON-INFRINGEMENT, TITLE, ANDFITNESS FOR A PARTICULAR PURPOSE AND ANY REPRESENTATION, WARRANTY, OR COVENANTBASED ON COURSE OF DEALING OR USAGE IN TRADE.

| Contents | iii

Contents

Overview....................................................................................................... 5

Cloudera Runtime Component Versions.................................................. 5

What's New In Cloudera Runtime 7.1.0.0..................................................6What's New in Apache Atlas...............................................................................................................6What's new in DAS............................................................................................................................. 6What's New in Apache HBase............................................................................................................6What's New in Apache Hadoop HDFS............................................................................................... 6What's New in Apache Hive............................................................................................................... 6What's New in Hue............................................................................................................................. 7What's New in Apache Impala............................................................................................................8What's New in Apache Kafka............................................................................................................. 9What's New in Apache Knox............................................................................................................ 10What's New in Apache Kudu............................................................................................................ 10What's New in Apache Oozie........................................................................................................... 10What's New in Apache Phoenix........................................................................................................10What's New in Cloudera Search.......................................................................................................10What's New in Apache Spark........................................................................................................... 10What's New in Apache Hadoop YARN.............................................................................................10What's New in Apache ZooKeeper...................................................................................................11

Fixed Issues In Cloudera Runtime 7.1.0.0.............................................. 11Atlas................................................................................................................................................... 11Fixed issues in DAS..........................................................................................................................12Hadoop.............................................................................................................................................. 12HDFS................................................................................................................................................. 12Fixed Issues in Apache Hive............................................................................................................ 12Zeppelin............................................................................................................................................. 12

Known Issues In Cloudera Runtime 7.1.0.0............................................13Known Issues in Apache Atlas......................................................................................................... 13Known Issues in DAS....................................................................................................................... 15Known Issues in Apache HBase.......................................................................................................16Known Issues in HDFS.....................................................................................................................29Known Issues in Hue........................................................................................................................ 29Known Issues in Apache Impala.......................................................................................................30Known Issues in Apache Kafka........................................................................................................ 34Known Issues in Apache Knox......................................................................................................... 36Known Issues in Apache Kudu.........................................................................................................37Known Issues in Apache Oozie........................................................................................................37Known Issues in Apache Ranger......................................................................................................37Known Issues in Cloudera Search....................................................................................................38Known Issues in Apache Spark........................................................................................................41Known Issues for Apache Sqoop......................................................................................................42

Known Issues in MapReduce and YARN......................................................................................... 43Known Issues in Apache Zeppelin....................................................................................................45Known Issues in Apache ZooKeeper................................................................................................45

Overview

Overview

Welcome to the Cloudera Runtime Release Notes. This document provides you with the latest informationabout Cloudera Runtime 7.1.0.0. This document includes improvements and describes new features, bugfixes, tech previews and more. For detailed information about the runtime components themselves, seeCloudera documentation.

Cloudera Runtime Component Versions

List of the official Apache component versions for Cloudera Runtime. To know the Apache componentversions for compatibility with other applications, you must be familiar with the latest Apache componentversions in Cloudera Runtime. You should also be aware of the available Technical Preview componentsand use them only in a testing environment. Apache versions of Cloudera Runtime 7.1.0.0. components.

Component Apache Version

Apache Accumulo 1.7.0

Apache Atlas 2.0.0

Apache Arrow 0.8.0

Apache Avatica 1.10.0

Apache Avro 1.8.2

Apache Calcite 1.19.0

Apache Crunch 0.11.0

Apache DataFu 1.3.0

Apache Druid 0.15.1

Apache Hadoop 3.1.1

Apache HBase 2.2.0

Apache Hive 3.1.3000

Hue 4.5.0

Apache Impala 3.3.0

Apache Kafka 2.3.0

Apache Knox 1.3.0

Apache Kudu 1.11.0

Apache Livy 0.6.0

Apache Oozie 5.1.0

Apache ORC 1.5.1

Apache Parquet 1.10.99

Apache Phoenix 5.0.0

Apache Ranger 2.0.0

Apache Solr 8.4.1

Apache Spark 2.4.4

Apache Sqoop 1.4.7

Apache Superset 0.34.0

5


What's New In Cloudera Runtime 7.1.0.0

Component Apache Version

Apache Tez 0.9.1

Apache ZooKeeper 3.5.5

Apache Zeppelin 0.8.2

YARN 3.1.1


This version of Cloudera Runtime provides you with several new capabilities. Learn how the new featuresand improvements benefit you.

What's New in Apache AtlasThis topic lists new features for Apache Atlas in this release of Cloudera Runtime.

Associated Classification Counts and Search Links in Statistics

Atlas Statistics page now includes a list of the classifications that have been associated with entities. Foreach classification, Atlas shows a count of the associated entities; clicking the count opens search resultsfor a search on that classification.

See Apache Atlas Statistics reference

New Atlas Metrics for Cloudera Manager Charts

Atlas now provides a number of metrics that you can use to create time-series charts in Cloudera Manager.Add these charts to a dashboard in Cloudera Manager to see Atlas resource use at a glance.

Metrics include the rate of entity creation, update, and deletion and the size of Java heap used by Atlas.

See Charting Time-Series Data

What's new in DASThere are no new features in DAS in this release of Cloudera Runtime.

What's New in Apache HBaseThere are no new features for Apache HBase in this release of Cloudera Runtime.

For more information about Apache HBase, see Apache HBase Overview.

What's New in Apache Hadoop HDFSThere are no new features for Apache Hadoop HDFS in this release of Cloudera Runtime.

For more information about HDFS, see HDFS Overview

What's New in Apache HiveThere are no new features for Apache Hive in this release of Cloudera Runtime.

6

https://docs.cloudera.com/runtime/7.0.3/atlas-reference/topics/atlas-reference-statistics.html

https://docs.cloudera.com/cloudera-manager/7.0.3/monitoring-and-diagnostics/topics/cm-charting-time-series-data.html

https://docs.cloudera.com/runtime/7.1.0/hbase-overview/topics/hbase-overview.html

https://docs.cloudera.com/runtime/7.1.0/hdfs-overview/topics/hdfs-overview-of-apache-hdfs.html


What's New in HueThis topic lists new features for Hue in this release of Cloudera Runtime.

Support for Hive on Tez

Hue in Cloudera Runtime now supports Hive using Tez as its execution engine.

Integration with Apache Atlas data catalog

You can now locate tables by searching for Atlas classifications in Hue by specifying theclassification search term in the search box at the top of the page:

You can also locate tables by searching for Atlas classifications by specifying the tag search term:

7


Hive Language Reference

A language reference has been added for Hive:

When you select the Hive query editor, click the book icon to the right of the editor window to launchthe Hive language reference.

What's New in Apache ImpalaThis topic lists new features for Apache Impala in this release of Cloudera Runtime.

April, 2020

Increased scratch capacity

To help reduce spilling to disk:

• Added startup parameter to support Spill-to-disk compression to increase effective scratch capacity by2.5x.

• Added startup parameter to reclaim space in scratch files.

Improved data cache performance

• Improved the efficiency of the data cache by providing an option to use a different cache evictionalgorithm (LIRS).

Support for Kudu Date and Varchar column types

Support for reading ZSTD-compressed text files

For details, see Using Text Data Files.

Improved read performance of ORC tables

Improved Impala resiliency

8

https://docs.cloudera.com/runtime/7.1.0/impala-reference/topics/impala-textfile.html


This release adds client retry support in the impala-shell. For details about installing the impala-shell, seeUsing Impala shell.

March, 2020

broadcast_bytes_limit query option

In this release, you can set a limit for the size of a broadcast input. For details, see Impala Query Options.

ORC stability and performance improvements

ORC reads enabled by default

Impala stability and performance have been improved. Consequently, ORC reads are now enabled inImpala by default. To disable, set --enable_orc_scanner to false when starting the cluster.

February, 2020

Constraints

This release adds support for primary and foreign key constraints, but in this release the constraints areadvisory and intended for estimating cardinality during query planning in a future release. There is noattempt to enforce constraints. For details, see the “Constraints” section of Create Table Statement.

Enhanced external Kudu table

By default HMS implicitly translates internal Kudu tables to external Kudu tables with the'external.table.purge' property set to true. These tables behave similar to internal tables. You can explicitlycreate such external Kudu tables. For details, see the “External Kudu Tables” section of Create TableStatement.

Ranger column masking

This release supports Ranger column masking, which hides sensitive columnar data in Impala queryoutput. For example, you can define a policy that reveals only the first or last four characters of columndata. Column masking is enabled by default. For details, see the "Ranger Column Masking" section inImpala Authorization.

What's New in Apache KafkaThis topic lists new features for Apache Kafka in this release of Cloudera Runtime.

Zookeeper TLS/SSL support for Kafka

A new Kafka configuration parameter, Enable Secure Connection to ZooKeeper is added. This propertyenables or disables secure communication between Kafka and Zookeeper. The property is enabled bydefault, however, it will only take effect if the Enable TLS/SSL for ZooKeeper property is also enabled forthe dependent ZooKeeper service. For more information, see the related Zookeeper documentation andKafka documentation.

Enable Delegation Tokens is set to true by default

The Enable Delegation Tokens property that enables and disables delegation token based authenticationis set to true by default. Note however, that this does not by default enable the feature itself as delegationtokens require Kerberos authentication to function.

For more information, see Enable or Disable Authentication with Delegation Tokens.

9

https://docs.cloudera.com/data-warehouse/cloud/querying-data/topics/dw-using-impala-shell.html

https://docs.cloudera.com/runtime/7.1.0/impala-sql-reference/topics/impala-query-options.html

https://docs.cloudera.com/runtime/7.1.0/impala-sql-reference/topics/impala-create-table.html



https://docs.cloudera.com/runtime/7.1.0/impala-manage/topics/impala-authorization.html

https://docs.cloudera.com/runtime/7.1.0/zookeeper-security/topics/zookeeper-configure-tls-ssl.html

https://docs.cloudera.com/runtime/7.1.0/kafka-securing/topics/kafka-secure-tls-zk.html

https://docs.cloudera.com/runtime/7.1.0/kafka-securing/topics/kafka-secure-deltokens-enable.html


What's New in Apache KnoxThere are no new features for Apache Knox in this release of Cloudera Runtime.

What's New in Apache KuduThis topic lists new features for Apache Kudu in this release of Cloudera Runtime.

Support for EC-based ciphers

Kudu supports EC-based ciphers in the web UI. Previously, both kudu-tserver and kudu-masterfailed to start if supplied with server certificates based on ciphers that squeasel does not support.

What's New in Apache OozieThere are no new features for Apache Oozie in this release of Cloudera Runtime.

For more information about Oozie, see Overview of Oozie.

What's New in Apache PhoenixThis release of Cloudera Runtime introduces Apache Phoenix, a massively parallel, relational databaseengine supporting OLTP use cases using Apache HBase as its backing store.

For more information about Apache Phoenix, see Phoenix Overview.

Note: Apache Phoenix in CDP is a technical preview and considered under development. Do notuse this in your production systems. If you have feedback, contact Support by logging a case on theCloudera Support Portal at https://my.cloudera.com/support.html. Technical preview features arenot guaranteed troubleshooting and fixes.

What's New in Cloudera SearchThere are no new features for Cloudera Search in this release of Cloudera Runtime.

For more information about Cloudera Search, see Cloudera Search Overview.

What's New in Apache SparkThis topic lists new features for Apache Spark in this release of Cloudera Runtime.

Data engineering cluster

You can create a data engineering cluster in Amazon AWS from within CDP by selecting the DataEngineering cluster template. A data engineering includes Spark, Livy, Hive, Zeppelin, and Oozie, alongwith supporting services (HDFS, YARN, and Zookeeper).

See Creating a Cluster on AWS.

What's New in Apache Hadoop YARNThere are no new features in this release.

10

https://docs.cloudera.com/runtime/7.1.0/configuring-oozie/topics/oozie-introduction.html

https://docs.cloudera.com/runtime/7.1.0/phoenix-overview/topics/phoenix-overview.html

https://my.cloudera.com/support.html

https://docs.cloudera.com/runtime/7.1.0/search-overview/topics/search-introducing.html

https://docs.cloudera.com/data-hub/cloud/create-cluster-aws/index.html

Fixed Issues In Cloudera Runtime 7.1.0.0

More reading

For more information about Apache Hadoop YARN, see Apache Hadoop YARN Overview

What's New in Apache ZooKeeperThis topic lists new features for Apache ZooKeeper in this release of Cloudera Runtime.

ZooKeeper TLS/SSL

TLS/SSL encryption between the ZooKeeper client and the ZooKeeper server and within the ZooKeeperQuorum is supported. TLS/SSL encryption is automatically enabled when AutoTLS is enabled, or you canenable it manually using Cloudera Manager. ClientSSL requires you to enable and configure it by othercomponents that you want to use this feature with. For more information, see Configure ZooKeeper TLS/SSL using Cloudera Manager.


Fixed issues represent issues reported by Cloudera customers that are addressed in this release.

AtlasThis section lists the issues that have been fixed since the previous version.

OPSAPS-53788: Lineage info for HBase notavailable in a TLS-enabled environment

Atlas will not collect metadata from HBase(namespaces, tables, or column families) whenCloudera Manager is configured for TLS/SSL.This issue has no effect on HBase functionality,however, tag-based authorization through Rangerpolicies will not be available for HBase.

CDPD-6023: Slight differences in metadatacollected for Hive database entities dependingon source

When a Hive database is created using Beeline,the ownerType is sent to ATLAS_HOOK and inrecorded in the hive_db entity. However, when aHive database is created through Spark-shell, theownerType is missing in the Atlas entity.

OPSAPS-51224: Atlas custom propertiesignored in client services

When adding a custom property for the atlas-application.properties in Atlas hook-basedservices such as Hive, HBase, and Impala, thecustom property is not reflected in the actualconfiguration file that Cloudera Manager generates,causing these properties to be ignored.

CDPD-5314: Unexpected Results from AtlasAdvanced Search Query with UnderscoreCharacter

An Atlas Advanced Search with an equal operator(=) and a like operator where the like objectincludes an underscore does not return correctresults. For example, the following search does notreturn expected results:

db.name : "=" finance andQualifiedName :"like" *finance_09*

CDPD-5309: Unexpected Results from AtlasAdvanced Search Query on String Attributes

When new entity types are defined with string typeattributes, Advanced Searches against the stringattributes return unexpected results. For example, if

11

https://docs.cloudera.com/runtime/7.1.0/yarn-overview/topics/yarn-introduction-yarn.html




attribute values include "caterpillar", "alleycat", and"altercation", all values are returned when searchingfor "cat*". The expected result would be that only"caterpillar" would be returned.

Fixed issues in DASThis section lists the issues that have been fixed since the previous version.

• Fixed several stability issues in the reporting pipeline which caused the report pipeline to get stuck.• Fixed performance issue in the events pipeline when events are on the S3 filesystem.• DWX-2508: DAS now shows accurate compilation time for a query on the Query Details > Timeline

tab.• DWX-2220: The Compose page UI has been enriched with tooltips and also displays the database

count.

HadoopThis section lists the issues that have been fixed since the previous version.

• CDPD-7383: The ABFS and ADL connectors compatible with Alpine Linux and other platformswhich have libssl1.1-1.1.1b-r1 as their native OpenSSL implementation, see HADOOP-16460 andHADOOP-16438

HDFSThis section lists the issues that have been fixed since the previous version.

CDPD-6100 This improvement makes HDFS NameNode leavesafemode immediately if all blocks have reported in,reducing the cluster startup time.

CDPD-2946: Slow reading and writing oferasure-coded files

The ISA-L library is not packaged with HDFS as aresult of which HDFS erasure coding falls back tothe Java implementation which is much slower thanthe native Hadoop implementation. This slows downthe reading and writing of erasure-coded files.

Fixed Issues in Apache HiveThis section lists the issues that have been fixed since the previous version.

• CDPD-676: Generate Oozie workflow for microstrategy• CDPD-7963: Fixes memory leak in HiveServer2 caused by external or unmanaged sessions in Tez..

ZeppelinThis section lists the issues that have been fixed since the previous version.

CDPD-1683: Zeppelin demo users have beenremoved

Use cluster users to access Zeppelin. Forinformation on provisioning users in CDP, seeOnboarding users.

12

https://jira.cloudera.com/browse/DWX-2508

https://jira.cloudera.com/browse/DWX-2220

https://jira.cloudera.com/browse/CDPD-7383





https://docs.cloudera.com/management-console/cloud/user-management/topics/mc-onboarding-users.html

Known Issues In Cloudera Runtime 7.1.0.0

CDPD-880, CDPD-1685: Shell, JDBC, and Sparkinterpreters have been removed

Workaround: Use an available interpreter. ForSpark functionality, use the Livy interpreter.

CDPD-3047: Markdown interpreter does nothandle certain numbered list syntax correctly

Using the plus sign (+) or asterisk (*) to continuea numbered list using the %md interpreter results inbullet point entries instead.

Workaround: None.


Known Issues in Apache AtlasThis topic describes known issues and workarounds for using Atlas in this release of Cloudera Runtime.

Bridge importing metadata from HBase failswhen it encounters an HBase table for whicha column family was previously dropped. Theerror indicates:

Metadata service APIorg.apache.atlas.AtlasClientV2$API_V2@58112bc4failed with status 404 (Not Found) Response Body({"errorCode":"ATLAS-404-00-007","errorMessage":"Invalidinstance creation/updation parameters passed :hbase_column_family.table: mandatory attributevalue missing in type hbase_column_family"})

Workaround: None.

Cloudera JIRA: CDPD-7982

Hive Default Database Location Incorrect inAtlas Metadata

The location of the default Hive database asreported through the HMS-Atlas plugin does notmatch the actual location of the database. Thisproblem does not affect non-default databases.

Workaround: None.


Unexpected Search Results When UsingRegular Expressions in Basic Searches onClassifications

When you include a regular expression or wildcardin the search criteria for a classification in theBasic Search, the results may differ unexpectedlyfrom when full classification names are included.For example, the Exclude sub-classificationsoption is respected when using a full classificationname as the search criteria; when using part ofthe classification name and the wildcard (*) withExclude sub-classifications turned off, entitiesmarked with sub-classifications are not included inthe results. Other instances of unexpected resultsinclude case-sensitivity.

Workaround: None.

Cloudera JIRA: CDPD-5933, CDPD-5931

Spark metadata order may affect lineage Atlas may record unexpected lineage relationshipswhen metadata collection from the Spark AtlasConnector occurs out of sequence from metadatacollection from HMS. For example, if an ALTERTABLE operation in Spark changing a table nameand is reported to Atlas before HMS has processed

13


the change, Atlas may not show the correct lineagerelationships to the altered table.

Workaround: None.


Searches for Qualified Names with "@" doesn'tfetch the correct results

When searching Atlas qualifiedName values thatinclude an "at" character (@), Atlas does not returnthe expected results or generate appropriate searchsuggestions.

Workaround: Consider leaving out the portion of thesearch string that includes the @ sign, using thewildcard character * instead.


Missing Impala and Spark lineage betweentables and their data files

Atlas does not create lineage between Hive tablesand their backing HDFS files for CTAS processesrun in Impala or Spark.

Workaround: None.

Cloudera JIRA: CDP-5027, CDPD-3700,IMPALA-9070

Table alias values are not found in search When table names are changed, Atlas keeps theold name of the table in a list of aliases. Thesevalues are not included in the search index inthis release, so after a table name is changed,searching on the old table name will not return theentity for the table.

Workaround: None.


Hive lineage missing for INSERT OVERWRITEqueries

Lineage is not generated for Hive INSERTOVERWRITE queries on partitioned tables. Lineageis generated as expected for CTAS queries frompartitioned tables.

Workaround: None.


Logging out of Atlas does not manage theexternal authentication

At this time, Atlas does not communicate a log-outevent with the external authentication management,Apache Knox. When you log out of Atlas, you canstill open the instance of Atlas from the same webbrowser without re-authentication.

Workaround: To prevent access to Atlas afterlogging out, close all browser windows and exit thebrowser.


Ranking of top results in free-text search notintuitive

The Free-text search feature ranks results basedon which attributes match the search criteria. Theattribute ranking is evolving and therefore thechoice of top results may not be intuitive in thisrelease.

Workaround: If you don't find what you need inthe top 5 results, use the full results or refine thesearch.

14



Free text search in Atlas is case sensitive The free text search bar in the top of the screenallows you to search across entity types andthrough all text attributes for all entities. The searchshows the top 5 results that match the search termsat any place in the text (*term* logic). It also showssuggestions that match the search terms that beginwith the term (term* logic). However, in this release,the search results are case-sensitive.

Workaround: If you don't see the results you expect,repeat the search changing the case of the searchterms.

Workaround: None.


Queries with ? wildcard return unexpectedresults

DSL queries in Advanced Search return incorrectresults when the query text includes a questionmark (?) wildcard character. This problem occursin environments where trusted proxy for Knox isenabled, which is always the case for CDP.

Workaround: None.


Guest users are redirected incorrectly Authenticated users logging in to Atlas areredirected to the CDP Knox-based login page.However, if a guest user (without Atlas privileges)attempts to log in to Atlas, the user is redirectedinstead to the Atlas login page.

Workaround: To avoid this problem, open theAtlas Dashboard in a private or incognito browserwindow.


All Spark Queries from the Same Spark Sessionare Included in a Single Atlas Process

A Spark session can include multiple queries. WhenAtlas reports the Spark metadata, it creates a singleprocess entity to correspond to the Spark session.The result is that an Atlas lineage picture may showmultiple input entities or multiple output entitiesfrom a process that are only related by the fact thatthey were included in operations in the same Sparksession. The consequence of this behavior is thatclassifications will be propagated from any inputentity to all output entities, even if the output entitiesaren't derived from the input entity.

Workaround: you can manually stop classificationpropagation to the inappropriate entity or choosenot to propagate classifications that might be usedin these scenarios.


Known Issues in DASThis topic describes known issues and workarounds for using DAS in this release of Cloudera Runtime.

15


• The task logs for a particular task may not be available in the task swimlane. And the zip file generatedby download logs artifact may not have task logs, but instead contain an error-reports.json filewith the error log of the download failures.

• You may not see any data for a report for any new queries that you run. This can happen especially forthe last one day's report.

Workaround:

1. Shut down the DAS Event Processor.2. Run the following command from the Postgres server:

update das.report_scheduler_run_audit set status = 'FAILED' where status = 'READING';

3. Start the DAS Event Processor.• On clusters secured with Knox proxy only: You might not be able to save the changes to the JDBC URL

in the DAS UI to change the server interface (HS2 or LLAP) on which you are running your queries.• You may be unable to upload tables or get an error while browsing files to upload tables in DAS on a

cluster secured using Knox proxy.• DAS does not parse semicolons (;) and double hyphens (--) in strings and comments.

For example, if you have a semicolon in query such as the following, the query might fail: select *from properties where prop_value = "name1;name2";

If a semicolon is present in a comment, then execute the query after removing the semicolon from thecomment, or removing the comment altogether. For example:

select * from test; -- select * from test;select * from test; /* comment; comment */

Queries with double hyphens (--) might also fail. For example:

select * from test where option = '--name';

• You might face UI issues on Google Chrome while using faceted search. We recommend you to usethe latest version of Google Chrome (version 71.x or higher).

• You are unable to see databases or the query editor on the Compose page even after following thetroubleshooting procedure. This can be caused due to incorrect hive service name in ZooKeeper.

• Visual Explain for the same query shows different graphs on the Compose page and the QueryDetails page.

• While running some queries, if you restart HSI, the query execution is stopped. However, DAS does notreflect this change and the queries appear to be in the same state forever.

• After a fresh installation, when there is no data and you try to access the Reports tab, DAS displays an"HTTP 404 Not Found" error.

• Join count does not get updated for tables with partitioned columns.

Known Issues in Apache HBaseThis topic describes known issues and workarounds for using HBase in this release of Cloudera Runtime.

OpDB Data Hub cluster fails to initialize if youare reusing a cloud storage location that wasused by an older OpDB Data Hub cluster

Workaround: Stop HBase using Cloudera Managerbefore deleting an OpDB Data Hub cluster.

IntegrationTestReplication fails ifreplication does not finish before the verifyphase begins

During IntegrationTestReplication, if theverify phase starts before the replicationphase finishes, the test will fail because the targetcluster does not contain all of the data. If the HBase

16


services in the target cluster does not have enoughmemory, long garbage-collection pauses mightoccur.

Workaround: Use the -t flag to set the timeoutvalue before starting verification.

HDFS encryption with HBase Cloudera has tested the performance impactof using HDFS encryption with HBase. Theoverall overhead of HDFS encryption on HBaseperformance is in the range of 3 to 4% for both readand update workloads. Scan performance has notbeen thoroughly tested.

Workaround: N/A

AccessController postOperation problems inasynchronous operations

When security and Access Control are enabled, thefollowing problems occur:

• If a Delete Table fails for a reason otherthan missing permissions, the access rights areremoved but the table may still exist and may beused again.

• If hbaseAdmin.modifyTable() is usedto delete column families, the rights are notremoved from the Access Control List (ACL)table. The portOperation is implemented onlyfor postDeleteColumn().

• If Create Table fails, full rights for that tablepersist for the user who attempted to create it. Ifanother user later succeeds in creating the table,the user who made the failed attempt still hasthe full rights.

Workaround: N/A

Apache Issue: HBASE-6992

Bulk load is not supported when the source isthe local HDFS

The bulk load feature (the completebulkloadcommand) is not supported when the source is thelocal HDFS and the target is an object store, suchas S3/ABFS.

Workaround: Use distcp to move the HFiles fromHDFS to S3 and then run bulk load from S3 to S3.

Apache Issue: N/A

Storing Medium Objects (MOBs) in HBase iscurrently not supported

Storing MOBs in HBase relies on bulk loading files,and this is not currently supported when HBase isconfigured to use cloud storage (S3).

Workaround: N/A

Apache Issue: N/A

Known incompatibilities when upgrading from CDH and HDP

This section describes incompatible changes when you upgrade from CDH and HDP. Use this list tounderstand some of the deprecated items and incompatibilities if you are moving from HDP 2.x or CDH5.x. Note that Cloudera Runtime uses HBase 2.x.x whereas older CDH and HDP used Apache HBase 1.x.

17

https://issues.apache.org/jira/browse/HBASE-6992


Important: Some APIs that are listed as deprecated, but these APIs do not block your upgrade.You must stop using the deprecated APIs in your existing applications after upgrade, and not usethese APIs in new development.

List of Major Changes

• HBASE-16189 and HBASE-18945- You cannot open the HBase 2.x HFiles through HBase 1.x version.• HBASE-18240 - Changed the ReplicationEndpoint Interface. It also introduces a new hbase-third party

1.0 that packages all the third party utilities, which are expected to run in the hbase cluster.

Co-processor API changes

• HBASE-16769 - Deprecated PB references from MasterObserver and RegionServerObserver.• HBASE-17312 - [JDK8] Use default method for Observer Coprocessors. The interface

classes of BaseMasterAndRegionObserver, BaseMasterObserver, BaseRegionObserver,BaseRegionServerObserver and BaseWALObserver uses JDK8’s 'default' keyword to provide emptyand no-op implementations.

• Interface HTableInterface introduces following changes to the methods listed below:

[#] interface CoprocessorEnvironment changes

Change Result

Abstract method getTable ( TableName ) has been removed. A client program may be interrupted by NoSuchMethodErrorexception.

Abstract method getTable ( TableName, ExecutorService ) hasbeen removed.

A client program may be interrupted by NoSuchMethodErrorexception.

• Public Audience

The following tables describes the coprocessor changes.

[#] class CoprocessorRpcChannel (1)

Change Result

This class has become interface. A client program may be interrupted byIncompatibleClassChangeError or InstantiationError exceptiondepending on the usage of this class.

Class CoprocessorHost<E> changes

Classes that were Audience Private but were removed.

Change Result

Type of field coprocessors has beenchanged from java.util.SortedSet<E> toorg.apache.hadoop.hbase.util.SortedList<E>.

A client program may be interrupted by NoSuchFieldErrorexception.

MasterObserver changes

The following changes are introduced to the MasterObserver interface.

[#] interface MasterObserver (14)

Change Result

Abstract method voidpostCloneSnapshot( ObserverContext<MasterCoprocessorEnvironment>,HBaseProtos.SnapshotDescription, HTableDescriptor ) hasbeen removed from this interface.


18


Abstract method voidpostCreateTable( ObserverContext<MasterCoprocessorEnvironment>,HTableDescriptor, HRegionInfo[ ] ) has been removed from thisinterface.

A client program may be interrupted byNoSuchMethodErrorexception.

Abstract method voidpostDeleteSnapshot(ObserverContext<MasterCoprocessorEnvironment>,HBaseProtos.SnapshotDescription ) has been removed from thisinterface.


Abstract method voidpostGetTableDescriptors( ObserverContext<MasterCoprocessorEnvironment>,List<HTableDescriptor> ) has been removed from this interface.


Abstract method voidpostModifyTable( ObserverContext<MasterCoprocessorEnvironment>,TableName, HTableDescriptor ) has been removed from thisinterface.


Abstract method voidpostRestoreSnapshot( ObserverContext<MasterCoprocessorEnvironment>,HBaseProtos.SnapshotDescription, HTableDescriptor ) hasbeen removed from this interface.


Abstract method voidpostSnapshot( ObserverContext<MasterCoprocessorEnvironment>,HBaseProtos.SnapshotDescription, HTableDescriptor ) hasbeen removed from this interface.


Abstract method voidpreCloneSnapshot( ObserverContext<MasterCoprocessorEnvironment>,HBaseProtos.SnapshotDescription, HTableDescriptor ) hasbeen removed from this interface.


Abstract method voidpreCreateTable( ObserverContext<MasterCoprocessorEnvironment>,HTableDescriptor, HRegionInfo[ ] ) has been removed from thisinterface.


Abstract method voidpreDeleteSnapshot( ObserverContext<MasterCoprocessorEnvironment>,HBaseProtos.SnapshotDescription ) has been removed from thisinterface.


Abstract method voidpreGetTableDescriptors( ObserverContext<MasterCoprocessorEnvironment>,List<TableName>, List<HTableDescriptor> ) has been removedfrom this interface.


Abstract method voidpreModifyTable( ObserverContext<MasterCoprocessorEnvironment>,TableName, HTableDescriptor ) has been removed from thisinterface.


Abstract method voidpreRestoreSnapshot( ObserverContext<MasterCoprocessorEnvironment>,HBaseProtos.SnapshotDescription, HTableDescriptor ) hasbeen removed from this interface.


Abstract method voidpreSnapshot( ObserverContext<MasterCoprocessorEnvironment>,HBaseProtos.SnapshotDescription, HTableDescriptor ) hasbeen removed from this interface.


RegionObserver interface changes

The following changes are introduced to the RegionObserver interface.

[#] interface RegionObserver (13)

Change Result

19


Abstract method voidpostCloseRegionOperation( ObserverContext<RegionCoprocessorEnvironment>,HRegion.Operation ) has been removed from this interface.


Abstract method voidpostCompactSelection( ObserverContext<RegionCoprocessorEnvironment>, Store,ImmutableList<StoreFile> ) has been removed from thisinterface.


Abstract method voidpostCompactSelection( ObserverContext<RegionCoprocessorEnvironment>, Store,ImmutableList<StoreFile>, CompactionRequest ) has beenremoved from this interface.


Abstract method voidpostGetClosestRowBefore( ObserverContext<RegionCoprocessorEnvironment>, byte[ ],byte[ ], Result ) has been removed from this interface.


Abstract method DeleteTrackerpostInstantiateDeleteTracker( ObserverContext<RegionCoprocessorEnvironment>,DeleteTracker ) has been removed from this interface.


Abstract method voidpostSplit( ObserverContext<RegionCoprocessorEnvironment>, HRegion,HRegion ) has been removed from this interface.


Abstract method voidpostStartRegionOperation( ObserverContext<RegionCoprocessorEnvironment>,HRegion.Operation ) has been removed from this interface.


Abstract method StoreFile.ReaderpostStoreFileReaderOpen( ObserverContext<RegionCoprocessorEnvironment>,FileSystem, Path, FSDataInputStreamWrapper, long,CacheConfig, Reference, StoreFile.Reader ) has been removedfrom this interface.


Abstract method voidpostWALRestore( ObserverContext<RegionCoprocessorEnvironment>,HRegionInfo, HLogKey, WALEdit ) has been removed from thisinterface.


Abstract method InternalScannerpreFlushScannerOpen( ObserverContext<RegionCoprocessorEnvironment>, Store,KeyValueScanner, InternalScanner ) has been removed fromthis interface.


Abstract method voidpreGetClosestRowBefore( ObserverContext<RegionCoprocessorEnvironment>, byte[ ],byte[ ], Result ) has been removed from this interface.


Abstract method StoreFile.ReaderpreStoreFileReaderOpen( ObserverContext<RegionCoprocessorEnvironment>,FileSystem, Path, FSDataInputStreamWrapper, long,CacheConfig, Reference, StoreFile.Reader ) has been removedfrom this interface.


Abstract method voidpreWALRestore( ObserverContext<RegionCoprocessorEnvironment>,HRegionInfo, HLogKey, WALEdit ) has been removed from thisinterface.


WALObserver interface changes

The following changes are introduced to the WALObserver interface.

[#] interface WALObserver

Change Result

20


Abstract method voidpostWALWrite( ObserverContext<WALCoprocessorEnvironment>,HRegionInfo, HLogKey, WALEdit ) has been removed from thisinterface.


Abstract method booleanpreWALWrite( ObserverContext<WALCoprocessorEnvironment>,HRegionInfo, HLogKey, WALEdit ) has been removed from thisinterface.


Scheduler changes

Following methods are now changed to abstract.

[#]class RpcScheduler (1)

Change Result

Abstract method void dispatch ( CallRunner ) has been removedfrom this class.


hbase-server-1.0.0.jar, RpcScheduler.class package org.apache.hadoop.hbase.ipc

[#] RpcScheduler.dispatch ( CallRunner p1 ) [abstract] : void 1

org/apache/hadoop/hbase/ipc/RpcScheduler.dispatch:(Lorg/apache/hadoop/hbase/ipc/CallRunner;)V

Change Result

Return value type has been changed from void to boolean. This method has been removed because the return type is partof the method signature. A client program may be interrupted byNoSuchMethodError exception.

The following abstract methods have been removed.

[#]interface PriorityFunction (2)

Change Result

Abstract method longgetDeadline ( RPCProtos.RequestHeader,Message ) has been removed from this interface.


Abstract method int getPriority ( RPCProtos.RequestHeader,Message ) has been removed from this interface.


Server API changes

[#] class RpcServer (12)

Change Result

Type of field CurCall has been changed fromjava.lang.ThreadLocal<RpcServer.Call> tojava.lang.ThreadLocal<RpcCall>.


This class became abstract. A client program may be interrupted by InstantiationErrorexception.

Abstract method int getNumOpenConnections ( ) has beenadded to this class.

This class became abstract and a client program may beinterrupted by InstantiationError exception.

Field callQueueSize of typeorg.apache.hadoop.hbase.util.Counter has been removed fromthis class.


Field connectionList of typejava.util.List<RpcServer.Connection> has been removed fromthis class.


21


Field maxIdleTime of type int has been removed from this class. A client program may be interrupted by NoSuchFieldErrorexception.

Field numConnections of type int has been removed from thisclass.


Field port of type int has been removed from this class. A client program may be interrupted by NoSuchFieldErrorexception.

Field purgeTimeout of type long has been removed from thisclass.


Field responder of type RpcServer.Responder has beenremoved from this class.


Field socketSendBufferSize of type int has been removed fromthis class.


Field thresholdIdleConnections of type int has been removedfrom this class.


Following abstract method has been removed:

Change Result

Abstract method Pair<Message,CellScanner>call( BlockingService, Descriptors.MethodDescriptor, Message,CellScanner, long, MonitoredRPCHandler ) has been removedfrom this interface.


Replication and WAL changes

HBASE-18733: WALKey has been purged completely. Following are the changes to the WALKey:

[#] classWALKey (8)

Change Result

Access level of field clusterIds has been changed from protectedto private.

A client program may be interrupted by IllegalAccessErrorexception.

Access level of field compressionContext has been changedfrom protected to private.


Access level of field encodedRegionName has been changedfrom protected to private.


Access level of field tablename has been changed fromprotectedto private.


Access level of field writeTime has been changed fromprotectedto private.


Following fields have been removed:

Change Result

Field LOG of type org.apache.commons.logging.Log has beenremoved from this class.


Field VERSION of type WALKey.Version has been removedfrom this class.


Field logSeqNum of type long has been removed from this class. A client program may be interrupted by NoSuchFieldErrorexception.

Following are the changes to the WALEdit.class: hbase-server-1.0.0.jar, WALEdit.class packageorg.apache.hadoop.hbase.regionserver.wal

22


Deprecated APIs or coprocessor

HBASE-16769 - Protocol Buffers (PB) references from MasterObserver and RegionServerObserver hasbeen removed.

Admin Interface API changes

You cannot administer a CDP Runtime Data Hub cluster using a client that includes RelicationAdmin,ACC, Thrift and REST usage of Admin ops. Methods returning protobufs have been changed toreturn POJOs instead. Returns have changed from void to Future for async methods. HBASE-18106 -Admin.listProcedures and Admin.listLocks were renamed to getProcedures and getLocks. MapReducemakes use of Admin doing following admin.getClusterStatus() to calcluate Splits.

• Thrift usage of Admin API:

compact(ByteBuffer) createTable(ByteBuffer, List<ColumnDescriptor>) deleteTable(ByteBuffer) disableTable(ByteBuffer) enableTable(ByteBuffer) getTableNames() majorCompact(ByteBuffer)

• REST usage of Admin API:

hbase-rest org.apache.hadoop.hbase.rest RootResource getTableList() TableName[] tableNames = servlet.getAdmin().listTableNames(); SchemaResource delete(UriInfo) Admin admin = servlet.getAdmin(); update(TableSchemaModel, boolean, UriInfo) Admin admin = servlet.getAdmin(); StorageClusterStatusResource get(UriInfo) ClusterStatus status = servlet.getAdmin().getClusterStatus(); StorageClusterVersionResource get(UriInfo) model.setVersion(servlet.getAdmin().getClusterStatus().getHBaseVersion()); TableResource exists() return servlet.getAdmin().tableExists(TableName.valueOf(table));

[#] interface Admin (9)

Following are the changes to the Admin interface:

Change Result

Abstract method createTableAsync ( HTableDescriptor, byte[ ][ ] ) has been removed from this interface.


Abstract method disableTableAsync ( TableName ) has beenremoved from this interface.


Abstract method enableTableAsync ( TableName ) has beenremoved from this interface.


Abstract method getCompactionState ( TableName ) has beenremoved from this interface.


Abstract method getCompactionStateForRegion ( byte[ ] ) hasbeen removed from this interface.


Abstract method isSnapshotFinished( HBaseProtos.SnapshotDescription ) has been removed fromthis interface.


Abstract method snapshot ( String, TableName,HBaseProtos.SnapshotDescription.Type ) has been removedfrom this interface.


Abstract method snapshot ( HBaseProtos.SnapshotDescription )has been removed from this interface.


Abstract method takeSnapshotAsync( HBaseProtos.SnapshotDescription ) has been removed fromthis interface.


23


[#] Admin.createTableAsync ( HTableDescriptor p1, byte[ ][ ] p2 ) [abstract] : void 1

org/apache/hadoop/hbase/client/Admin.createTableAsync:(Lorg/apache/hadoop/hbase/HTableDescriptor;[[B)V

Change Result

Return value type has been changed from void tojava.util.concurrent.Future<java.lang.Void>.

This method has been removed because the return type is partof the method signature. A client program may be interrupted byNoSuchMethodError exception.

[#] Admin.disableTableAsync ( TableName p1 ) [abstract] : void 1

org/apache/hadoop/hbase/client/Admin.disableTableAsync:(Lorg/apache/hadoop/hbase/TableName;)V

Change Result



Admin.enableTableAsync ( TableName p1 ) [abstract] : void 1

org/apache/hadoop/hbase/client/Admin.enableTableAsync:(Lorg/apache/hadoop/hbase/TableName;)V

Change Result



Admin.enableTableAsync ( TableName p1 ) [abstract] : void 1

org/apache/hadoop/hbase/client/Admin.getCompactionState:(Lorg/apache/hadoop/hbase/TableName;)Lorg/apache/hadoop/hbase/protobuf/generated/AdminProtos$GetRegionInfoResponse$CompactionState;

Change Result

Return value type has been changed fromorg.apache.hadoop.hbase.protobuf.generated.AdminProtos.GetRegionInfoResponse.CompactionStateto CompactionState.


[#] Admin.getCompactionStateForRegion ( byte[ ] p1 ) [abstract] :AdminProtos.GetRegionInfoResponse.CompactionState 1

org/apache/hadoop/hbase/client/Admin.getCompactionStateForRegion:([B)Lorg/apache/hadoop/hbase/protobuf/generated/AdminProtos$GetRegionInfoResponse$CompactionState;

Change Result

Return value type has been changed fromorg.apache.hadoop.hbase.protobuf.generated.AdminProtos.GetRegionInfoResponse.CompactionStateto CompactionState.


HTableDescriptor and HColumnDescriptor changes

HTableDescriptor and HColumnDescriptor has become interfaces and you can create it through Builders.HCD has become CFD. It no longer implements writable interface. package org.apache.hadoop.hbase[#]class HColumnDescriptor (1)

Change Result

Removed super-interfaceorg.apache.hadoop.io.WritableComparable<HColumnDescriptor>.


24


HColumnDescriptor in 1.0.0 {code} @InterfaceAudience.Public @InterfaceStability.Evolving public classHColumnDescriptor implements WritableComparable<HColumnDescriptor> { {code}

HColumnDescriptor in 2.0 {code} @InterfaceAudience.Public @Deprecated // remove it in 3.0 public classHColumnDescriptor implements ColumnFamilyDescriptor, Comparable<HColumnDescriptor> { {code}

For META_TABLEDESC, the maker method had been deprecated already in HTD in 1.0.0. OWNER_KEYis still in HTD.

class HTableDescriptor (3)

hbase-client-1.0.0.jar, HTableDescriptor.class package org.apache.hadoop.hbase

Change Result

Removed super-interfaceorg.apache.hadoop.io.WritableComparable<HTableDescriptor>.


Field META_TABLEDESC of type HTableDescriptor has beenremoved from this class.


[#] HTableDescriptor.getColumnFamilies ( ) : HColumnDescriptor[ ] (1)

org/apache/hadoop/hbase/HTableDescriptor.getColumnFamilies:()[Lorg/apache/hadoop/hbase/HColumnDescriptor;[#] class HColumnDescriptor (1)

Change Result

Return value type has been changed fromHColumnDescriptor[]to client.ColumnFamilyDescriptor[].


[#] interface Table (4)

Change Result

Abstract method batch ( List<?> ) has been removed from thisinterface.


Abstract method batchCallback ( List<?>,Batch.Callback<R> )has been removed from this interface.


Abstract method getWriteBufferSize ( ) has been removed fromthis interface.


Abstract method setWriteBufferSize ( long ) has been removedfrom this interface.


Deprecated buffer methods

• HBASE-13298- Clarify if Table.{set|get}WriteBufferSize() is deprecated or not.• LockTimeoutException and OperationConflictException classes have been removed.

class OperationConflictException (1)

Change Result

This class has been removed. A client program may be interrupted byNoClassDefFoundErrorexception.

class class LockTimeoutException (1)

Change Result This class has been removed. A client program may be interrupted byNoClassDefFoundErrorexception.

Filter API changes

Following methods have been removed: package org.apache.hadoop.hbase.filter

25


[#] class Filter (2)

Change Result

Abstract method getNextKeyHint ( KeyValue ) has beenremoved from this class.


Abstract method transform ( KeyValue ) has been removed fromthis class.


• HBASE-12296 Filters should work with ByteBufferedCell.• HConnection is removed in HBase 2.0.• RegionLoad and ServerLoad internally moved to shaded PB.

[#] class RegionLoad (1)

Change Result

Type of field regionLoadPB has been changed fromprotobuf.generated.ClusterStatusProtos.RegionLoad toshaded.protobuf.generated.ClusterStatusProtos.RegionLoad.


HBASE-15783:AccessControlConstants#OP_ATTRIBUTE_ACL_STRATEGY_CELL_FIRST is not usedany more. package org.apache.hadoop.hbase.security.access

[#] interface AccessControlConstants (3)

Change Result

Field OP_ATTRIBUTE_ACL_STRATEGY of typejava.lang.Stringhas been removed from this interface.


Field OP_ATTRIBUTE_ACL_STRATEGY_CELL_FIRST of typebyte[] has been removed from this interface.


Field OP_ATTRIBUTE_ACL_STRATEGY_DEFAULT of typebyte[] has been removed from this interface.


ServerLoad returns long instead of int 1

hbase-client-1.0.0.jar, ServerLoad.class package org.apache.hadoop.hbase

[#] ServerLoad.getNumberOfRequests ( ) : int 1

org/apache/hadoop/hbase/ServerLoad.getNumberOfRequests:()I

Change Result

Return value type has been changed from int to long. This method has been removed because the return type is partof the method signature. A client program may be interrupted byNoSuchMethodError exception.

[#] ServerLoad.getNumberOfRequests ( ) : int 1

org/apache/hadoop/hbase/ServerLoad.getReadRequestsCount:()I

Change Result


b[#] ServerLoad.getTotalNumberOfRequests ( ) : int 1

org/apache/hadoop/hbase/ServerLoad.getTotalNumberOfRequests:()I

Change Result

26



[#]ServerLoad.getWriteRequestsCount ( ) : int 1

org/apache/hadoop/hbase/ServerLoad.getWriteRequestsCount:()I

Change Result


• HBASE-13636 Remove deprecation for HBASE-4072 (Reading of zoo.cfg)• HConstants are removed. HBASE-16040 Remove configuration "hbase.replication"

[#]class HConstants (6)

Change Result

FieldDEFAULT_HBASE_CONFIG_READ_ZOOKEEPER_CONFIG oftype boolean has been removed from this class.


Field HBASE_CONFIG_READ_ZOOKEEPER_CONFIG of typejava.lang.String has been removed from this class.


Field REPLICATION_ENABLE_DEFAULT of type boolean hasbeen removed from this class.


Field REPLICATION_ENABLE_KEY of type java.lang.String hasbeen removed from this class.


Field ZOOKEEPER_CONFIG_NAME of type java.lang.Stringhas been removed from this class.


Field ZOOKEEPER_USEMULTI of type java.lang.String hasbeen removed from this class.


HBASE-18732: [compat 1-2] HBASE-14047 removed Cell methods without deprecation cycle.

[#]interface Cell 5

Change Result

Abstract method getFamily ( ) has been removed from thisinterface.


Abstract method getMvccVersion ( ) has been removed from thisinterface.


Abstract method getQualifier ( ) has been removed from thisinterface.


Abstract method getRow ( ) has been removed from thisinterface.


Abstract method getValue ( ) has been removed from thisinterface.


HBASE-18795:Expose KeyValue.getBuffer() for tests alone. Allows KV#getBuffer in tests only that wasdeprecated previously.

Region scanner changes

[#]interface RegionScanner (1)

Change Result

27


Abstract method boolean nextRaw ( List<Cell>, int ) has beenremoved from this interface.


StoreFile changes

[#] class StoreFile (1)

Change Result

This class became interface. A client program may be interrupted byIncompatibleClassChangeError or InstantiationError exceptiondependent on the usage of this class.

Mapreduce changes

HFile*Format has been removed.

ClusterStatus changes

HBASE-15843: Replace RegionState.getRegionInTransition() Map with a Set hbase-client-1.0.0.jar,ClusterStatus.class package org.apache.hadoop.hbase

[#] ClusterStatus.getRegionsInTransition ( ) : Map<String,RegionState> 1

org/apache/hadoop/hbase/ClusterStatus.getRegionsInTransition:()Ljava/util/Map;

Change Result

Return value type has been changed fromjava.util.Map<java.lang.String,master.RegionState> tojava.util.List<master.RegionState>.


Other changes in ClusterStatus include removal of convert methods that were no longer necessary afterpurge of PB from API.

Purge of Protocol Buffers from API

Protocol Buffers (PB) has been deprecated in APIs.

[#] HBaseSnapshotException.getSnapshotDescription ( ) : HBaseProtos.SnapshotDescription 1

org/apache/hadoop/hbase/snapshot/HBaseSnapshotException.getSnapshotDescription:()Lorg/apache/hadoop/hbase/protobuf/generated/HBaseProtos$SnapshotDescription;

Change Result

Return value type has been changed from

org.apache.hadoop.hbase.protobuf.generated.HBaseProtos.SnapshotDescription to org.apache.hadoop.hbase.client.SnapshotDescription

.


HBASE-15609: Remove PB references from Result, DoubleColumnInterpreter and any such public facingclass for 2.0. hbase-client-1.0.0.jar, Result.class package org.apache.hadoop.hbase.client

[#] Result.getStats ( ) : ClientProtos.RegionLoadStats 1

org/apache/hadoop/hbase/client/Result.getStats:()Lorg/apache/hadoop/hbase/protobuf/generated/ClientProtos$RegionLoadStats;

28


Change Result

Return value type has been changed from

org.apache.hadoop.hbase.protobuf.generated.ClientProtos.RegionLoadStats to RegionLoadStats

.


PrettyPrinter changes

hbase-server-1.0.0.jar, HFilePrettyPrinter.class package org.apache.hadoop.hbase.io.hfile

Change Result

Return value type has been changed from void to int. This method has been removed because the return type is partof the method signature. A client program may be interrupted byNoSuchMethodError exception.

Known Issues in HDFSThis topic describes known issues and unsupported features for using HDFS in this release of ClouderaRuntime.

Unsupported Features The following HDFS features are currently notsupported in Cloudera Data Platform:

• ACLs for the NFS gateway (HADOOP-11004)• Aliyun Cloud Connector (HADOOP-12756)• Allow HDFS block replicas to be provided by an

external storage system (HDFS-9806)• Consistent standby Serving reads

(HDFS-12943)• Cost-Based RPC FairCallQueue (HDFS-14403)• HDFS Router Based Federation (HDFS-10467)• More than two NameNodes (HDFS-6440)• NameNode Federation (HDFS-1052)• NameNode Port-based Selective Encryption

(HDFS-13541)• Non-Volatile Storage Class Memory (SCM) in

HDFS Cache Directives (HDFS-13762)• OpenStack Swift (HADOOP-8545)• SFTP FileSystem (HADOOP-5732)• Storage policy satisfier (HDFS-10285)• Upgrade Domain (HDFS-7541)

Known Issues in HueThis topic describes known issues and workarounds for using Hue in this release of Cloudera Runtime.

Hue Importer is not supported in the DataEngineering template

When you create a Data Hub cluster using the DataEngineering template, the Importer application isnot supported in Hue:

29

https://issues.apache.org/jira/browse/HADOOP-11004


https://issues.apache.org/jira/browse/HDFS-9806













Known Issues in Apache ImpalaThis topic describes known issues and workarounds for using Impala in this release of Cloudera Runtime.

Queries stuck on failed HDFS calls and nottiming out

In Impala 3.2 and higher, if the following errorappears multiple times in a short duration whilerunning a query, it would mean that the connectionbetween the impalad and the HDFS NameNode isin a bad state.

"hdfsOpenFile() for <filename> at backend <hostname:port> failed to finish before the <hdfs_operation_timeout_sec> second timeout "

In Impala 3.1 and lower, the same issue wouldcause Impala to wait for a long time or hang withoutshowing the above error message.

Workaround: Restart the impalad.

Apache JIRA: HADOOP-15720

Impala should tolerate bad locale settings If the LC_* environment variables specify anunsupported locale, Impala does not start.

Workaround: Add LC_ALL="C" to the environmentsettings for both the Impala daemon and theStatestore daemon.

Apache JIRA: IMPALA-532

Configuration to prevent crashes caused bythread resource limits

Impala could encounter a serious error due toresource usage under very high concurrency. Theerror message is similar to:

F0629 08:20:02.956413 29088 llvm-codegen.cc:111] LLVM hit fatal error: Unable to allocate section memory!terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::thread_resource_error> >'

30


Workaround: To prevent such errors, configureeach host running an impalad daemon with thefollowing settings:

echo 2000000 > /proc/sys/kernel/threads-max echo 2000000 > /proc/sys/kernel/pid_max echo 8000000 > /proc/sys/vm/max_map_count

Add the following lines in /etc/security/limits.conf:

impala soft nproc 262144 impala hard nproc 262144


Avro Scanner fails to parse some schemas The default value in Avro schema must match typeof first union type, e.g. if the default value is null,then the first type in the UNION must be "null".

Workaround: Swap the order of the fields in theschema specification. For example, use ["null","string"] instead of ["string", "null"].Note that the files written with the problematicschema must be rewritten with the new schemabecause Avro files have embedded schemas.


Process mem limit does not account for theJVM's memory usage

Some memory allocated by the JVM used internallyby Impala is not counted against the memory limitfor the impalad daemon.

Workaround: To monitor overall memory usage, usethe top command, or add the memory figures in theImpala web UI /memz tab to JVM memory usageshown on the /metrics tab.


Ranger audit logs for applying column maskingpolicies missing

Impala is not producing these logs.

Workaround: None.


Impala BE cannot parse Avro schema thatcontains a trailing semi-colon

If an Avro table has a schema definition with atrailing semicolon, Impala encounters an error whenthe table is queried.

Workaround: Remove trailing semicolon from theAvro schema.


31


Incorrect results with basic predicate on CHARtyped column

When comparing a CHAR column value to a stringliteral, the literal value is not blank-padded and sothe comparison might fail when it should match.

Workaround: Use the RPAD() function to blank-pad literals compared with CHAR columns to theexpected length.


ImpalaODBC: Can not get the value inthe SQLGetData(m-x th column) after theSQLBindCol(m th column)

If the ODBC SQLGetData is called on a seriesof columns, the function calls must follow thesame order as the columns. For example, if datais fetched from column 2 then column 1, theSQLGetData call for column 1 returns NULL.

Workaround: Fetch columns in the same order theyare defined in the table.


Casting scenarios with invalid/inconsistentresults

Using a CAST() function to convert large literalvalues to smaller types, or to convert special valuessuch as NaN or Inf, produces values not consistentwith other database systems. This could lead tounexpected results from queries.


A failed CTAS does not drop the table if theinsert fails

If a CREATE TABLE AS SELECT operationsuccessfully creates the target table but an erroroccurs while querying the source table or copyingthe data, the new table is left behind rather thanbeing dropped.

Workaround: Drop the new table manually after afailed CREATE TABLE AS SELECT


% escaping does not work correctly whenoccurs at the end in a LIKE clause

If the final character in the RHS argument of a LIKEoperator is an escaped \% character, it does notmatch a % final character of the LHS argument.


Crash:impala::Coordinator::ValidateCollectionSlots

A query could encounter a serious error if includesmultiple nested levels of INNER JOIN clausesinvolving subqueries.


Incorrect result due to constant evaluation inquery with outer join

Workaround: An OUTER JOIN query could omitsome expected result rows due to a constant suchas FALSE in another join clause. For example:

explain SELECT 1 FROM alltypestiny a1 INNER JOIN alltypesagg a2 ON a1.smallint_col = a2.year AND false

32


RIGHT JOIN alltypes a3 ON a1.year = a1.bigint_col;+-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-+| Explain String |+-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-+| Estimated Per-Host Requirements: Memory=1.00KB VCores=1 || || 00:EMPTYSET |+-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-+


Breakpad minidumps can be very large whenthe thread count is high

The size of the breakpad minidump files growslinearly with the number of threads. By default, eachthread adds 8 KB to the minidump size. Minidumpfiles could consume significant disk space when thedaemons have a high number of threads.

Workaround: Add -\-minidump_size_limit_hint_kb=size to seta soft upper limit on the size of each minidumpfile. If the minidump file would exceed that limit,Impala reduces the amount of information for eachthread from 8 KB to 2 KB. (Full thread informationis captured for the first 20 threads, then 2 KB perthread after that.) The minidump file can still growlarger than the "hinted" size. For example, if youhave 10,000 threads, the minidump file can be morethan 20 MB.


Impala requires FQDN from hostname commandon Kerberized clusters

The method Impala uses to retrieve the host namewhile constructing the Kerberos principal is thegethostname() system call. This function mightnot always return the fully qualified domain name,depending on the network configuration. If thedaemons cannot determine the FQDN, Impala doesnot start on a Kerberized cluster.

Workaround: Test if a host is affectedby checking whether the output of thehostname command includes the FQDN.On hosts where hostname, only returnsthe short name, pass the command-line flag

33


##hostname=fully_qualified_domain_namein the startup options of all Impala-related daemons.


Metadata operations block read-only operationson unrelated tables

Metadata operations that change the state of atable, like COMPUTE STATS or ALTER RECOVERPARTITIONS, may delay metadata propagation ofunrelated unloaded tables triggered by statementslike DESCRIBE or SELECT queries.

Workaround:


Impala does not support Heimdal Kerberos Apache JIRA: IMPALA-7072

Known Issues in Apache KafkaThis topic describes known issues and unsupported features for using Kafka in this release of ClouderaRuntime.

Known Issues

Topics created with the kafka-topics tool areonly accessible by the user who created themwhen the deprecated --zookeeper option isused

By default all created topics are secured. However,when topic creation and deletion is done withthe kafka-topics tool using the --zookeeperoption, the tool talks directly to Zookeeper.Because security is the responsibility of ZooKeeperauthorization and authentication, Kafka cannotprevent users from making ZooKeeper changes. Asa result, if the --zookeeper option is used, onlythe user who created the topic will be able to carryout administrative actions on it. In this scenarioKafka will not have permissions to perform tasks ontopics created this way.

Workaround: Use kafka-topics with the --bootstrap-server option that does not requiredirect access to Zookeeper.

Certain Kafka command line tools require directaccess to Zookeeper

The following command line tools talk directly toZooKeeper and therefore are not secured via Kafka:

• kafka-configs

• kafka-reassign-partitions

Workaround:None.

The offsets.topic.replication.factorproperty must be less than or equal to thenumber of live brokers

The offsets.topic.replication.factorbroker configuration is now enforced upon autotopic creation. Internal auto topic creation will failwith a GROUP_COORDINATOR_NOT_AVAILABLEerror until the cluster size meets this replicationfactor requirement.

Workaround: None.

34


Requests fail when sending to a nonexistenttopic with auto.create.topics.enable set totrue

The first few produce requests failwhen sending to a nonexistent topic withauto.create.topics.enable set to true.

Workaround: Increase the number of retries in theproducer configuration setting retries.

Custom Kerberos principal names cannotbe used for kerberized ZooKeeper and Kafkainstances

When using ZooKeeper authentication and acustom Kerberos principal, Kerberos-enabledKafka does not start. You must disable ZooKeeperauthentication for Kafka or use the default Kerberosprincipals for ZooKeeper and Kafka.

Workaround: None.

Performance degradation when SSL Is enabled In some configuration scenarios, significantperformance degradation can occur when SSLis enabled. The impact varies depending on yourCPU, JVM version, Kafka configuration, andmessage size. Consumers are typically moreaffected than producers.

Workaround: Configure brokers and clients withssl.secure.random.implementation =SHA1PRNG. It often reduces this degradationdrastically, but its effect is CPU and JVMdependent.

Apache JIRA: KAFKA-2561

OPSAPS-43236: Kafka garbage collection logsare written to the process directory

By default Kafka garbage collection logs are writtento the agent process directory. Changing the defaultpath for these log files is currently unsupported.

Workaround: None.

CDPD-8546: Repeated ZooKeeper client logtrace in Kafka server logs

If the Enable Secure Connection to ZooKeeperproperty is set to true and the RANGERService property is configured, both the KafkaZookeeper client and Ranger Zookeeper clientwill be configured to connect to Zookeepervia secure channels. However, the RangerZookeeper client will try to establish a TLS/SSLconnection to an unsecure port (2181). Thisresults in the client repeatedly trying and failingto connect to Zookeeper, which in turn causesthe org.apache.zookeeper.Login: TGTrefresh thread started. log message aswell as other related log messages to repeatedlyappear in the Kafka logs.

Note: This issue affects Data Hub clustersonly. Data Lake clusters are not affected.

Workaround:

1. In Cloudera Manager, select the Kafka service.

35


2. Select Configuration and find the Kafka BrokerAdvanced Configuration Snippet (SafetyValve) for ranger-kafka-audit.xml property.

3. Add the following to the property:

name : xasecure.audit.destination.solr.zookeepersvalue: ZOOKEEPER_HOST:SECURE_PORT/solr

Replace ZOOKEEPER_HOST:SECURE_PORT withthe hostname and secure port of the ZooKeeperhost that Solr depends on.

4. Enter a Reason for change, and click SaveChanges to commit the changes.

5. Restart the service.

Result: The Ranger Zookeeper client connects tothe correct secure port.

RELENG-8748: Kafka command line toolalternatives unavailable

The following Kafka command line tool alternativesare currently unavailable:

• kafka-dump-log

• kafka-producer-perf-test

• kafka-streams-application-reset

Workaround: Run the scripts located in /opt/cloudera/parcels/CDH/lib/kafka/bininstead. For example:

/opt/cloudera/parcels/CDH/lib/kafka/bin/kafka-producer-perf-test.sh

Unsupported Features

The following Kafka features are not supported inCloudera Data Platform:

• Only Java based clients are supported. Clientsdeveloped with C, C++, Python, .NET and otherlanguages are currently not supported.

• Although Kafka Connect is available in the formof a Kafka service role, it is not supported atthis time. NiFi is a proven solution for batchand real time data loading that complementsKafka's message broker capability. For moreinformation, see Cloudera Flow Management.

• The Kafka default authorizer is not supported.This includes setting ACLs and all related APIs,broker functionality, and command-line tools.

Known Issues in Apache KnoxThis topic describes known issues and workarounds for using Knox in this release of Cloudera Runtime.

36

https://docs.cloudera.com/cfm/1.0.1/index.html


CDPD-3125: Logging out of Atlas does notmanage the external authentication

At this time, Atlas does not communicate a log-outevent with the external authentication management,Apache Knox. When you log out of Atlas, you canstill open the instance of Atlas from the same webbrowser without re-authentication.

Workaround: To prevent additional access to Atlas,close all browser windows and exit the browser.

Known Issues in Apache KuduThis topic describes known issues and workarounds for using Kudu in this release of Cloudera Runtime.

• Kudu supports only coarse grain authorization

• Kudu does not yet support integration with Ranger• Kudu does not yet support integration with Atlas

• Kudu HMS Sync is disabled and is not yet supported

Known Issues in Apache OozieThis topic describes known issues and unsupported features for using Oozie in this release of ClouderaRuntime.

Oozie jobs fail (gracefully) on secure YARNclusters when JobHistory server is down

If the JobHistory server is down on a YARN (MRv2)cluster, Oozie attempts to submit a job, by default,three times. If the job fails, Oozie automatically putsthe workflow in a SUSPEND state.

Workaround: When the JobHistory server is runningagain, use the resume command to inform Oozie tocontinue the workflow from the point at which it leftoff.

CDPD-5340: The resourceManager propertydefined in an Oozie workflow might not workproperly if the workflow is submitted throughKnox proxy.

An Oozie workflow defined to use theresourceManager property might not workas expected in situations when the workflow issubmitted through Knox proxy.

Workaround: Define the jobTracker property withthe same value as that of the resourceManagerproperty.

Unsupported Feature The following Oozie features are currently notsupported in Cloudera Data Platform:

• Non-support for Pig action (CDPD-1070)• Conditional coordinator input logic

Known Issues in Apache RangerThis topic describes known issues and workarounds for using Ranger in this release of Cloudera Runtime.

CDPD-3296: Audit files for Ranger plugincomponents do not appear immediately in S3after cluster creation

For Ranger plugin components (Atlas, Hive, HBase,etc.), audit data is updated when the applicableaudit file is rolled over. The default Ranger audit

37


rollover time is 24 hours, so audit data appears 24hours after cluster creation.

Workaround:

To see the audit logs in S3 before the defaultrollover time of 24 hours, use the following steps tooverride the default value in the Cloudera Managersafety valve for the applicable service.

1. On the Configuration tab in the applicableservice, select Advanced under CATEGORY.

2. Click the + icon for the <service_name>Advanced Configuration Snippet (SafetyValve) for ranger-<service_name>-audit.xmlproperty.

3. Enter the following property in the Name box:

xasecure.audit.destination.hdfs.file.rollover.sec.4. Enter the desired rollover interval (in seconds) in

the Value box. For example, if you specify 180,the audit log data is updated every 3 minutes.

5. Click Save Changes and restart the service.

Known Issues in Cloudera SearchThis topic describes known issues and unsupported features for using Cloudera Search in this release ofCloudera Runtime.

Known Issues

Processing UpdateRequest with delegationtoken throws NullPointerException

When using the Spark Crunch Indexer or anotherclient application which utilizes the SolrJ API tosend Solr Update requests with delegation tokenauthentication, the server side processing of therequest might fail with a NullPointerException.

Workaround: None.

Apache Issue: SOLR-13921

Solr service with no added collections causesthe upgrade process to fail

Upgrade fails while performing the bootstrapcollections step of the solr-upgrade.sh scriptwith the error message:

Failed to execute command Bootstrap Solr Collections on service Solr

if there are no collections present in Solr.

Workaround: If there are no collections added toit, remove the Solr service from your cluster beforeyou start the upgrade.

HBase Lily indexer might fail to write role logfiles

In certain scenarios the HBase Lily Indexer (Key-Value Store Indexer) fails to write its role log files.

Workaround: None.

38


Solr SQL, Graph, and Stream Handlers areDisabled if Collection Uses Document-LevelSecurity

The Solr SQL, Graph, and Stream handlers donot support document-level security, and aredisabled if document-level security is enabled onthe collection. If necessary, these handlers can bere-enabled by setting the following Java systemproperties, but document-level security is notenforced for these handlers:

• SQL: solr.sentry.enableSqlQuery=true• Graph:

solr.sentry.enableGraphQuery=true

• Stream:solr.sentry.enableStreams=true

Workaround: None.

Collection Creation No Longer SupportsAutomatically Selecting A Configuration If OnlyOne Exists

Before CDH 5.5.0, a collection could be createdwithout specifying a configuration. If no -c valuewas specified, then:

• If there was only one configuration, thatconfiguration was chosen.

• If the collection name matched a configurationname, that configuration was chosen.

Search now includes multiple built-in configurations.As a result, there is no longer a case in which onlyone configuration can be chosen by default.

Workaround: Explicitly specify the collectionconfiguration to use by passing -c <configName>to solrctl collection --create.

CrunchIndexerTool which includes Sparkindexer requires specific input file formatspecifications

If the --input-file-format option is specifiedwith CrunchIndexerTool, then its argument must betext, avro, or avroParquet, rather than a fullyqualified class name.

Workaround: None.

The quickstart.sh file does not validateZooKeeper and the NameNode on someoperating systems

The quickstart.sh file uses the timeoutfunction to determine if ZooKeeper and theNameNode are available. To ensure this check canbe complete as intended, the quickstart.shdetermines if the operating system on which thescript is running supports timeout. If the scriptdetects that the operating system does not supporttimeout, the script continues without checkingif the NameNode and ZooKeeper are available. Ifyour environment is configured properly or you areusing an operating system that supports timeout,this issue does not apply.

Workaround: This issue only occurs in someoperating systems. If timeout is not available, thequickstart continues and final validation is alwaysdone by the MapReduce jobs and Solr commandsthat are run by the quickstart.

39


Field value class guessing and Automaticschema field addition are not supportedwith the MapReduceIndexerTool nor with theHBaseMapReduceIndexerTool

The MapReduceIndexerTool and theHBaseMapReduceIndexerTool can be used witha Managed Schema created via NRT indexing ofdocuments or via the Solr Schema API. However,neither tool supports adding fields automatically tothe schema during ingest.

Workaround: Define the schema beforerunning the MapReduceIndexerTool orHBaseMapReduceIndexerTool. In non-schemalessmode, define in the schema using the schema.xmlfile. In schemaless mode, either define the schemausing the Solr Schema API or index sampledocuments using NRT indexing before invokingthe tools. In either case, Cloudera recommendsthat you verify that the schema is what you expect,using the List Fields API command.

The Browse and Spell Request Handlers are notenabled in schemaless mode

The Browse and Spell Request Handlers requirecertain fields to be present in the schema. Sincethose fields cannot be guaranteed to exist in aSchemaless setup, the Browse and Spell RequestHandlers are not enabled by default.

Workaround: If you require the Browse andSpell Request Handlers, add them to thesolrconfig.xml configuration file. Generatea non-schemaless configuration to see the usualsettings and modify the required fields to fit yourschema.

Enabling blockcache writing may result inunusable indexes

It is possible to create indexes withsolr.hdfs.blockcache.write.enabledset to true. Such indexes may appear corruptto readers, and reading these indexes mayirrecoverably corrupt indexes. Blockcache writing isdisabled by default.

Workaround: None.

Users with insufficient Solr permissions mayreceive a "Page Loading" message from theSolr Web Admin UI

Users who are not authorized to use the Solr AdminUI are not given a page explaining that access isdenied to them, instead receive a web page thatnever finishes loading.

Workaround: None.

Using MapReduceIndexerTool orHBaseMapReduceIndexerTool multiple timesmay produce duplicate entries in a collection

Repeatedly running the MapReduceIndexerToolon the same set of input files can result induplicate entries in the Solr collection. This occursbecause the tool can only insert documentsand cannot update or delete existing Solrdocuments. This issue does not apply to theHBaseMapReduceIndexerTool unless it is run withmore than zero reducers.

40


Workaround: To avoid this issue, useHBaseMapReduceIndexerTool with zero reducers.This must be done without Kerberos.

Deleting collections might fail if hosts areunavailable

It is possible to delete a collection when hosts thathost some of the collection are unavailable. Aftersuch a deletion, if the previously unavailable hostsare brought back online, the deleted collection maybe restored.

Workaround: Ensure all hosts are online beforedeleting collections.

Saving search results is not supported Cloudera Search does not support the ability tosave search results.

Workaround: None.

HDFS Federation is not supported Cloudera Search does not support HDFSFederation.

Workaround: None.

Collection state goes down after Solr SSL If you enable TLS/SSL on a Solr instance withexisting collections, the collections will break andbecome unavailable. Collections created afterenabling TLS/SSL are not affected by this issue.

Workaround: Recreate the collection after enablingTLS.


The following Search features are currently notsupported in Cloudera Data Platform:

• Solr SQL/JDBC• Graph Traversal• Cross Data Center Replication (CDCR)• SolrCloud Autoscaling• HDFS Federation• Saving search results• Solr contrib modules (Spark, MapReduce and

Lily HBase indexers are not contrib modulesbut part of the Cloudera Search product itself,therefore they are supported).

• Logging Slow Queries

Known Issues in Apache SparkThis topic describes known issues and workarounds for using Spark in this release of Cloudera Runtime.

CDPD-217: HBase/Spark connectors are notsupported

The Spark HBase Connector (SHC) from HDPand the hbase-spark module from CDH are notsupported.

41

https://lucene.apache.org/solr/guide/7_0/parallel-sql-interface.html

https://lucene.apache.org/solr/guide/7_0/graph-traversal.html

https://lucene.apache.org/solr/guide/7_0/cross-data-center-replication-cdcr.html

https://lucene.apache.org/solr/guide/7_0/solrcloud-autoscaling.html

https://lucene.apache.org/solr/guide/7_4/configuring-logging.html#logging-slow-queries


Workaround: Migrate to the Apache HBaseConnectors integration for Apache Spark (hbase-connectors/spark) available in CDP. Moredetails on the integration for working with HBasedata from Spark in CDP is available in the ClouderaCommunity article, HBase and Spark in CDP.

CDPD-3038: Launching pyspark displaysseveral HiveConf warning messages

When pyspark starts, several Hive configurationwarning messages are displayed, similar to thefollowing:

19/08/09 11:48:04 WARN conf.HiveConf: HiveConf of name hive.vectorized.use.checked.expressions does not exist19/08/09 11:48:04 WARN conf.HiveConf: HiveConf of name hive.tez.cartesian-product.enabled does not exist

Workaround: These errors can be safely ignored.

CDPD-2650: Spark cannot write ZSTD and LZ4compressed Parquet to dynamically partitionedtables

Workaround: Use a different compressionalgorithm.

CDPD-3293: Cannot create views (CREATEVIEW statement) from Spark

Apache Ranger in CDP disallows Spark users fromrunning CREATE VIEW statements.

Workaround: Create the view using Hive or Impala.

CDPD-3783: Cannot create databases fromSpark

Attempting to create a database using Spark resultsin an error similar to the following:

org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Permission denied: user [sparkuser] does not have [ALL] privilege on [hdfs://ip-10-1-2-3.cloudera.site:8020/tmp/spark/warehouse/spark_database.db]);

Workaround: Create the database using Hive orImpala, or specify the external data warehouselocation in the create command. For example:

sql("create database spark_database location '/warehouse/tablespace/external/hive/spark_database.db'")

Known Issues for Apache SqoopThis topic describes known issues and workarounds for using Parquet and Avro imports in this release ofCloudera Runtime.

42

https://community.cloudera.com/t5/Community-Articles/HBase-Spark-in-CDP/ta-p/294868


Avro, S3, and HCat do not work togetherproperly

Problem: Importing an Avro file into S3 with HCatfails with Delegation Token not available.

CDPD-3089

Parquet columns inadvertently renamed Problem: Column names that start with a numberare renamed when you use the --as-parquetfileoption to import data.

Workaround: Prepend column names in Parquettables with one or more letters or underscorecharacters.

Apache JIRA: None

Importing Parquet files might cause out-of-memory (OOM) errors

Problem: Importing multiple megabytes per rowbefore initial-page-run check (ColumnWriter) cancause OOM. Also, rows that vary significantly bysize so that the next-page-size check is based onsmall rows, and is set very high, followed by manylarge rows can also cause OOM.

PARQUET-99

Known Issues in MapReduce and YARNThis topic describes known issues, unsupported features and limitations for using MapReduce and YARNin this release of Cloudera Runtime.

Known Issues

JobHistory URL mismatch after serverrelocation

After moving the JobHistory Server to a new host,the URLs listed for the JobHistory Server on theResourceManager web UI still point to the oldJobHistory Server. This affects existing jobs only.New jobs started after the move are not affected.

Workaround: For any existing jobs that havethe incorrect JobHistory Server URL, there is nooption other than to allow the jobs to roll off thehistory over time. For new jobs, make sure that allclients have the updated mapred-site.xml thatreferences the correct JobHistory Server.

CDH-49165: History link in ResourceManagerweb UI broken for killed Spark applications

When a Spark application is killed, the history link inthe ResourceManager web UI does not work.

Workaround: To view the history for a killed Sparkapplication, see the Spark HistoryServer web UIinstead.

CDH-6808: Routable IP address required byResourceManager

ResourceManager requiresroutable host:port addresses foryarn.resourcemanager.scheduler.address,and does not support using the wildcard 0.0.0.0address.

Workaround: Set the address, in the formhost:port, either in the client-side configuration,or on the command line when you submit the job.

43


OPSAPS-52066: Stacks under Logs Directoryfor Hadoop daemons are not accessible fromKnox Gateway.

Stacks under the Logs directory for Hadoopdaemons, such as NameNode, DataNode,ResourceManager, NodeManager, andJobHistoryServer are not accessible from KnoxGateway.

Workaround: Administrators can SSH directly to theHadoop Daemon machine to collect stacks underthe Logs directory.

CDPD-2936: Application logs are not accessiblein WebUI2 or Cloudera Manager

Running Containers Logs from NodeManager localdirectory cannot be accessed either in ClouderaManager or in WebUI2 due to log aggregation.

Workaround: Use the YARN log CLI to accessapplication logs. For example:

yarn logs -applicationId <Application ID>

Apache Issue: YARN-9725

OPSAPS-50291: Environment variablesHADOOP_HOME, PATH, LANG, and TZ are notgetting whitelisted

It is possible to whitelist the environment variablesHADOOP_HOME, PATH, LANG, and TZ, but thecontainer launch environments do not have thesevariables set up automatically.

Workaround: You can manually add the requiredenvironment variables to the whitelist usingCloudera Manager.

1. In Cloudera Manager, select the YARN service.2. Click the Configuration tab.3. Search for Containers Environment

Variable Whitelist.4. Add the environment variables (HADOOP_HOME,

PATH, LANG, TZ) which are required to the list.5. Click Save Changes.6. Restart all NodeManagers.7. Check the YARN aggregated logs to ensure that

newly whitelisted environment variables are setup for container launch.

Limitations

Capacity Scheduler • As Capacity Scheduler is the default scheduler,the Dynamic Resource Pool User Interface is notavailable by default.

• Capacity Scheduler can be configured onlythrough safety-valves in Cloudera Manager.


The following YARN features are currently notsupported in Cloudera Data Platform:

• GPU support for Docker• Hadoop Pipes

44

https://issues.apache.org/jira/browse/YARN-9725


• Fair Scheduler

Known Issues in Apache ZeppelinThis topic describes known issues and workarounds for using Zeppelin in this release of ClouderaRuntime.

CDPD-3090: Due to a configuration typo,functionality involving notebook repositoriesdoes not work

Due to a missing closing brace, access to thenotebook repositories API is blocked by default.

Workaround: From the CDP Management Console,go to Cloudera Manager for the cluster runningZeppelin. On the Zeppelin configuration page(Zeppelin service > Configuration), enter shirourls in the Search field, and then add the missingclosing brace to the notebook-repositoriesURL, as follows:

/api/notebook-repositories/** = authc, roles[{{zeppelin_admin_group}}]

Click Save Changes, and restart the Zeppelinservice.

CDPD-2406: Logout button does not work Clicking the Logout button in the Zeppelin UI logsyou out, but then immediately logs you back inusing SSO.

Workaround: Close the browser.

Known Issues in Apache ZooKeeperThis topic describes known issues and workarounds for using Zeppelin in this release of ClouderaRuntime.

Zookeeper-client does not use ZooKeeper TLS/SSL automatically

The command-line tool ‘zookeeper-client’ isinstalled to all Cloudera Nodes and it can be usedto start the default Java command line ZooKeeperclient. However even when ZooKeeper TLS/SSL isenabled, the zookeeper-client command connectsto localhost:2181, without using TLS/SSL.

Workaround:

Manually configure the 2182 port, when zookeeper-client connects to a ZooKeeper cluster.Thefollowing is an example of connecting to a specificthree-node ZooKeeper cluster using TLS/SSL:

CLIENT_JVMFLAGS="-Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty -Dzookeeper.ssl.keyStore.location=<path to your configured keystore> -Dzookeeper.ssl.keyStore.password=<the password you configured for the keystore> -

45


Dzookeeper.ssl.trustStore.location=<path to your configured truststore> -Dzookeeper.ssl.trustStore.password=<the password you configured for the truststore> -Dzookeeper.client.secure=true" zookeeper-client -server <your.zookeeper.server-1>:2182,<your.zookeeper.server-2>:2182,<your.zookeeper.server-3>:2182

46

Documents

Release Notes - ClouderaSee Charting Time-Series Data What's new in DAS There are no new features in DAS in this release of Cloudera Runtime. What's New in Apache HBase There are no