Upload
hortonworks
View
799
Download
2
Embed Size (px)
Citation preview
1 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Ambari 2.4.0What’s NewAugust 2016
2 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
What is Apache Ambari?
A completely open source management platform for provisioning, managing, monitoring and securing Apache Hadoop clusters. Apache Ambari takes the guesswork out of operating Hadoop.
3 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
What Ambari Does
Simplified Installation, Configuration and Management
Centralized Security Setup
Full Visibility into Cluster Health
Highly Extensible and Customizable
• Wizard-driven and automated cluster provisioning• Smart Configurations and Cluster Recommendations• Automated Rolling and Express cluster upgrades
• Reduce complexity to administer security across the platform• Automate setup Kerberos• Simplify the configuration of Apache Ranger
• Predefined alerts based on operational best practices• Advanced metrics visualization with Grafana
• Seamlessly fit into your enterprise environment• Bring custom Services under management via Ambari Stacks• Customize the UI with Ambari Views
4 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
What’s New in Ambari 2.4
Alerts: Customizable SCRIPT Parameters (AMBARI-14898)
Alerts: Retry Check Counts (AMBARI-15686)
Alerts: New HDFS Alerts (AMBARI-14800)
New Host Page Filtering (AMBARI-15210)
Remove Service (AMBARI-14759)
Support for SLES 12 Technical Preview (AMBARI-16007)
Stability: Database Consistency Checking (AMBARI-16258)
Customizable Ambari Log + PID Dirs (AMBARI-15300)
New Version Registration Experience (AMBARI-15724)
Log Search Technical Preview (AMBARI-14927)
Operational Audit Logging (AMBARI-15241)
Role-Based Access Control (AMBARI-13977)
Automated Setup of Ambari Kerberos (AMBARI-15561)
Automated Setup of Ambari Proxy User (AMBARI-15561)
Customizable Host Reg. SSH Port (AMBARI-13450)
Core Features Security Features
View URLs (AMBARI-15821), View Refresh (AMBARI-15682)
Inherit Cluster Permissions (AMBARI-16177)
Remote Cluster Registration (AMBARI-16274)
Views Framework Features
5 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
AgendaWhat’s New in Ambari 2.4.0
Feature Highlights: Alert Retry Check Counts
6 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Alert Check Counts
Customize the number of times an alert is checked before dispatching a notification Avoid dispatching an alert notification (email, snmp) in case of transient issues
7 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Configuring the Check Count
Set globally for all alerts, or override for a specific alert
Global Setting
Alert Override
8 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
State Change Types
SOFT state changes do not perform a dispatch HARD state changes (to non-OK) perform dispatch Regardless of change:
– The Ambari Web UI will show the current state (OK/WARN/CRIT)– The state change is written to ambari-alerts.log
2016-05-31 13:20:52,294 [CRITICAL] [SOFT] [AMBARI_METRICS] [grafana_webui] (Grafana Web UI) Connection failed to http://c6401.ambari.apache.org:3000 (<urlopen error [Errno 111] Connection refused>)2016-05-31 13:22:52,290 [CRITICAL] [HARD] [AMBARI_METRICS] [grafana_webui] (Grafana Web UI) Connection failed to http://c6401.ambari.apache.org:3000 (<urlopen error [Errno 111] Connection refused>)
Note: check counts are not configurable for AGGREGATE alert types. All state changes are considered HARD.
9 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Example: Check Count = 3
Check 1/3State: OK
Change: n/a
Check 1/3State: OK
Change: n/a
Check 1/3State: CRIT
Change: SOFT
Check 2/3State: CRIT
Change: n/a
Check 3/3State: CRIT
Change: HARD
Check 1/3State: OK
Change: HARD
DISPATCH
Check Interval Check Interval Check Interval Check Interval Check Interval
no state changestate changes to CRIT
performing multiple checks
back to OK
10 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
AgendaWhat’s New in Ambari 2.4.0
Feature Highlights: Alert Customizable Params
11 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Alert Types and Thresholds
Ability to customize Thresholds for SCRIPT and SERVER alerts Ability to customize Connection Timeout for METRIC alerts
Alert Type Description Thresholds (units)
WEB Connects to a Web URL. Alert status is based on the HTTP response code. Response Code (n/a)Connection Timeout (seconds)
PORT Connects to a port. Alert status is based on response time. Response (seconds)
METRIC Checks the value of a service metric. Units vary, based on the metric being checked.
Metric Value (units vary)Connection Timeout (seconds)
AGGREGATE Aggregates the status for another alert. % Affected (percentage)
SCRIPT Executes a script to handle the alert check. Varies
SERVER Executes a server-side runnable class to handle the alert check. Varies
NEW!
12 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Alerts: Customizable METRIC Connection Timeout
Ability to set Connection Timeout threshold via Ambari Web UI
NEW!
13 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Alerts: Customizable SCRIPT Thresholds
Ability to set various thresholds via Ambari Web UI
14 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Alerts: NEW!!! Ambari Server Performance Alert
Measures the Ambari Server REST API and Backend Database response
15 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
AgendaWhat’s New in Ambari 2.4.0
Feature Highlights: New HDFS Alerts
16 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
New HDFS Alerts Watch Trends
NameNode Client RPC Queue Latency (Hourly/Daily) NameNode Client RPC Processing Latency (Hourly/Daily) NameNode Service RPC Queue Latency (Hourly/Daily) NameNode Service RPC Processing Latency (Hourly/Daily) NameNode Heap Usage (Daily/Weekly) HDFS Storage Capacity Usage (Daily/Weekly)
17 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
AgendaWhat’s New in Ambari 2.4.0
Feature Highlights: New Host Filtering
18 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
New Host Filtering Control in Ambari Web
Ability to perform complex host filtering from Ambari Web Make it easier to find hosts
NEW!
19 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Search by Host Attribute, Service or Component
20 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Host Attribute Filtering
Host Name IP Host Status Cores RAM Stack Version + Version State Rack
21 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Service Filtering
22 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Component Filtering
23 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Host Filter: Examples
24 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
AgendaWhat’s New in Ambari 2.4.0
Feature Highlights: Remove Service
25 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Remove Service
Ability to perform Remove Service from Ambari Web Eliminates need to use Ambari REST API Checks for Service dependencies Service must be stopped All configuration information and history is also removed This operation is not reversible
NEW!
26 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
AgendaWhat’s New in Ambari 2.4.0
Feature Highlights: Other Items
27 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Customizable Ambari Log + PID Dirs (AMBARI-15300)
Ambari Server and Agents write log activity output to log files and use a PID-file that contains the process identification number (PID) for their running process.
Log Location PID Location
Ambari Server /var/log/ambari-server/ambari-server.log /var/run/ambari-server/ambari-server.pid
Ambari Agent /var/log/ambari-agent/ambari-agent.log /var/run/ambari-agent/ambari-agent.pid
28 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Customize Ambari Server Log + PID
vi /etc/ambari-server/conf/ambari.properties
pid.dir=/var/run/ambari-server
vi /etc/ambari-server/conf/log4j.properties
ambari.log.dir=${ambari.root.dir}/var/log/ambari-server
Ambari Server PID Ambari Server Log
1. Stop Ambari Server prior to modifying log or pid directories.2. You must manually create the new directories and be sure to set the directory
ownership + permissions to allow the Ambari Server process access.
29 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Customize Ambari Agent Log + PID
vi /etc/ambari-agent/conf/ambari-agent.ini
[agent]
logdir=/var/log/ambari-agent
piddir=/var/run/ambari-agent
1. Stop Ambari Agent prior to modifying log or pid directories.2. You must manually create the new directories and be sure to set the directory
ownership + permissions to allow the Ambari Agent process access.
30 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Customizable Host Registration SSH Port
Customize SSH Port when performing Host Registration automatically
NEW!
31 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Stability: Database Consistency Checking
On Ambari Server start, Ambari runs a database consistency check looking for issues. If any issues are found, Ambari Server start will abort and a message will be printed to
console “DB configs consistency check failed.” Check Ambari Server log file for more details:/var/log/ambari-server/ambari-server-check-database.log
Ability to “skip” check and force Ambari Server startambari-server start --skip-database-check
Important: if you “skip” the check to force Ambari Server start, do not make any changes to your cluster topology or perform a cluster upgrade until you correct
the database consistency issues.
32 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
AgendaWhat’s New in Ambari 2.4.0
Feature Highlights: View Framework Enhancements
33 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
View URLs (AMBARI-15821)
Ability to create a “short URL” or “vanity URL” for view instances Provide users with a non-version or instance specific URL to a view
/#/main/views/{viewName}/{viewVersion}/{viewInstanceName}/#/main/view/{viewName}/{shortURL}
34 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
View Refresh (AMBARI-15682)
Automatically deploy new views into Ambari Server w/o a restart
1. Copy view archive to: /var/lib/ambari-server/resources/views/
2. Ambari Server detects the new view, automatically extracts + deploys
3. View is available for creating instances
4. Click “Refresh” in Views UI
35 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Remote Cluster ConfigurationAMBARI-16274
36 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Background: View <-> Cluster Communication
Deployed Views “talk” with cluster using REST APIs (as applicable)
CLUSTER
ATS
RM
Ambari Server
Tez UIView
Tez UI View talks with cluster using
REST APIs toATS and ResourceManager
Ambari DB
LDAPAuthN
37 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Background: Operational vs. Standalone Ambari Server
Ambari Agent
Host
Ambari Agent
Host
Ambari Agent
Host
Standalone Ambari ServerOne or More Ambari Server Instances
No Agents, no requirement to operate the cluster
Operational AmbariOne Ambari Server Instance
Talking with Agents, Managing the cluster
Ambari Server
Ambari DB
LDAPAuthN
Ambari Server
Ambari DB
LDAPAuthN
38 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Background: Local Cluster vs. Non-Local
Ambari Server
Ambari DB
LDAPAuthN
Ambari Server
Ambari DB
LDAPAuthN
Standalone Ambari ServerOne or More Ambari Server Instances
No Agents, no requirement to operate the cluster
Operational AmbariOne Ambari Server Instance
Talking with Agents, Managing the cluster
LOCALCLUSTER
NON-LOCAL
CLUSTER
39 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Introducing Remote Cluster Configuration (AMBARI-16274)Option DescriptionLocal Cluster When you select this Local Cluster option, Ambari will automatically determine the
cluster configuration properties needed for the view instance.Criteria:• Ambari Server running the views is also managing the cluster
Remote Cluster When you select Remote Cluster option, Ambari will automatically determine the cluster configuration properties needed for the view instance.Criteria:• The cluster is not local to the Ambari Server running the views (i.e. Standalone)• Cluster is being managed by Ambari
Custom When you select Custom option, you must enter all configuration information, and are responsible for updating if the cluster configuration changes.Criteria:• The cluster running the view is not local to the Ambari Server• The cluster is not being managed by Ambari
NEW!
40 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Local vs Remote View Configuration
Ambari Server
Views
Cluster
Ambari Server
Views
ClusterAmbari Server
LOCAL CLUSTER
REMOTE CLUSTER
Operational Ambari
Manages cluster
Standalone Ambari
Manages cluster
Talks to cluster
Obtains view config
Obtains view config
Talks to cluster
Operational Ambari
41 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
View Configuration: Minimizing Need for Custom
Cluster Config Ambari Server Cluster Mgmt Ambari 2.2 or Earlier Ambari 2.4
No HA, No Kerberos Operational Ambari Local Local
HA or Kerberos Operational Ambari Custom Local
No HA, No Kerberos Standalone Ambari Custom Remote
HA or Kerberos Standalone Ambari Custom Remote
No HA, No Kerberos Standalone Non-Ambari Custom Custom
HA or Kerberos Standalone Non-Ambari Custom Custom
42 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Inherit Cluster PermissionsAMBARI-16177
43 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Inherit Cluster Permissions (AMBARI-16177)
Ability to automatically grant View “Use” permission based on Cluster role Note: Option is only available when using a Local Cluster Configuration
Explicitly grant users and groups Use
permission
Automatically grant users and groups Use permission based on
Cluster roles
44 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
AgendaWhat’s New in Ambari 2.4.0
Feature Highlights: Log Search
TECH PREVIEW
45 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Log Search
Solr
A M B A R ILog
Search
Search Cluster Component Logs from within Ambari
Goal: When issues arise, be able to quickly find issues across all components
⬢ Capabilities– Rapid Search of all cluster component logs– Search across time ranges, log levels, and for keywords
⬢ Core Technologies: – Apache Ambari– Apache Solr– Apache Ambari Log Search
Tech Preview
46 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Log Search Architecture
A M B A R I
L O G F E E D E R
L O G F E E D E R
L O GF E E D E R
L O GF E E D E R
L O G F E E D E R
L O G F E E D E R
W O R K E RN O D E
W O R K E RN O D E
W O R K E RN O D E
W O R K E RN O D E
W O R K E RN O D E
W O R K E RN O D E
Solr
L O G S E A R C H
U I
Tech Preview
47 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Log Search Details
W O R K E RN O D E
L O G F E E D E R
Solr
L O G S E A R C H
U I
Solr
Solr
A M B A R I
Java ProcessMulti-output SupportGrok
Solr CloudLocal Disk StorageTTL
Tech Preview
48 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Considerations
Log Feeders are CPU intensive, consider 1 dedicated core Solr instances should use dedicated hardware with at least 32GB of RAM dedicated to
the Solr instance By default, logs will age out after 7 days
Tech Preview
49 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
AgendaWhat’s New in Ambari 2.4.0
Feature Highlights: RBAC
50 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
New Role Based Access Control
Introducing new “roles” for more granular division of control for cluster operations
Old Permission New Role Notable Permissions
Operator Cluster Administrator Full operational control, including upgrades. Ambari Admins are implicitly granted this Role.
Cluster Operator Adding and removing hosts.
Service Administrator Manage configurations, move components.
Service Operator Service stop and start and service-specific operations such as HDFS Rebalance.
Read-Only Cluster User View cluster service and host information.
Note: Users flagged as “Ambari Administrators / Ambari Admins” are implicitly granted Cluster Administrator permission.
51 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Managing Cluster RolesAssign roles to users or groups
Manage roles in Block or List View
layouts
52 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Managing Cluster Roles View users or groups
Change current role assignment
53 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
AgendaWhat’s New in Ambari 2.4.0
Feature Highlights: Security Enhancements
54 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Summary of Security Enhancements
Automatic Setup of Ambari Server as a Proxyuser Automatic Setup of Ambari Server for Kerberos
55 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Automatic Setup of Ambari Server as a Proxyuser
56 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Background: Proxyusers
HDFS and WebHCat (as part of Hive) support the concept of a Proxyuser Proxyuser allows UserA to access the service on behalf of UserB (i.e. the proxyuser is
allowed to impersonate other users) Proxyuser is a commonly used capability of Hadoop
HDFS
“UserA” is setup as a proxyuser
UserA can access HDFS as “UserA” on behalf of “UserB”
HDFS ops performed are as “UserB”
57 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Background: HDFS Proxyuser Setup
A proxyuser needs to be configured in core-site.xml configuration:hadoop.proxyuser.{proxyuser-name}.hosts
hadoop.proxyuser.{proxyuser-name}.groups
If these settings are not present, impersonation will not be allowed and connection to the service via proxyuser will fail
58 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Background: Ambari + Proxyuser
Ambari Views use proxyuser to access the cluster (such as Hive View and Pig View) Ambari Server needs to access a service on behalf of an authenticated user
Ambari Server HDFS
(running as user “ambari”)
“joe” authenticates to Ambari
(setup for proxyuser “ambari”)hadoop.proxyuser.ambari.hosts=*hadoop.proxyuser.ambari.groups=*
Ambari Server can talk to HDFS as “ambari” proxyuser on behalf of “joe”
Configuration of proxyuser is commonly “missed” when setting up Ambari Views
59 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
New: Automatic Ambari Server Proxyuser Setup
Proxyuser configurations are automatically added for HDFS and WebHCat For example: Ambari Server as running as “ambari”, the following configurations are
added during HDFS service installhadoop.proxyuser.ambari.hosts
hadoop.proxyuser.ambari.groups
60 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Automatic Setup of Ambari Server for Kerberos
61 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Background: Hadoop + Kerberos
Strongly authenticating and establishing a user’s identity is the basis for secure access in Hadoop. Users need to be able to reliably “identify” themselves and then have that identity propagated throughout the Hadoop cluster.
Once this is done, those users can access resources (such as files or directories) or interact with the cluster (like running MapReduce jobs).
Besides users, Hadoop cluster resources themselves (such as Hosts and Services) need to authenticate with each other to avoid potential malicious systems or daemon’s “posing as” trusted components of the cluster to gain access to data.
62 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Background: Hadoop + Kerberos
Service Component
A
Service Component
B
Hadoop Cluster
KDC
keytabkeytab
Service Component
C
keytab
Service Component
D
keytab
Service Component
X
Service Component
X
keytabkeytab
Service Component
X
keytab
Service Component
X
keytab
Kerberos is used to secure the
Components in the cluster. Kerberos
identities are managed via
“keytabs” on the Component hosts.
Principals for the
cluster are managed in
the KDC.
63 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Background: Automated Kerberos Setup with Ambari
64 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Background: Principal and Keytab Generation & Distribution
1. User provides KDC Admin Account credentials to Ambari
2. Ambari connects to KDC, creates principals (Service and Ambari) needed for cluster
3. Ambari generates keytabs for the principals
4. Ambari distributes keytabs to Ambari Server and cluster hosts
5. Ambari discards the KDC Admin Account credentials (optional)
AmbariServer KDC
1 2
4
3
5
Cluster
65 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Background: Ambari + Hadoop + Kerberos
Ambari Server communicates with the cluster to retrieve information (such as metrics) Especially important for Ambari Views (e.g. Files, Hive, Pig) Therefore: Ambari Server ALSO needs to be “setup for Kerberos”
Ambari Server Cluster
Kerberos enabled
66 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Background: Manual Setup of Ambari Server for Kerberos
Manual setup of Ambari Server for Kerberos (outside of “Enable Kerberos” wizard):
1. Create principal for Ambari Server
2. Generate keytab for Ambari Server
3. Place keytab on Ambari Server host
4. Run “ambari-server setup-security” on Ambari Server
5. Restart Ambari Server
Configuration of Ambari Server for Kerberos is commonly “missed” when setting up Ambari Views
67 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
New: Automatic Setup of Ambari Server for Kerberos
When enabling Kerberos and choosing an automated option (MIT or AD), Ambari Server will be setup for Kerberos automatically:
1. A principal will be created for Ambari Server
2. A keytab will be generated and placed on Ambari Server
3. Ambari Server is setup for Kerberos
Note: you will still need to perform the Ambari Server restart for the Kerberos identity to get picked-up by Ambari.
68 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
What about Proxyuser + Kerberos?
69 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
New: Automatic Proxyuser Setup with Kerberos
When a cluster has Kerberos enabled, the proxyuser needs to be configured based on the primary part of the Kerberos principal name
hadoop.proxyuser.{principal-name-primary}.hosts
hadoop.proxyuser.{principal-name-primary}.groups
Ambari will adjust proxyuser configurations during Kerberos setup
Ambari Server HDFS
(running as user “ambari”)(setup with principal “[email protected]”
“joe” authenticates to Ambari
(setup for proxyuser “ambari-server”)hadoop.proxyuser.ambari-server.hosts=*hadoop.proxyuser.ambari-server.groups=*
Ambari Server can talk to HDFS as “ambari-server” proxyuser on behalf
of “joe”
70 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
AgendaWhat’s New in Ambari 2.4.0
Feature Highlights: Ops Audit Logging
71 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Operational Audit Logging
Ambari will create entries in an audit log as Ambari + Cluster operations are performed Using the audit log, you can determine who performed the operation and when the
operation was performed as well as other operation-specific information The Ambari Audit log can be found at: /var/log/ambari-server/ambari-audit.log
72 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
List of Operations
Stop/Start Service Stop all Services Add Service Move Component Turn On/Off Maintenance Mode Download Client Configurations Blueprint Export Update Configuration **
Login (success/failed) / Logout Create User, Group Delete User, Group Change Group Membership Change User Status, Admin Change User Password Grant/Revoke User, Group Cluster Roles
Service Operations User Operations
** Note: When a Service Configuration change is made, an entry is also written to a specific log file ambari-config-changes.log for configuration changes that provides even more detail on the change.
73 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
List of Operations (continued)
Add/Remove Host Enable/Disable/Edit Alert Add/Update/Delete Alert Group Add/Upgrade/Delete Notification Enable/Disable Kerberos Regenerate Kerberos Keytabs Rename Cluster Add/Remove Remote Clusters
Register/Deregister Version Cluster Upgrade
Cluster Operations Upgrade Operations
Create/Delete View Instance Edit View Instance Grant/Revoke View Permissions Create/Delete View URLs
View Operations
74 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Example: Change Group Membership
Add/Remove group members creates a “Membership change” audit entry
2016-06-02T23:12:09.930Z, User(admin), RemoteIp(192.168.64.1), Operation(Membership change), RequestType(PUT), url(http://c6401.ambari.apache.org:8080/api/v1/groups/customgroup/members), ResultStatus(200 OK), Group(customgroup), Members(joeuser)2016-06-02T23:12:34.700Z, User(admin), RemoteIp(192.168.64.1), Operation(Membership change), RequestType(PUT), url(http://c6401.ambari.apache.org:8080/api/v1/groups/customgroup/members), ResultStatus(200 OK), Group(customgroup), Members(joeuser, mike)
75 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Example: Stop ZooKeeper
A single operation (like “Stop ZooKeeper”) might generate multiple audit entries Relate entries via RequestId()
2016-06-02T23:14:35.206Z, User(admin), RemoteIp(192.168.64.1), Operation(INSTALLED: ZOOKEEPER_SERVER/ZOOKEEPER on c6401.ambari.apache.org (MyCluster)), Host name(c6401.ambari.apache.org), RequestId(7), Status(Successfully queued)2016-06-02T00:31:56.016Z, User(admin), Operation(Stop ZooKeeper Server), Status(IN_PROGRESS), RequestId(7)2016-06-02T00:31:56.025Z, User(admin), Operation(STOP ZOOKEEPER_SERVER), Status(QUEUED), RequestId(7), TaskId(52), Hostname(c6401.ambari.apache.org)2016-06-02T00:31:57.370Z, User(admin), Operation(Stop ZooKeeper Server), Status(COMPLETED), RequestId(7)2016-06-02T00:31:57.370Z, User(admin), Operation(STOP ZOOKEEPER_SERVER), Status(COMPLETED), RequestId(7), TaskId(52), Hostname(c6401.ambari.apache.org)
76 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
AgendaWhat’s New in Ambari 2.4.0
Feature Highlights: Version Registration Experience
77 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Introducing the Version Definition File (VDF)
This is a meta file describing which Services are included and at which version
78 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Ambari will “discover” Available Versions
Tabs for list of available Stacks
List of discovered Versions
List of Services w/version #
79 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
“Default Version Definition” for Backwards Compat
Ambari provides a “default” Version
Definition.
80 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Add New Version via File Upload or URL
81 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Changes in Install / Version Registration Flow
Scenario Ambari 2.4 ChangeInternet Access / Public Repositories No change.
No Internet Access / Local repositories - Upload a VDF for the Local Repository you created- Set the Local Repository URLs
OR
- Choose the Default Version Definition- Set the Local Repository URLs
82 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Other UX Changes: Local vs. Public Repository Radio
Explicit Choice
83 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Other UX Changes: Local vs. Public Repository Radio
Choose Local
Must enter Base URLs
84 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Other UX Changes: OS Add/Remove
85 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Other UX Changes: RedHat Satellite/Spacewalk
Explicit Choice
- Ambari will not write the .repo files
- User must register the repositories channels via Satellite
86 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Other UX Changes: Viewing, Install and Upgrade
87 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Other UX Changes: Managing Versions
88 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Thank You