Integration with HDFSUsing NFS and WebHDFS

Christophe Marchal | Software Architect

Access your files in HDFS

Network File System WebHDFS

NFS support Motivations

1. File browsing and downloading

2. File uploading

3. Data streaming

Network File System Stack

NFS Gateway

NFS Gateway

Hands on

mount -t nfs -o vers=3,proto=tcp,nolock HOSTIP:/ /PATH/TO/MOUNTPOINT

WebHDFS

API

HTTP

webhdfs://<HOST>:<HTTP_PORT>/<PATH>

WebHDFS hdfs://<HOST>:<RPC_PORT>/<PATH>

http://<HOST>:<HTTP_PORT>/webhdfs/v1/<PATH>?op=

API

HTTP GET

OPENGETFILESTATUSLISTSTATUSGETCONTENTSUMMARYGETFILECHECKSUMGETHOMEDIRECTORYGETDELEGATIONTOKEN

HTTP PUT

CREATEMKDIRSRENAMESETREPLICATIONSETOWNERSETPERMISSIONSETTIMESRENEWDELEGATIONTOKENCANCELDELEGATIONTOKEN

HTTP POST APPEND

HTTP DELETE DELETE

File status

Read File

Append

Append to a File

Append

Example of use

Christophe Marchal | Software Architect @toff63

Thanks