Ben Chuanlong Du's Blog

It is never too late to learn.

Spark Issue: AccessControlException: Permission Denied

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Symptom

org.apache.hadoop.security.AccessControlException: Permission denied ...

Cause

The user of the Spark application has no permission to the query a table or HDFS path.

Solution

Apply to access to …

Access Control in Spark SQL

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Grant Permission to Users

GRANT
    priv_type [, priv_type ] ...
    ON database_table_or_view_name
    TO principal_specification [, principal_specification] ...
    [WITH GRANT OPTION];

Examples:

GRANT SELECT ON table1 TO USER user1;
GRANT SELECT ON DATABASE db1 TO USER user1 …

Configure Jupyte Notebook Permissions

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

  1. umask in /etc/profile or /etc/bashrc doesn't work in docker, which is strange. Changing UMASK 022 in /etc/login.defs doesn't work either ...

Had issues, tried to set /etc/profile …