Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
py4j.protocol.Py4JJavaError: An error occurred while calling o156.getQuotaUsage.
org.apache.hadoop.ipc.RemoteException(java.io.IOException): The quota system is disabled in Router.
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Caused by: java.lang.RuntimeException: Could not find any configured addresses for URI hdfs://clustername-router/
This is due to missing clustername-router settings in the property dfs.nameservices in …
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: hdfs::/cluster-name/user/dclong/feature_example/features/train/2022-03-11
As the error message points out, there's a syntax …
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
LightGBM support pandas columns of category type.
As a matter of fact,
this is the suggested way of handling categorical columns in LightGBM.
data[feature] = pd.Series(data[feature], dtype="category")
A LightGBM model (which is a Booster object) records categories of each categorical feature. This information is used to set categories of each categorical feature during prediction, which ensures that a LightGBM model can always handle categorical features correctly.
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Categorical variables are very common in a machine learning project. On a high level, there are two ways to handle a categorical variable.
Drop a categorical variable if a categorical variable …
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
It is strongly suggested that you load data into a pandas DataFrame
and handle categorical variables
by specifying a dtype of "category" for those categorical variables.
df.cat_var = df.cat_var.astype …