Ben Chuanlong Du's Blog

It is never too late to learn.

Data Sources

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Data Platforms

https://www.oxen.ai/

https://github.com/quiltdata/quilt

https://registry.opendata.aws/

https://www.google.com/publicdata/directory

https://proxycrawl.com

Data for Computer Vison

Data for NLP …

Spark Issue: Too Many Containers Asked

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Error Message

org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException: Too many containers asked, 16731530.

image

Possible Causes

"Too many containers asked" is a protection mechanism of the Resource Manager. It might be triggered …