Hi, to further investigate some of the performance bottlenecks,
it would be interesting to see if using multiple Spark workers on
each node improves performance.
I wonder if using ~20GB of Java heap memory for each
Spark worker is too much.
-Brandon.
Hi, to further investigate some of the performance bottlenecks,
it would be interesting to see if using multiple Spark workers on
each node improves performance.
I wonder if using ~20GB of Java heap memory for each
Spark worker is too much.
-Brandon.