To address the errors – like the one below

6 WARN scheduler.TaskSetManager: Lost task 0.3 in stage 2.0 (TID 16, n06.domain.com): org.apache.spark.SparkException: Kryo serialization failed: Buffer overflow. Available: 0, required: 3. To avoid this, increase spark.kryoserializer.buffer.max value.

Set In CDH under SPARK, look for spark-defaults.conf, add the below.

One of the two values below shuld work (not sure which one)
spark.kryoserializer.buffer.max=64m
spark.kryoserializer.buffer.mb=64m

Deploy the config once done the modify

Note: The file below gets modified
/etc/spark/conf/spark-defaults.conf