java转换JavaObject`scala。收集映射<对象,RDD<?>`到python字典
在pyspark中,调用getPersistentRDDs()
JavaSparkContext方法返回scala.collection.Map<Object,RDD<?>>
的JavaObject实例
from pyspark.sql import SparkSession
from pyspark import StorageLevel
spark = SparkSession.builder.master('yarn').getOrCreate()
sc = spark.sparkContext
df = spark.range(0, 25000000, 1)
df.persist(StorageLevel.MEMORY_ONLY)
df.limit(1).count()
sc._jsc.sc().getPersistentRDDs()
返回JavaObject id=o477
如何将scala.collection.Map<Object,RDD<?>>
的JavaObject转换为python字典
# 1 楼答案