pyspark sql createGlobalTempView和createOrReplaceTempView

左手的ㄟ右手 2022-06-08 11:36 223阅读 0赞

createGlobalTempView ( *name* )

Creates a global temporary view with this DataFrame.

The lifetime of this temporary view is tied to this Spark application. throws TempTableAlreadyExistsException, if the view name already exists in the catalog.

>>> df.createGlobalTempView("people")
    >>> df2 = spark.sql("select * from global_temp.people")
    >>> sorted(df.collect()) == sorted(df2.collect())
    True
    >>> df.createGlobalTempView("people")  
    Traceback (most recent call last):
    ...
    AnalysisException: u"Temporary table 'people' already exists;"
    >>> spark.catalog.dropGlobalTempView("people")

*New in version 2.1.*

createOrReplaceGlobalTempView ( *name* )

Creates or replaces a global temporary view using the given name.

The lifetime of this temporary view is tied to this Spark application.

>>> df.createOrReplaceGlobalTempView("people")
    >>> df2 = df.filter(df.age > 3)
    >>> df2.createOrReplaceGlobalTempView("people")
    >>> df3 = spark.sql("select * from global_temp.people")
    >>> sorted(df3.collect()) == sorted(df2.collect())
    True
    >>> spark.catalog.dropGlobalTempView("people")

*New in version 2.2.*

createOrReplaceTempView ( *name* )

Creates or replaces a local temporary view with this DataFrame.

The lifetime of this temporary table is tied to the [SparkSession][] that was used to create this [DataFrame][].

>>> df.createOrReplaceTempView("people")
    >>> df2 = df.filter(df.age > 3)
    >>> df2.createOrReplaceTempView("people")
    >>> df3 = spark.sql("select * from people")
    >>> sorted(df3.collect()) == sorted(df2.collect())
    True
    >>> spark.catalog.dropTempView("people")

*New in version 2.0.*

createTempView ( *name* )

Creates a local temporary view with this DataFrame.

The lifetime of this temporary table is tied to the [SparkSession][] that was used to create this [DataFrame][]. throws TempTableAlreadyExistsException, if the view name already exists in the catalog.

>>> df.createTempView("people")
    >>> df2 = spark.sql("select * from people")
    >>> sorted(df.collect()) == sorted(df2.collect())
    True
    >>> df.createTempView("people")  
    Traceback (most recent call last):
    ...
    AnalysisException: u"Temporary table 'people' already exists;"
    >>> spark.catalog.dropTempView("people")

*New in version 2.0.*

[SparkSession]: http://spark.apache.org/docs/latest/api/python/pyspark.sql.html#pyspark.sql.SparkSession
[DataFrame]: http://spark.apache.org/docs/latest/api/python/pyspark.sql.html#pyspark.sql.DataFrame