Запись данных в MongoDB с использованием DBWriter

Для записи данных в MongoDB используйте DBWriter.

Warning

Пожалуйста, учитывайте типы данных MongoDB

Примеры

    from onetl.connection import MongoDB
    from onetl.db import DBWriter
    mongodb = MongoDB(...)
    df = ...  # здесь находятся данные
    writer = DBWriter(
        connection=mongodb,
        target="schema.table",
        options=MongoDB.WriteOptions(
            if_exists="append",
        ),
    )
    writer.run(df)  

Параметры записи

Метод выше принимает MongoDB.WriteOptions

onetl.connection.db_connection.mongodb.options.MongoDBWriteOptions

Bases: GenericOptions

Writing options for MongoDB connector.

Warning

Options uri, database, collection are populated from connection attributes, and cannot be overridden by the user in WriteOptions to avoid issues.

Added in 0.7.0

Examples

Note

You can pass any value supported by connector, even if it is not mentioned in this documentation. Option names should be in camelCase!

The set of supported options depends on connector version.

from onetl.connection import MongoDB

options = MongoDB.WriteOptions(
    if_exists="append",
    sampleSize=500,
    localThreshold=20,
)

Source code in onetl/connection/db_connection/mongodb/options.py

class MongoDBWriteOptions(GenericOptions):
    """Writing options for MongoDB connector.

    !!! warning

        Options `uri`, `database`, `collection` are populated from connection attributes,
        and cannot be overridden by the user in `WriteOptions` to avoid issues.

    !!! success "Added in 0.7.0"

    Examples
    --------

    !!! note

        You can pass any value
        [supported by connector](https://www.mongodb.com/docs/spark-connector/current/batch-mode/batch-write-config/),
        even if it is not mentioned in this documentation. **Option names should be in** `camelCase`!

        The set of supported options depends on connector version.

    ```python
    from onetl.connection import MongoDB

    options = MongoDB.WriteOptions(
        if_exists="append",
        sampleSize=500,
        localThreshold=20,
    )
    ```
    """

    if_exists: MongoDBCollectionExistBehavior = Field(  # type: ignore[literal-required]
        default=MongoDBCollectionExistBehavior.APPEND,
        alias=avoid_alias("mode"),
    )
    """Behavior of writing data into existing collection.

    Possible values:
        * `append` (default)
            Adds new objects into existing collection.

            ??? note "Behavior in details"

                * Collection does not exist
                    Collection is created using options provided by user
                    (`shardkey` and others).

                * Collection exists
                    Data is appended to a collection.

                    !!! warning

                        This mode does not check whether collection already contains
                        objects from dataframe, so duplicated objects can be created.

        * `replace_entire_collection`
            **Collection is deleted and then created**.

            ??? note "Behavior in details"

                * Collection does not exist
                    Collection is created using options provided by user
                    (`shardkey` and others).

                * Collection exists
                    Collection content is replaced with dataframe content.

        * `ignore`
            Ignores the write operation if the collection already exists.

            ??? note "Behavior in details"

                * Collection does not exist
                    Collection is created using options provided by user

                * Collection exists
                    The write operation is ignored, and no data is written to the collection.

        * `error`
            Raises an error if the collection already exists.

            ??? note "Behavior in details"

                * Collection does not exist
                    Collection is created using options provided by user

                * Collection exists
                    An error is raised, and no data is written to the collection.

    !!! info "Changed in 0.9.0"
        Renamed `mode` → `if_exists`
    """

    class Config:
        prohibited_options = PROHIBITED_OPTIONS
        known_options = KNOWN_WRITE_OPTIONS
        extra = "allow"

    @root_validator(pre=True)
    def _mode_is_deprecated(cls, values):
        if "mode" in values:
            warnings.warn(
                "Option `MongoDB.WriteOptions(mode=...)` is deprecated since v0.9.0 and will be removed in v1.0.0. "
                "Use `MongoDB.WriteOptions(if_exists=...)` instead",
                category=UserWarning,
                stacklevel=5,
            )
        return values

if_exists = Field(default=(MongoDBCollectionExistBehavior.APPEND), alias=(avoid_alias('mode'))) class-attribute instance-attribute

Behavior of writing data into existing collection.

Possible values

• append (default) Adds new objects into existing collection.

  Behavior in details

  • Collection does not exist Collection is created using options provided by user (shardkey and others).

  • Collection exists Data is appended to a collection.

    Warning

    This mode does not check whether collection already contains objects from dataframe, so duplicated objects can be created.

• replace_entire_collection Collection is deleted and then created.

  Behavior in details

  • Collection does not exist Collection is created using options provided by user (shardkey and others).

  • Collection exists Collection content is replaced with dataframe content.

• ignore Ignores the write operation if the collection already exists.

  Behavior in details

  • Collection does not exist Collection is created using options provided by user

  • Collection exists The write operation is ignored, and no data is written to the collection.

• error Raises an error if the collection already exists.

  Behavior in details

  • Collection does not exist Collection is created using options provided by user

  • Collection exists An error is raised, and no data is written to the collection.

Changed in 0.9.0

Renamed mode → if_exists