Skip to content

File Downloader Result

Bases: FileResult

Representation of file download result.

Container for file paths, divided into certain categories:

  • [successful][]
  • [failed][]
  • [skipped][]
  • [missing][]

Added in 0.3.0

Examples

>>> from onetl.file import FileDownloader
>>> downloader = FileDownloader(local_path="/local", ...)
>>> download_result = downloader.run(
...     [
...         "/remote/file1",
...         "/remote/file2",
...         "/failed/file",
...         "/existing/file",
...         "/missing/file",
...     ]
... )
>>> download_result
DownloadResult(
    successful=FileSet([
        LocalPath("/local/file1"),
        LocalPath("/local/file2"),
    ]),
    failed=FileSet([
        FailedLocalFile("/failed/file")
    ]),
    skipped=FileSet([
        RemoteFile("/existing/file")
    ]),
    missing=FileSet([
        RemotePath("/missing/file")
    ]),
)
Source code in onetl/file/file_downloader/result.py 
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
class DownloadResult(FileResult):
    """
    Representation of file download result.

    Container for file paths, divided into certain categories:

    * [successful][]
    * [failed][]
    * [skipped][]
    * [missing][]

    !!! success "Added in 0.3.0"

    Examples
    --------

    ```python
    >>> from onetl.file import FileDownloader
    >>> downloader = FileDownloader(local_path="/local", ...)
    >>> download_result = downloader.run(
    ...     [
    ...         "/remote/file1",
    ...         "/remote/file2",
    ...         "/failed/file",
    ...         "/existing/file",
    ...         "/missing/file",
    ...     ]
    ... )
    >>> download_result
    DownloadResult(
        successful=FileSet([
            LocalPath("/local/file1"),
            LocalPath("/local/file2"),
        ]),
        failed=FileSet([
            FailedLocalFile("/failed/file")
        ]),
        skipped=FileSet([
            RemoteFile("/existing/file")
        ]),
        missing=FileSet([
            RemotePath("/missing/file")
        ]),
    )
    ```
    """

    successful: FileSet[LocalPath] = Field(default_factory=lambda: FileSet({}))
    "File paths (local) which were downloaded successfully"

    failed: FileSet[FailedRemoteFile] = Field(default_factory=lambda: FileSet({}))
    "File paths (remote) which were not downloaded because of some failure"

    skipped: FileSet[RemoteFile] = Field(default_factory=lambda: FileSet({}))
    "File paths (remote) which were skipped because of some reason"

    missing: FileSet[RemotePath] = Field(default_factory=lambda: FileSet({}))
    "File paths (remote) which are not present in the remote file system"

successful = Field(default_factory=(lambda: FileSet({}))) class-attribute instance-attribute

File paths (local) which were downloaded successfully

failed = Field(default_factory=(lambda: FileSet({}))) class-attribute instance-attribute

File paths (remote) which were not downloaded because of some failure

skipped = Field(default_factory=(lambda: FileSet({}))) class-attribute instance-attribute

File paths (remote) which were skipped because of some reason

missing = Field(default_factory=(lambda: FileSet({}))) class-attribute instance-attribute

File paths (remote) which are not present in the remote file system