Skip to content

File Downloader Result

Bases: FileResult

Representation of file download result.

Container for file paths, divided into certain categories:

  • :obj:~successful
  • :obj:~failed
  • :obj:~skipped
  • :obj:~missing

.. versionadded:: 0.3.0

Examples

from onetl.file import FileDownloader downloader = FileDownloader(local_path="/local", ...) download_result = downloader.run( ... [ ... "/remote/file1", ... "/remote/file2", ... "/failed/file", ... "/existing/file", ... "/missing/file", ... ] ... ) download_result DownloadResult( successful=FileSet([ LocalPath("/local/file1"), LocalPath("/local/file2"), ]), failed=FileSet([ FailedLocalFile("/failed/file") ]), skipped=FileSet([ RemoteFile("/existing/file") ]), missing=FileSet([ RemotePath("/missing/file") ]), )

Source code in onetl/file/file_downloader/result.py 
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
class DownloadResult(FileResult):
    """
    Representation of file download result.

    Container for file paths, divided into certain categories:

    * :obj:`~successful`
    * :obj:`~failed`
    * :obj:`~skipped`
    * :obj:`~missing`

    .. versionadded:: 0.3.0

    Examples
    --------

    >>> from onetl.file import FileDownloader
    >>> downloader = FileDownloader(local_path="/local", ...)
    >>> download_result = downloader.run(
    ...     [
    ...         "/remote/file1",
    ...         "/remote/file2",
    ...         "/failed/file",
    ...         "/existing/file",
    ...         "/missing/file",
    ...     ]
    ... )
    >>> download_result
    DownloadResult(
        successful=FileSet([
            LocalPath("/local/file1"),
            LocalPath("/local/file2"),
        ]),
        failed=FileSet([
            FailedLocalFile("/failed/file")
        ]),
        skipped=FileSet([
            RemoteFile("/existing/file")
        ]),
        missing=FileSet([
            RemotePath("/missing/file")
        ]),
    )
    """

    successful: FileSet[LocalPath] = Field(default_factory=FileSet)
    "File paths (local) which were downloaded successfully"

    failed: FileSet[FailedRemoteFile] = Field(default_factory=FileSet)
    "File paths (remote) which were not downloaded because of some failure"

    skipped: FileSet[RemoteFile] = Field(default_factory=FileSet)
    "File paths (remote) which were skipped because of some reason"

    missing: FileSet[RemotePath] = Field(default_factory=FileSet)
    "File paths (remote) which are not present in the remote file system"

successful = Field(default_factory=FileSet) class-attribute instance-attribute

File paths (local) which were downloaded successfully

failed = Field(default_factory=FileSet) class-attribute instance-attribute

File paths (remote) which were not downloaded because of some failure

skipped = Field(default_factory=FileSet) class-attribute instance-attribute

File paths (remote) which were skipped because of some reason

missing = Field(default_factory=FileSet) class-attribute instance-attribute

File paths (remote) which are not present in the remote file system