Skip to content

Базовый интерфейс

Bases: ABC

Base file limit interface.

Limits used by several onETL components, including [file-downloader][] and [file-mover][], to determine if internal loop should be stopped.

Unlike file filters, limits have internal state which can be updated or reset.

Added in 0.8.0

Source code in onetl/base/base_file_limit.py 
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
class BaseFileLimit(ABC):
    """
    Base file limit interface.

    Limits used by several onETL components, including [file-downloader][] and [file-mover][],
    to determine if internal loop should be stopped.

    Unlike file filters, limits have internal state which can be updated or reset.

    !!! success "Added in 0.8.0"
    """

    @abstractmethod
    def reset(self) -> Self:
        """
        Resets the internal limit state.

        !!! success "Added in 0.8.0"

        Returns
        -------
        Self
            Returns a filter of the same type, but with non-reached state.

            It could be the same filter or a new one, this is an implementation detail.

        Examples
        --------

        ```python
        >>> limit.is_reached
        True
        >>> new_limit = limit.reset()
        >>> new_limit.is_reached
        False
        ```
        """

    @abstractmethod
    def stops_at(self, path: PathProtocol) -> bool:
        """
        Update internal state and return current state.

        !!! success "Added in 0.8.0"

        Parameters
        ----------
        path : [onetl.base.path_protocol.PathProtocol][]
            Path to check

        Returns
        -------
        bool
            `True` if limit is reached, `False` otherwise.

        Examples
        --------

        ```python
        >>> from onetl.impl import LocalPath
        >>> # limit is not reached yet
        >>> limit.stops_at(LocalPath("/path/to/file.csv"))
        False
        >>> # after limit is reached
        >>> limit.stops_at(LocalPath("/path/to/another.csv"))
        True
        >>> # at this point, .stops_at() and .is_reached will always return True,
        >>> # even on inputs that returned False before.
        >>> # it will be in the same state until .reset() is called
        >>> limit.stops_at(LocalPath("/path/to/file.csv"))
        True
        ```
        """

    @property
    @abstractmethod
    def is_reached(self) -> bool:
        """
        Check if limit is reached.

        !!! success "Added in 0.8.0"

        Returns
        -------
        bool
            `True` if limit is reached, `False` otherwise.

        Examples
        --------

        ```python
        >>> from onetl.impl import LocalPath
        >>> limit.is_reached
        False
        >>> limit.stops_at(LocalPath("/path/to/file.csv"))
        False
        >>> limit.is_reached
        False
        >>> # after limit is reached
        >>> limit.stops_at(LocalPath("/path/to/file.csv"))
        True
        >>> limit.is_reached
        True
        ```
        """

is_reached abstractmethod property

Check if limit is reached.

Added in 0.8.0

Returns

bool True if limit is reached, False otherwise.

Examples

>>> from onetl.impl import LocalPath
>>> limit.is_reached
False
>>> limit.stops_at(LocalPath("/path/to/file.csv"))
False
>>> limit.is_reached
False
>>> # after limit is reached
>>> limit.stops_at(LocalPath("/path/to/file.csv"))
True
>>> limit.is_reached
True

reset() abstractmethod

Resets the internal limit state.

Added in 0.8.0

Returns

Self Returns a filter of the same type, but with non-reached state.

It could be the same filter or a new one, this is an implementation detail.

Examples

>>> limit.is_reached
True
>>> new_limit = limit.reset()
>>> new_limit.is_reached
False
Source code in onetl/base/base_file_limit.py 
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
@abstractmethod
def reset(self) -> Self:
    """
    Resets the internal limit state.

    !!! success "Added in 0.8.0"

    Returns
    -------
    Self
        Returns a filter of the same type, but with non-reached state.

        It could be the same filter or a new one, this is an implementation detail.

    Examples
    --------

    ```python
    >>> limit.is_reached
    True
    >>> new_limit = limit.reset()
    >>> new_limit.is_reached
    False
    ```
    """

stops_at(path) abstractmethod

Update internal state and return current state.

Added in 0.8.0

Parameters

path : [onetl.base.path_protocol.PathProtocol][] Path to check

Returns

bool True if limit is reached, False otherwise.

Examples

>>> from onetl.impl import LocalPath
>>> # limit is not reached yet
>>> limit.stops_at(LocalPath("/path/to/file.csv"))
False
>>> # after limit is reached
>>> limit.stops_at(LocalPath("/path/to/another.csv"))
True
>>> # at this point, .stops_at() and .is_reached will always return True,
>>> # even on inputs that returned False before.
>>> # it will be in the same state until .reset() is called
>>> limit.stops_at(LocalPath("/path/to/file.csv"))
True
Source code in onetl/base/base_file_limit.py 
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
@abstractmethod
def stops_at(self, path: PathProtocol) -> bool:
    """
    Update internal state and return current state.

    !!! success "Added in 0.8.0"

    Parameters
    ----------
    path : [onetl.base.path_protocol.PathProtocol][]
        Path to check

    Returns
    -------
    bool
        `True` if limit is reached, `False` otherwise.

    Examples
    --------

    ```python
    >>> from onetl.impl import LocalPath
    >>> # limit is not reached yet
    >>> limit.stops_at(LocalPath("/path/to/file.csv"))
    False
    >>> # after limit is reached
    >>> limit.stops_at(LocalPath("/path/to/another.csv"))
    True
    >>> # at this point, .stops_at() and .is_reached will always return True,
    >>> # even on inputs that returned False before.
    >>> # it will be in the same state until .reset() is called
    >>> limit.stops_at(LocalPath("/path/to/file.csv"))
    True
    ```
    """