Spooled Temporary Files are file-like objects that start out mapped to in-memory objects, but automatically roll over to a temporary file once they reach a certain (configurable) threshhold. Unfortunately the built-in SpooledTemporaryFile class in Python does not implement the exact API that some common classes like StringIO do. SpooledTemporaryFile also spools all of it's in-memory files as cStringIO instances. cStringIO instances cannot be deep-copied, and they don't work with the zip library either. This along with the incompatible api makes it useless for several use-cases.

To combat this but still gain the memory savings and usefulness of a true spooled file-like-object, two custom classes have been implemented which have a compatible API.

SpooledBytesIO is a spooled file-like-object that only accepts bytes. On Python 2.x this means the 'str' type; on Python 3.x this means the 'bytes' type. Bytes are written in and retrieved exactly as given, but it will raise TypeErrors if something other than bytes are written.

SpooledStringIO is a spooled file-like-object that only accepts unicode values. On Python 2.x this means the 'unicode' type and on Python 3.x this means the 'str' type. Values are accepted as unicode and then coerced into utf-8 encoded bytes for storage. On retrieval, the values are returned as unicode.

I've always thought it was weird these were absent from the standard library, even though they're so simple. I added two functions to timeutils.py to transform between seconds since the UNIX epoch and datetime objects. Currently, they ignore any timezone information, but if you think these functions are a good fit for the package, I might be able to work on adding that kind of functionality.

Hi!

As as discussed in #242, here is my suggestion to add an edge case to the usage of funcutils.wraps. The situation is that if the wrapped and wrapper function have the same arguments but differ as to which are keyword-only or positional-only, the function returned by wraps will fail as the wrong "invocation string" was inserted in the process. A solution to this is to create the internal FunctionBuilderinstance from the wrapper instead of the wrapped. As it didn't make sense to simply add an argument to wraps, I decided to replicate the structure of the built-in functools and add a update_wrapper function, of which wraps is only a partial call.

This solves a problem that originated from using partials. Code that now works to wrap those is as follow:

import functools
from boltons import funcutils

def func(a, b=1, c=1):  # Has the signature "a, b=1, c=1"
    return a, b, c

wrapper = partial(func, b=2)
functools.update_wrapper(wapper, func)  # Needed as FunctionBuilder will look for some attributes like __name__ that are not updated directly with `partial`.
# wrapper has the signature; "a, *, b=2, c=1"

new_func = funcutils.update_wrapper(wrapper, func, build_from=wrapper, injected='b')
# new_func now has the signature : "a, *, c=1"
new_func(1)  # Prints : 1, 2, 1, 

Also, another small improvement I made here is to add an option to completely hide the wrapped function in the new one returned. Most IDEs use inspect.signature to extract the signature functions that they display to the user. However, by default, signature follows all wrapped functions until the innermost, so the user doesn't see the updated signature. One could use the follow_wrapped=False option, but I believe offering the possibility here could be nice.

Closes #242

Given the following decorator:

def inject_loop(func):
      sig = inspect.signature(func)
      loop_param = sig.parameters['loop'].replace(default=None)
      sig = sig.replace(parameters=[loop_param])

      def add_loop(
          args: typing.Tuple[typing.Any, ...],
          kwargs: typing.Dict[str, typing.Any]
      ) -> collections.OrderedDict:
          bargs = sig.bind(*args, **kwargs)
          bargs.apply_defaults()
          if bargs.arguments['loop'] is None:
              bargs.arguments['loop'] = asyncio.get_event_loop()

          return bargs.arguments

      if inspect.isasyncgenfunction(func):
          async def async_gen_loop_wrapper(*args, **kwargs):
              async for elem in func(**add_loop(args, kwargs)):
>>                yield elem
          ret = async_gen_loop_wrapper

      elif inspect.iscoroutinefunction(func):
          async def async_loop_wrapper(*args, **kwargs):
              return await func(**add_loop(args, kwargs))
          ret = async_loop_wrapper

      elif inspect.isgeneratorfunction(func):
          def gen_loop_wrapper(*args, **kwargs):
              yield from func(**add_loop(args, kwargs))
          ret = gen_loop_wrapper

      else:
          def func_loop_wrapper(*args, **kwargs):
              return func(**add_loop(args, kwargs))
          ret = func_loop_wrapper

      return boltons.funcutils.wraps(func, injected=['loop'])(ret)

The following code fails:

@inject_loop
def example(*, loop):  # loop is a keyword-only argument
    return loop

with the following stacktrace:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
.../python3.6/site-packages/boltons/funcutils.py in remove_arg(self, arg_name)
    565         try:
--> 566             self.args.remove(arg_name)
    567         except ValueError:

ValueError: list.remove(x): x not in list

During handling of the above exception, another exception occurred:

MissingArgument                           Traceback (most recent call last)
<ipython-input-3-6ef3f4f5dc19> in <module>()
----> 1 @lib.inject_loop
      2 def example(*, loop):
      3     return loop

... in inject_loop(func)
    169         ret = func_loop_wrapper
    170
--> 171     return boltons.funcutils.wraps(func, injected=['loop'])(ret)
    172
    173

.../python3.6/site-packages/boltons/funcutils.py in wraps(func, injected, **kw)
    295     for arg in injected:
    296         try:
--> 297             fb.remove_arg(arg)
    298         except MissingArgument:
    299             if inject_to_varkw and fb.varkw is not None:

.../python3.6/site-packages/boltons/funcutils.py in remove_arg(self, arg_name)
    569                                   % (arg_name, self.name, self.args))
    570             exc.arg_name = arg_name
--> 571             raise exc
    572         d_dict.pop(arg_name, None)
    573         self.defaults = tuple([d_dict[a] for a in self.args if a in d_dict])

MissingArgument: arg 'loop' not found in example argument list: []

However, it works if I do this:

@inject_loop
def example(loop):  # Not a keyword-only argument
    return loop

I believe the solution should also check the kwonly args when attempting to mark a parameter as "injected", so that keyword only arguments can be listed in the injected parameter to wraps().

This is another tool I have built a number of times in various permutations; I don't know if it's all that interesting but I've found it quite useful in a number of situations including immutable constant maps without heinous amounts of namedtuple boilerplate, and some cases with threading.

A FrozenDict is a dictionary whose values are set during instance creation and are fixed from that point on. Setting and altering values is not allowed. This can be useful in a number of scenarios, including setting up mapping constants without worrying that the values will get mutated by a misbehaving function.

There are a lot of online recipes for creating a FrozenDict class, but most still rely on a real dictionary under the hood for storage. They also tend to be extra weighty because they have an underlying object dict.

One common solution is to use a named tuple, but this requires setting up boilerplate for every type or relying on factory functions. The FrozenDict utilizes a named tuple for storage and then is further made lighter by utilizing slots to eliminate the underlying object dict.

What is the motivation of not updating the current index of items when they are removed? It's this a bug? Because i starting using IndexedSet to pop items from a list that i have and i start get index of range errors.

https://github.com/mahmoud/boltons/blob/bf8a65942cd8078dba7dc45543ae0923fe2fabbc/boltons/setutils.py#L201-L209

If I try this little code on Windows 7:

from boltons.fileutils import AtomicSaver
with AtomicSaver('foo.txt') as f:
    f.write('whatever')

I get the following Exception:

Traceback (most recent call last):
  File "C:\Python34\lib\site-packages\boltons\fileutils.py", line 272, in setup
    overwrite=self.overwrite_part)
  File "C:\Python34\lib\site-packages\boltons\fileutils.py", line 194, in _atomic_rename
    os.rename(path, new_path)
PermissionError: [WinError 32] Der Prozess kann nicht auf die Datei zugreifen, da sie von einem ande
ren Prozess verwendet wird: 'C:\\Windows\\System32\\tmphia0tzzm' -> 'foo.txt.part'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python34\lib\site-packages\boltons\fileutils.py", line 281, in __enter__
    self.setup()
  File "C:\Python34\lib\site-packages\boltons\fileutils.py", line 274, in setup
    os.unlink(tmp_part_path)
PermissionError: [WinError 32] Der Prozess kann nicht auf die Datei zugreifen, da sie von einem ande
ren Prozess verwendet wird: 'C:\\Windows\\System32\\tmphia0tzzm'

Add a more flexible interface to subprocess, in the style of subprocess.call() but better.

I find subprocess.call(), check_call() and check_output() really badly lacking, but I end up using them anyway due to the considerable extra complexity in using my own Popen object.

For example:

• check_* does not report an exit code or stderr in the exception, so you may know "something failed" but it's hard to tell what or why.
• *_call use the process's stdout, resulting in often unwanted extra output
• with any of these calls, it is impossible to interact with stdin

I have endeavoured to create a function that is very flexible but with excellent defaults, in the style of requests.get(), where the 90% use case is very simple, but the 99% use case is still possible. In getting into specifics (such as having None redirect output to /dev/null), I'm pretty sure I broke windows support. It's probably possible to add with minor changes in interface though.

I'd like strip / rstrip / lstrip that works with arbitrary iterables:

def iter_lstrip(li, it): 
    pos = 0
    while li[pos] == it:
        pos += 1
        if pos == len(li):
            return []
    return li[pos:]


def iter_rstrip(li, it):
    return list(reversed(iter_lstrip(list(reversed(li)), it)))


def iter_strip(li, it):
    return iter_lstrip(iter_rstrip(li, it), it)

iter_strip([0,0,0,1,2,3,4,0,0], 0)
# [1,2,3,4]

Adds failing tests to exercise issues with LRI

To fix the issue, I have a trivial solution that has LRI inherit from LRU. After a couple hours of thinking about it I don't think there's a faster solution that using the linked list that LRU uses without introducing an unbounded memory size of the underlying datastructure stored.

That being said LRU has a bunch of thread locking logic that I'm uncertain you would want to add to the LRI cache. So the current solution is more of a conversation piece. Open to feedback on how to go forward on this:

1. Extract common logic from LRU and have LRI and LRU utilize that? (lets us omit thread safety from LRI if we want)
2. Keep naive solution (though thread locking might introduce unwanted overhead).
3. Maybe make thread safety an optional feature on both LRU and LRI. Would be tricky not to break existing features. Probably would require names like: LRU, ThreadUnsafeLRU, LRI, ThreadSafeLRI.

the idea is complements of sets -- like inverse sets -- they support all of the set API with each other and with regular sets with the exception of iteration and len (b/c they contain "everything")

I really badly wanted one of these to be able to pass in a "universal set" as a null value rather than None -- would have saved a bunch of code like

for item in things:
    if filter is not None:
        if item not in filter:
            continue

instead, complement(set()) would simply return True for everything (or everything that is hashable)

the next day a co-worker wanted one of these for a different purpose, so I took a couple hours to bang out some code and the implementation actually seems pretty straight forward

still need to implement the operator overloads, more docs and better tests

opening the PR early to allow for feedback ASAP

A function that I frequently want is to print a list of strings, separating them with commas, plus a final conjunction like 'and' or 'or'. The Oxford comma should be the default but some users might like to disable it.

>>> human_list(['spam'])
'spam'
>>> human_list(['egg', 'spam'])
'egg and spam'
>>> human_list(['egg', 'bacon', 'spam'], conjunction='or')
'egg, bacon, or spam'
>>> human_list(['spam', 'spam', 'egg', 'bacon', 'spam'])
'spam, spam, egg, bacon, and spam'
>>> human_list(['lobster thermidor aux crevettes', 'garnished with truffle paté', 'with a fried egg on top', 'spam'], oxford=False)
'lobster thermidor aux crevettes, garnished with truffle paté, with a fried egg on top and spam'

Hi would you please fix your remap cookbook (https://sedimental.org/remap.html), it looks like this would be so amazingly useful but all the code examples are showing up as one liners.

Thank you in advance.

I noticed README says:

Boltons is tested against [...] CPython nightly

But I cannot find where this test occurs. There seem to be no 3.10 test runs or CPython nightly test runs. My concern is the traceback string format has changed in 3.11 so the parser may not work as is.

This PR adds a chunk_ranges function to iterutils, which doesn't chunk an iterable or list directly, but produces the relevant indices to chunk it. It contains functionality for windowing/overlap (see also issue https://github.com/mahmoud/boltons/issues/310) and alignment of the chunks. I'm not sure if input_size or input_end is more appropriate for the arguments, also the function name might need improvement.

It would be great to incorporate this function into boltons IMO, thanks a lot for considering this.

This PR includes the function itself, a test, and updates the docs.