Since you dropped python 2 support, you have to stop making universal wheels for this module, otherwise pypi is offering wrong version for python2 setups. I see that this was addressed in commit 50fa96bf370146602b02ece8378cdafa9f57684f

Please push new version to pypi, otherwise we are having issues with installation on python2.

I have a task where I need to perform complex decoration with patching function spec. But this library is still very useful in my case and allow to not reimplement a lot of decoration logic.

I'm not sure if this is by design, but I have a lot of decorators that look like this

@decorator
def wrap(func_, *args, **kwargs):
    if not args[-1]:
        new_args = list(args)
        new_args[-1] = ENVIRONMENT_DEPENDENT_VAR
        args = tuple(new_args)
    return func_(*args, **kwargs)

@wrap
def do_stuff(requiredvar, optionalvar=None):
    pass # real work
    return 

basically checking if a certain variable is set and, if not, setting it to some appropriate veriable depending on whatever situation.

In v4, when calling do_stuff(some_var)

the args value within wrap would be (some_var, None)

in v5, the args value within wrap is now (some_var, )

Is this intentional to no longer supply a None (or whatever the function's default is) for named positional arguments?

It would be great if @decorator could be used to create decorators which produce coroutine functions. Is there already such functionality and I just didn't find it? If not, would this be feasible? If so, are there any plans for this? Would you accept PRs?

Example:

from inspect import iscoroutinefunction
from decorator import decorator

@decorator
async def ensure_coro(function, *args, **kwargs): 
    ret = function(*args, **kwargs)
    if iscoroutinefunction(function):
        ret = await ret
    return ret

@ensure_coro
def foo():
    return 42

assert iscoroutinefunction(foo)

It looks like the kw array is not being preserved or passed at all in 3.6. I have not tested in previous versions, but based on the documentation it is a different behavior.

Python 3.6.0 |Anaconda 4.3.1 (64-bit)| (default, Dec 23 2016, 11:57:41) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from decorator import decorator
>>> @decorator
... def trace(f, *args, **kw):
...     kwstr = ', '.join('%r: %r' % (k, kw[k]) for k in sorted(kw))
...     print("calling %s with args %s, {%s}" % (f.__name__, args, kwstr))
...
>>> @trace
... def samplefn(x, a="a1"):
...     pass
...
>>> samplefn(1)
calling samplefn with args (1, 'a1'), {}
>>> samplefn(1, "inline")
calling samplefn with args (1, 'inline'), {}
>>> samplefn(1, a="askeyword") # notice kwstr is coming back blank
calling samplefn with args (1, 'askeyword'), {}

In Python 3.5, many such limitations have been removed, but I still think that it is cleaner and safer to decorate only functions. If you want to decorate things like classmethods/staticmethods and general callables - which I will never support in the decorator module - I suggest you to look at the wrapt project by Graeme Dumpleton.

from decorator import decorator
import asyncio
import time


@decorator
async def log_start_stop(func, *args, **kwargs):
    t0 = int(time.time())
    print(t0, f'core={func}, args={args},kwargs={kwargs}')
    result = await func(*args, **kwargs)
    t1 = int(time.time())
    print(t1, f'result={result}')


class A:
    @staticmethod
    @log_start_stop
    async def make_task(n):
        for i in range(n):
            await asyncio.sleep(1)
        return 100


a = A()
print(a)
loop = asyncio.get_event_loop()
try:
    loop.run_until_complete(A.make_task(5))
finally:
    loop.run_until_complete(loop.shutdown_asyncgens())
    loop.close()

This still needs a review and possibly improvements (such as also removing arguments by name and adding tests). Nevertheless it shows how it could be done.

Fixes #55

On Python 3, functools.wraps nominally preserves function signatures, e.g.,

import functools

def original(x):
    return x

@functools.wraps(original)
def wrapper(*args, **kwargs):
    return original(*args, **kwargs)

The function has the right associated signature object (so introspection works properly) but abstraction is still a little leaky, e.g., for handling invalid function arguments:

In [11]: wrapper(y=1)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-11-1e0b778ae20b> in <module>()
----> 1 wrapper(y=1)

<ipython-input-4-878d38897098> in wrapper(*args, **kwargs)
      6 @functools.wraps(original)
      7 def wrapper(*args, **kwargs):
----> 8     return original(*args, **kwargs)

TypeError: original() got an unexpected keyword argument 'y'

I think the decorator library can produce fully compatible function signatures, but the interface is a little different:

@functools.partial(decorate, original)
def wrapper(original, *args, **kwargs):
    return original(*args, **kwargs)

In particular, it's awkward that I need to include the original function in the wrapper signature (e.g., def wrapper(original, *args, **kwargs) rather than def wrapper(*args, **kwargs) .

Is there an easy way to write a drop-in replacement for functools.wraps with the decorator library?

Hello,

I ran into a issue when I try to assess certain kwargs inside the decorator. In my situation I had one kwarg passed to the function, but it was inside args inside the decorator. Am I doing something wrong or is this a bug or is this a feature?

The test below fails for me and prints args (True,) kwargs {}

def test_decorator_kwargs():
    @decorator.decorator
    def loader(func, *args, **kwargs):
        print(f"args {args} kwargs {kwargs}")
        return kwargs

    @loader
    def my_loader(test=False):
        return test

    kwargs = my_loader(test=True)
    assert "test" in kwargs
    assert kwargs["test"]

    kwargs = my_loader(test=False)
    assert "test" in kwargs
    assert not kwargs["test"]

I'm using decorator 4.4.2 on python 3.7.1

Hey, I'm working on deleting Python 2 code in numpy/scipy, which includes a copy of decorator (yay!).

I see you mention full backwards support to python 2.6 in src/tests/documentation.py, but that seems to have been written ~5 years ago. With Python 2 no longer officially supported would you be interested in dropping Python 2 support? It looks like it's possible it would allow for the clean up of a few pieces of logic. I'm happy to help out if so.

Possibly related to #138 , but I'm creating a new issue here in case the root cause is fundamentally different.

I'm using the third-party decorator example like so:

In [1]: def foo(a, *, b, c=0):
   ...:     return a + b * c
   ...: 

In [2]: from decorator import FunctionMaker

In [3]: def decorator_apply(dec, func):
   ...:      """
   ...:      Decorate a function by preserving the signature even if dec
   ...:      is not a signature-preserving decorator.
   ...:      """
   ...:      return FunctionMaker.create(
   ...:          func, 'return decfunc(%(signature)s)',
   ...:          dict(decfunc=dec(func)), __wrapped__=func)
   ...: 

In [4]: def dec(f):
   ...:     print(f)
   ...:     return f

In [5]: bar = decorator_apply(dec, foo)
<function foo at 0x7f520b0f5f70>
Error in generated code:
def foo(a, *, b=None, c=None):
    return decfunc(a, *, b=None, c=None)

╭──────────────────────────────────────────────────────────────────────────────╮
│  <decorator-gen-0>:2                                                         │
│     return decfunc(a, *, b=None, c=None)                                     │
│                        ▲                                                     │
╰──────────────────────────────────────────────────────────────────────────────╯
SyntaxError: iterable argument unpacking follows keyword argument unpacking

I'm on decorator==5.1.0, python 3.9.9.

Is there a recommended workaround here? Or is there something more fundamental that will prevent this from working?

I looked at the code and it wasn't supported. Can we add a judgment here that supports partial functions？ src/decorator.py:74 f inspect.isroutine(func):

(Sorry if the title is somewhat misleading or I did get some jargon wrong)

In version 5.1.1 methods decorated by decorate created with the decorator lib stopped working with qt(pyqt?) signals that inspect the signature of the connected slot (also tested with 4.0.2 with a slight different error).

Failing snippet:

@decorator
def dummy_decorator(f, *args, **kwargs):
    return f(*args, **kwargs)

 class Foo:
    def __init__(self):
        self.action = QAction()
        self.action.triggered.connect(self.ActionCallbackNoArg)
    @dummy_decorator
    def ActionCallbackNoArg(self):
        pass

foo = Foo()
foo.action.trigger()

This was not an issue in 4.4.2. While porting our code base to python 3.10 the decorator version also got upgraded, decorator version prior to this was 4.4.2. We will probably be able to downgrade back for now.

I created a test to reproduce the error. add_qt_test_patch.txt

Create env and install decorator in editable mode:

conda create -n decorator python=3.6 pyqt
conda activate decorator
pip install --editable .

Test with 4.4.2 (pass):

W:\my\Projects\decorator (master -> origin)
(decorator) λ git checkout -- src/tests/test.py

W:\my\Projects\decorator (master -> origin)
(decorator) λ git checkout 4.4.2
Note: switching to '4.4.2'.
...
HEAD is now at 55a68b5 Updated setup.py [skip CI]

W:\my\Projects\decorator (HEAD detached at 55a68b5)
(decorator) λ patch -i add_qt_test_patch.txt -p 1
patching file src/tests/test.py
Hunk #1 succeeded at 18 with fuzz 1 (offset 6 lines).

W:\my\Projects\decorator (HEAD detached at 55a68b5)
(decorator) λ python src/tests/test.py -v QtActionTestCase.test_qt_decorator_signature_preserving_interaction_methods
test_qt_decorator_signature_preserving_interaction_methods (__main__.QtActionTestCase) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.001s

OK

Test with 5.1.1 (fail, at the time of this writing origin/master will also fail):

W:\my\Projects\decorator (HEAD detached at 55a68b5)
(decorator) λ git checkout -- src/tests/test.py

W:\my\Projects\decorator (HEAD detached at 55a68b5)
(decorator) λ git checkout 5.1.1
Previous HEAD position was 55a68b5 Updated setup.py [skip CI]
HEAD is now at ad013a2 Updated changelog and bumped version to 5.1.1

W:\my\Projects\decorator (HEAD detached at ad013a2)
(decorator) λ patch -i add_qt_test_patch.txt -p 1
patching file src/tests/test.py

W:\my\Projects\decorator (HEAD detached at ad013a2)
(decorator) λ python src/tests/test.py -v QtActionTestCase.test_qt_decorator_signature_preserving_interaction_methods
test_qt_decorator_signature_preserving_interaction_methods (__main__.QtActionTestCase) ... Traceback (most recent call last):
  File "w:\my\projects\decorator\src\decorator.py", line 231, in fun
    args, kw = fix(args, kw, sig)
  File "w:\my\projects\decorator\src\decorator.py", line 203, in fix
    ba = sig.bind(*args, **kwargs)
  File "W:\my\envs\decorator\lib\inspect.py", line 2997, in bind
    return args[0]._bind(args[1:], kwargs)
  File "W:\my\envs\decorator\lib\inspect.py", line 2918, in _bind
    raise TypeError('too many positional arguments') from None
TypeError: too many positional arguments

I also was somewhat curious. "Plain" functions:

>>> plain.ActionCallbackNoArg.__func__.__code__.co_argcount
1
>>> plain.ActionCallbackNoArg.__func__.__code__.co_varnames
('self',)
>>> plain.ActionCallbackAnyArgs.__func__.__code__.co_argcount
1
>>> plain.ActionCallbackAnyArgs.__func__.__code__.co_varnames
('self', 'args', 'kwargs')

Decorated with 4.4.2:

>>> decorated.ActionCallbackNoArg.__func__.__code__.co_argcount
1
>>> decorated.ActionCallbackNoArg.__func__.__code__.co_varnames
('self',)
>>> decorated.ActionCallbackAnyArgs.__func__.__code__.co_argcount
1
>>> decorated.ActionCallbackAnyArgs.__func__.__code__.co_varnames
('self', 'args', 'kwargs')

Decorated with 5.1.1:

>>> decorated.ActionCallbackNoArg.__func__.__code__.co_argcount
0
>>> decorated.ActionCallbackNoArg.__func__.__code__.co_varnames
('args', 'kw')
>>> decorated.ActionCallbackAnyArgs.__func__.__code__.co_argcount
0
>>> decorated.ActionCallbackAnyArgs.__func__.__code__.co_varnames
('args', 'kw')

Hi,

I was wondering if you would like to integrate continuous fuzzing by way of OSS-Fuzz? Fuzzing is a way to automate test-case generation and has been heavily used for memory unsafe languages. Recently efforts have been put into fuzzing memory safe languages and Python is one of the languages that has recently gotten better tool support.

In this https://github.com/google/oss-fuzz/pull/7998 we did an initial integration into OSS-Fuzz. OSS-Fuzz is a free service run by Google that performs continuous fuzzing of important open source projects. If you would like to integrate, the only thing I need is a list of email(s) that will get access to the data produced by OSS-Fuzz, such as bug reports, coverage reports and more stats. Notice the emails affiliated with the project will be public in the OSS-Fuzz repo, as they will be part of a configuration file.

About

In current FunctionMaker.create, if the function to create from is an async function and if the function body contains any occurrence of return, it will blindly replace it with return await. This will raise a SyntaxError in edge cases where return is present not as the keyword return but a substring.

Reproduce

from decorator import FunctionMaker


async def IDENTITY(return_value):
    return return_value

FunctionMaker.create('WILL_RAISE_SYNTAX_ERROR()', 'return IDENTITY(return_value=actual_return_value)', dict(IDENTITY=IDENTITY, _call_=IDENTITY, actual_return_value=10), addsource=True)

Error Message

Error in generated code:
async def WILL_RAISE_SYNTAX_ERROR():
    return await IDENTITY(return await_value=actual_return await_value)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/conda/lib/python3.9/site-packages/decorator.py", line 196, in create
    return self.make(body, evaldict, addsource, **attrs)
  File "/opt/conda/lib/python3.9/site-packages/decorator.py", line 159, in make
    code = compile(src, filename, 'single')
  File "<decorator-gen-0>", line 2
    return await IDENTITY(return await_value=actual_return await_value)
                          ^
SyntaxError: invalid syntax

Analysis

FunctionMaker.create has the following:

if caller and iscoroutinefunction(caller):
    body = ('async def %(name)s(%(signature)s):\n' + ibody).replace(
        'return', 'return await')

However, this replacement is blind, if a parameter is named like return_value, it will be also replaced as it contains return (like the above example).

Moreover, the following lines can produce similar errors:

• return_value = ... A simple assignment statement
• "This is a long message containing a return word"

Fix

To fix, we need to match return as a keyword rather than as any substring. A straightforward fix looks like the following (not fully tested):

if caller and iscoroutinefunction(caller):
    rawbody = "async def %(name)s(%(signature)s):\n" + ibody
    body = re.sub(r"(^|\s)return(\s)", r"\1return await\2", rawbody)

I think I may have found an issue with FunctionMaker.create(). When you try to pass a string containing a return type annotation, it breaks this function. Consider this example:

from decorator import FunctionMaker

def add(a: int, b: int) -> int:
    return a + b

add2 = FunctionMaker.create("add2(a: int, b: int)", "return add(a, b)", evaldict={"add": add}) # No problem
assert add2(6, 8) == add(6, 8) # add2 does the same as add

add3 = FunctionMaker.create("add3(a: int, b: int) -> int", "return add(a, b)", evaldict={"add": add}) # ERROR

I get a message like this:

  File "/nfs/iil/proj/dt/prv08/airon/git/socb_client/venv/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3457, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)

  File "<ipython-input-9-5d5ad6515055>", line 9, in <module>
    add3 = FunctionMaker.create("add3(a: int, b: int) -> int", "return add(a, b)", evaldict={"add": add})

  File "/nfs/iil/proj/dt/prv08/airon/git/socb_client/venv/lib/python3.9/site-packages/decorator.py", line 222, in create
    return self.make(body, evaldict, addsource, **attrs)

  File "/nfs/iil/proj/dt/prv08/airon/git/socb_client/venv/lib/python3.9/site-packages/decorator.py", line 184, in make
    code = compile(src, filename, 'single')

  File "<decorator-gen-123>", line 1
    def add3(a: int, b: int) -> in):
                                ^
SyntaxError: invalid syntax

It seems like the code that decomposes the signature at https://github.com/micheles/decorator/blob/9cf8151de8fd26551cb43288b2010965ad346839/src/decorator.py#L181

assumes the end of the signature is always a right parenthesis, which is not the case when we specify a return type annotation.

Then, the code at https://github.com/micheles/decorator/blob/9cf8151de8fd26551cb43288b2010965ad346839/src/decorator.py#L195 unconditionally adds the right parenthesis that was (actually or supposedly) removed previously.

I could fix this issue in my own environment, but the fix is kind of dirty, and has not gone through all your testing, obviously.