[python] How do I pass a variable by reference?



Answers

The problem comes from a misunderstanding of what variables are in Python. If you're used to most traditional languages, you have a mental model of what happens in the following sequence:

a = 1
a = 2

You believe that a is a memory location that stores the value 1, then is updated to store the value 2. That's not how things work in Python. Rather, a starts as a reference to an object with the value 1, then gets reassigned as a reference to an object with the value 2. Those two objects may continue to coexist even though a doesn't refer to the first one anymore; in fact they may be shared by any number of other references within the program.

When you call a function with a parameter, a new reference is created that refers to the object passed in. This is separate from the reference that was used in the function call, so there's no way to update that reference and make it refer to a new object. In your example:

def __init__(self):
    self.variable = 'Original'
    self.Change(self.variable)

def Change(self, var):
    var = 'Changed'

self.variable is a reference to the string object 'Original'. When you call Change you create a second reference var to the object. Inside the function you reassign the reference var to a different string object 'Changed', but the reference self.variable is separate and does not change.

The only way around this is to pass a mutable object. Because both references refer to the same object, any changes to the object are reflected in both places.

def __init__(self):         
    self.variable = ['Original']
    self.Change(self.variable)

def Change(self, var):
    var[0] = 'Changed'
Question

The Python documentation seems unclear about whether parameters are passed by reference or value, and the following code produces the unchanged value 'Original'

class PassByReference:
    def __init__(self):
        self.variable = 'Original'
        self.change(self.variable)
        print(self.variable)

    def change(self, var):
        var = 'Changed'

Is there something I can do to pass the variable by actual reference?




Effbot (aka Fredrik Lundh) has described Python's variable passing style as call-by-object: http://effbot.org/zone/call-by-object.htm

Objects are allocated on the heap and pointers to them can be passed around anywhere.

  • When you make an assignment such as x = 1000, a dictionary entry is created that maps the string "x" in the current namespace to a pointer to the integer object containing one thousand.

  • When you update "x" with x = 2000, a new integer object is created and the dictionary is updated to point at the new object. The old one thousand object is unchanged (and may or may not be alive depending on whether anything else refers to the object).

  • When you do a new assignment such as y = x, a new dictionary entry "y" is created that points to the same object as the entry for "x".

  • Objects like strings and integers are immutable. This simply means that there are no methods that can change the object after it has been created. For example, once the integer object one-thousand is created, it will never change. Math is done by creating new integer objects.

  • Objects like lists are mutable. This means that the contents of the object can be changed by anything pointing to the object. For example, x = []; y = x; x.append(10); print y will print [10]. The empty list was created. Both "x" and "y" point to the same list. The append method mutates (updates) the list object (like adding a record to a database) and the result is visible to both "x" and "y" (just as a database update would be visible to every connection to that database).

Hope that clarifies the issue for you.




There are no variables in Python

The key to understanding parameter passing is to stop thinking about "variables". There are names and objects in Python and together they appear like variables, but it is useful to always distinguish the three.

  1. Python has names and objects.
  2. Assignment binds a name to an object.
  3. Passing an argument into a function also binds a name (the parameter name of the function) to an object.

That is all there is to it. Mutability is irrelevant for this question.

Example:

a = 1

This binds the name a to an object of type integer that holds the value 1.

b = x

This binds the name b to the same object that the name x is currently bound to. Afterwards, the name b has nothing to do with the name x any more.

See sections 3.1 and 4.2 in the Python 3 language reference.


So in the code shown in the question, the statement self.Change(self.variable) binds the name var (in the scope of function Change) to the object that holds the value 'Original' and the assignment var = 'Changed' (in the body of function Change) assigns that same name again: to some other object (that happens to hold a string as well but could have been something else entirely).




In this case the variable titled var in the method Change is assigned a reference to self.variable, and you immediately assign a string to var. It's no longer pointing to self.variable. The following code snippet shows what would happen if you modify the data structure pointed to by var and self.variable, in this case a list:

>>> class PassByReference:
...     def __init__(self):
...         self.variable = ['Original']
...         self.change(self.variable)
...         print self.variable
...         
...     def change(self, var):
...         var.append('Changed')
... 
>>> q = PassByReference()
['Original', 'Changed']
>>> 

I'm sure someone else could clarify this further.




Since your example happens to be object-oriented, you could make the following change to achieve a similar result:

class PassByReference:
    def __init__(self):
        self.variable = 'Original'
        self.change('variable')
        print(self.variable)

    def change(self, var):
        setattr(self, var, 'Changed')

# o.variable will equal 'Changed'
o = PassByReference()
assert o.variable == 'Changed'



It is neither pass-by-value or pass-by-reference - it is call-by-object. See this, by Fredrik Lundh:

http://effbot.org/zone/call-by-object.htm

Here is a significant quote:

"...variables [names] are not objects; they cannot be denoted by other variables or referred to by objects."

In your example, when the Change method is called--a namespace is created for it; and var becomes a name, within that namespace, for the string object 'Original'. That object then has a name in two namespaces. Next, var = 'Changed' binds var to a new string object, and thus the method's namespace forgets about 'Original'. Finally, that namespace is forgotten, and the string 'Changed' along with it.




A simple trick I normally use is to just wrap it in a list:

def Change(self, var):
    var[0] = 'Changed'

variable = ['Original']
self.Change(variable)      
print variable[0]

(Yeah I know this can be inconvenient, but sometimes it is simple enough to do this.)




Here is the simple (I hope) explanation of the concept pass by object used in Python.
Whenever you pass an object to the function, the object itself is passed (object in Python is actually what you'd call a value in other programming languages) not the reference to this object. In other words, when you call:

def change_me(list):
   list = [1, 2, 3]

my_list = [0, 1]
change_me(my_list)

The actual object - [0, 1] (which would be called a value in other programming languages) is being passed. So in fact the function change_me will try to do something like:

[0, 1] = [1, 2, 3]

which obviously will not change the object passed to the function. If the function looked like this:

def change_me(list):
   list.append(2)

Then the call would result in:

[0, 1].append(2)

which obviously will change the object. This answer explains it well.




There is a little trick to pass an object by reference, even though the language doesn't make it possible. It works in Java too, it's the list with one item. ;-)

class PassByReference:
    def __init__(self, name):
        self.name = name

def changeRef(ref):
    ref[0] = PassByReference('Michael')

obj = PassByReference('Peter')
print obj.name

p = [obj] # A pointer to obj! ;-)
changeRef(p)

print p[0].name # p->name

It's an ugly hack, but it works. ;-P




As you can state you need to have a mutable object, but let me suggest you to check over the global variables as they can help you or even solve this kind of issue!

http://docs.python.org/3/faq/programming.html#what-are-the-rules-for-local-and-global-variables-in-python

example:

>>> def x(y):
...     global z
...     z = y
...

>>> x
<function x at 0x00000000020E1730>
>>> y
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'y' is not defined
>>> z
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'z' is not defined

>>> x(2)
>>> x
<function x at 0x00000000020E1730>
>>> y
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'y' is not defined
>>> z
2



While pass by reference is nothing that fits well into python and should be rarely used there are some workarounds that actually can work to get the object currently assigned to a local variable or even reassign a local variable from inside of a called function.

The basic idea is to have a function that can do that access and can be passed as object into other functions or stored in a class.

One way is to use global (for global variables) or nonlocal (for local variables in a function) in a wrapper function.

def change(wrapper):
    wrapper(7)

x = 5
def setter(val):
    global x
    x = val
print(x)

The same idea works for reading and deleting a variable.

For just reading there is even a shorter way of just using lambda: x which returns a callable that when called returns the current value of x. This is somewhat like "call by name" used in languages in the distant past.

Passing 3 wrappers to access a variable is a bit unwieldy so those can be wrapped into a class that has a proxy attribute:

class ByRef:
    def __init__(self, r, w, d):
        self._read = r
        self._write = w
        self._delete = d
    def set(self, val):
        self._write(val)
    def get(self):
        return self._read()
    def remove(self):
        self._delete()
    wrapped = property(get, set, remove)

# left as an exercise for the reader: define set, get, remove as local functions using global / nonlocal
r = ByRef(get, set, remove)
r.wrapped = 15

Pythons "reflection" support makes it possible to get a object that is capable of reassigning a name/variable in a given scope without defining functions explicitly in that scope:

class ByRef:
    def __init__(self, locs, name):
        self._locs = locs
        self._name = name
    def set(self, val):
        self._locs[self._name] = val
    def get(self):
        return self._locs[self._name]
    def remove(self):
        del self._locs[self._name]
    wrapped = property(get, set, remove)

def change(x):
    x.wrapped = 7

def test_me():
    x = 6
    print(x)
    change(ByRef(locals(), "x"))
    print(x)

Here the ByRef class wraps a dictionary access. So attribute access to wrapped is translated to a item access in the passed dictionary. By passing the result of the builtin locals and the name of a local variable this ends up accessing a local variable. The python documentation as of 3.5 advises that changing the dictionary might not work but it seems to work for me.




Related