Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Why all the __double_underscored_vars__?

10 views
Skip to first unread message

kj

unread,
Aug 8, 2009, 3:21:39 AM8/8/09
to

Python is chock-full of identifiers beginning and ending with double
underscores, such as __builtin__, __contains__, and __coerce__.

Using underscores to signal that an identifier is somehow "private"
to an implementation is pretty common in languages other than
Python. But in these cases the understanding is that the use of
these variables in external code amounts to "going behind the API",
and is therefore unwise.

But as far as I can tell, Python's double-underscore variables are
very much part of Python's API. E.g., programmers are specifically
instructed to override "double-underscore methods" to achieve
certain functionalities.

In this case, I wonder: what justifiies the double-underscore
convention? For example, what would be the problem with changing
the name of the complex.__gt__ method to complex.gt?

TIA!

kynn

Chris Rebert

unread,
Aug 8, 2009, 3:43:41 AM8/8/09
to pytho...@python.org
On Fri, Aug 7, 2009 at 5:51 PM, kj<no.e...@please.post> wrote:
> Python is chock-full of identifiers beginning and ending with double
> underscores, such as __builtin__, __contains__, and __coerce__.
>
> Using underscores to signal that an identifier is somehow "private"
> to an implementation is pretty common in languages other than
> Python.  But in these cases the understanding is that the use of
> these variables in external code amounts to "going behind the API",
> and is therefore unwise.
>
> But as far as I can tell, Python's double-underscore variables are
> very much part of Python's API.  E.g., programmers are specifically
> instructed to override "double-underscore methods" to achieve
> certain functionalities.

Right, but the *users* of the functionality provided by __methods__
typically should not invoke such methods directly.
For example, one should write `a > b`, not `a.__gt__(b)`. The lack of
underscores in just plain `gt` would suggest that it's a normal method
that could/should be called directly. Also, if the underscore
requirement were removed, then people might unknowingly overload an
operator without knowing it.
The double-underscores indicate that the Python interpreter itself
usually is the caller of the method, and as such some level of "magic"
may be associated with it. Other languages have you do the equivalent
of `def +():` or `def operator +()` to override an operator, the
keyword or symbol serving a similar warning that "here be magic". To
avoid adding another keyword and the confusion of having punctuation
as method names, Python uses a different convention, double leading
and trailing underscores.

Cheers,
Chris
--
http://blog.rebertia.com

r

unread,
Aug 8, 2009, 5:10:14 AM8/8/09
to
On Aug 7, 5:13 pm, Chris Rebert <c...@rebertia.com> wrote:

> On Fri, Aug 7, 2009 at 5:51 PM, kj<no.em...@please.post> wrote:
> > Python is chock-full of identifiers beginning and ending with double
> > underscores, such as __builtin__, __contains__, and __coerce__.
...(snip)

> Right, but the *users* of the functionality provided by __methods__
...(snip)
> Cheers,
> Chris

Yes and Python's way is by-far the proper way to handle such
functionailty. Don't thank god, thank Guido

kj

unread,
Aug 8, 2009, 5:41:19 PM8/8/09
to

>The double-underscores indicate that the Python interpreter itself
>usually is the caller of the method, and as such some level of "magic"
>may be associated with it. Other languages have you do the equivalent
>of `def +():` or `def operator +()` to override an operator, the
>keyword or symbol serving a similar warning that "here be magic".

In this case, then I hope that some of these __items__ get demoted
to a more mundane level, so that the notion of "magic" doesn't get
trivialized by everyday idioms like:

if __name__ == '__main__':
# etc

There are a few in this category... I figure that they are cases
of "atavistic magic".

I bring this up because I find it quite difficult to explain to my
students (who are complete newcomers to programming) all the
__underscored__ stuff that even rank noobs like them have to deal
with. (Trust me, to most of them your reply to my post would be
as clear as mud.) This suggests to me that there's something a
bit unnatural about some of these __items__.

Anyway, thanks for your post. I see your point.

kynn

David Cournapeau

unread,
Aug 8, 2009, 6:32:37 PM8/8/09
to kj, pytho...@python.org
On Sat, Aug 8, 2009 at 9:11 PM, kj<no.e...@please.post> wrote:
> In <mailman.4446.1249683...@python.org> Chris Rebert <cl...@rebertia.com> writes:
>
>>The double-underscores indicate that the Python interpreter itself
>>usually is the caller of the method, and as such some level of "magic"
>>may be associated with it. Other languages have you do the equivalent
>>of `def +():` or `def operator +()` to override an operator, the
>>keyword or symbol serving a similar warning that "here be magic".
>
> In this case, then I hope that some of these __items__ get demoted
> to a more mundane level, so that the notion of "magic" doesn't get
> trivialized by everyday idioms like:
>
> if __name__ == '__main__':
>    # etc
>
> There are a few in this category...  I figure that they are cases
> of "atavistic magic".
>
> I bring this up because I find it quite difficult to explain to my
> students (who are complete newcomers to programming) all the
> __underscored__ stuff that even rank noobs like them have to deal
> with.  (Trust me, to most of them your reply to my post would be
> as clear as mud.)

Maybe your students do not need to know about it, at least at the
beginning ? I heavily use python, and do not use the underscore
methods so much most of the time, except for __init__,

cheers,

David

kj

unread,
Aug 9, 2009, 12:19:28 AM8/9/09
to

>On Sat, Aug 8, 2009 at 9:11 PM, kj<no.e...@please.post> wrote:

>> In <mailman.4446.1249683...@python.org> Chris Rebert <cl=


>p...@rebertia.com> writes:
>>
>>>The double-underscores indicate that the Python interpreter itself
>>>usually is the caller of the method, and as such some level of "magic"
>>>may be associated with it. Other languages have you do the equivalent
>>>of `def +():` or `def operator +()` to override an operator, the
>>>keyword or symbol serving a similar warning that "here be magic".
>>
>> In this case, then I hope that some of these __items__ get demoted
>> to a more mundane level, so that the notion of "magic" doesn't get
>> trivialized by everyday idioms like:
>>

>> if __name__ =3D=3D '__main__':
>> =C2=A0 =C2=A0# etc
>>
>> There are a few in this category... =C2=A0I figure that they are cases


>> of "atavistic magic".
>>
>> I bring this up because I find it quite difficult to explain to my
>> students (who are complete newcomers to programming) all the
>> __underscored__ stuff that even rank noobs like them have to deal

>> with. =C2=A0(Trust me, to most of them your reply to my post would be
>> as clear as mud.)

>Maybe your students do not need to know about it, at least at the
>beginning ? I heavily use python, and do not use the underscore
>methods so much most of the time, except for __init__,

Believe me, it's not me who's bringing this stuff up: *they*
specifically ask. That's precisely my point: it is *they* who
somehow feel they can't avoid finding out about this stuff; they
must run into such __arcana__ often enough to cause them to wonder.
If at least some rank beginners (i.e. some of my students) feel
this way, I suggest that some of this alleged __arcana__ should be
demoted to a more mundane everyday status, without the scare-underscores.
E.g. maybe there should be a built-in is_main(), or some such, so
that beginners don't have to venture into the dark underworld of
__name__ and "__main__".

kynn

Scott David Daniels

unread,
Aug 9, 2009, 3:20:32 AM8/9/09
to
kj wrote:
>>> ... I find it quite difficult to explain to my

>>> students (who are complete newcomers to programming) all the
>>> __underscored__ stuff that even rank noobs like them have to deal
>>> with. =C2=A0(Trust me, to most of them your reply to my post would be
>>> as clear as mud.)
> Believe me, it's not me who's bringing this stuff up: *they*
> specifically ask. That's precisely my point: it is *they* who
> somehow feel they can't avoid finding out about this stuff; they
> must run into such __arcana__ often enough to cause them to wonder.
> If at least some rank beginners (i.e. some of my students) feel
> this way, I suggest that some of this alleged __arcana__ should be
> demoted to a more mundane everyday status, without the scare-underscores.
> E.g. maybe there should be a built-in is_main(), or some such, so
> that beginners don't have to venture into the dark underworld of
> __name__ and "__main__".

Do you know about Kirby Urner's technique of calling such symbols,
"ribs," -- the access to the "stuff" Python is built from? One nice
thing about Python is that you can experiment with what these
"__ribs__" do without having to learn yet another language.

It seems nice to me that you can use a rule that says, "stick to
normal names and you don't have to worry about mucking with the
way Python itself works, but if you are curious, looks for those
things and fiddle with them."

--Scott David Daniels
Scott....@Acm.Org

Zac Burns

unread,
Aug 9, 2009, 7:22:17 AM8/9/09
to Scott David Daniels, pytho...@python.org
As I understand it, the double underscores is to create a namespace
"reserved for python's internal use". That way python can add more
variables and methods in the future and as long as people respect the
namespace their code will not break with future revisions.

--
Zachary Burns
(407)590-4814
Aim - Zac256FL
Production Engineer (Digital Overlord)
Zindagi Games

Steven D'Aprano

unread,
Aug 9, 2009, 2:18:16 PM8/9/09
to
On Sat, 08 Aug 2009 12:11:19 +0000, kj wrote:

> In <mailman.4446.1249683...@python.org> Chris Rebert
> <cl...@rebertia.com> writes:
>
>>The double-underscores indicate that the Python interpreter itself
>>usually is the caller of the method, and as such some level of "magic"
>>may be associated with it. Other languages have you do the equivalent of
>>`def +():` or `def operator +()` to override an operator, the keyword or
>>symbol serving a similar warning that "here be magic".
>
> In this case, then I hope that some of these __items__ get demoted to a
> more mundane level, so that the notion of "magic" doesn't get
> trivialized by everyday idioms like:
>
> if __name__ == '__main__':
> # etc

But that is magic, and just because it's magic doesn't mean it's not
useful every day.

I don't see what's so difficult about telling your students that double
underscore names have special meaning to the Python interpreter. That
doesn't mean you're forbidden from using them, or that you have to use
them, it just means that they have a special meaning to the interpreter,
and you usually don't call them directly.

--
Steven

0 new messages