Output a log message every x% of a long computation

import logging

logging.basicConfig(level=logging.INFO)


def time_to_log(data, percent=10):
    percent = percent if percent > 1 else percent * 100
    data_size = len(data)
    log_every = data_size * percent / 100
    return float(data_size), lambda i: not i % log_every


data = range(100)
data_size, is_time_to_log = time_to_log(data, 0.2)

for i, d in enumerate(data):
    if is_time_to_log(i):
        logging.info("processing... {:.2%}".format(i / data_size))
    pass  # do whatever is needed
logging.info("processing... 100.00%")

another reader's digest

The borg pattern

https://www.safaribooksonline.com/library/view/python-cookbook/0596001673/ch05s23.html

Neural network, 11 LOC

http://iamtrask.github.io/2015/07/12/basic-python-network/

Best practices

to manage a Python code library

http://pbpython.com/best-practices.html

0x5f3759df

Another magic number

http://h14s.p5r.org/2012/09/0x5f3759df.html

comprendo comprehension ?

http://blog.lerner.co.il/want-to-understand-pythons-comprehensions-think-like-an-accountant/

 Youtube File System

https://github.com/rasguanabana/ytfs

 

A peek in the future

Print

>>> print

>>> from __future__ import print_function

>>> print

<built-in function print>

division

>>> 3/2

1

>>> from __future__ import division

>>> 3/2

1.5

Hold your brace

>>> from __future__ import braces

SyntaxError: not a chance (<input>, line 1)

30 000 % speed improvement

https://www.ibm.com/developerworks/community/blogs/jfp/entry/Python_Is_Not_C?lang=en

What’s a bug ?

http://www.yegor256.com/2015/06/18/good-programmers-bug-free.html

Compile ?

http://ruslanspivak.com/lsbasi-part1/

String formatting

http://pyformat.info/

Cutting off the internet

https://www.youtube.com/watch?v=YHbKxFcDltM

True or False ?

Here are different way to transform values from a config file to a boolean

Which is your favourite ?

devpi install z3c.appconfig

>>> from z3c.appconfig.utils import asBool

>>> asBool("true")

True

>>> asBool(1)

True

>>> asBool(-1)

True

>>> asBool("no")

False

>>> asBool("yes")

True

>>> asBool("Yes")

True

>>> asBool("YES")

True

>>> asBool("NO")

False

>>> asBool("off")

False

>>> asBool("on")

True

>>> asBool('Lorem')

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

  File "/home/nlaurance/PycharmProjects/dev_venv/local/lib/python2.7/site-packages/z3c/appconfig/utils.py", line 18, in asBool

    raise ValueError("Invalid boolean: %s" % value)

ValueError: Invalid boolean: lorem

>>> asBool(456)

True

>>> asBool(0)

False

devpi install distutils

>>> from distutils.util import strtobool

>>> strtobool('true')

1

>>> strtobool('False')

0

>>> strtobool('off')

0

>>> strtobool('on')

1

>>> strtobool('y')

1

>>> strtobool('n')

0

>>> strtobool('lorem')

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

  File "/usr/lib/python2.7/distutils/util.py", line 325, in strtobool

    raise ValueError, "invalid truth value %r" % (val,)

ValueError: invalid truth value 'lorem'

>>> strtobool('1')

1

>>> strtobool(456)

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

  File "/usr/lib/python2.7/distutils/util.py", line 319, in strtobool

    val = string.lower(val)

  File "/usr/lib/python2.7/string.py", line 226, in lower

    return s.lower()

AttributeError: 'int' object has no attribute 'lower'

Roll your own

>>> def asbool(value):

...     return str(value).lower() in ('true', 't', 'on', 'yes', 'y', '1', '1.0')

...

>>> asbool('f')

False

>>> asbool('YES')

True

>>> asbool(True)

True

>>> asbool(False)

False

>>> asbool(0)

False

>>> asbool(1)

True

>>> asbool(-1)

False

>>> asbool('On')

True

Another panda sweet treat

http://pbpython.com/pandas-pivot-table-explained.html

Gesture Typing, the swipe keyboard

http://nbviewer.ipython.org/url/norvig.com/ipython/Gesture Typing.ipynb

Retro Gaming

http://arunrocks.com/python-one-liner-games/

Sanitize the inputs

https://github.com/TomAugspurger/engarde

Validate your CSV

https://pypi.python.org/pypi/chkcsv

https://pythonhosted.org/chkcsv/

Global variables are BAD

http://c2.com/cgi/wiki?GlobalVariablesAreBad

Profiler

https://github.com/what-studio/profiling

 

By Reference or by value ?

Last but not least, there is no such thing as passing variable by value or by reference.

They are names, take 25 minutes of attention and listen to Ned Batchelder, a name you’ll cross often

https://www.youtube.com/watch?v=_AEJHKGk9ns&index=8&list=PLwyG5wA5gIzjhW36BxGBoQwUZHnPDFux3

Crunching the crashes

A nice tutorial for pandas, showing how to manipulate and analyse a big CSV file ( 6 000 000 rows +)

https://realpython.com/blog/python/working-with-large-excel-files-in-pandas/

lost in data

This one is fun! Another tutorial around data analysis with python, this time working against a TV series database

http://nbviewer.ipython.org/github/pmbaumgartner/LOST/blob/master/WE%20HAVE%20TO%20GO%20BACK.ipynb

 

playing or learning ? both !

Willing to learn python from the start by example on a nice slope ?

A great site

www.checkio.org

has loads of puzzles to solve in python, it starts at elementary level.

You can then publish (or not) your solution and vote for the best proposal.

Great to see others answers and learn from it.