import logging logging.basicConfig(level=logging.INFO) def time_to_log(data, percent=10): percent = percent if percent > 1 else percent * 100 data_size = len(data) log_every = data_size * percent / 100 return float(data_size), lambda i: not i % log_every data = range(100) data_size, is_time_to_log = time_to_log(data, 0.2) for i, d in enumerate(data): if is_time_to_log(i): logging.info("processing... {:.2%}".format(i / data_size)) pass # do whatever is needed logging.info("processing... 100.00%")
python thoughts
here I will gather some notes on my python use. I will give my impressions on different libs when I can give them a try to solve the problem at hand. Also I will announce some releases when I think some of my packages might be useful to others.
Search This Blog
12 October 2015
Output a log message every x% of a long computation
10 October 2015
another reader's digest
The borg pattern
Neural network, 11 LOC
Best practices
to manage a Python code library
0x5f3759df
Another magic number
comprendo comprehension ?
Youtube File System
A peek in the future
Print
>>> print
>>> from __future__ import print_function
>>> print
<built-in function print>
division
>>> 3/2
1
>>> from __future__ import division
>>> 3/2
1.5
Hold your brace
>>> from __future__ import braces
SyntaxError: not a chance (<input>, line 1)
21 September 2015
30 000 % speed improvement
String formatting
Cutting off the internet
Here are different way to transform values from a config
file to a boolean
Which is your favourite ?
devpi
install z3c.appconfig
>>>
from z3c.appconfig.utils import asBool
>>>
asBool("true")
True
>>>
asBool(1)
True
>>>
asBool(-1)
True
>>>
asBool("no")
False
>>>
asBool("yes")
True
>>>
asBool("Yes")
True
>>>
asBool("YES")
True
>>>
asBool("NO")
False
>>>
asBool("off")
False
>>>
asBool("on")
True
>>>
asBool('Lorem')
Traceback
(most recent call last):
File "<stdin>", line 1, in <module>
File
"/home/nlaurance/PycharmProjects/dev_venv/local/lib/python2.7/site-packages/z3c/appconfig/utils.py",
line 18, in asBool
raise ValueError("Invalid boolean: %s" % value)
ValueError:
Invalid boolean: lorem
>>>
asBool(456)
True
>>>
asBool(0)
False
devpi
install distutils
>>>
from distutils.util import strtobool
>>>
strtobool('true')
1
>>>
strtobool('False')
0
>>>
strtobool('off')
0
>>>
strtobool('on')
1
>>>
strtobool('y')
1
>>>
strtobool('n')
0
>>>
strtobool('lorem')
Traceback
(most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/distutils/util.py", line 325, in strtobool
raise ValueError, "invalid truth value %r" % (val,)
ValueError:
invalid truth value 'lorem'
>>>
strtobool('1')
1
>>>
strtobool(456)
Traceback
(most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/distutils/util.py", line 319, in strtobool
val = string.lower(val)
File "/usr/lib/python2.7/string.py", line 226, in lower
return s.lower()
AttributeError:
'int' object has no attribute 'lower'
Roll
your own
>>>
def asbool(value):
...
return str(value).lower() in ('true', 't', 'on', 'yes', 'y', '1', '1.0')
...
>>>
asbool('f')
False
>>>
asbool('YES')
True
>>>
asbool(True)
True
>>>
asbool(False)
False
>>>
asbool(0)
False
>>>
asbool(1)
True
>>>
asbool(-1)
False
>>>
asbool('On')
True
08 September 2015
Another
panda sweet treat
Gesture Typing, the swipe keyboard
Retro Gaming
Sanitize the inputs
Validate your CSV
Global variables are BAD
Profiler
By Reference or by value ?
Last but not least, there is no such thing as passing
variable by value or by reference.
They are names, take 25 minutes of attention and listen to
Ned Batchelder, a name you’ll cross often
31 August 2015
Crunching the crashes
A nice tutorial for pandas, showing how to manipulate and
analyse a big CSV file ( 6 000 000 rows +)
lost in data
This one is fun! Another tutorial around data analysis with
python, this time working against a TV series database
playing or learning ? both
!
Willing to learn python from the start by example on a nice
slope ?
A great site
has loads of puzzles to solve in python, it starts at
elementary level.
You can then publish (or not) your solution and vote for the
best proposal.
Great to see others answers and learn from it.
Subscribe to:
Posts (Atom)