概述
Tqdm 是一个快速,可扩展的Python进度条,可以在 Python 长循环中添加一个进度提示信息,用户只需要封装任意的迭代器 tqdm(iterator)。
我的系统是window环境,首先安装python,接下来就是pip。
pip安装:
在python根目录下创建一个get-pip.py的文件,内容:
https://bootstrap.pypa.io/get-pip.py
然后在CMD窗口进入python下面:
输出:
python -m pip install -U pip
由于Tqdm要求的pip版本是9.0所以需要手动安装pip9.0
http://pypi.python.org/pypi/pip
下载安装包9.0
然后解压进入,CMD窗口输入:python setup.py install
然后就可以安装Tqdm了,
pip install tqdm
安装最新的开发版的话
pip install -e git+https://github.com/tqdm/tqdm.git@master#egg=tqdm
最后看看怎么用呢?https://pypi.python.org/pypi/tqdm
基本用法:
from tqdm import tqdm for i in tqdm(range(10000)): sleep(0.01)
当然除了tqdm,还有trange,使用方式完全相同
for i in trange(100): sleep(0.1)
只要传入list都可以:
pbar = tqdm(["a","b","c","d"]) for char in pbar: pbar.set_description("Processing %s" % char)
也可以手动控制更新
with tqdm(total=100) as pbar: for i in range(10): pbar.update(10)
也可以这样:
pbar = tqdm(total=100) for i in range(10): pbar.update(10) pbar.close()
在Shell的tqdm用法
统计所有python脚本的行数:
$ time find . -name '*.py' -exec cat \{} \; | wc -l 857365 real 0m3.458s user 0m0.274s sys 0m3.325s $ time find . -name '*.py' -exec cat \{} \; | tqdm | wc -l 857366it [00:03,246471.31it/s] 857365 real 0m3.585s user 0m0.862s sys 0m3.358s
使用参数:
$ find . -name '*.py' -exec cat \{} \; | tqdm --unit loc --unit_scale --total 857366 >> /dev/null 100%|| 857K/857K [00:04<00:00,246Kloc/s]
备份一个目录:
$ 7z a -bd -r backup.7z docs/ | grep Compressing | tqdm --total $(find docs/ -type f | wc -l) --unit files >> backup.log 100%|| 8014/8014 [01:37<00:00,82.29files/s]
通过看示范的代码,我们能发现使用的核心是tqdm和trange这两个函数,从代码层面分析tqdm的功能,那首先是init.py
__all__ = ['tqdm','tqdm_gui','trange','tgrange','tqdm_pandas','tqdm_notebook','tnrange','main','TqdmKeyError','TqdmTypeError','__version__']
跟踪到_tqdm.py,能看到tqdm类的声明,首先是初始化
def __init__(self,iterable=None,desc=None,total=None,leave=True,file=sys.stderr,ncols=None,mininterval=0.1,maxinterval=10.0,miniters=None,ascii=None,disable=False,unit='it',unit_scale=False,dynamic_ncols=False,smoothing=0.3,bar_format=None,initial=0,position=None,gui=False,**kwargs):
Parameters iterable : iterable,optional Iterable to decorate with a progressbar. 可迭代的进度条。 Leave blank to manually manage the updates. 留空手动管理更新?? desc : str,optional Prefix for the progressbar. 进度条的描述 total : int,optional The number of expected iterations. If unspecified,len(iterable) is used if possible. As a last resort,only basic progress statistics are displayed (no ETA,no progressbar). If gui is True and this parameter needs subsequent updating,specify an initial arbitrary large positive integer,e.g. int(9e9). 预期的迭代数目,默认为None,则尽可能的迭代下去,如果gui设置为True,这里则需要后续的更新,将需要指定为一个初始随意值较大的正整数,例如int(9e9) leave : bool,optional If [default: True],keeps all traces of the progressbar upon termination of iteration. 保留进度条存在的痕迹,简单来说就是会把进度条的最终形态保留下来,默认为True file : io.TextIOWrapper or io.StringIO,optional Specifies where to output the progress messages [default: sys.stderr]. Uses file.write(str) and file.flush() methods. 指定消息的输出 ncols : int,optional The width of the entire output message. If specified,dynamically resizes the progressbar to stay within this bound. If unspecified,attempts to use environment width. The fallback is a meter width of 10 and no limit for the counter and statistics. If 0,will not print any meter (only stats). 整个输出消息的宽度。如果指定,动态调整的进度停留在这个边界。如果未指定,尝试使用环境的宽度。如果为0,将不打印任何东西(只统计)。 mininterval : float,optional Minimum progress update interval,in seconds [default: 0.1]. 最小进度更新间隔,以秒为单位(默认值:0.1)。 maxinterval : float,optional Maximum progress update interval,in seconds [default: 10.0]. 最大进度更新间隔,以秒为单位(默认值:10)。 miniters : int,in iterations. If specified,will set mininterval to 0. 最小进度更新周期 ascii : bool,optional If unspecified or False,use unicode (smooth blocks) to fill the meter. The fallback is to use ASCII characters 1-9 #. 如果不设置,默认为unicode编码 disable : bool,optional Whether to disable the entire progressbar wrapper [default: False]. 是否禁用整个进度条包装(如果为True,进度条不显示) unit : str,optional String that will be used to define the unit of each iteration [default: it]. 将被用来定义每个单元的字符串??? unit_scale : bool,optional If set,the number of iterations will be reduced/scaled automatically and a metric prefix following the International System of Units standard will be added (kilo,mega,etc.) [default: False]. 如果设置,迭代的次数会自动按照十、百、千来添加前缀,默认为false dynamic_ncols : bool,constantly alters ncols to the environment (allowing for window resizes) [default: False]. 不断改变ncols环境,允许调整窗口大小 smoothing : float,optional Exponential moving average smoothing factor for speed estimates (ignored in GUI mode). Ranges from 0 (average speed) to 1 (current/instantaneous speed) [default: 0.3]. bar_format : str,optional Specify a custom bar string formatting. May impact performance. If unspecified,will use ‘{l_bar}{bar}{r_bar}',where l_bar is ‘{desc}{percentage:3.0f}%|' and r_bar is ‘| {n_fmt}/{total_fmt} [{elapsed_str}<{remaining_str},{rate_fmt}]' Possible vars: bar,n,n_fmt,total,total_fmt,percentage,rate,rate_fmt,elapsed,remaining,l_bar,r_bar,desc. 自定义栏字符串格式化…默认会使用{l_bar}{bar}{r_bar}的格式,格式同上 initial : int,optional The initial counter value. Useful when restarting a progress bar [default: 0]. 初始计数器值,默认为0 position : int,optional Specify the line offset to print this bar (starting from 0) Automatic if unspecified. Useful to manage multiple bars at once (eg,from threads). 指定偏移,这个功能在多个条中有用 gui : bool,optional WARNING: internal parameter - do not use. Use tqdm_gui(…) instead. If set,will attempt to use matplotlib animations for a graphical output [default: False]. 内部参数… Returns out : decorated iterator. 返回为一个迭代器
其实不用分析更多代码,多看看几个例子:(官网的例子)
7zx.py压缩进度条
# -*- coding: utf-8 -*- """Usage: 7zx.py [--help | options] <zipfiles>... Options: -h,--help Print this help and exit -v,--version Print version and exit -c,--compressed Use compressed (instead of uncompressed) file sizes -s,--silent Do not print one row per zip file -y,--yes Assume yes to all queries (for extraction) -D=<level>,--debug=<level> Print varIoUs types of debugging information. Choices: CRITICAL|FATAL ERROR WARN(ING) [default: INFO] DEBUG NOTSET -d,--debug-trace Print lots of debugging information (-D NOTSET) """ from __future__ import print_function from docopt import docopt import logging as log import subprocess import re from tqdm import tqdm import pty import os import io __author__ = "Casper da Costa-Luis <casper.dcl@physics.org>" __licence__ = "MPLv2.0" __version__ = "0.2.0" __license__ = __licence__ RE_SCN = re.compile("([0-9]+)\s+([0-9]+)\s+(.*)$",flags=re.M) def main(): args = docopt(__doc__,version=__version__) if args.pop('--debug-trace',False): args['--debug'] = "NOTSET" log.basicConfig(level=getattr(log,args['--debug'],log.INFO),format='%(levelname)s: %(message)s') log.debug(args) # Get compressed sizes zips = {} for fn in args['<zipfiles>']: info = subprocess.check_output(["7z","l",fn]).strip() finfo = RE_SCN.findall(info) # builtin test: last line should be total sizes log.debug(finfo) totals = map(int,finfo[-1][:2]) # log.debug(totals) for s in range(2): assert(sum(map(int,(inf[s] for inf in finfo[:-1]))) == totals[s]) fcomp = dict((n,int(c if args['--compressed'] else u)) for (u,c,n) in finfo[:-1]) # log.debug(fcomp) # zips : {'zipname' : {'filename' : int(size)}} zips[fn] = fcomp # Extract cmd7zx = ["7z","x","-bd"] if args['--yes']: cmd7zx += ["-y"] log.info("Extracting from {:d} file(s)".format(len(zips))) with tqdm(total=sum(sum(fcomp.values()) for fcomp in zips.values()),unit="B",unit_scale=True) as tall: for fn,fcomp in zips.items(): md,sd = pty.openpty() ex = subprocess.Popen(cmd7zx + [fn],bufsize=1,stdout=md,# subprocess.PIPE,stderr=subprocess.STDOUT) os.close(sd) with io.open(md,mode="rU",buffering=1) as m: with tqdm(total=sum(fcomp.values()),disable=len(zips) < 2,leave=False,unit_scale=True) as t: while True: try: l_raw = m.readline() except IOError: break l = l_raw.strip() if l.startswith("Extracting"): exname = l.lstrip("Extracting").lstrip() s = fcomp.get(exname,0) # 0 is likely folders t.update(s) tall.update(s) elif l: if not any(l.startswith(i) for i in ("7-Zip ","p7zip Version ","Everything is Ok","Folders: ","Files: ","Size: ","Compressed: ")): if l.startswith("Processing archive: "): if not args['--silent']: t.write(t.format_interval( t.start_t - tall.start_t) + ' ' + l.lstrip("Processing archive: ")) else: t.write(l) ex.wait() main.__doc__ = __doc__ if __name__ == "__main__": main()
tqdm_wget.py
"""An example of wrapping manual tqdm updates for urllib reporthook. # urllib.urlretrieve documentation > If present,the hook function will be called once > on establishment of the network connection and once after each block read > thereafter. The hook will be passed three arguments; a count of blocks > transferred so far,a block size in bytes,and the total size of the file. Usage: tqdm_wget.py [options] Options: -h,--help Print this help message and exit -u URL,--url URL : string,optional The url to fetch. [default: http://www.doc.ic.ac.uk/~cod11/matryoshka.zip] -o FILE,--output FILE : string,optional The local file path in which to save the url [default: /dev/null]. """ import urllib from tqdm import tqdm from docopt import docopt def my_hook(t): """ Wraps tqdm instance. Don't forget to close() or __exit__() the tqdm instance once you're done with it (easiest using `with` Syntax). Example ------- >>> with tqdm(...) as t: ... reporthook = my_hook(t) ... urllib.urlretrieve(...,reporthook=reporthook) """ last_b = [0] def inner(b=1,bsize=1,tsize=None): """ b : int,optional Number of blocks just transferred [default: 1]. bsize : int,optional Size of each block (in tqdm units) [default: 1]. tsize : int,optional Total size (in tqdm units). If [default: None] remains unchanged. """ if tsize is not None: t.total = tsize t.update((b - last_b[0]) * bsize) last_b[0] = b return inner opts = docopt(__doc__) eg_link = opts['--url'] eg_file = eg_link.replace('/',' ').split()[-1] with tqdm(unit='B',unit_scale=True,miniters=1,desc=eg_file) as t: # all optional kwargs urllib.urlretrieve(eg_link,filename=opts['--output'],reporthook=my_hook(t),data=None)
examples.py
""" # Simple tqdm examples and profiling # Benchmark for i in _range(int(1e8)): pass # Basic demo import tqdm for i in tqdm.trange(int(1e8)): pass # Some decorations import tqdm for i in tqdm.trange(int(1e8),miniters=int(1e6),ascii=True,desc="cool",dynamic_ncols=True): pass # Nested bars from tqdm import trange for i in trange(10): for j in trange(int(1e7),unit_scale=True): pass # Experimental GUI demo import tqdm for i in tqdm.tgrange(int(1e8)): pass # Comparison to https://code.google.com/p/python-progressbar/ try: from progressbar.progressbar import ProgressBar except ImportError: pass else: for i in ProgressBar()(_range(int(1e8))): pass # Dynamic miniters benchmark from tqdm import trange for i in trange(int(1e8),smoothing=0): pass # Fixed miniters benchmark from tqdm import trange for i in trange(int(1e8),miniters=4500000,smoothing=0): pass """ from time import sleep from timeit import timeit import re # Simple demo from tqdm import trange for i in trange(16,leave=True): sleep(0.1) # Profiling/overhead tests stmts = filter(None,re.split(r'\n\s*#.*?\n',__doc__)) for s in stmts: print(s.replace('import tqdm\n','')) print(timeit(stmt='try:\n\t_range = xrange' '\nexcept:\n\t_range = range\n' + s,number=1),'seconds')
pandas_progress_apply.py
import pandas as pd import numpy as np from tqdm import tqdm df = pd.DataFrame(np.random.randint(0,100,(100000,6))) # Register `pandas.progress_apply` and `pandas.Series.map_apply` with `tqdm` # (can use `tqdm_gui`,`tqdm_notebook`,optional kwargs,etc.) tqdm.pandas(desc="my bar!") # Now you can use `progress_apply` instead of `apply` # and `progress_map` instead of `map` df.progress_apply(lambda x: x**2) # can also groupby: # df.groupby(0).progress_apply(lambda x: x**2) # -- Source code for `tqdm_pandas` (really simple!) # def tqdm_pandas(t): # from pandas.core.frame import DataFrame # def inner(df,func,*args,**kwargs): # t.total = groups.size // len(groups) # def wrapper(*args,**kwargs): # t.update(1) # return func(*args,**kwargs) # result = df.apply(wrapper,**kwargs) # t.close() # return result # DataFrame.progress_apply = inner
引用tqdm并非强制作为依赖:
include_no_requirements.py
# How to import tqdm without enforcing it as a dependency try: from tqdm import tqdm except ImportError: def tqdm(*args,**kwargs): if args: return args[0] return kwargs.get('iterable',None)
参考:
https://github.com/tqdm/tqdm/tree/master/examples
https://pypi.python.org/pypi/tqdm
https://github.com/tqdm/tqdm
以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持编程小技巧。
总结
以上是编程之家为你收集整理的python的Tqdm模块的使用全部内容,希望文章能够帮你解决python的Tqdm模块的使用所遇到的程序开发问题。
如果您也喜欢它,动动您的小指点个赞吧