这是我的流式迭代器,它是urllib3的实验分支,它支持通过可迭代的流式分块请求:
class IterStreamer(object):
"""
File-like streaming iterator.
"""
def __init__(self, generator):
self.generator = generator
self.iterator = iter(generator)
self.leftover = ''
def __len__(self):
return self.generator.__len__()
def __iter__(self):
return self.iterator
def next(self):
return self.iterator.next()
def read(self, size):
data = self.leftover
count = len(self.leftover)
if count < size:
try:
while count < size:
chunk = self.next()
data += chunk
count += len(chunk)
except StopIteration:
pass
self.leftover = data[size:]
return data[:size]
带有上下文的源代码:https : //github.com/shazow/urllib3/blob/filepost- stream/urllib3/filepost.py#L23
相关单元测试:https : //github.com/shazow/urllib3/blob/filepost- stream/test/test_filepost.py#L9
遗憾的是,此代码尚未进入稳定分支,因为无法支持无大小的分块请求,但它应该为您要执行的操作奠定良好的基础。有关显示如何使用它的示例,请参见源链接。