import zlib
def stream_gzip_decompress(stream):
dec = zlib.decompressobj(32 + zlib.MAX_WBITS) # offset 32 to skip the header
for chunk in stream:
rv = dec.decompress(chunk)
if rv:
yield rv
到zlib
标头的32个偏移量表示gzip标头是预期的但已跳过。
S3键对象是一个迭代器,因此您可以执行以下操作:
for data in stream_gzip_decompress(k):
# do something with the decompressed data