您应该将文件对象用作迭代器,以减少文件的内存使用量。然后,您可以分块而不是全部处理数据库列表。例如:
results = []
database = []
for line in open("database/db.hdb"):
line = line.split(':')
#You Could then manage database in chunks?
database.append(line)
if len(database) > MAX:
#dosomething with database list so far to get result
results.append(process_database(database))
database = []
#do something Now with individual results to make one result
combine_results(results)