python - python3 memoryerror when producing a large list -
i'm beginner. see mandelbrot set fantastic, decide draw set python. there problem,i got 'memoryerror' when run code.
this statement num_set = gen_num_set(10000)
produce large list, 20000*20000*4 = 1600000000. when use '1000' instead of '10000', can run code successfully.
my computer's memory 4gb , operating system window7 32bit. want know if problem limit of computer or there way optimize code.
thanks.
#!/usr/bin/env python3.4 import matplotlib.pyplot plt import numpy np import random,time multiprocessing import * def first_quadrant(n): start_point = 1 / n n = 2*n return gen_complex_num(start_point,n,1) def second_quadrant(n): start_point = 1 / n n = 2*n return gen_complex_num(start_point,n,2) def third_quadrant(n): start_point = 1 / n n = 2*n return gen_complex_num(start_point,n,3) def four_quadrant(n): start_point = 1 / n n = 2*n return gen_complex_num(start_point,n,4) def gen_complex_num(start_point,n,quadrant): complex_num = [] if quadrant == 1: in range(n): real = i*start_point j in range(n): imag = j*start_point complex_num.append(complex(real,imag)) return complex_num elif quadrant == 2: in range(n): real = i*start_point*(-1) j in range(n): imag = j*start_point complex_num.append(complex(real,imag)) return complex_num elif quadrant == 3: in range(n): real = i*start_point*(-1) j in range(n): imag = j*start_point*(-1) complex_num.append(complex(real,imag)) return complex_num elif quadrant == 4: in range(n): real = i*start_point j in range(n): imag = j*start_point*(-1) complex_num.append(complex(real,imag)) return complex_num def gen_num_set(n): return [first_quadrant(n), second_quadrant(n), third_quadrant(n), four_quadrant(n)] def if_man_set(num_set): iteration_n = 10000 man_set = [] z = complex(0,0) c in num_set: if_man = 1 in range(iteration_n): if abs(z) > 2: if_man = 0 z = complex(0,0) break z = z*z + c if if_man: man_set.append(c) return man_set def plot_scatter(x,y): #plt.plot(x,y) color = ran_color() plt.scatter(x,y,c=color) plt.show() def ran_num(): return random.random() def ran_color(): return [ran_num() in range(3)] def plot_man_set(man_set): z_real = [] z_imag = [] z in man_set: z_real.append(z.real) z_imag.append(z.imag) plot_scatter(z_real,z_imag) if __name__ == "__main__": start_time = time.time() num_set = gen_num_set(10000) pool(processes=4) pool: #use multiprocess set_part = pool.map(if_man_set, num_set) man_set = [] in set_part: man_set += plot_man_set(man_set) end_time = time.time() use_time = end_time - start_time print(use_time)
you creating list 1.6 billion elements. each of complex number contains 2 floats. python complex number takes 24 bytes (at least on system: sys.getsizeof(complex(1.0,1.0))
gives 24
), you'll need on 38gb store values, , that's before start looking @ list itself.
your list 1.6 billion elements won't fit @ on 32-bit system (6.4gb 4 byte pointers), need go 64-bit system 8 byte pointers , @ need 12.8gb pointers.
so, no way you're going unless upgrade 64-bit os maybe 64gb ram (though might need more).
Comments
Post a Comment