python - convert itertools array into numpy array -
i'm creating array:
a=itertools.combinations(range(6),2)
and have manipulate array numpy, like:
a.reshape(..
if dimensions is high, command list(a)
slow.
how can "convert" itertools array numpy array?
update 1: i've tried solution of hpaulj, in specific situation little bit slower, idea?
start=time.clock() a=it.combinations(range(495),3) a=np.array(list(a)) print stop=time.clock() print stop-start start=time.clock() a=np.fromiter(it.chain(*it.combinations(range(495),3)),dtype=int).reshape (-1,3) print stop=time.clock() print stop-start
results:
[[ 0 1 2] [ 0 1 3] [ 0 1 4] ..., [491 492 494] [491 493 494] [492 493 494]] 10.323822 [[ 0 1 2] [ 0 1 3] [ 0 1 4] ..., [491 492 494] [491 493 494] [492 493 494]] 12.289898
i'm reopening because dislike linked answer. accepted answer suggests using
np.array(list(a)) # producing (15,2) array
but op aparently has tried list(a)
, , found slow.
another answer suggests using np.fromiter
. buried in comments note fromiter
requires 1d array.
in [102]: a=itertools.combinations(range(6),2) in [103]: np.fromiter(a,dtype=int) --------------------------------------------------------------------------- valueerror traceback (most recent call last) <ipython-input-103-29db40e69c08> in <module>() ----> 1 np.fromiter(a,dtype=int) valueerror: setting array element sequence.
so using fromiter
itertools requires somehow flattening iterator.
a quick set of timings suggests list
isn't slow step. it's converting list array slow:
in [104]: timeit itertools.combinations(range(6),2) 1000000 loops, best of 3: 1.1 µs per loop in [105]: timeit list(itertools.combinations(range(6),2)) 100000 loops, best of 3: 3.1 µs per loop in [106]: timeit np.array(list(itertools.combinations(range(6),2))) 100000 loops, best of 3: 14.7 µs per loop
i think fastest way use fromiter
flatten combinations
idiomatic use of itertools.chain
:
in [112]: timeit np.fromiter(itertools.chain(*itertools.combinations(range(6),2)),dtype=int) .reshape(-1,2) 100000 loops, best of 3: 12.1 µs per loop
not of time savings, @ least on small size. (fromiter
takes count
, shaves off µs. larger case, range(60)
, fromiter
takes half time of array
.
a quick search on [numpy] itertools
turns number of suggestions of pure numpy ways of generating combinations. itertools
fast, generating pure python structures, converting arrays slow step.
a picky point question.
a
generator, not array. list(a)
produce nested list, can described loosely array. isn't np.array
, , not have reshape
method.
Comments
Post a Comment